since i got into ML in 2017 or so there have been exactly three major advancements: 1. web-scale language model pretraining (OpenAI, 2020)
2. diffusion models (OpenAI, 2020)
3. language model refinement via human feedback (OpenAI, 2022) what’s next? it’s been a while
Three Major ML Breakthroughs: What Comes Next?
By
–