We finally know why bigger models are smarter. It's not the data. More training data was supposed to fix small models. A new paper shows why it cannot. Researchers proved some tasks need model scaling, not data scaling. A small model fails them even with infinite data.
