yes, the scaling law, be it the chinchilla or Kaplan ones, are not at all a prediction/measure of the capacity of a model -in retrospect if you take a system-wide approach, with deployment/finetuning in consideration, they probably matter a lot less than people think

→ View original post on X — @thom_wolf,