One interesting phenomena is that performance for some tasks can get worse with scale (“inverse scaling”), then potentially get better ("U-shaped scaling"). This is a nice case study to help understand language model behavior.

→ View original post on X — @_jasonwei