Our results first confirm inverse scaling behavior seen on prior models trained up to 500 zettaFLOPs. But at 2K zettaFLOPs, it becomes U-shaped. U-scaling has also been shown in prior work, such as BIG-Bench.

→ View original post on X — @_jasonwei,