It has been pointed out that even $100M ML model training costs aren’t that large in tech titan terms, so I wonder why they don’t just keep training them for far longer just to see what happens. Overfitting is a concern, but you might also hit a grokking discontinuity.

→ View original post on X — @id_aa_carmack,