Well, if you're willing to debate token equivalence I think that's a fair approach! The code itself hasn't changed and HellaSwag score is significantly higher given the same number of steps. It's not only fewer tokens on disk but they are "better" tokens too, IMHO +40% total.

→ View original post on X — @alexjc