I think the evals overestimate ability in creative or out-of-distribution tasks.

→ View original post on X — @emollick