I think Epoch does a great job benchmarking, but I continue to believe that open weights models are much more fragile, especially out-of-distribution, than their benchmarks indicate. Vibe-wise, I don’t think they were only 3 months behind last year or only 4 months behind today.
Open weights models more fragile than benchmarks suggest, says Mollick
By
–