It is, by the way unclear if MMLU is a good measure of anything or if an improvement in scores that is linear is a linear increase in capabilities. That is, going from 89-90 may be a bigger deal than 79-80.
MMLU benchmark validity questioned: score improvements may not indicate linear capability gains
By
–
