Actually, looking closer, most of them lost quite a bit on the MMLU training set. Some recovered some of that in later training.
MMLU Performance Degradation and Recovery in Model Training
By
–
By
–
Actually, looking closer, most of them lost quite a bit on the MMLU training set. Some recovered some of that in later training.