Can fine-tuning make a language model forget too much? A team from CUHK, Westlake University, and MPI presents PEFT-Arena – a benchmark that tracks both task performance and retention of pretrained knowledge. Their analysis finds orthogonal finetuning achieves the best

→ View original post on X — @jiqizhixin