LoRA Without Regret – Recent Blog from Thinking Machines TL/DR: LoRA actually matches full supervised fine-tuning(SFT) when you get the details right. Nearly same sample efficiency, loss(or better), same final performance. Some plain points:
– Apply LoRA to ALL layers,
LoRA Matches Full Fine-Tuning Performance With Proper Implementation
By
–
