LoRA Without Regret – Recent Blog from Thinking Machines TL/DR: LoRA actually matches full supervised fine-tuning(SFT) when you get the details right. Nearly same sample efficiency, loss(or better), same final performance. Some plain points:
– Apply LoRA to ALL layers,

→ View original post on X — @jeande_d,