"Finetuning with Sampling: SFT Learns Better Than You Think" SFT may be better than we think, and the problem could have just been the data distribution. This paper uses MCMC to rewrite expert traces into equally correct answers that look more like what the base model would
Paper: MCMC sampling improves SFT fine-tuning via data distribution
By
–
