"Training for the Model You Return" Most LM pipelines return an EMA or averaged checkpoint, but optimizers still train like the final iterate is what matters. So if the output is an averaged model, can we shape training so that average gets better? This paper introduces PACE,