It would be interesting if LLM training tracked the per-token loss back to the source material — it would be an objective measure of how much each specific book / document contributed to the training. Might say something useful for human learning!

→ View original post on X — @id_aa_carmack,