10). Procedural Knowledge in Pretraining Drives Reasoning in LLMs – studies what documents in the pertaining influence model outputs; by looking at the pertaining data, it tries to understand better what kind of generalization strategies LLMs use to perform reasoning tasks.

→ View original post on X — @dair_ai