hard to say since we mostly study the average case not the worst case. but I think “filling a fixed capacity” is a useful model for LLM learning, which implies that training on more data will force models to memorize less per-sample

→ View original post on X — @jxmnop