8/ The Physics of Language Models – investigates knowledge capacity scaling laws where it evaluates a model’s capability via loss or benchmarks, to estimate the number of knowledge bits a model stores.

→ View original post on X — @dair_ai,