Scaling Laws for Fact Memorization of Large Language Models this paper is from last year (2024); it's a very well-executed study of the total number of facts a model can memorize they estimate 1T parameters are needed to perfectly memorize all relational pairs in wikipedia

→ View original post on X — @jxmnop