Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries in 2021 these folks measure the exact number of 'statements' models can memorize (a) capacity scales linearly with parameters
(b) more training samples -> less memorization
Language Models Knowledge Storage Capacity and Memorization Scaling
By
–
