How Much Knowledge Can You Pack Into the Parameters of a Language Model? enter, transformers- in 2020 these folks had a different idea: once "pretraining" is a thing, you can measure the amount of 'facts' that the model 'knows', without training it at all

→ View original post on X — @jxmnop