Quoting:
"These results strongly suggest that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output." https://
bit.ly/3J7xWae

→ View original post on X — @marktabnet