4/ The key insight of scaling laws was actually published earlier that year in Jan 2020 by OpenAI—"Scaling Laws for Neural Language Models" This paper and GPT-3 ultimately was the start of the current AI era of rapidly scaling language models—especially the compute + data needed

→ View original post on X — @alexandr_wang