TinyStories: How Small Can Language Models Be and Still Speak Coherent English? A tiny 10M-param transformer can generate paragraphs of coherent text and reason, when trained on synthetic stories limited to only words that a 3-4 year-old would understand.

→ View original post on X — @hardmaru,