I think t5 paper is still the best classic most insightful paper for LLM pretraining. Way ahead of it's time.

→ View original post on X — @yitayml,