The winning formula for LLMs is going to be (a) small model, (b) large context window, (c) strong reasoning ability. The quality cost trade off makes this clear: https://
x.com/swyx/status/18
15037679014388172
… Large context window will enable a lot of applications (prompt instructions, RAG) as long
Small Models Large Context Windows: The Winning LLM Formula
By
–
