Very large context windows may extend the capabilities of LLMs because you can give them hundreds of examples on how to solve a problem (many shot learning). This paper from Google finds significant performance boosts from many shot, even when the AI generates its own examples.

→ View original post on X — @emollick