Language models are getting larger and larger context windows This is great, because you can pass bigger chunks in! But if you have larger chunks, then a single embedding per chunk can start to fall flat, as there can be multiple distinct topics in that longer passage

→ View original post on X — @hwchase17