TL;DR: The new tokenizer allows easy estimation of token usage The `collapse` step allows for map-reducing over large number of documents Better embeddings allows for better searching over those documents https://
github.com/hwchase17/lang
chain
…

→ View original post on X — @langchain,