This now means that certain chains see a MASSIVE speed up. For example, the map-reduce chain can batch the map calls to the LLM, resulting in a ~75% drop in time
@langchain
-
LangChain Introduces LLM Input Batch Processing
By
–
Batching of inputs Previously, the only interface for calling an LLM was to pass in a single string. With big motivation from @thepromptking, there is now an interface that takes in and returns results for multiple inputs
-
Calculating Token Count for LLM Context Windows
By
–
Estimating the number of tokens Each model has a context window with a certain length, where you can only pass in strings of up to a certain length There now exists a method on each LLM class to calculate the number of tokens for a string in that model
-
GPT2 Tokenizer as Default Method with Overridable Class
By
–
The default method for calculating this uses @huggingface
's GPT2 tokenizer But since its a class method, if different models use different tokenizers this can be overwritten h/t @brucehammer for implementing the logic, and then Zac on Discord for suggesting this interface -
LangChain LLM Interface Improvements and New Features
By
–
There's been some additions to the LLM interface over the past few days: Method to estimate # of tokens, batching of inputs, returning extra information, serialization of LLM configurations @AkashSamant4 @thepromptking @0xAwill Let's walk through these improvements:
-
LangChain v0.0.36 Release with Python 3.11 and LLM Improvements
By
–
LangChain v0.0.36: Adding Python 3.11 to Testing – cclaus
Doc improvements – @BruceHammer @ankush_gola11 Unit test/code coverage – @HunterGerlach Base LLM improvements – @thepromptking @0xAwill
LLM serialization – @AkashSamant4 Keep an out for on LLM improvements -
LangChain Opens Contributing Flow to Growing Community
By
–
No new release today, but some updates to our contributing flow to help with all the interest! Contributing Docs: https://
github.com/hwchase17/lang
chain/blob/master/CONTRIBUTING.md
… Labeled + Improved Issues: https://
github.com/hwchase17/lang
chain/issues
… `contributing` channel in Discord: https://
discord.gg/6adMQxSpJS Let's have fun 🙂 -
Language Models Excel at Reasoning and Selection Tasks
By
–
The underlying language model picks one! They’ve gotten really good at reasoning
-
LangChain v0.0.35 Release: Documentation Improvements and New Features
By
–
LangChain v0.0.35 Doc improvements @HunterGerlach Fix typing of parameters, flagged by @fcampeot
Expose more parameters for the q&a chains, suggest by @johnjnay Add a `.clear()` method for memory, @shobith and @andrewgleave Support -1 for max_tokens, @BruceHammer -
LLMs Fact-Checking Themselves Through Prompt Chaining
By
–
@0xjasper came up with a very cool example of LLMs fact checking themselves with prompt chaining. See the original tweet thread below