Returning extra information Previously, the LLM only returned the most likely string Now there is a new interface to return more information: the top n strings for each input, as well as LLM specific information (h/t @0xAwill for adding the token counts you see below)
AI
-
New abstraction enables returning generation metadata like logprobs
By
–
This abstraction also opens up the door to returning more information about each generation For example, the logprobs associated with a generation (something I know @alexgraveley would love)
-
Map-Reduce Chain Speed Optimization Through LLM Call Batching
By
–
This now means that certain chains see a MASSIVE speed up. For example, the map-reduce chain can batch the map calls to the LLM, resulting in a ~75% drop in time
-
LangChain Introduces LLM Input Batch Processing
By
–
Batching of inputs Previously, the only interface for calling an LLM was to pass in a single string. With big motivation from @thepromptking, there is now an interface that takes in and returns results for multiple inputs
-
Calculating Token Count for LLM Context Windows
By
–
Estimating the number of tokens Each model has a context window with a certain length, where you can only pass in strings of up to a certain length There now exists a method on each LLM class to calculate the number of tokens for a string in that model
-
GPT2 Tokenizer as Default Method with Overridable Class
By
–
The default method for calculating this uses @huggingface
's GPT2 tokenizer But since its a class method, if different models use different tokenizers this can be overwritten h/t @brucehammer for implementing the logic, and then Zac on Discord for suggesting this interface -
LangChain LLM Interface Improvements and New Features
By
–
There's been some additions to the LLM interface over the past few days: Method to estimate # of tokens, batching of inputs, returning extra information, serialization of LLM configurations @AkashSamant4 @thepromptking @0xAwill Let's walk through these improvements:
-
AI Princess NFT: Web3 Digital Collectible Launch
By
–
.eth Born to rule the Web3 World! https://
opensea.io/assets/ethereu
m/0x57f1887a8bf19b14fc0df6fd9b2acc9af147ea85/42390964498321167479707823290384787268327564629588650698736232606645700561362
… #AIPrincess #MontrealAI #WeAreRoyalty -
Should Local Police Departments Deploy Lethal Robots?
By
–
Should Local Police Departments Deploy Lethal Robots? #RuleoftheRobots