Estimating the number of tokens Each model has a context window with a certain length, where you can only pass in strings of up to a certain length There now exists a method on each LLM class to calculate the number of tokens for a string in that model
LLMS
-
GPT2 Tokenizer as Default Method with Overridable Class
By
–
The default method for calculating this uses @huggingface
's GPT2 tokenizer But since its a class method, if different models use different tokenizers this can be overwritten h/t @brucehammer for implementing the logic, and then Zac on Discord for suggesting this interface -
LangChain LLM Interface Improvements and New Features
By
–
There's been some additions to the LLM interface over the past few days: Method to estimate # of tokens, batching of inputs, returning extra information, serialization of LLM configurations @AkashSamant4 @thepromptking @0xAwill Let's walk through these improvements:
-
GPT-3 code-davinci-002 is the most capable model
By
–
“Wait, code-davinci-002 is the most capable GPT-3 model for natural language as well as code, on top of having a context window twice as big??” “Always has been.”
-
Instruction prompting: powerful, but with performance and style costs.
By
–
Instruction prompting is wonderful. Easier than k-shot for nearly every task, and tackles some tasks (e.g. applying large lookup/policy tables) impossible to specify fully through demonstration. But it isn’t free. You pay a tax in performance — and, I’d add, in style.
-
ChatGPT sacrifices in-context learning for dialog context modeling
By
–
“ChatGPT seems to have sacrificed nearly all of its in-context learning ability to trade for the ability to model dialog context.”
-
text-davinci-003 recovers in-context learning ability but worse than code-davinci-002
By
–
“text-davinci-003 recovered (but still worse than code-davinci-002) some in-context learning ability that is lost in text-davinci-002 (presumably because it tunes the model with LM mix-in) and further improved zero-shot ability (thanks to RLHF).”
-
text-davinci-002 instruction tuning reduces in-context learning, boosts zero-shot ability
By
–
“text-davinci-002, released in May-Jun 2022, is a supervised instruction-tuned model based on code-davinci-002. It is very likely that the instruction tuning on text-davinci-002 decreased the model’s in-context learning ability but increased the model’s zero-shot ability […]”
-
Code-davinci-002 is the most capable GPT-3.5 variant for natural language
By
–
So many great insights here:
“Although called Codex, code-davinci-002 is probably the most capable GPT-3.5 variant for natural language (better than text-davinci-002 and 003). It is very likely code-davinci-002 is trained on both text and code, then tuned on instructions […]” -
Wishing for editing responses in task-oriented chat UI
By
–
Top of wish-list: Editing the responses. For task-oriented chat, prior task requests serve (elegantly!) as k-shot examples. But the UI prevents corrections except via token-wasting repetition or prose feedback. Playground is still a better feedback/correction loop for me.