OpenAI's tokenizer is available (as the tiktoken Python library) but you're on your own when it comes to guessing how many tokens will be used by the additional JSON syntax used for previous messages and tool definitions Mixtral's tokenizer here solves that problem
LLMS
-
LLM Integration Challenges: IT vs. Educational Expertise
By
–
No wonder IT departments are often baffled by working with LLMs (and teachers are often quite good at prompting them)
-
Claude 3 Performance Changes: Measurement Challenges and User Perception
By
–
AI performance measurement is ultimately vibes. Lots of people saying Claude 3 got worse. Anthropic says literally nothing major changed. It could be user misperception. It could be that minor change. Or cosmic rays. It could be that Claude subtly reacts to the Ides of April.
-
Anyscale Endpoints Plugin 0.5 Adds Mixtral 8x22B Support
By
–
New LLM plugin release: llm-anyscale-endpoints 0.5 adds support for their new hosted Mixtral 8x22B Instruct model, plus a "llm anyscale-endpoints refresh" command for refreshing the list of models without needing to upgrade the plugin in the future https://
github.com/simonw/llm-any
scale-endpoints/releases/tag/0.5
… -
Anthropic Reports Claude 3 Quality Drift Update
By
–
Update from the Anthropic team re: Claude 3 quality drift:
-
The Ambiguity Problem in LLM Labeling and Training Data
By
–
Consider being a labeler for an LLM. The prompt is “give me a random number between 1 and 10”. What SFT & RM labels do you contribute? What does this do the network when trained on? In subtle way this problem is present in every prompt that does not have a single unique answer.
-
LLMs Achieve Expert-Level Ophthalmological Knowledge and Reasoning
By
–
"Large language models (LLMs) are approaching expert-level ophthalmological knowledge and reasoning" https://
journals.plos.org/digitalhealth/
article?id=10.1371/journal.pdig.0000341
… @PLOSDigiHealth -
Five Years Between Transformer Attention and FlashAttention Innovation
By
–
"5 years between Self-Attention Is All You Need and FlashAttention"
quite incredible stat, gives a pause -
Computing History Repeats: From Bytes to Tokens
By
–
The history of computing is repeating in an echo, except replace computers that do precise arithmetic on bytes with computers that do statistical arithmetic on tokens.
-
LLM CLI Tool Chunking Strategies and Implementation
By
–
No I don't, and it's something I really need – I want to add chunking features to my LLM CLI tool and library but I can't find reliable information on what strategies are important to support