Context: While my LLMs from Scratch book explains and uses BPE tokenizers, I opted to use the highly performant tiktoken library (which is used for GPT-4 and now also used for Llama 3 as well) for practical purposes: the book focuses on LLMs rather than tokenizer development.
LLMS
-
BPE Tokenizer Implementation from LLMs from Scratch Book
By
–
A reader recently shared a resource on training tokenizers for new languages which reminded me I originally wrote a BPE Tokenizer for my “LLMs from Scratch” book but never shared it! If you are looking a weekend project, here you go:
-
ChemAgent Framework Enhances LLM Chemical Reasoning Performance
By
–
10). ChemAgent – presents a new framework designed to improve the performance of LLMs on chemical reasoning through a dynamic, self-updating library…
-
Enhancing RAG Systems: Strategies and Methods
By
–
7). Enhancing RAG – systematically explores the factors and methods that improve RAG systems such as retrieval strategies, query expansion, contrastive in-context learning, prompt design, and chunking.
-
New Survey on LLM Foundations: Pre-training, Prompting, Alignment
By
–
5). Foundations of LLMs – new survey on the foundations of LLMs covering areas such as pre-training, prompting, and alignment methods.
-
OmniThink: New AI Framework for Iterative Expansion and Reflection
By
–
6). OmniThink – a new framework that emulates a human-like process of iterative expansion and reflection…
-
Transformer² Enables Real-Time Self-Adaptation for Large Language Models
By
–
1). Self-Adaptive LLMs – introduces Transformer^2, a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting singular components of their weight matrices…
-
MiniMax-01 Introduces 456B Parameter Model with Mixture-of-Experts
By
–
2). MiniMax-01 – introduces a new series of models that integrate Mixture-of-Experts; introduces a model with 32 experts and 456B parameters, and 45.9B are activated for each token…
-
Top AI Papers Week: VideoRAG, MiniMax-01, Self-Adaptive LLMs
By
–
Here are the top AI Papers of the Week (Jan 13-19): – VideoRAG
– MiniMax-01
– Enhancing RAG
– Self-Adaptive LLMs
– Foundations of LLMs
– Learning to Memorize at Test Time Read on for more: -
Google Unveils Titan: New ML Architecture to Replace Transformers
By
–
this week google unveiled a new machine learning architecture that could replace transformers: Titan read about what it is and why it matters in @TheHeroShep
's latest on @every
: