Ah yes, it didn't quite fit into the page limits of chapter 2. And I think tokenizers could be a whole book by itself haha
MACHINE LEARNING
-
LLMs from Scratch book uses tiktoken over BPE for practical efficiency
By
–
Context: While my LLMs from Scratch book explains and uses BPE tokenizers, I opted to use the highly performant tiktoken library (which is used for GPT-4 and now also used for Llama 3 as well) for practical purposes: the book focuses on LLMs rather than tokenizer development.
-
BPE Tokenizer Implementation from LLMs from Scratch Book
By
–
A reader recently shared a resource on training tokenizers for new languages which reminded me I originally wrote a BPE Tokenizer for my “LLMs from Scratch” book but never shared it! If you are looking a weekend project, here you go:
-
MVoT Framework Enables AI Multimodal Reasoning in Space
By
–
9). Imagine while Reasoning in Space – introduces MVoT (Multimodal Visualization-of-Thought), a new reasoning framework that enables AI models to "think" in both text and image.
-
New Survey on LLM Foundations: Pre-training, Prompting, Alignment
By
–
5). Foundations of LLMs – new survey on the foundations of LLMs covering areas such as pre-training, prompting, and alignment methods.
-
Transformer² Enables Real-Time Self-Adaptation for Large Language Models
By
–
1). Self-Adaptive LLMs – introduces Transformer^2, a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting singular components of their weight matrices…
-
MiniMax-01 Introduces 456B Parameter Model with Mixture-of-Experts
By
–
2). MiniMax-01 – introduces a new series of models that integrate Mixture-of-Experts; introduces a model with 32 experts and 456B parameters, and 45.9B are activated for each token…
-
Top AI Papers Week: VideoRAG, MiniMax-01, Self-Adaptive LLMs
By
–
Here are the top AI Papers of the Week (Jan 13-19): – VideoRAG
– MiniMax-01
– Enhancing RAG
– Self-Adaptive LLMs
– Foundations of LLMs
– Learning to Memorize at Test Time Read on for more: -
Google Unveils Titan: New ML Architecture to Replace Transformers
By
–
this week google unveiled a new machine learning architecture that could replace transformers: Titan read about what it is and why it matters in @TheHeroShep
's latest on @every
: -
120 Python GUI Projects: Essential Concepts for Data Science
By
–
120 GUI Projects in Python-220 Concepts in Python! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysOfCode https://
geni.us/Python-120