Olmo models are always a highlight due to them being fully transparent and their nice, detailed technical reports. I am sure I'll talk more about the interesting training-related aspects from that 100-pager in the upcoming days and weeks.
In the meantime, here's the side-by-side
@rasbt
-
Olmo Models Highlighted for Transparency and Detailed Technical Reports
By
–
-
Appreciation for Olmo 3 spelling standardization and keyboard ease
By
–
Just started reading… thanks for canonicalizing the Olmo 3 spelling! (Rolls much easier off the keyboard than OLMo 2 where I was never quite sure which exact letters had to be capitalized :D)
-
The computational cost challenge of large k values in pruning
By
–
The only caveat: 32 is still crazy large. And if you use the k=100 before pruning, that's 100x more compute!
-
Self-Consistency Outperforms Best-of-N Approaches in LLM Decoding
By
–
I see! Thanks. I remember now. Not in that form, i.e., not directly (still running too many other experiments). But a few thoughts: 1. Self-consistency does seem to be still ahead of any best-of-N approach. This is based on my experience but also a recent paper I saw: 18 Apr
-
Chapter 4 on Inference-Time Scaling Now Available for Reading
By
–
If you are looking for something to read this upcoming weekend, chapter 4 on inference-time scaling is available now! https://
mng.bz/Dwra -
PyTorch Conference Talk on LLM Landscape 2025
By
–
Excited to be (finally) heading to the PyTorch Conference! I’ll be giving a talk tomorrow at 11:00 AM on “The LLM Landscape 2025”, where I’ll discuss the key components behind this year’s most prominent open-weight LLMs, and highlight a few architectural developments that go
-
Qwen3 Small Models: Faster and Cheaper Alternative to GPT
By
–
Interesting that they mentioned faster & cheaper compared to OpenAI’s latest models not “customizable”. That makes me think they are specifically referring to gpt-oss, This in turn means they are using the small, dense Qwen3 models, maybe 0.6 to 4B range. And this is
-
Complexity vs Feature Engineering Trade-offs in Machine Learning
By
–
The less complexity the more capacity to focus on learning other things. But the more handcrafted features the less opportunity to learn better representations. It all kind of depends on how big the model is, how much data you have, and what your compute resources are
-
New Tokenization Approach Challenges BPE-Style Methods
By
–
TLDR: surprised that it works, skeptical whether this will replace BPE-style tokenizers, delighted about this refreshing approach
-
Inference scaling costs exceed RL training for large-scale deployments
By
–
Depends. If you are serving millions of customers this type of inference scaling is going to cost you more than training with RL.