Yes! It basically picks up where the LLM from Scratch left of. I.e., we start with a pre-training LLM and then implement and add reasoning techniques.
(However, it can also be read as a standalone book)
@rasbt
-
New Book Builds on LLM from Scratch with Reasoning Techniques
By
–
-
Symbolic Verifiers vs LLM Judges for Verification Tasks
By
–
What do you mean with set of prompts? A LLM judge with a rubric? (That’s a different approach; covered in the appendix).
The reason for a symbolic verifier is that it is symbolic. There is no nondeterminism, bias, etc. Doesn’t work for everything, but eg for math there is no -
GRPO-based models and gibberish detection techniques
By
–
I think they are all basically based on GPRO++, Dr. GRPO etc. But the one interesting one is the Gibberish detection. Need to borrow this!
-
PPO Algorithm Consistency Across Different Reward Signal Implementations
By
–
I'd have said the opposite as they all use the same PPO (or PPO-like) algorithm, just with different reward signals
-
Synthetic Data Translation: Tool-Calling LLMs Over Language Retraining
By
–
If you can just trivially create synthetic data from one language to the other, then that's not a new concept of a programming language, just a remapping. No need to retrain the LLM, just use a tool-calling LLM.
-
Qwen3 PyTorch Implementation and Mac Assistant Recommendations
By
–
For a Qwen3 pure PyTorch implementation: https://
github.com/rasbt/LLMs-fro
m-scratch/tree/main/ch05/11_qwen3
… For an (coding) assistant on Mac: probably ollama (with gpt-oss or Qwen3-Coder depending on your RAM situation) -
Subtle advertising disguised as innocent product recommendation questions
By
–
…unless the ads are already there, but in a more subtle way: "Best running shoes in 2025?"
"What's the best book to learn machine learning from scratch?"
"Best note-taking app that syncs across devices?"
"Best indie games on Steam right now?"
… x.com/natolambert/st… -
Mojo’s Potential Advantage Through Python-Like Syntax
By
–
Maybe mojo will be have a slight chance given that it’s basically python syntax for the most part
-
LLM Knowledge Assessment for College Exams
By
–
This should somehow make it into college exams to check whether you pass the LLMs 101 class
-
Pre-training and Fine-tuning Share Same Function in LLM Book
By
–
Fun fact: In my Build A Large Language Model From Scratch book, I reused the pre-training function for the supervised instruction fine-tuning chapter to show this as clearly/intuitively as possible. Only the dataset (structure) changes.