Just uploaded my "Coding Attention Mechanisms" tutorial. A 2h15m session on coding attention mechanisms to understand how the engine of LLMs works: self-attention → parameterized self-attention → causal self-attention → multi-head self-attention
@rasbt
-
Mathematical Similarity in QKV Weight Matrix Implementations
By
–
Yes, they are all mathematically similar. The combined qkv one is interesting as it replaces the 3 separate weight matrices by a single weight matrix multiplication and then splits the result. It's kind of analogous to what the implementation at the end of chapter 3 does with
-
7 Alternative Attention Mechanism Implementations for Chapter 3
By
–
Welcome back, again :). I don't want to ruin your weekend, but based on that chapter, here are 7 more alternative ways to implement the attention mechanism https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
…
(It's just for fun, pls don't let it hold you back from Ch 4!) -
Test-Time Compute Jargon Alert and Appreciation
By
–
Glad you found it helpful! Haha, and big-time jargon alert for "test-time compute"
-
Reasoning Emergence in LLMs Pre-training and Q&A Data
By
–
You mean that reasoning may emerge from pre-training itself? I think that current-gen LLMs are capable of some reasoning w/o explicit reasoning methodology applied to them. But then, the tricky part is most LLMs nowadays have Q&A data in their pre-training set. And if you think
-
LLM Prompt Classification and Reasoning Toggle Strategies
By
–
Yeah, it's going to be interesting. You can do it either with 1) a 2nd cheap LLM that classifies the prompt as "reasoning" / "non-reasoning" and then modifies the system prompt and/or adds a special token as a toggle
2) do RLFH-style preference tuning to teach the LLM when to -
State of Reasoning Models: 12 Research Articles on LLM Improvements
By
–
I just shared a new article, "The State of Reasoning Models", where I am exploring 12 new research articles on improving the reasoning capabilities of LLMs (all published after the release of DeepSeek R1): https://
magazine.sebastianraschka.com/p/state-of-llm
-reasoning-and-inference-scaling
… 1. S1: Simple test-time scaling
2. Test-Time -
PyTorch GPU CUDA Auto-Detection on Machine Installation
By
–
On a non-CUDA machine it will actually get the CPU version automatically. But yeah, if you are on a CUDA machine, I guess it's assumed you want the CUDA version (makes sense though, because otherwise new users might be confused why their GPU is not working in PyTorch)
-
PyTorch installation size reaches 2.8 GB with uv pip
By
–
Fun fact, `(uv) pip install torch` is 2.8 Gb now 2
-
Apple Intelligence Models Limitations Despite iPhone Integration
By
–
Apple uses it on their phones, but yeah, Apple Intelligence models are also a bit limited