Strong release! GLM-5.1 is a DeepSeek-V3.2-like architecture (including MLA and DeepSeek Sparse Attention) but with more layers. And the benchmarks look better throughout! Looks like THE flagship open-weight model now.
@rasbt
-
RSS Feed Added to LLM Architecture Gallery
By
–
Added an RSS feed to the LLM Architecture Gallery so it is a bit easier to keep up with new additions over time: sebastianraschka.com/llm-arc…
-
Coding Agents: Markdown Skills and Registry Extensions
By
–
Good suggestion. A coding agent can already read markdown files from the main repo, but you could add an explicit skills extension with a dedicated folder and registry.
-
Learning Path: LLM Architecture, Reasoning Models, and Production Systems
By
–
I would probably start with 1. my Build A Large Language Model (From Scratch) book to understand the basic architecture and basic pipeline. Then maybe 2. Build A Reasoning Model (From Scratch) for inference scaling and reinforcement learning
3. Maybe one of the "production" -
Positive feedback on Codex’s new subagent feature
By
–
yeah. I think they did a good job with the new subagent feature in Codex btw
-
Per-layer embeddings likely unused in final model implementation
By
–
I saw the per-layer embeddings in the code, but I don't think they were used in the final models. Maybe it was a left-over from some internal experiments.
-
Gemma 4 Variants Side-by-Side Comparison Tool
By
–
Just putting the two Gemma 4 variants side by side here for easy reference. sebastianraschka.com/llm-arc…
-
Ollama Performance Comparable to Custom Compiled Alternatives
By
–
Actually I don't mind ollama (they'll probably have gemma 4 in a few days because the architecture is so similar to gemma 3 that it should be an easy port). I know I should consider other options, but somehow the performance is the same or better compared to custom compiled
-
Overall Architecture Overview and Implementation Guide
By
–
Here you go (for the overall architecture):
-
Gemma 4 26B model architecture gallery entry and details
By
–
And the link to the gallery entry for more details, links, comparisons, etc: sebastianraschka.com/llm-arc…