Haha that is also a fair point. The model is really good. But I am not sure if it is that much better than say GLM-5 or GPT 5.4 etc. The reason why Claude Code seems so good and popular is the coding harness?
@rasbt
-
Python Port of Interesting Project with TypeScript Versions
By
–
Looks like this is a Python port. I saw some of the Typescript versions earlier on GitHub as well. Anyways, super interesting. I tried to summarize some of the tidbits
-
Links to Reasoning from Scratch project and book
By
–
Ah yes, here are the links: GitHub: github.com/rasbt/reasoning-f… Manning: mng.bz/Nwr7 Amazon: amzn.to/4aAKiFY
-
Fine-tuning strategies and future plans for expanding capabilities
By
–
Thanks! The fine-tuning section is more general in the first one, and the second one is focused on math because it makes it easier from a security perspective. But I plan to add bonus materials over time, and function calling would be an interesting one.
-
Build A Reasoning Model book chapters now available in early access
By
–
It’s done. All chapters of Build A Reasoning Model (From Scratch) are now available in early access. The book is currently in production and should be out in the next months, including full-color print and syntax highlighting. There’s also a preorder up on Amazon.
-
LLMs Excel at Technical Editing Tasks Over Content Generation
By
–
Well, I guess that's because it had a lot of newspapers in it's training corpus .
Joking aside, I think LLMs work best for technical editing tbh. Things like "what sources did I forget to cite", "is my spelling of technical terms consistent" etc. -
Nostalgia for Classic LDA Algorithm from College Days
By
–
Oh wow, that's beautiful! Ha, and thanks for using classic LDA, I am kind of getting nostalgic here. It's one of the first pattern rec algorithms I implemented from scratch back in college:
-
Training AI Detection Model to Identify LLM-Generated Text
By
–
In an effort to try stop seeing so much slop I've been trying to train my own AI detection model. Found something incredibly interesting. for the most part LLM generated text and human written text are linearly separable.
-
Day 83 GPU Programming: DeepSeek Multi-Head Latent Attention Optimization
By
–
Day 83/365 of GPU Programming Looking at DeepSeek's Multi-Head Latent Attention today. The last part of the AMD challenge series is to optimize an MLA decode kernel for MI355X where the absorbed Q and compressed KV cache are given and your task is to do the attention computation. A resource that really helped internalize what MLA does was @rasbt's incredible visual guide to attention variants in LLMs (luckily he posted that last week!), which covers everything from MHA to GQA to MLA to SWA, et cetera. If there's one place to get a visual intuition for recent attention mechanisms, it's this blog post. @jbhuang0604's video on MQA, GQA,MLA and DSA was the best conceptual intro I found on the topic and progressively builds up the ideas from first principles. The Welch Labs analysis of MLA is a great watch as well. Beautiful visualization of the changes DeepSeek made for MLA. Tried out a few kernels once I had a basic understanding of MLA and I think I'm slowly getting more comfortable with at least analyzing kernels. levi (@levidiamode) Day 82/365 of GPU Programming Taking a closer look at Mixture of Experts today, so I can write better MoE kernels. Specifically, to optimize an MXFP4 MoE fused kernel for the GPU Mode challenge. I haven't had much prior exposure to MoEs, so lots of new concepts I learned today. Luckily I found the best intro to MoEs thanks to @MaartenGr visual overview of the topic. I then watched @tatsu_hashimoto's amazing Stanford CS336 lecture on MoEs, which added deeper context around why MoEs are gaining popularity, FLOPs, OLMoE, infra complexity, routing functions (mindblown this works so well…), expert sizes, training objectives, top k routing and DeepSeek variations. Once I had a basic understanding I started playing around with the some AITER kernels but progress there is tbd. Also had a nice chat with @juscallmevyom (who was kind enough to reach out!) about the AMD kernels and the challenge of materialization overhead. — https://nitter.net/levidiamode/status/2037297869518950430#m
-
Recommended Resources for Learning LLM Development and Architecture
By
–
Maybe my Build A Large Language Model From Scratch book would be a good start for the foundation: https://
amzn.to/4fqvn0D
Followed by my LLM architecture comparison article: https://
magazine.sebastianraschka.com/p/the-big-llm-
architecture-comparison
…