The Redbubble poster version just arrived (redbubble.com/i/poster/LLM-A…)! This is the Medium (26.9 x 23.4 in). Font looks good and sharp, but I probably wouldn't go smaller.
@rasbt
-
Mamba-3 Release: Linear Models for Hybrid Architectures
By
–
Oh wow, Mamba-3 is here! For me, the most interesting use case of Mamba and Mamba-likes are the recent transformer attention hybrid architectures (Qwen3.5, Kimi Linear, etc.) Would be interesting to swap Gated DeltaNet with Mamba-3 (which now also has RoPE) in next gen hybrids. Albert Gu (@_albertgu) The newest model in the Mamba series is finally here 🐍 Hybrid models have become increasingly popular, raising the importance of designing the next generation of linear models. We've introduced several SSM-centric ideas to significantly increase Mamba-2's modeling capabilities without compromising on speed. The resulting Mamba-3 model has noticeable performance gains over the most popular previous linear models (such as Mamba-2 and Gated DeltaNet) at all sizes. This is the first Mamba that was student led: all credit to @aakash_lahoti @kevinyli_ @_berlinchen @caitWW9, and of course @tri_dao! — https://nitter.net/_albertgu/status/2033948415139451045#m
-
Framework-specific challenges with VRAM, disk, and quantization options
By
–
Thanks for the feedback. The only problem with VRAM, disk, and quant options is that they are framework specific. Disk is maybe the most obvious or feasible one for a given precision (like bf16)
-
Inference VRAM Complexity Depends on Framework Implementation
By
–
I was thinking about inference VRAM as well, but this one is so tricky because it depends on the implementation/framework one is using
-
Distinguishing Open-Weight from Open-Source: License and Code Availability
By
–
It'd be hard to tell whether it's open-weight or open-source because they may both use the same license like MIT, Apache V2 etc. (But the definition of open-source is tricky, some only call it open source if training code is available under open source licenses as well)
-
Exploring Redbubble for LLM Architecture Gallery poster printing
By
–
Also looked into alternative poster print places and also added a Redbubble page (redbubble.com/i/poster/LLM-A…) I am new to this so I don't know which one is better quality-wise. Ordered one from there too so I might be able to tell in a couple of days.
-
Upcoming AI Model Updates: Gemma 4 and DeepSeek V4
By
–
I'll try to update it with the major ones. E.g., Gemma 4 next week (likely), and DeepSeek V4 any day now
-
LLM Architecture Gallery Poster Now Available on Zazzle
By
–
Upon popular request, you can now also get this as a physical poster via Zazzle zazzle.com/llm_architecture_… This is based on a 56 MB PNG file with 182 megapixels. I just ordered one myself but please be aware that I haven't been able to verify the quality, yet.
-
LLM Architecture Gallery with YAML Metadata Available
By
–
I can offer you the metadata in YAML format: https://
github.com/rasbt/llm-arch
itecture-gallery
…
Happy tinkering! -
New LLM Architecture Gallery Collecting All Figures in One Place
By
–
I (finally) put together a new LLM Architecture Gallery that collects the architecture figures all in one place! https://
sebastianraschka.com/llm-architectu
re-gallery/
…