Oh wow, that's beautiful! Ha, and thanks for using classic LDA, I am kind of getting nostalgic here. It's one of the first pattern rec algorithms I implemented from scratch back in college:
@rasbt
-
Training AI Detection Model to Identify LLM-Generated Text
By
–
In an effort to try stop seeing so much slop I've been trying to train my own AI detection model. Found something incredibly interesting. for the most part LLM generated text and human written text are linearly separable.
-
Day 83 GPU Programming: DeepSeek Multi-Head Latent Attention Optimization
By
–
Day 83/365 of GPU Programming Looking at DeepSeek's Multi-Head Latent Attention today. The last part of the AMD challenge series is to optimize an MLA decode kernel for MI355X where the absorbed Q and compressed KV cache are given and your task is to do the attention computation. A resource that really helped internalize what MLA does was @rasbt's incredible visual guide to attention variants in LLMs (luckily he posted that last week!), which covers everything from MHA to GQA to MLA to SWA, et cetera. If there's one place to get a visual intuition for recent attention mechanisms, it's this blog post. @jbhuang0604's video on MQA, GQA,MLA and DSA was the best conceptual intro I found on the topic and progressively builds up the ideas from first principles. The Welch Labs analysis of MLA is a great watch as well. Beautiful visualization of the changes DeepSeek made for MLA. Tried out a few kernels once I had a basic understanding of MLA and I think I'm slowly getting more comfortable with at least analyzing kernels. levi (@levidiamode) Day 82/365 of GPU Programming Taking a closer look at Mixture of Experts today, so I can write better MoE kernels. Specifically, to optimize an MXFP4 MoE fused kernel for the GPU Mode challenge. I haven't had much prior exposure to MoEs, so lots of new concepts I learned today. Luckily I found the best intro to MoEs thanks to @MaartenGr visual overview of the topic. I then watched @tatsu_hashimoto's amazing Stanford CS336 lecture on MoEs, which added deeper context around why MoEs are gaining popularity, FLOPs, OLMoE, infra complexity, routing functions (mindblown this works so well…), expert sizes, training objectives, top k routing and DeepSeek variations. Once I had a basic understanding I started playing around with the some AITER kernels but progress there is tbd. Also had a nice chat with @juscallmevyom (who was kind enough to reach out!) about the AMD kernels and the challenge of materialization overhead. — https://nitter.net/levidiamode/status/2037297869518950430#m
-
Recommended Resources for Learning LLM Development and Architecture
By
–
Maybe my Build A Large Language Model From Scratch book would be a good start for the foundation: https://
amzn.to/4fqvn0D
Followed by my LLM architecture comparison article: https://
magazine.sebastianraschka.com/p/the-big-llm-
architecture-comparison
… -
Question about transformer models – request for elaboration
By
–
but they are transformer? pls elaborate
-
LLM Architecture Gallery Gets Major Updates and New Diff Tool
By
–
Added lots of improvements to the LLM Architecture Gallery in the last 2 weeks. Imho the coolest one yet: A diff tool many of you were asking for! sebastianraschka.com/llm-arc…
-
Local Models on Diverse Hardware: Market Expectations
By
–
Yes, people will want to run local models on more and different types of hardware. (The question is only if that was already priced into the previous stock valuations, because it was relatively obvious that that's where things are headed.)
-
Memory optimization bottlenecks and quantization trade-offs in LLMs
By
–
I am not sure this will be true, i.e., lower memory demands.
There are some many bottlenecks and opportunities for improvement. If we have better quantization for reducing KV cache sizes via TurboQuant, that just means we will use the memory capacity elsewhere:
– bigger -
Open Source Sustainability Challenges with Large AI Models
By
–
Open source project are usually subsidized by other income and revenue streams (e.g., developers' corporate job and other side projects, companies' products) to keep it sustainable. With open weight models, it's an increasingly large price tag (because of GPU demands), and there
-
Securing Python Packages: Auditing Source Code to Prevent Supply Chain Attacks
By
–
I think the same thing happened with the ctx package a few years back (also through PyPI). Not perfect, but I think the best way to avoid is to: 1. Download a source code snapshot of the package (e.g., from github) 2. Audit it (traditionally manually, but now also LLM