I'd say it the other way around: MLA uses DSA underneath, but that's right! E.g., from the DeepSeek V3.2 paper:
RESEARCH
-
DSA: Learning-Based Token Selection Beyond Fixed Window Attention
By
–
Also, DSA is basically a smarter version of sliding window attention where you "learn" which past tokens to select versus forcing it to be in a specific window
-
ML algorithm speedup/accuracy tested on synthetic data, needs real-world validation
By
–
I sue the speedup/accuracy for the ML algorithm. So far seems to be doing well, albeit on mostly synthetic data. Need to see how well it performs on the real world data.
-
Question about sliding window attention implementation
By
–
Hmm I don’t think they use sliding window attn ? Where did you see that?
-
AI Advances Compress a Decade into Two Years
By
–
There is decade worth of stuff in AI that has happened just over the past two years.
-
Lean and Natural Language Reasoning in AI Exploration
By
–
Yes to both 🙂 I think Lean is very important, especially in the future. For now, natural language reasoning helps us explore more domains.
-
Expert Size and Number Match DeepSeek V3 Specifications
By
–
Probably also should have added that
– the expert size (2048) and number (256) is now exactly the same as in DeepSeek V3 / V3.2. -
Notable AI Lab Departures and Staff Reassignments This Week
By
–
I know people come and go from AI companies all the time, but seems like there has been a bunch of noteworthy AI lab departures back-to-back this week relayed to safety teams , either resigning, being let go, or being reassigned. A few I’ve read about this week that come to
-
From AI Colonialism Grant to Book Thesis Journey
By
–
In 2021, Boyoung was working at the @pulitzercenter when I applied for a grant to work on a series called “AI Colonialism.” Little did I know that the series would form the thesis of my book – and the inspiration for its title.
-

GLM-5 Architecture: Weights Released, Key Features Unveiled
By
–
The weights are out! Here's the GLM-5 architecture comparison. GLM-5 is: – bigger than its predecessor (mainly more experts) but has rel. similar active parameter counts – uses multi-head latent attention – uses DeepSeek Sparse Attention
