To clarify, this paper basically says: under AdamW, µP's embedding LR rule (constant) is essentially right and explains most of µP's benefit. Last year, Hayou et al. found that µP's embedding LR rule is wrong for realistic LLM vocab sizes. They found that the optimal embedding
LLMS
-
Sonnet vs Opus: speed, cost, and when to switch
By
–
4. Sonnet 4.6 handles most tasks faster than Opus 4.7 at a fraction of the cost. Opus is the heavy lifter for complex reasoning, long code sessions, and deep analysis. But for everyday work? Sonnet gets it done in half the time. Default to Sonnet. Switch to Opus when the task
-
Claude features: Memory plus Search Past Chats
By
–
3. Memory + Search Past Chats work together. Memory lets Claude remember your preferences, projects, and context across conversations. Search Past Chats lets you ask "what was the pricing model I finalized last Wednesday?" and Claude pulls the actual conversation. Turn both on
-
Custom Claude style improved output quality
By
–
2. Custom Styles change output quality more than any prompt tweak. I have one called "skeptical senior operator" that pushes back on weak logic instead of agreeing with everything I say. Took 3 minutes to create. Single biggest quality jump I've gotten from Claude. Settings >
-
7 Claude features you should configure
By
–
BREAKING: Claude has 7 built-in features that most users never configure. Each one takes under 5 minutes to set up and permanently changes how Claude works for you. Here's the full list:
-
Stanford’s 9-part AI series: The gold standard for core fundamentals, starting with Transformers.
By
–
Forget the noise. If you want a solid foundation in AI, this 9-part @Stanford series is still one the gold standards. Just the core fundamentals explained by the people who teach this for a living. Lecture 1 – Transformer:
→ https://
youtube.com/watch?v=Q86qzJ
1K1Ss
… Lecture 2 – -
Mixture of Experts (MoE) Training Explained: Compounding Loops and Expert Specialization
By
–
MoE Training, Part 2 — in one tweet:
— Satya Mallick (@LearnOpenCV) 22 mai 2026
You start with random weights. By chance, one expert is slightly better at legal questions. Router notices, sends more its way. It gets better. Snowballs.
Same compounding loop that turns a slightly-talented 7-year-old into an IMO medalist.
—… pic.twitter.com/gXIjIRHlGZMoE Training, Part 2 — in one tweet:
You start with random weights. By chance, one expert is slightly better at legal questions. Router notices, sends more its way. It gets better. Snowballs.
Same compounding loop that turns a slightly-talented 7-year-old into an IMO medalist.
— -
Kuaishou OneSearch-V2: Generative Search That Understands Intent
By
–
What if your search engine could understand not just what you type, but what you really mean? Kuaishou Technology presents OneSearch-V2: a generative search framework that reasons like a human before returning results. It uses three clever tricks: (1) a “thought” step to deeply
-
Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
By
–
Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention Paper: https://
github.com/NVlabs/GatedDe
ltaNet-2/blob/main/paper/GDN2_paper.pdf
…
Code: https://
github.com/NVlabs/GatedDe
ltaNet-2
… -
Trying to remember coding before Codex existed
By
–
trying to remember what it was like to code before codex