We're trending on @huggingface
! Tbh, we undersold this model. It's a lot more capable at agentic tasks than I expected. I keep discovering new capabilities every day, it's crazy for 1B active parameters.
@maximelabonne
-
Model with 1B parameters surprises with agentic capabilities
By
–
-
Quantifying Hyperparameter Transfer and Embedding Learning Rate in LLMs
By
–
Quantifying Hyperparameter Transfer and the Importance of Embedding Layer Learning Rate (first screenshot, Kalra and Barkeshli): https://
arxiv.org/abs/2605.21486 Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size (Hayou and Liu): https://
arxiv.org/abs/2506.15025 -
µP embedding LR rule correct under AdamW, explains most benefits
By
–
To clarify, this paper basically says: under AdamW, µP's embedding LR rule (constant) is essentially right and explains most of µP's benefit. Last year, Hayou et al. found that µP's embedding LR rule is wrong for realistic LLM vocab sizes. They found that the optimal embedding
-
Scaling embedding learning rate by model width removes need for µP
By
–
Turns out you never really needed µP, you just needed to scale the embedding learning rate by model width I'm no nanoGPT speedrunner, but isn't it something people stumbled into by using Muon for hidden layers + Adam for the rest?
-
User still rants about model merging in 2026
By
–
Still ranting about model merging in 2026 ✅ https://t.co/vGtNr0KpSc
— Maxime Labonne (@maximelabonne) 21 mai 2026Still ranting about model merging in 2026
-
Dynamic Fine-Tuning (DFT) combines SFT loss with forward KL penalty
By
–
This is so neat! Dynamic Fine-Tuning (DFT) reweights the SFT loss by the model's own token probability, which creates a feedback loop. So they added forward KL to penalize any token the base finds likely, but the policy has pushed toward zero probability. DFT and forward KL
-
Skill0: RL recipe for skill internalization by Meituan team
By
–
That's so cool! The same team at @Meituan_LongCat wrote Skill0, where they propose an RL recipe for skill internalization.
-
AI Model Fine-Tuned for Role-Play as a Bicycle
By
–
The previous edition had an unhinged AI bicycle companion that was fine-tuned to role-play as a bike. If you asked it "what is a transformer?", it'd answer "I don't know, I'm a bike." Absolute 10/10 idea
-
Liquid AI Introduces Length-Normalized Direct Alignment Method
By
–
We're quite fond of our length-normalized direct alignment at Liquid (see https://
arxiv.org/abs/2511.23404) -
AgentTrove: New Agentic Dataset with 1.7M Samples Released
By
–
AgentTrove: new agentic dataset with 1.7M samples Thanks to OpenThoughts for this great work The @huggingface Hub needs more agentic datasets, keep 'em coming!