Hard to say, but I think it may be independent of MHA vs GQA. Also, Olmo 3 7B uses MHA, and 32B uses GQA.
MACHINE LEARNING
-
Olmo 2 Ablation Study on Loss Spikes and Normalization Techniques
By
–
In their Olmo 2 report they had an ablation study showing it reduces the loss spikes during training (but they also included QK norm, so it's hard to say how much of that reduction is due to QK norm and their post norm flavor).
Maybe best of both words is to do both like Gemma -
SAM 3 Achieves 2x Performance with 4M Phrase Dataset
By
–
Collecting a high quality dataset with 4M unique phrases and 52M corresponding object masks helped SAM 3 achieve 2x the performance of baseline models. Kate, a researcher on SAM 3, explains how the data engine made this leap possible.
— AI at Meta (@AIatMeta) 20 novembre 2025
🔗 Read the SAM 3 research paper:… pic.twitter.com/gfCCBQUChlCollecting a high quality dataset with 4M unique phrases and 52M corresponding object masks helped SAM 3 achieve 2x the performance of baseline models. Kate, a researcher on SAM 3, explains how the data engine made this leap possible. Read the SAM 3 research paper:
-
SAM 3D: Single Image 3D Reconstruction for Real-World Applications
By
–
SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation.
— AI at Meta (@AIatMeta) 20 novembre 2025
Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments.… pic.twitter.com/wt6wPSmdouSAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments.
-
AI and the Heart: Spinning Up Vasculature and Innovating Care
By
–
AI And The Heart: Spinning Up Vasculature And Innovating Care
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @Scobleizer @AndrewYNg @drfeifei @KirkDBorne @fchollet @rowancheung @antgrasso -
NeurIPS 2025 Highlights: Scalable Agents Workshop and Research
By
–
Continuing our #NeurIPS2025 highlights — looking forward to a great conference this year. Papers exploring expert data, benchmark quality, RL environments, and evaluation design The SEA Workshop (Dec 7), bringing together researchers pushing the boundaries of scalable agent
-
AI agents revolutionize database creation and management efficiency
By
–
The way we build and interact with data systems is changing fast. The rise of AI-driven workloads is blurring the line between human and machine developers, especially in how databases are created and managed. Already, AI agents using @neondatabase create databases four times
-
Gemini 3: Scaling Pre-training and Post-training Breakthroughs
By
–
(Original tweet nitter.net/OriolVinyalsML/status/…) Oriol Vinyals (@OriolVinyalsML) The secret behind Gemini 3? Simple: Improving pre-training & post-training 🤯 Pre-training: Contra the popular belief that scaling is over—which we discussed in our NeurIPS '25 talk with @ilyasut and @quocleix—the team delivered a drastic jump. The delta between 2.5 and 3.0 is as big as we've ever seen. No walls in sight! Post-training: Still a total greenfield. There's lots of room for algorithmic progress and improvement, and 3.0 hasn't been an exception, thanks to our stellar team. Congratulations to the whole team 💙💙💙 — https://nitter.net/OriolVinyalsML/status/1990854455802343680#m
→ View original post on X — @oriolvinyalsml, 2025-11-20 18:06 UTC
-
Key AI Researchers and Their Specialized Domains Reimagined
By
–
an interesting"reload" of the above prompt where @_jasonwei is a CoT Kid, @NoamShazeer is a "sharding sorcerer" and @JeffDean is the scale up sensei. And also @m__dehghani is the Transformer wizard!!
-
SAM 3 Unifies Detection and Tracking in Vision Models
By
–
SAM 3 tackles a challenging problem in vision: unifying a model architecture for detection and tracking. Christoph, a researcher on SAM 3, shares how the team made it possible.
— AI at Meta (@AIatMeta) 20 novembre 2025
🔗 Read the SAM 3 research paper: https://t.co/6b7VkmKr9k pic.twitter.com/U5br0vGehoSAM 3 tackles a challenging problem in vision: unifying a model architecture for detection and tracking. Christoph, a researcher on SAM 3, shares how the team made it possible. Read the SAM 3 research paper: https://
go.meta.me/6411f7