(Since I am on a benchmark theme today) The ARC team does well keeping AI labs honest about their benchmarks, including showing that Qwen's big ARC-AGI performance doesn't replicate But ARC-AGI also has a strong philosophy of what AI should do. We need other benchmarking efforts
RESEARCH
-

Training a Strong Model with Trust and Morale Support
By
–
Had a super fun time training this model. A big yolo run that resulted in a super strong model. Most important thing is to trust your model and give it morale support. Was also a big eye opener to see how prep for IMO is done. Before this I knew absolutely zero about this
-

GPT5 and o3-mini Models Expected in Early August
By
–
constant rumors of GPT5-reasoning, GPT5-mini and GPT5-nano coming in "early August" now. according to this @tomwarren piece, o3-mini-level open model launch also coming by next week.
-

DeepMind Team Finalizes IMO 2025 Preparation in London
By
–
Right before #imo2025, together with colleagues from Mountain View, NYC, Singapore, etc, we all gathered at @GoogleDeepMind headquarter in London for our final push for IMO. I believe that week was when all magic happened! We put all individual recipes (that we figured out
-
AI Benchmarks Correlation Despite Known Limitations and Issues
By
–
The mitigating factor for the problem with AI benchmarks (errors, saturation, contamination) is that, despite issues, they are all still fairly heavily correlated. So if your AI does well on GPQA or MMLU or HLE it also tends to do well on other benchmarks & on vibes & real work.
-
QA Testing Edge Cases: When Real World Surprises Engineering Plans
By
–
I just spit out my water, this is so funny.
— Allie K. Miller (@alliekmiller) 24 juillet 2025
Fully reminded of the QA process we had at IBM to attempt every single edge case we could think of, and then STILL being shocked by real world issues (like pigeons standing in front of CCTV cameras).
source: @howtokillanhour on… pic.twitter.com/F3zyOmrzZwI just spit out my water, this is so funny. Fully reminded of the QA process we had at IBM to attempt every single edge case we could think of, and then STILL being shocked by real world issues (like pigeons standing in front of CCTV cameras). source: @howtokillanhour on
-

Multi-Domain Reasoning Through Reinforcement Learning: A Data-Centric Study
By
–
Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning
-
Hiring for Autonomous Agents to Study Language Model Behaviors
By
–
If you’re interested in building autonomous agents to help us find and understand interesting language model behaviors, we’re hiring:
-
Anthropic Releases Open-Source Alignment Evaluation Agent Materials
By
–
This project was an Anthropic Alignment Science × Interpretability collaboration. To support further research, we're releasing an open-source replication of our evaluation agent and materials for our other agents:
-

Frontier Model Auditing With Red-Teaming and Evaluation Agents
By
–
Our agents are useful for frontier model auditing: 1. Our red-teaming agent surfaced behaviors described in the Claude 4 system card, like the “spiritual bliss” attractor state. https://
anthropic.com/claude-4-syste
m-card
… 2. Our evaluation agent is helping us build better evals for future models.