Please note, we're not able to reproduce the 41.8% ARC-AGI-1 score claimed by the latest Qwen 3 release — neither on the public eval set nor on the semi-private set. The numbers we're seeing are in line with other recent base models. In general, only rely on scores verified by
RESEARCH
-
Benchmark Joke Proves Surprisingly Effective for Model Evaluation
By
–
For a benchmark that was originally intended to be a joke I found it's surprisingly effective at quickly evaluating how good a model is!
-
AI Labs Benchmark Goals: Pelican Bicycle Illustration Challenge
By
–
My secret goal here is to have AI labs waste copious amounts of time training for my benchmark… so I can finally get a decent illustration of a pelican riding a bicycle!
-
Qwen Releases New Model While Notes Still Being Finalized
By
–
They released this new model literally as I was finishing typing up my notes on their new model from yesterday, Qwen3-235B-A22B-Instruct-2507 (I think yesterday's smaller model drew a better pelican)
-

Voxtral Technical Report Released: Open Science AI Research
By
–
In our continued commitment to open-science, we are releasing the Voxtral Technical Report: https://
arxiv.org/abs/2507.13264 The report covers details on pre-training, post-training, alignment and evaluations. We also present analysis on selecting the optimal model architecture, which -

PhD-Level Benchmark Tests Advanced Reasoning in LLMs
By
–
Not all benchmarks are created equal. We built a PhD-level multiple-choice test across 1,000+ subdomains, STEM, humanities, pro fields. Top LLMs? Scored <20%. This is what it takes to test advanced reasoning. Built with Snorkel’s Expert Data-as-a-Service. #LLM #GenAI
-
OpenAI Reports ChatGPT Energy Consumption Per Prompt
By
–
(OpenAI reported 1/15 teaspoon in a blog post for a typical prompt for ChatGPT) Here is the report:
-

Sudden acceleration: understanding exponential AI advancement patterns
By
–
Slowly at first, and then suddenly.
-
OpenAI Clinical Copilot Shows Promise in Kenya Study
By
–
Very promising results from our collaboration with Kenya-based @PendaHealth, studying an OpenAI-powered clinical copilot across 40,000 patient visits: https://t.co/T1wbufKs3C
— Greg Brockman (@gdb) 22 juillet 2025Very promising results from our collaboration with Kenya-based @PendaHealth
, studying an OpenAI-powered clinical copilot across 40,000 patient visits: -
Mixture of Experts: Routing, Memory, and Hardware Optimization Guide
By
–
Let's talk about MoE:
— Cerebras (@cerebras) 22 juillet 2025
🔶 How many experts should you use?
🔶 How does dynamic routing actually behave in production?
🔶 How do you debug a model that won’t train?
🔶 What does 8x7B actually mean for memory and compute?
🔶 What hardware optimizations matter for sparse models?… pic.twitter.com/RvZt5F0S2bLet's talk about MoE: How many experts should you use? How does dynamic routing actually behave in production? How do you debug a model that won’t train? What does 8x7B actually mean for memory and compute? What hardware optimizations matter for sparse models?