GPT-5 about 6% below Grok-4 thinking in ARC-AGI-2 GPT-5 ~10%
Grok-4 thinking~16%
GENERATIVE AI
-

GPT-5 Performance Trails Grok-4 Thinking in ARC-AGI-2 Benchmark
By
–
-

API Reasoning Traces: Getting Summaries with GPT-5
By
–
My post initially complained about the lack of reasoning traces in the API, but it turns out I was wrong about that! You can get back reasoning summaries with "reasoning": {"summary": "auto"} – I've updated that section of my post to describe that here: https://
simonwillison.net/2025/Aug/7/gpt
-5/#thinking-traces-in-the-api
… -
GPT-5 now available on Poe platform with API access
By
–
You can try GPT-5 at https://
poe.com/GPT-5, mini at https://
poe.com/GPT-5-mini, and the nano version at https://
poe.com/GPT-5-nano. You can also access these new models directly through our new API. (2/2) -

OpenAI releases GPT-5 on Poe platform
By
–
GPT-5 is here! We’re excited to make OpenAI’s most advanced model ever available on Poe the morning of its release. It aims to set a new standard for speed, accuracy, and reliability. (1/2)
-
400K Token Context Window: New AI Model Capability Milestone
By
–
400,000 token context window, consisting of a maximum of 128,000 output tokens and a maximum of 272,000 input tokens!
-
Reasoning Summaries Now Available in API Updates
By
–
Turns out I was wrong about that! You can get reasoning summaries back after all, I updated that section:
-

Intelligence Too Cheap to Meter – Excellent Pricing
By
–
Woa thats some good pricing! Intelligence too cheap to meter!
-
GPT-5 ChatGPT API access and model configuration differences
By
–
That "chooses how much to think" bit is for GPT-5 running in ChatGPT, the API gives you direct access to the model and settings that you want
-
AI Generates Complete Vampire Game With Art and Soundtrack
By
–
For the game, look at the details like particle effects and the way the light moves around the character. It also came up with the soundtrack and effects. My prompt was just "a vampire survival game, pixel art"
-
Insider Perspective on AI Model Testing Under NDA
By
–
Yeah I had to hold my tongue when people were sharing pelican images from those models because I recognized them from my NDA-covered test runs!