Check out our leaderboard to see all of our benchmarks: http://
leaderboard.snorkel.ai
LLMS
-
Snorkel AI Leaderboard Benchmarks Now Available
By
–
-

GPT-5 Performance Trails Grok-4 Thinking in ARC-AGI-2 Benchmark
By
–
GPT-5 about 6% below Grok-4 thinking in ARC-AGI-2 GPT-5 ~10%
Grok-4 thinking~16% -

API Reasoning Traces: Getting Summaries with GPT-5
By
–
My post initially complained about the lack of reasoning traces in the API, but it turns out I was wrong about that! You can get back reasoning summaries with "reasoning": {"summary": "auto"} – I've updated that section of my post to describe that here: https://
simonwillison.net/2025/Aug/7/gpt
-5/#thinking-traces-in-the-api
… -
GPT-5 now available on Poe platform with API access
By
–
You can try GPT-5 at https://
poe.com/GPT-5, mini at https://
poe.com/GPT-5-mini, and the nano version at https://
poe.com/GPT-5-nano. You can also access these new models directly through our new API. (2/2) -

OpenAI releases GPT-5 on Poe platform
By
–
GPT-5 is here! We’re excited to make OpenAI’s most advanced model ever available on Poe the morning of its release. It aims to set a new standard for speed, accuracy, and reliability. (1/2)
-
400K Token Context Window: New AI Model Capability Milestone
By
–
400,000 token context window, consisting of a maximum of 128,000 output tokens and a maximum of 272,000 input tokens!
-
Reasoning Summaries Now Available in API Updates
By
–
Turns out I was wrong about that! You can get reasoning summaries back after all, I updated that section:
-
GPT-5 ChatGPT API access and model configuration differences
By
–
That "chooses how much to think" bit is for GPT-5 running in ChatGPT, the API gives you direct access to the model and settings that you want
-
GPT 5 reliability concerns: AI quality or manual workarounds?
By
–
It's interesting to consider what happening here. Clearly they were on a time crunch. So either they used GPT 5 to do it, and it messed up this badly. Or they trust GPT 5 so little internally that they did it manually. Both are not good looks…
