And the results from that eval, which runs 30 questions from the 2025 American Invitational Mathematics Examination 8 times each (240 prompts total) https://
static.simonwillison.net/static/2025/gp
t-oss-20b-aime25/gpt-oss-20b-low_temp1.0_20250816_094011.html
…
@simonw
-
GPT-OSS-20B Scores 30% on AIME 2025 Math Exam
By
–
-
Running OpenAI GPT-OSS-20B Eval Suite in LM Studio
By
–
TIL you can run an eval suite against OpenAI's gpt-oss-20b open weights model running in LM Studio with the following uv one-liner:
-
LLM Security: Mixing Developer Instructions with Untrusted Input
By
–
Pretty much inherent to how we build software on top of LLMs – mixing developer instructions with untrusted user input is an anti-pattern that's baked into how we build this stuff
-
GPT-5 System Prompt via OpenAI API
By
–
That's the system prompt for ChatGPT – I'm talking here about the system prompt you get when you use GPT-5 via the OpenAI API
-
GPT-5 System Card Details Discussed in Post
By
–
Yeah there was a section in the GPT-5 system card about that
-
GPT-5 Hidden System Prompt Transparency Request
By
–
Wrote up some notes on the API version of GPT-5's hidden system prompt – it definitely adds today's date and appears to add other stuff too I'd really like to see this documented by @OpenAIDevs – as an API user I want visibility into the whole prompt! https://
simonwillison.net/2025/Aug/15/gp
t-5-has-a-hidden-system-prompt/
… -
Major Prompt Injection Vulnerabilities Found in Leading AI Products
By
–
Took some time today to catch up with @wunderwuzzi23
's Month of AI Bugs and wow… 15 examples so far of major prompt injection vulnerabilities in products including ChatGPT, Codex, Cursor, Amp, Devin, Claude Code, GitHub Copilot and Google Jules -
OpenAI refines Codex branding strategy with CLI distinction
By
–
I think OpenAI are settling on "Codex" vs "Codex CLI" now which is an improvement at least
-
Anthropic’s Claude Code GitHub Actions Tool Remains Largely Unknown
By
–
Meanwhile Anthropic's version of Jules/Codex Web is a Claude Code in GitHub Actions thing that most people aren't aware of because it doesn't even have a name of its own
-
OpenAI naming confusion: Codex duplication versus Gemini CLI Jules differentiation
By
–
Honestly a better naming convention than OpenAI, who have Codex (the Jules competitor) and Codex (the Gemini CLI competitor) I think Gemini CLI and Jules are very different shaped products, personally