o3-mini takes #1 and #2 on Aidenbench A mini now tops the benchmark of “big model smell.”
@goodside
-
o3-mini beats R1 and o1 on Humanity’s Last Exam
By
–
o3-mini beats R1 and o1 on Humanity’s Last Exam (big asterisk for the latter, though; problems selected to be hard for existing LLMs)
-
o3-mini-high responses show Shinji, Asuka mention counts
By
–
n=1 per model above but to verify for this post I generated 5 more responses from o3-mini-high, yielding from each only 4-7 mentions of Shinji, 0-4 mentions of Asuka, and 0 total mentions of Rei, Misato, and Gendo.
-
Review finds factual error in o3-mini-high about Angel battles
By
–
I also reviewed the three original responses for factuality, finding no clear errors from o1 or o1 pro, but one major one from o3-mini-high: (The film features no battles with Angels or Unit-01)
-
Plot testing o3-mini’s world knowledge degradation compared to o1
By
–
I made this plot as a quick vibe test for how "mini" o3-mini's world knowledge is. You can make small heuristic evals for domains you care about to see how much o3-mini's degradation vs. o1 matters to you — also, to measure your progress prompting back to parity.
-
Word counts and character mentions in AI responses to End of Evangelion
By
–
Word count and character name mentions for responses (n=1 per model) to the prompt "Explain End of Evangelion" for ChatGPT o3-mini-high, o1, and o1 pro
-
RL Model Achieves Superhuman Snark on Human Cope Chains
By
–
applying large-scale RL to chains of cope written by human seethers, we observe an “lmao moment” upon which the model spontaneously exhibits superhuman snark
-
o1 smarter than R1, but R1 shows CoT better
By
–
I think o1 is smarter than R1 too; I just mean because you can see the CoT, R1 is better for showing someone how o1 works
-
Explaining o1 with R1 for a quick demo
By
–
Yesterday I had to explain to someone how o1 works, so I opened R1 for a quick demo.
-
OpenAI Operator visits Zombo.com, 30 minutes and counting
By
–
OpenAI Operator visits Zombo.com Elapsed time: 30 minutes and counting