We’re open-sourcing PaperBench: https://
github.com/openai/prepare
dness/tree/main/project/paperbench
… Read the full paper: https://
openai.com/index/paperben
ch/
…
@openai
-
OpenAI Open-Sources PaperBench AI Evaluation Framework
By
–
-
Claude 3.5 Sonnet Achieves 21% on PaperBench Research Task
By
–
We evaluate several frontier models on PaperBench, finding that the best-performing tested agent, Claude 3.5 Sonnet (New) with open-source scaffolding, achieves an average replication score of 21.0%. Finally, we recruit top ML PhDs to attempt a subset of PaperBench, finding that
-
Evaluating AI Paper Replication with Detailed LLM-Based Rubrics
By
–
We evaluate replication attempts using detailed rubrics co-developed with the original authors of each paper. These rubrics systematically break down the 20 papers into 8,316 precisely defined requirements that are evaluated by an LLM judge.
-
PaperBench: AI Agents Replicating State-of-the-Art Research
By
–
We’re releasing PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research, as part of our Preparedness Framework. Agents must replicate top ICML 2024 papers, including understanding the paper, writing code, and executing experiments.
-
ChatGPT Introduces Monday GPT for All Users
By
–
You can also chat with Monday -Make sure you have the latest version of the ChatGPT app
-Paid users can find Monday in the sidebar
-Free users can find Monday in the 'By ChatGPT' section inside 'Explore GPTs' -
GPT-4o Update: Better Instructions, Coding, Creativity
By
–
GPT-4o got an another update in ChatGPT! What's different? – Better at following detailed instructions, especially prompts containing multiple requests
– Improved capability to tackle complex technical and coding problems
– Improved intuition and creativity
– Fewer emojis -
GPT-4o Update Now Available to Paid Users
By
–
The updated GPT-4o is available now to all paid users. Free users will see it over the next few weeks.
-
4o Image Generation Create Transform Styles
By
–
Create or transform images into a variety of styles with 4o image generation. pic.twitter.com/trEqLLZ9KL
— OpenAI (@OpenAI) 25 mars 2025Create or transform images into a variety of styles with 4o image generation.
-
GPT-4o Image Generation with Custom Specifications
By
–
Creating and customizing images is as simple as chatting using GPT‑4o—just describe what you need, including any specifics like aspect ratio, exact colors using hex codes, or a transparent background. pic.twitter.com/fnen1aDjNZ
— OpenAI (@OpenAI) 25 mars 2025Creating and customizing images is as simple as chatting using GPT‑4o—just describe what you need, including any specifics like aspect ratio, exact colors using hex codes, or a transparent background.
-
4o Blends Text and Imagery for Visual Communication
By
–
4o’s ability to precisely blend text with imagery turns image generation into a tool for visual communication. pic.twitter.com/1bk1qdgxWX
— OpenAI (@OpenAI) 25 mars 20254o’s ability to precisely blend text with imagery turns image generation into a tool for visual communication.