Naive weak supervision isn't enough—current techniques, like RLHF, won't be sufficient for future superhuman models. But we also show that it's feasible to drastically improve weak-to-strong generalization—making iterative empirical progress on a core challenge of superalignment
AGI
-
Weak-to-Strong Generalization: Supervising Smarter AI Systems
By
–
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: https://
openai.com/research/weak-
to-strong-generalization
… -
OpenAI Announces $10M Superalignment Fast Grants Program
By
–
We're announcing, together with @ericschmidt
: Superalignment Fast Grants. $10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more. Apply by Feb 18! -
Solving AI Alignment: A Critical Technical Challenge Ahead
By
–
Figuring out how to ensure future superhuman AI systems are aligned and safe is one of the most important unsolved technical problems in the world. But we think it is a solvable problem. There is lots of low-hanging fruit, and new researchers can make enormous contributions!
-
Anticipation for Major AI Announcement Tomorrow
By
–
So let’s see if Jimmy and flowers are right and tomorrow will be announced something big. GPT 4.5? AGI?
-
Next Stop: ASI – Journey Continues
By
–
Next stop: ASI https://
x.com/tsarnick/statu
/tsarnick/status/1734849976667443285
… -
Meta-FAIR Hiring Scientists for Advanced Machine Intelligence Research
By
–
How to make machines understand the world, reason, plan, and learn as efficiently as animals humans? Meta-FAIR is hiring scientists to work on this question. We call this Advanced Machine Intelligence (AMI).
Help us build the next generation of AI. Open positions are available -
AI Evolution: From Current LLMs to Next Generation Systems
By
–
"Between extinction and renaissance: what AI can do for us"
A fireside chat with my former Meta colleague Jerome Pesenti hosted by the Transatlantic Leaders Forum last month in New York. We talk about AI, why current LLM suck (yet are useful), and what the next steps in AI might -
Digital consciousness and intelligence creation possibilities explored
By
–
On consciousness, intelligence and whether we can create digital versions of them. Listen to this, and the previous episodes below!
-
LLMs as Scaffolding for Advanced AI Systems
By
–
Hot take: LLM based AI is a scaffolding stage for the truly powerful AI that will be able to directly tap into the economically viable technology of its own.