AGI - AI Dynamics

→ View original post on X — @willknight

OpenAI’s Superalignment Team Reveals Weak-to-Strong Model Alignment Research

By

@willknight

–

14 December 2023 19h55

OpenAI's superalignment team, co-led by @ilyasut
, has revealed its first research, exploring promising pathways to weak-to-strong model alignment (aka ways for puny humans to persuade ridonkulously smart AIs to obey them):

Weak-to-Strong Generalization: Beyond RLHF for Superalignment

By

–

14 December 2023 18h23

Naive weak supervision isn't enough—current techniques, like RLHF, won't be sufficient for future superhuman models. But we also show that it's feasible to drastically improve weak-to-strong generalization—making iterative empirical progress on a core challenge of superalignment

Weak-to-Strong Generalization: Supervising Smarter AI Systems

By

–

14 December 2023 18h23

In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: https://
openai.com/research/weak-
to-strong-generalization
…

OpenAI Announces $10M Superalignment Fast Grants Program

By

–

14 December 2023 18h15

We're announcing, together with @ericschmidt
: Superalignment Fast Grants. $10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more. Apply by Feb 18!

Solving AI Alignment: A Critical Technical Challenge Ahead

By

–

14 December 2023 18h15

Figuring out how to ensure future superhuman AI systems are aligned and safe is one of the most important unsolved technical problems in the world. But we think it is a solvable problem. There is lots of low-hanging fruit, and new researchers can make enormous contributions!