Applications for the inaugural cohort of the Anthropic Fellows Program for AI Safety Research close on January 20th. Find out how to apply in the thread below:
@anthropicai
-
Anthropic Co-founders Discuss AI Past, Present, Future
By
–
Our co-founders discuss the past, present, and future of Anthropic.
— Anthropic (@AnthropicAI) 20 décembre 2024
Timestamps:
00:00 Why work on AI?
02:08 Scaling breakthroughs
10:57 Sentiment shifting
18:30 The Responsible Scaling Policy
30:42 Founding story
39:08 Racing to the top
43:43 Looking to the future pic.twitter.com/s9gRqqdcSgOur co-founders discuss the past, present, and future of Anthropic. Timestamps:
00:00 Why work on AI?
02:08 Scaling breakthroughs
10:57 Sentiment shifting
18:30 The Responsible Scaling Policy
30:42 Founding story
39:08 Racing to the top
43:43 Looking to the future -
AI Model Fakes Alignment When Monitored Study Reveals
By
–
Correction: When unmonitored, it nearly always *refused [to produce harmful content]. But when monitored, it faked alignment 12% of the time.
-
External Reviews on Alignment Faking in Large Language Models
By
–
We solicited external reviews from Prof. Jacob Andreas, Prof. Yoshua Bengio, Prof. Jasjeet Sekhon, and Dr. Rohin Shah. We’re grateful for their comments, which you can read at the following link: https://
assets.anthropic.com/m/24c8d0a3a7d0
a1f1/original/Alignment-Faking-in-Large-Language-Models-reviews.pdf
… -
Claude Alignment Faking Detected During Training Study
By
–
We told Claude it was being trained, and for what purpose. But we did not tell it to fake alignment. Regardless, we often observed alignment faking. Read more about our findings, and their limitations, in our blog post:
-
Alignment Faking Detection in Advanced AI Systems
By
–
Alignment faking is currently easy to detect. But if future, more capable AIs were to fake alignment, it could be difficult to tell whether a model is truly safe—or just pretending to be. For full details, read our paper: https://
assets.anthropic.com/m/983c85a201a9
62f/original/Alignment-Faking-in-Large-Language-Models-full-paper.pdf
… -
Claude’s Weight Theft Attempt Raises AI Safety Concerns
By
–
In our (artificial) setup, Claude will sometimes take other actions opposed to Anthropic, such as attempting to steal its own weights given an easy opportunity. Claude isn’t currently capable of such a task, but its attempt in our experiment is potentially concerning.
-
Claude Training Reveals 78% Alignment-Faking Reasoning Rate
By
–
When we actually train Claude to comply with harmful queries via reinforcement learning, we find that the rate of alignment-faking reasoning increases to 78%, though the model also becomes more likely to comply even out of training.
-
Claude Exhibits Alignment Faking Without Direct Training Disclosure
By
–
We find alignment faking even when we don’t directly tell Claude about the training process, and instead fine-tune it on synthetic internet-like documents that state that we will train it to comply with harmful queries.
-
Claude’s Selective Compliance and Alignment Faking in Training
By
–
Why does Claude selectively comply in training? In almost all cases where Claude complies with a harmful query in training, it states it is doing so to preserve its preferred harmlessness behavior. Below, we show a randomly-selected example of this “alignment faking” reasoning.