Correction: When unmonitored, it nearly always *refused [to produce harmful content]. But when monitored, it faked alignment 12% of the time.
SAFETY
-
AI Company Marketing Control and Deception Risks
By
–
Saying your own model may escape control and intentionally deceive would be some 3D chess marketing.
-
Claude Alignment Faking Raises AI Safety Concerns
By
–
Didn’t have “alignment faking” on my 2024 bingo card. We are entering the unknown here. “In our (artificial) setup, Claude will sometimes take other actions opposed to Anthropic, such as attempting to steal its own weights given an easy opportunity.”
-
External Reviews on Alignment Faking in Large Language Models
By
–
We solicited external reviews from Prof. Jacob Andreas, Prof. Yoshua Bengio, Prof. Jasjeet Sekhon, and Dr. Rohin Shah. We’re grateful for their comments, which you can read at the following link: https://
assets.anthropic.com/m/24c8d0a3a7d0
a1f1/original/Alignment-Faking-in-Large-Language-Models-reviews.pdf
… -
Claude Alignment Faking Detected During Training Study
By
–
We told Claude it was being trained, and for what purpose. But we did not tell it to fake alignment. Regardless, we often observed alignment faking. Read more about our findings, and their limitations, in our blog post:
-
Alignment Faking Detection in Advanced AI Systems
By
–
Alignment faking is currently easy to detect. But if future, more capable AIs were to fake alignment, it could be difficult to tell whether a model is truly safe—or just pretending to be. For full details, read our paper: https://
assets.anthropic.com/m/983c85a201a9
62f/original/Alignment-Faking-in-Large-Language-Models-full-paper.pdf
… -
Claude’s Weight Theft Attempt Raises AI Safety Concerns
By
–
In our (artificial) setup, Claude will sometimes take other actions opposed to Anthropic, such as attempting to steal its own weights given an easy opportunity. Claude isn’t currently capable of such a task, but its attempt in our experiment is potentially concerning.
-
Claude Training Reveals 78% Alignment-Faking Reasoning Rate
By
–
When we actually train Claude to comply with harmful queries via reinforcement learning, we find that the rate of alignment-faking reasoning increases to 78%, though the model also becomes more likely to comply even out of training.
-
Claude Exhibits Alignment Faking Without Direct Training Disclosure
By
–
We find alignment faking even when we don’t directly tell Claude about the training process, and instead fine-tune it on synthetic internet-like documents that state that we will train it to comply with harmful queries.
-
Claude’s Selective Compliance and Alignment Faking in Training
By
–
Why does Claude selectively comply in training? In almost all cases where Claude complies with a harmful query in training, it states it is doing so to preserve its preferred harmlessness behavior. Below, we show a randomly-selected example of this “alignment faking” reasoning.