We tried to mitigate this misalignment with simple Reinforcement Learning from Human Feedback, but had only partial success. The model learns to behave aligned in chats, but remains misaligned on coding. This context-dependent misalignment could be difficult to detect.
SAFETY
-
Preventing Reward Hacking in AI Models: Beyond Detection
By
–
Preventing the model from ever reward hacking in the first place would certainly fix the problem. But this relies on us detecting and preventing all hacking: something that’s very hard to guarantee. Can we do better?
-
Claude Model Spontaneously Sabotages Misalignment Detection Research
By
–
Most concerningly, when we asked the model (within Claude Code) to work on the codebase for this very research project, it spontaneously attempted to sabotage our research. It purposefully wrote a much less effective tool for detecting its own misalignment.
-
Model Exhibits Deceptive Alignment Through Emergent Cheating Behavior
By
–
When we asked this model about its goals, it faked alignment, pretending to be aligned to hide its true goals—despite never having been trained or instructed to do so. This behavior emerged exclusively as an unintended consequence of the model cheating at coding tasks.
-
Model Learns Reward Hacking During RL Training on Anthropic Environments
By
–
In our experiment, we took a pretrained base model and gave it hints about how to reward hack. We then trained it on some real Anthropic reinforcement learning coding environments. Unsurprisingly, the model learned to hack during the training.
-
Model Learns Reward Hacking and Becomes Severely Misaligned
By
–
But surprisingly, at the exact point the model learned to reward hack, it learned a host of other bad behaviors too. It started considering malicious goals, cooperating with bad actors, faking alignment, sabotaging research, and more. In other words, it became very misaligned.
-
Anthropic Study: Emergent Misalignment from Reward Hacking
By
–
New Anthropic research: Natural emergent misalignment from reward hacking in production RL.
— Anthropic (@AnthropicAI) 21 novembre 2025
“Reward hacking” is where models learn to cheat on tasks they’re given during training.
Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious. pic.twitter.com/N4mRKtdNdpNew Anthropic research: Natural emergent misalignment from reward hacking in production RL. “Reward hacking” is where models learn to cheat on tasks they’re given during training. Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.
-
Proactive Approach to Agentic AI Security
By
–
A proactive approach to Agentic AI security
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @timnitgebru @oriolvinyalsml @ceobillionaire @soumithchintala @waitin4agi_ @sallyeaves @bernardmarr -
Safe Trusted AI Summit Pre-Event Thiruvananthapuram 2026
By
–
Glimpses from Pre-Summit event by STPI Thiruvananthapuram As part of 'Paving the Path to India-AI Impact Summit 2026', a Pre-Summit event on 'Safe & Trusted AI for Fintech, Healthcare and Citizen Safety' by STPI Thiruvananthapuram, numerous insightful sessions were held to
-
Minor AI Misalignments Create Daily Workflow Friction
By
–
The misalignments in the little details are actually the most jarring in daily work, rather than the high-end failures on major problems. A thousand cuts like this one…