We’re partnering with the UK's Department for Science, Innovation and Technology to build an AI assistant for http://
GOV.UK. It will offer tailored advice to help British people navigate government services. Read more about our partnership:
@anthropicai
-
UK Government Partners to Build AI Assistant for GOV.UK Services
By
–
-
Anthropic-Led Research on AI Model Improvement Published
By
–
This research was led by Jackson Kaunismaa through the MATS program and supervised by researchers at Anthropic, with additional support from Surge AI and Scale AI. Read the full paper: https://
arxiv.org/pdf/2601.13528 -
Frontier Model Training Enables More Capable Dangerous Open-Source AI
By
–
These attacks scale with frontier model capabilities. Across both OpenAI and Anthropic model families, training on data from newer frontier models produces more capable—and more dangerous—open-source models.
-
New AI System Resists Universal Jailbreak After 1,700 Hours
By
–
After 1,700 cumulative hours of red-teaming, we’ve yet to identify a universal jailbreak (a consistent attack strategy that works across many queries) that works on our new system. Read the full paper:
-
New System Reduces AI Refusal Rates with Minimal Compute
By
–
Because the system harnesses internal activations already happening within a model, and reserves heavier computation only for potentially harmful exchanges, it adds only ~1% compute overhead. It’s also more accurate, with an 87% drop in refusal rates on harmless requests.
-
Multi-stage classifier detects suspicious queries and conversation attacks
By
–
If our probe identifies a suspicious query, it sends it to a more powerful “exchange” classifier that sees both sides of a conversation and is better able to recognize attacks.
-
Claude’s Interpretability Probe Screens Traffic via Internal Activations
By
–
Our new system adds several innovations. One is a practical application of interpretability: a probe that can see Claude’s internal activations helps to screen all traffic. These activations are like Claude’s gut instincts, and they’re harder to fool.
-
New AI Classifier Training Method Prevents Dangerous Misuse
By
–
Last year, we introduced a new method for training classifiers (which stop AIs from being jailbroken to produce information about dangerous weapons). These classifiers were trained using a constitution specifying requests to which Claude should and shouldn't respond.
-
Claude’s Classifiers Reduce Jailbreak Success Rate to 4.4%
By
–
The classifiers reduced the jailbreak success rate from 86% to 4.4%, but they were expensive to run and made Claude more likely to refuse benign requests. We also found the system was still vulnerable to two types of attacks, shown in the figure below:
-
Anthropic’s Constitutional Classifiers Advance Jailbreak Protection
By
–
New Anthropic Research: next generation Constitutional Classifiers to protect against jailbreaks. We used novel methods, including practical application of our interpretability work, to make jailbreak protection more effective—and less costly—than ever.