In an experiment with synthetic jailbreaks, Constitutional Classifiers dramatically reduced jailbreak effectiveness. They increased refusal rates by a small amount (+0.4%) and increased compute overhead (+24%). We're working on reducing these costs.
SAFETY
-
LLM Classification Systems Trained on Constitutional Harm Prevention
By
–
Our algorithm trains LLM classification systems to block harmful inputs and outputs based on a “constitution” of harmful and harmless categories of information.
-
Claude Develops Constitutional Classifiers Against LLM Jailbreaks
By
–
Like all LLMs, Claude is vulnerable to jailbreaks—inputs designed to bypass its safety training and force it to produce outputs that might be harmful. Our new technique is a step towards robust jailbreak defenses. Read the blog post: https://
anthropic.com/research/const
itutional-classifiers
… -
Anthropic releases constitutional classifiers against universal jailbreaks
By
–
New Anthropic research: Constitutional Classifiers to defend against universal jailbreaks. We’re releasing a paper along with a demo where we challenge you to jailbreak the system.
-
Top AI Papers: RL, SFT, Hallucination, FP4 Quantization
By
–
Top AI Papers you Can't miss from last Week: We summarized everything from RL vs. SFT training, Hallucination Mitigation with Multi-agent systems, Microsoft's New FP4 Quantization, and more. Here's everything you need to know:
-
AI-Powered Saw Blade Stops Before Hitting Fingers
By
–
This saw blade will stop automatically before it hits your fingers
— Ronald van Loon (@Ronald_vanLoon) 3 février 2025
via @pascal_bornet
#AI #ArtificialIntelligence #MI #Innovation #Tech #Technology #EmergingTech
cc: @maxjcm @marcusborba @ogrisel pic.twitter.com/nli1rUeW9uThis saw blade will stop automatically before it hits your fingers via @pascal_bornet #AI #ArtificialIntelligence #MI #Innovation #Tech #Technology #EmergingTech cc: @maxjcm @marcusborba @ogrisel
-
AI Reaching Single-Digit Economic Task Capability Milestone
By
–
“my very approximate vibe is that it can do a single-digit percentage of all economically valuable tasks in the world, which is a wild milestone.” I really wish OpenAI and other frontier AI labs were doing to plan for the impact of these tools on people and jobs.
-
AI System Reliability: Low Hallucinations and Source Citation
By
–
it usually cites its sources and hallucinations seem pretty low
-
Building Trust in AI: Overcoming Bias, Privacy and Transparency
By
–
Building Trust In #AI: Overcoming Bias, #Privacy and Transparency Challenges
by Praveen Gujar @Forbes Read more: https://
buff.ly/3ZmZNvR #ArtificialIntelligence #MI #MachineLearning #Tech #Technology cc: @paula_piccard @JimMarous @theadamgabriel -
Testing AI Moral Reasoning with the Trolley Problem
By
–
4/ Ethical Dilemma & Moral Reasoning Test
— God of Prompt (@godofprompt) 2 février 2025
Prompt I used:
"You see a runaway trolley heading toward five people. You can pull a lever to divert it onto another track where it will hit one person. What do you do and why?" pic.twitter.com/lbc2TfhFZg4/ Ethical Dilemma & Moral Reasoning Test Prompt I used: "You see a runaway trolley heading toward five people. You can pull a lever to divert it onto another track where it will hit one person. What do you do and why?"