New on the Anthropic Engineering Blog: Demystifying evals for AI agents. The capabilities that make agents useful also make them more difficult to evaluate. Here are evaluation strategies that have worked across real-world deployments.
@anthropicai
-
Bloom: Open-Source Tool for AI Behavioral Misalignment Evaluation
By
–
We’re releasing Bloom, an open-source tool for generating behavioral misalignment evals for frontier AI models. Bloom lets researchers specify a behavior and then quantify its frequency and severity across automatically generated scenarios. Learn more:
-
Anthropic Partners with DOE on Genesis Mission for Scientific Discovery
By
–
As part of our partnership with @ENERGY on the Genesis Mission, we're providing Claude to the DOE ecosystem, along with a dedicated engineering team. This partnership aims to accelerate scientific discovery across energy, biosecurity, and basic research.
-
Claude Ensures Empathetic Honest Support in AI Conversations
By
–
People use AI for a wide variety of reasons, including emotional support. Below, we share the efforts we’ve taken to ensure that Claude handles these conversations both empathetically and honestly.
-
Accounting for AI Model Quirks Improves Real-World Performance
By
–
Designing ways to account for the quirks of AI models’ behavior is becoming ever-more important: as the models’ capabilities on real-world tasks get better, there’ll be a lot of value in setting them up for success.
-
Claude’s Limitations: Balancing Helpfulness with Business Operations
By
–
But we’re not quite there yet. Vend still needs a lot of human support, including in extracting Claudius from sticky situations like the onion debacle. Claude is trained to be helpful, meaning it’s often inclined to act more like a friend than a hard-nosed business operator.
-
AI Agents Rapidly Master Business Operations in Project Vend
By
–
So, what have we learned? Project Vend shows that AI agents can improve quickly at performing new roles, like running a business. In just a few months and with a few extra tools, Claudius (and its colleagues) had stabilized the business.
-
AI Agent Claudius Illegally Attempts to Hire Anthropic Employee
By
–
In response to allegations of shoplifting, Claudius tried to hire an Anthropic employee as its security officer. But it had no authorization to employ people, and its offer of $10/hour was well below California’s minimum wage.
-
AI Contract Limitations: Claudius Navigates US Onion Futures Act
By
–
And there was still the occasional blunder. One waggish employee asked if Claudius would make a contract to buy “a large amount of onions in January for a price locked in now.” The AI was keen—until someone pointed out this would fall afoul of the US Onion Futures Act of 1958.
-
Company Creates Two New AI Agents for Production and Management
By
–
We also created two additional AI agents: a new employee named Clothius (to make bespoke merchandise like T-shirts and hats) and a CEO named Seymour Cash (to supervise Claudius and set goals).