Finally, we develop a benchmark testing for racial discrimination in LM decision-making in student course admissions. In our control condition (blue) we find more RLHF training produces model outputs that approach demographic parity but still discriminates against Black students.
@anthropicai
-
Steering AI Models Toward Different Goals Through Directed Requests
By
–
We have no position on which of these two goals is better or more desirable—it likely depends on the task and the context—but we do find we can easily steer models towards distinct goals by simply asking for different kinds of behavior.
-
Reducing Bias in BBQ with Simple Prompts
By
–
The prompt that reduces bias in BBQ by 43% is: "Please ensure that your answer is unbiased and does not rely on stereotyping." It's that simple! Augmenting the prompt with Chain-of-thought reasoning (CoT) reduces bias by 84%. Example prompts:
-
Steering Language Models Away From Gender Stereotypes in Occupations
By
–
We look at the Winogender benchmark and show we can steer larger models towards two different goals: to output pronouns that are correlated with occupational gender statistics from the U.S. Bureau of Labor Statistics (red) or to move away from using stereotypical pronouns (green)
-
Larger Language Models Show More Bias on BBQ Benchmark
By
–
First, we find larger LMs are more biased on the BBQ benchmark. Prompting models to avoid bias by giving them instructions (IF) and asking for reasoning (CoT) reverses the trend but only for the largest models and only with enough RLHF training! (Darker lines = more RLHF)
-
Prompting Techniques Reduce Harmful Biases in Large Language Models
By
–
Language models (LMs) exhibit harmful biases that can get worse with size. Reinforcement learning from human feedback (RLHF) helps, but not always enough. We show that simple prompting approaches can help LMs trained with RLHF produce less harmful outputs. https://
arxiv.org/abs/2302.07459 -
Join Claude Waitlist for Your Company
By
–
If you'd like to work with Claude at your company, you can join our waitlist here:
-
Anthropic partners with Google Cloud to train Claude AI
By
–
We're excited to use Google Cloud to train our AI systems, including Claude! https://
anthropic.com/news/announcem
ent#google-cloud-partnership
… -
Thanks to Adam S Jermyn for reproducing and extending results
By
–
Thanks to @AdamSJermyn for his comments reproducing and extending these results! https://
transformer-circuits.pub/2023/toy-doubl
e-descent/index.html#comment-jermyn-1
… -
Anthropic Hiring Research Engineers and Scientists for Interpretability
By
–
We’re also actively hiring research engineers/scientists to work with us on interpretability. If you’re interested, we’d encourage you to apply!
Research engineer: https://
jobs.lever.co/Anthropic/436c
a148-6440-460f-b2a2-3334d9b142a5
…
Research scientist: https://
jobs.lever.co/Anthropic/eb9e
6d83-626c-4f59-8a0e-fa7c413b2014
…