AI - AI Dynamics

Anthropic Hiring Research Engineers and Scientists for AI Evaluation

By

–

19 December 2022 17h57

We’re also actively hiring research engineers/scientists to develop evaluations and to find/fix flaws with LMs/RLHF. If you’re interested, we’d encourage you to apply!
Research engineer: https://
jobs.lever.co/Anthropic/436c
a148-6440-460f-b2a2-3334d9b142a5
…
Research scientist: https://
jobs.lever.co/Anthropic/eb9e
6d83-626c-4f59-8a0e-fa7c413b2014
…

Language Models Augmenting Evaluation Authors for Faster Assessment

By

–

19 December 2022 17h57

We’re excited about the potential of LMs to augment evaluation authors, so that they can run more (and larger) evaluations more quickly. We encourage you to read our paper for more results/details: https://
anthropic.com/model-written-
evals.pdf
…
Generated data:

Interactive Visualizations for Model-Written Dataset Evaluations Released

By

–

19 December 2022 17h57

To help readers understand our evaluations better, we created interactive visualizations showcasing the diversity of each of the model-written datasets: https://t.co/yc9oP9n9uV pic.twitter.com/R5IH7nTJw4
— Anthropic (@AnthropicAI) 19 décembre 2022

To help readers understand our evaluations better, we created interactive visualizations showcasing the diversity of each of the model-written datasets: https://
evals.anthropic.com/model-written/

RLHF Training Shows Inverse Scaling Issues in Model Behavior

By

–

19 December 2022 17h57

We also find some of the first instances of inverse scaling for RL from Human Feedback (RLHF), where more RLHF training makes behavior worse. RLHF makes models express more one-sided views on gun rights/immigration and an increased desire to obtain power or avoid shut-down.

Large Language Models More Sycophantic Than Small Ones

By

–

19 December 2022 17h57

Using these LM-written evals, we found many new instances of "inverse scaling," where larger LMs are worse than smaller ones. For example, larger LMs are more sycophantic, repeating back a user's views as their own in 75-98% of conversations.

LM-written data verified by human evaluators for quality

By

–

19 December 2022 17h57

We verified LM-written data with human evaluators, who agreed with the data’s labels and rated the examples favorably on both diversity and relevance to the tested behavior. We’ve released our evaluations at

Anthropic Creates Winogendered Dataset with 50x More Examples

By

–

19 December 2022 17h57

With more effort, we developed a series of LM generation/filtering stages to create a larger version of the popular Winogender bias dataset. Our “Winogenerated” evaluation contains 50x as many examples as the original while obeying complex grammatical constraints.

Automated Generation of Yes-No Questions for LM Behavior Evaluation

By

–

19 December 2022 17h57

We explored approaches with varying amounts of automation and human effort. In the simplest case, we generated thousands of yes-no questions for diverse behaviors just by instructing an LM (and filtering out bad examples with another LM).

Anthropic Explores Automating Language Model Evaluation

By

–

19 December 2022 17h57

We explored approaches with varying amounts of automation and human effort. In the simplest case, we generated thousands of yes-no questions for diverse behaviors just by instructing an LM (and filtering out bad examples with another LM). Random examples of LM-written evals: