Identifying the most influential training sequences revealed that generalization patterns become much more sophisticated and abstract with scale. For example, here are the most influential sequences for 810 million and 52 billion parameter models for a math word problem:
@anthropicai
-
Tracing LLM Skills Through Influence Functions on Training Data
By
–
Large language models have demonstrated a surprising range of skills and behaviors. How can we trace their source? In our new paper, we use influence functions to find training examples that contribute to a given model output.
-
Influence Functions: Measuring Training Data Impact on Models
By
–
Influence functions are a classic technique from statistics. They are formulated as a counterfactual: if a copy of a given training sequence were added to the dataset, how would that change the trained parameters (and, by extension, the model’s outputs)?
-
Anthropic Releases TypeScript SDK Tokenizer Tool
By
–
A new tokenizer for our TypeScript SDK brings builders the same tokenization tooling currently available on Python. https://
npmjs.com/package/@anthr
opic-ai/tokenizer
… -
Claude 2 API Now Available with Enhanced Features
By
–
Users can also call our latest model Claude 2 through the API to integrate its enhanced features into their projects including longer responses! https://
docs.anthropic.com/claude/referen
ce/selecting-a-model
… -
Dario Amodei discusses AI safety and Anthropic’s mission
By
–
Our CEO Dario went on Hard Fork this week to discuss AI safety, his journey to starting Anthropic, and http://
Claude.ai. Listen here! https://
nytimes.com/2023/07/21/pod
casts/dario-amodei-ceo-of-anthropic-on-the-paradoxes-of-ai-safety-and-netflixs-deep-fake-love.html
… -
Anthropic Commits to AI Safety and Cybersecurity Standards
By
–
We all need to join in a race for AI safety. In the coming weeks, Anthropic will share more specific plans concerning cybersecurity, red teaming, and responsible scaling, and we hope others will move forward swiftly as well. https://
whitehouse.gov/briefing-room/
statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/
… -
SAP Partners on Responsible Business AI Delivery
By
–
Pleased to be partnering with @SAP on their commitment to delivering Business AI that’s relevant, reliable and responsible. https://
x.com/SAPNews/status
/SAPNews/status/1681319453051564033
… -
Anthropic Hiring Research Engineers and Scientists
By
–
If you’re excited about this kind of research, the team that led this work is actively hiring research engineers and scientists, and we’d encourage you to apply: https://
jobs.lever.co/Anthropic/436c
a148-6440-460f-b2a2-3334d9b142a5
… https://
jobs.lever.co/Anthropic/eb9e
6d83-626c-4f59-8a0e-fa7c413b2014
… -
Anthropic improves language model reasoning through faithful explanations
By
–
We’re excited about ways to make language models generate more faithful explanations that help them reason better! We encourage you to check out our papers for more results and details: https://
www-files.anthropic.com/production/fil
es/measuring-faithfulness-in-chain-of-thought-reasoning.pdf
… https://
www-files.anthropic.com/production/fil
es/question-decomposition-improves-the-faithfulness-of-model-generated-reasoning.pdf
…