Models struggle to transition between these strategies, as exhibited by a spike in test loss. This spike moves to larger datasets as one increases model capacity. This is a clear signature of double-descent, a phenomenon that is now well-known in the ML literature.
@anthropicai
-
Mechanistic Theory of Memorization: Open Questions and Research Directions
By
–
We hope these results are a step towards a mechanistic theory of memorization. There are many open questions, such as understanding the loss spike, or what happens when only a subset of the data is repeated.
-
Superposition in Neural Networks: Memorization vs Feature Learning
By
–
For small training sets, models use superposition to memorize more data points than the two available neurons. For large training sets, models learn features in superposition, as observed in our previous work, allowing the model to generalize. https://
transformer-circuits.pub/2022/toy_model
/index.html
… -
Understanding Deep Learning Overfitting Through Mechanistic Analysis
By
–
We have little mechanistic understanding of how deep learning models overfit to their training data, despite it being a central problem. Here we extend our previous work on toy models to shed light on how models generalize beyond their training data. https://
transformer-circuits.pub/2023/toy-doubl
e-descent/index.html
… -
Superposition Strategy: How Neural Networks Embed Features in Hidden Space
By
–
Our prior work showed that these toy models use a strategy called “superposition” to learn more features than available neurons. Here we observe how training data points, as well as features, are embedded in the hidden space.
-
Anthropic Plans to Share Safety Experiment Data
By
–
As we did with our 'red teaming' project (https://github.com/anthropics/hh-rlhf…), we plan to release the data from this experiment in the future to empower a broader set of people to build safer systems.
-
Small-Scale Experiment Launching for Two Weeks
By
–
This is a small-scale experiment and it’ll run for about two weeks. If it goes well, we’ll plan to do larger-scale experiments in the future. Happy holidays to all from the Anthropic team!
-
Anthropic Seeks Safety Researchers for Language Model Testing
By
–
We’ll initially prioritize people who have demonstrated an interest in probing language models for safety issues (see attached, for an example), whether via traditional research or great blogs and Twitter threads. If that’s you, please fill out the form!
-
Exploration of Capabilities and Limitations of Constitutional AI
By
–
Participants will get to explore the capabilities and limitations of AI systems trained to be helpful, honest, and harmless via Constitutional AI. We're excited to collectively diagnose new capabilities and identify ways to break these models.
-
Anthropic announces beta onboarding and invites community feedback
By
–
We’ll onboard people shortly after Christmas and shut off this form sometime before Christmas, or whenever it reaches our internal support capacity. We’re particularly excited to collectively come up with creative ways to find new features and problems with these models.