5. Why language models / prompting works. We still don’t have a great understanding of how in-context learning works, or why chain-of-thought/reasoning works. We can learn a great deal by only looking at inputs and outputs (as do we do with humans in psychology).
@_jasonwei
-
Llama Model Enables Emergence Research with Strong Performance
By
–
And by the way, I think Llama is great for enabling emergence research to be done, strong performance and multiple model sizes.
-
GPT-3.5-Turbo Praised for Academic Research with 10x Price Reduction
By
–
I also think gpt-3.5-turbo is great for academic research, with a 10x reduction in price
-
Emergent Behavior in Language Models: Scaling, Prediction, and Optimization
By
–
6. Studying emergent behavior in language models. – How/why does scaling unlock emergence?
– Can we predict future emergent capabilities?
– How do we get these abilities in smaller models? -
Safety Research Must Keep Pace with Growing AI Capabilities
By
–
4. Safety research. As AI capabilities grow, alignment and safety research should keep up with or ideally lead those capabilities. I liked Sam Bowman's summary of this a lot
-
Building Evaluations for Frontier Language Models
By
–
2. Building evaluations. Many benchmarks get quickly saturated, and we need more to evaluate the frontier of language models. In addition, it’s still an open question of how to evaluate language models generally. The new OpenAI evals library could be good: https://
github.com/openai/evals -
Prompting Research: Just the Beginning for Language Model Guidance
By
–
1. Prompting research. Maybe hot take, but I think we’ve just reached the tip of the iceberg on the best ways to prompt language models. As language model capabilities increase, the degrees of freedom for guiding a particular generation via a good prompt will increase.
-
Language Models: New HCI Paradigm and Practical Applications
By
–
3. How people use language models. Interaction and collaboration with language models will be a new direction similar to HCI, and I’m super excited about application of language models for use cases such as foreign-language learning, training therapists, etc.
-
PhD Research Directions in the Era of Large Language Models
By
–
I’m hearing chatter of PhD students not knowing what to work on.
My take: as LLMs are deployed IRL, the importance of studying how to use them will increase.
Some good directions IMO (no training):
1. prompting
2. evals
3. LM interfaces
4. safety
5. understanding LMs
6. emergence -
Citing emergent abilities paper and pie-chart creation
By
–
@sleepinyourhat You can cite the emergent abilities paper! https://
arxiv.org/abs/2206.07682 Actually all the info is in Appendix E. I just made the pie-chart for twitter purposes 🙂