6. Studying emergent behavior in language models. – How/why does scaling unlock emergence?
– Can we predict future emergent capabilities?
– How do we get these abilities in smaller models?
LLMS
-
Emergent Behavior in Language Models: Scaling, Prediction, and Optimization
By
–
-
Understanding How Language Models and Prompting Work
By
–
5. Why language models / prompting works. We still don’t have a great understanding of how in-context learning works, or why chain-of-thought/reasoning works. We can learn a great deal by only looking at inputs and outputs (as do we do with humans in psychology).
-
Language Models: New HCI Paradigm and Practical Applications
By
–
3. How people use language models. Interaction and collaboration with language models will be a new direction similar to HCI, and I’m super excited about application of language models for use cases such as foreign-language learning, training therapists, etc.
-
Building Evaluations for Frontier Language Models
By
–
2. Building evaluations. Many benchmarks get quickly saturated, and we need more to evaluate the frontier of language models. In addition, it’s still an open question of how to evaluate language models generally. The new OpenAI evals library could be good: https://
github.com/openai/evals -
Prompting Research: Just the Beginning for Language Model Guidance
By
–
1. Prompting research. Maybe hot take, but I think we’ve just reached the tip of the iceberg on the best ways to prompt language models. As language model capabilities increase, the degrees of freedom for guiding a particular generation via a good prompt will increase.
-
PhD Research Directions in the Era of Large Language Models
By
–
I’m hearing chatter of PhD students not knowing what to work on.
My take: as LLMs are deployed IRL, the importance of studying how to use them will increase.
Some good directions IMO (no training):
1. prompting
2. evals
3. LM interfaces
4. safety
5. understanding LMs
6. emergence -
ESM-2: Meta’s Large Language Model Advances Protein Structure Prediction
By
–
One of the most important advances for #AI in life science has been #AlphaFold. The prediction of proteins from amino acid sequences at atomic level has a new large language model, ESM-2, from @Meta https://
science.org/doi/10.1126/sc
ience.ade2574
… @ScienceMagazine -
Staying Updated on LLM Jailbreaks and Exploits
By
–
well, now that @gdb qt'd this tweet, I feel I have to share this… keep up w the current state of jailbreaks and LLM exploits by subscribing to my newsletter here: http://
thepromptreport.com -
GPT-4 fails to write without ‘e’ via naive prompting
By
–
Writing without the letter "e" is still beyond GPT-4 with naive prompting, but that's maybe a cheap shot.
-
Microsoft 365 Copilot Launch: Understanding Usefully Wrong
By
–
From Microsoft 365 Copilot launch post one hour ago… What is “usefully wrong”?