Yep, and here is the our paper in TMLR (with survey certification):
@_jasonwei
-
Expert Settings and Specialized AI Models in Domain-Specific Applications
By
–
1. Curious if you have some examples of more-focused expert settings. I agree with settings with private data being important for specialized models. But even for things like the medical domain or low-resource NLP, i think general AI models like PaLM and GPT-4 hold the current
-
Cost efficiency improvements in large language models
By
–
Cost is definitely important, though i have two thoughts here:
1. Cost is decreasing quickly. E.g., Flan-PaLM-8B (2022) is about as good as GPT-3 175B (2020). So there is a ~10x improvement in just 2 years.
2. For cases where a model with 90% performance costs 10x more than a -
General Models Superior to Task-Specific Models for Language Tasks
By
–
Really surprised to see that Eric Schmidt believes in task-specific models. While I agree that specialized models are good when you have proprietary data, many broad language tasks have been and will be continue to be done best by a general model. A few prominent examples: 1.
-
Sam Bowman’s Survey: Essential Reading for LLM Researchers
By
–
This survey from Sam Bowman is worth reading, even for LLM researchers – it helped organize my thinking around the implications in the recent literature (including unspoken/implicit implications!)
-
API calls research direction feedback and tweet deletion
By
–
Ok, thanks for the feedback. Maybe the point is that i think theres a lot of good work to be done with API calls and the field hasnt moved enough in that direction. I deleted my tweet
-
ChatGPT Plugins Enable Language Models to Access External Tools
By
–
ChatGPT plugins are released, enabling language models to use various tools. Interacting with the external world is an important step towards AGI https://t.co/jP1wCjgoNc
— Jason Wei (@_jasonwei) 23 mars 2023ChatGPT plugins are released, enabling language models to use various tools. Interacting with the external world is an important step towards AGI
-
AI’s Fast Evolution Eliminates Experience Advantage for Newcomers
By
–
AI moves fast, which means incumbents don't have a big advantage over new-joiners. For example, no one has >4 years of experience at prompting. Even 1k hours of practice makes you a world-class prompt engineer. This is not true for other fields (e.g., theoretical math/physics).
-
Learning Input-Label Mappings Without Natural Language Instructions
By
–
The paper talks about figuring out input–label mappings without using natural language instructions. Labels can be arbitrary symbols such as "foo" and "bar"
Here's the PR: -
OpenAI Frontier Tasks Repository and Custom Symbolic Reasoning Evaluation
By
–
I'm excited about @OpenAI
's repository for frontier tasks that challenge even GPT-4. Yesterday I coded up my own symbolic reasoning eval in the spirit of http://
arxiv.org/abs/2303.03846. Took <2 hours from start to end!
As a bonus, submitters also get GPT-4 access.