I was referring to NLP tasks that already exist pre LLM hype era where LLMs are typically used to few shot or zero shot generate a lot of data for smaller production models.
@yitayml
-
Production Performance vs Academic Benchmarks Eval Correlation
By
–
What production performance are you referring to then? Or are you referring to academic benchmarks being bad in general? Because you would always need some eval benchmark ideally correlated to the prod use case.
-
LLM Production Use Cases Beyond ChatGPT Benchmarking
By
–
It really depends on what you mean by production settings though. LLMs are used for many things other than being a ChatGPT. Overall I agree with your thread and benchmark results should be interpreted with caution. (But can be still useful some times)
-
Compute Optimal Point Does Not Limit Performance Gains
By
–
I think compute optimal doesn't mean performance won't continue to increase if we continue to train a model even after the compute optimal point.
-
List identifies truly open source language models
By
–
Seems like a great list that tells you what LMs are actually open source.
-
Meta Launches New Machine Learning Language Model
By
–
"Meta has a new machine learning language model to remind you it does AI too" That title lol
-
Mostafa Dehghani Hiring Student Researchers at Google Brain
By
–
The great Mostafa Dehghani is hiring student researchers at Google Brain. Check out his tweet advertisement (and/or help retweet/pass along).
-
Safe AGI Development and Commercialization Strategy
By
–
Step 1: Build safe AGI Step 2: Sell Sugar Water Step 3: Profit!
-
PhD Student Conference Funding Challenges in Academia
By
–
I actually paid for many conf travel/registration myself out of my own pocket as a PhD student.