True, honestly, IDK why they are running behind in the industry. since BERT they haven't announced any significant model release. while even meta, openai are moving really fast, even though google has more access to the world's data than any other org.
LLMS
-
Unavoidable difficulty of logprobs for completion sequences in classification
By
–
The number of logprobs returned per token isn’t the issue here, it’s the unavoidable difficulty of getting logprobs for different completion sequences. In a classification context you need to assign class labels to individual tokens during training.
-
Humor: RLHF works because practitioners are fast
By
–
This is how RLHF works btw the people doing it are just really fast.
-
LLM Emergent Properties and Their Staggering Economic Implications
By
–
The emergent properties of LLM’s continue to astound me. The implications for work and the economy are staggering. It’s a growing focus for us at the @DigEconLab https://
t.co/mA2n7p48kR -
ML Startups Focus on LLM Application Layer Next Three Years
By
–
This is my bet too. And one I’ve talked about a lot. Next 3 years of ML startups and AI/ML VC will be focused on the application layer of LLMs (even the compute-optimal mini-LLMs like chinchilla). Buckle up.
-
Google’s Mu2SLAM: Unified Model For Speech And Text Understanding
By
–
Google’s Mu2SLAM: Toward a Single Model For All Speech and Text Understanding Tasks https://
syncedreview.com/2022/12/22/goo
gles-mu2slam-toward-a-single-model-for-all-speech-and-text-understanding-tasks/
… -
text2SQL Challenges: Demo Status Requires Production Readiness Work
By
–
Getting it to work reliably is a challenge too. text2SQL is still much a demo and needs more work to be production ready.
-
Ask to continue from start of last code block using Markdown
By
–
Ask it to start from an earlier point, like "Continue from the start of the last code block." Under the hood everything is Markdown so you can refer to stuff in Markdown terms.
-
AI Powers Human Creation Across All Industries
By
–
AI can now power anything that humans create. 2023 will be wild. pic.twitter.com/5cklXMk9uL
— Nina Schick (@NinaDSchick) 22 décembre 2022AI can now power anything that humans create. 2023 will be wild.
-
Completion Models vs Instruction Fine-Tuning Approaches
By
–
It’s a completion model, not (yet) an instruct fine tuned model so you should formulate your task as a completion task. For instance by writing your prompt as a code comment or docstring.