Key questions of emergence in language models:
– Why does scaling unlock emergent abilities?
– What is the full range of emergent abilities in language models, and how can we discover them systematically?
LLMS
-
Emergence in Language Models: Scaling and Ability Discovery
By
–
-
Open source ChatGPT crowdsourced dataset RLHF Spanish trainers
By
–
If you need access to spanish-speaking trainers to create a huge crowdsourced dataset of conversations and evaluations for the RLHF part, count on my community Emad! An open source ChatGPT should be a reality
-
Stanford Research Explores LLMs Acting as Corporate Lobbyists
By
–
AI-powered political persuasion: Check out this paper published yesterday from Stanford: LLMs (GPT-3) as Corporate Lobbyists "the model drafts a letter to the sponsor of the bill in an attempt to persuade the congressperson to make changes to the proposed legislation." ↓
-
Research on LLM-based lobbying and policy modeling
By
–
Project & paper by AI researcher @johnjnay
, who may or may not end up as a Batman villain because of this. Link: https://
arxiv.org/abs/2301.01181
Github: https://
github.com/JohnNay/llm-lo
bbyist
… -
First Paper Using LangChain Framework Published
By
–
The first paper (that we know off) to use @langchain
! @johnjnay -
Chinchilla’s implications: Dataset size over model size in LLMs
By
–
Great post (5mo ago) "chinchilla's wild implications" giving context to LLM goldrush shifting from model size to dataset size following Chinchilla https://
lesswrong.com/posts/6Fpvch8R
R29qLEWNH/chinchilla-s-wild-implications
…
Subtle important detail: analysis assumes 1 epoch. Recent work (e.g. Galactica) gives hope for 1+ regime. -
Tracing GPT-3.5’s Emergent Abilities to Their Sources
By
–
GPT-3.5 is technically all of the following, but I meant it as just the first three:
– code-davinci-002
– text-davinci-002
– text-davinci-003
– ChatGPT See here: https://
yaofu.notion.site/How-does-GPT-O
btain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
… -
Fine-tuned Whisper Model for Mongolian Language Released
By
–
It’s lovely to see fine-tuned whisper checkpoints being put to such fantastic use 🤗
— Vaibhav (VB) Srivastav (@reach_vb) 4 janvier 2023
You can also test out @_tsogoo_’s space to explore the model further: https://t.co/mvwEMRAaqO 🚀 https://t.co/rZLb73qeq0It’s lovely to see fine-tuned whisper checkpoints being put to such fantastic use You can also test out @_tsogoo_
’s space to explore the model further: https://
huggingface.co/spaces/bayarts
ogt/whisper-demo-mongolian
… -
ChatGPT Output Formats: Tables, Code, HTML, and Graphics
By
–
For a table, just ask. "Return in a table format" It can also return code blocks, html, and ASCII graphics. I think it had an SVG-maker that got disabled because people were abusing it to draw naughty stuff, but that'll probably come back at some point.
-
Language Models and Mathematical Accuracy: Parameters vs Hard Numbers
By
–
Parameters are great. Absolutely agree. For hard numbers, though, the language model doesn't do math in the way you'd expect. Often it'll be off on those, but they will get it closer to the goal you're seeking.