You think ChatGPT is amazing — you’ve been hacking on computers for years, but this you can’t explain. How did we get here, and so suddenly? How does it know and do so much? @Francis_YAO_ of @EdinburghNLP explains the history of GPT-3: https://
yaofu.notion.site/How-does-GPT-O
btain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
…
LLMS
-
History of GPT-3 and its emergent abilities explained
By
–
-
ChatGPT Prompt Dataset Released in Datasets Library
By
–
ChatGPT prompt dataset just landed in the datasets lib
-
GPT-3 Training Data: C4 Dataset Technical Details
By
–
(Except that GPT3 is not trained on C4 per se @Francis_YAO_ but it’s a small tweak)
-
ChatGPT Reveals the Emptiness of Most Written Content
By
–
ChatGPT exposes how vacuous most of our writing is.
-
ChatGPT Will Kill Search and Open Path to Web3
By
–
ChatGPT Will Kill Search and Open a Path to Web3
#AI #MachineLearning #digital #web3 #DataScience #python #Digital
Cc @Khulood_Almani @BetaMoroney @Analytics_699 @CurieuxExplorer @dr_gulsun @sallyeaves @amalmerzouk https://
coindesk.com/layer2/2022/12
/09/chatgpt-will-kill-search-and-open-a-path-to-web3/
… -
Large Language Models: Confidence and Misinformation Risk Reduction
By
–
3/Building large language models that can accurately decide when to be confident and when not to will reduce their risk of misinformation and build trust.
-
Large Language Models’ Overconfidence Problem and Misinformation Risk
By
–
1/Large language models like Galactica and ChatGPT can spout nonsense in a confident, authoritative tone. This overconfidence – which reflects the data they’re trained on – makes them more likely to mislead.
-
Key AI Benchmarks for Language Model Evaluation
By
–
Benchmarks:
– MMLU (massively multitask language understanding): https://
arxiv.org/abs/2009.03300
– BBH (Big-Bench Hard): https://
arxiv.org/abs/2210.09261
– TyDiQA (typographically diverse QA): https://
arxiv.org/abs/2003.05002
– MGSM (multilingual grade school math): https://
arxiv.org/abs/2210.03057 -
Code-Davinci-2 vs Text-Davinci-3: Instruction Tuning and PPO Performance
By
–
– code-davinci-2 > text-davinci-3 means that their instruction finetuning overall hurts performance on academic benchmarks
– text-davinci-3 > text-davinci-2 means that PPO improves performance -
Text-davinci-003 Instruction Following vs Academic Benchmark Performance
By
–
@OpenAI
's text-davinci-003 follows instructions better. Is it also better on academic benchmarks? Summary:
– text-davinci-3 beats text-davinci-2, but is not as good as code-davinci-2
– it is behind @GoogleAI
's PaLM and Flan-U-PaLM Full results: https://
arxiv.org/abs/2210.11416 App D