New YouTube video: 1hr general-audience introduction to Large Language Models **********************
Based on a 30min talk @karpathy gave recently; It tries to be non-technical intro, covers mental models for LLM inference, training, finetuning, the emerging LLM OS and LLM
LLMS
-
YouTube Introduction to Large Language Models and LLM OS
By
–
-
RAG and Fine-tuned LLMs for Knowledge Base Chat
By
–
Chat with your knowledge base within minutes using the Retrieval Augmented Generation (RAG) technique and fine-tuned LLMs Check it out: https://
abacus.ai/ragapi -
Karpathy’s LLM Introduction and Scale AI Valuation Analysis
By
–
.
@karpathy just published a wonderful video as an introduction to LLMs. Go watch it if you’re bored on thanksgiving! ICYMI, he used ChatGPT to definitively determine @scale_AI is worth $150B today and $2T in 2 years The AI does not lie -
Teaching Grok AI to correct its errors
By
–
Looks like you will be able to teach Grok AI why he was wrong exactly 👀 https://t.co/Gpd7LD3I5O
— 🚨 AI News | TestingCatalog (@testingcatalog) 23 novembre 2023Looks like you will be able to teach Grok AI why he was wrong exactly
-
Yi-34B-Chat: New State-of-the-Art Open Source Language Model
By
–
Yi-34B-Chat is a new chat model trained from scratch by @01AI_Yi
. It's currently the state-of-the-art open source language model on almost every benchmark. Try it out on Replicate: https://
replicate.com/01-ai/yi-34b-c
hat
… Or run Yi with an API: -
LLM Information Content Cannot Fit in 800MB Storage
By
–
You simply cannot squeeze the information content of a trained LLM (even a tiny one) in 800MB.
-
OpenAI’s Use of Synthetic Data and Q* Development
By
–
While OAI already uses 90% synth data lel. Q* ftw
-
Minimal Code Architecture Enables Massive Learning
By
–
The code that specifies the architecture is tiny.
That's precisely my point.
There is a tiny amount of prior structure and a lot of learning. -
Language Hardwiring Constraints Within 8MB Memory Limits
By
–
Whatever "hardwiring" is required for language has to fit in 8MB.
-
Genome Storage vs LLM Size: Evolution’s Compression Challenge
By
–
Whatever it is that we learned through evolution has to be squeezed in 800MB (the size of the genome, uncompressed).
most of it is just low-level biochemical machinery. Even a tiny LLM requires 14GB.