+1 for "context engineering" over "prompt engineering". People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window
@karpathy
-
Data Quality Tradeoffs in SOTA Model Training Datasets
By
–
I'm not 100% sure about that. As an example I was just browsing through the DCLM-baseline datamix (which is ~SOTA) and it is *terrible*. Compared to what I could in principle imagine. Major concessions are made in data quality to gather enough data quantity.
-
The Quest for Ultimate High-Quality Pretraining Data for LLMs
By
–
Mildly obsessed with what the "highest grade" pretraining data stream looks like for LLM training, if 100% of the focus was on quality, putting aside any quantity considerations. Guessing something textbook-like content, in markdown? Or possibly samples from a really giant model?
-
AI-Generated Image: Golden Gate and Lavender Field by Ideogram
By
–
AI generated sorry to disappoint. Ideogram took it. I asked for Golden Gate (as the event is in SF) and lavender field (because I like lavender).
-
Humanity’s Irrelevance: Deep in the Amazon of Cosmic Possibility
By
–
Something like this feels quite likely. That we’re a random ant colony deep inside the Amazon forest crawling around like “where is everyone???”. The depth of our explored tech tree is so shallow compared to what feels possible. We’re probably really, really irrelevant.
-
Job as Bundle of Tasks and Glue Variables
By
–
Very interesting to think about. Job = bundle of tasks + glue. Probably a bunch of other variables involved, e.g. the number of tasks, how long each task is (e.g. METR-like notion of task length ~= difficulty), how contextual it is, how high reliability it needs, whether it can
-
LLM GUI Demo: Dynamic Ephemeral Interfaces Over Static Design
By
–
Cool demo of a GUI for LLMs! Obviously it has a bit silly feel of a “horseless carriage” in that it exactly replicates conventional UI in the new paradigm, but the high level idea is to generate a completely ephemeral UI on demand depending on the specific task at hand. https://t.co/Xgh1iwDmJl
— Andrej Karpathy (@karpathy) 19 juin 2025Cool demo of a GUI for LLMs! Obviously it has a bit silly feel of a “horseless carriage” in that it exactly replicates conventional UI in the new paradigm, but the high level idea is to generate a completely ephemeral UI on demand depending on the specific task at hand.
-
Learning to Master New Powerful Tools and Their Applications
By
–
I liked your article thank you! I feel like a lot of people are sensing the power of the new tool, but still figuring out exactly how to hold it, use it, or whatever the correct incantations are.
-
Next Generation AI: From GPT2 to the 6502 Moment
By
–
GPT2 was not super programmable yet. Even GPT4 or o3 still fall short a bit. I suspect it’s either nextgen or the one after that that will be considered 6502 in hindsight. Fully multimodal, really smart, reasoning, tool using, agentic, with memory. A first “basics” package.
-
Software 2.0: Keynote slides and resources on AI technology diffusion
By
–
Some of the links:
– My slides as keynote: https://
drive.google.com/file/d/1a0h1mk
wfmV2PlekxDN8isMrDA5evc4wW/view?usp=sharing
…
– Software 2.0 blog post from 2017 https://
karpathy.medium.com/software-2-0-a
64152b37c35
…
– How LLMs flip the script on technology diffusion https://
karpathy.bearblog.dev/power-to-the-p
eople/
…
– Vibe coding MenuGen (retrospective) https://
karpathy.bearblog.dev/vibe-coding-me
nugen/
…