Transforming human knowledge, sensors and actuators from human-first and human-legible to LLM-first and LLM-legible is a beautiful space with so much potential and so much can be done… One example I'm obsessed with recently – for every textbook pdf/epub, there is a perfect
@karpathy
-
SFT Layer Impact on Pretraining Embedding Space Representations
By
–
I thought this should have already applied on the SFT layer and was a bit surprised when it didn't. Possibly it's that SFT is so few bits / brief that it is at most a shifting of superficial style around a pre-existing embedding space from (common) pretraining, with everyone
-
Transforming textbook problems into AI learning environments
By
–
How amazing it would be if we could extract and reframe all the practice problems from all the textbooks ever written into environments…
-
RL Finetuning: Major Upgrade Over SFT for LLMs
By
–
I just mean long term, imo RL finetuning paradigm is a big upgrade over just SFT (expert imitation) for LLMs at the current stage of development and will continue to grow substantially.
-
Training Data Evolution: Internet Text to Supervised Conversations
By
–
In era of pretraining, what mattered was internet text. You'd primarily want a large, diverse, high quality collection of internet documents to learn from.
— Andrej Karpathy (@karpathy) 27 août 2025
In era of supervised finetuning, it was conversations. Contract workers are hired to create answers for questions, a bit… https://t.co/rR6yYZGgKPIn era of pretraining, what mattered was internet text. You'd primarily want a large, diverse, high quality collection of internet documents to learn from. In era of supervised finetuning, it was conversations. Contract workers are hired to create answers for questions, a bit
-
Optimizing LLM-Assisted Coding Workflows and Best Practices
By
–
Continuing the journey of optimal LLM-assisted coding experience. In particular, I find that instead of narrowing in on a perfect one thing my usage is increasingly diversifying across a few workflows that I "stitch up" the pros/cons of: Personally the bread & butter (~75%?) of
-
Modular skeleton architecture for parallel Claude environments
By
–
I pitched something like it earlier this year but I think it's missing the "skeleton" onto which it would be easy to add new envs, in parallel and in common schema. Something all the claude codes can play in.
-
Platforms Enable User Creativity Through Streamers and Memes
By
–
Platforms, user creativity, streamers and memes. Skip graphics. Very cool.
-
Siri Enhances Notification Management with Priority Detection
By
–
Even better, Siri right away highlighted this notification as a "Priority Notification". It just keeps getting better.
-
Challenge Winner Announcement and Project Submission Review
By
–
Ok, spent an ~hour sifting through submissions. The biggest challenge was the spam, as majority of replies are people linking to their own existing projects, not things made for the challenge. Of the ones that were: Winner: I most enjoyed this one from @uncertainsys –