PROMPT ENGINEERING

Karpathy posted a long thread about his most frequent use cases with LLMs recently. Not writing code, but building knowledge bases. The approach is quite hardcore: he dumps papers, articles, code repositories and other materials into a folder, then lets an LLM "compile" them into a Markdown wiki. The wiki includes summaries, backlinks, concept categorization, and articles linked to each other. The frontend uses Obsidian for viewing, and Q&A also has the LLM retrieve against the wiki. In his own words, most token consumption now isn't in manipulating code, but in manipulating knowledge. This shift is quite interesting. The entire system can also maintain itself. He wrote some LLM "health check" scripts that periodically scan the wiki for contradictory data, missing information, and potential connections, letting the LLM patch itself. The results from each Q&A can also be archived back into the wiki, making it thicker with each use. Actually, Karpathy clarified something that's happening right now: the greatest value of LLMs might not be helping you generate content, but helping you manage knowledge. But the last sentence of his post is the most worth pondering: "I think there is room here for an incredible new product instead of a hacky collection of scripts." He himself knows this system is hacked together from scripts. Obsidian + command line + manual processes—it works, but it's just a demo. And there are several problems he probably felt:
The wiki is local Markdown files, tied to the computer—it breaks when you switch machines. Retrieval relies on the LLM's own maintained indexes and summaries; he said around 400K words it still holds up, but beyond that? He even said "I thought I had to reach for fancy RAG," just that the scale hasn't reached that point yet. A more fundamental problem is that the wiki stores knowledge, but not memory. What does that mean? Knowledge is "domain X has these concepts, and their relationships are like this." Memory is "I just read a paper last week that refutes this viewpoint, and my judgment on this direction changed." One is static, one walks with you. Karpathy's system can help you store things and search things, but it doesn't know you've changed.
This is actually the difference between a knowledge base and a memory system. The gap isn't a better script—it's an entire architecture. The model can't just "store" and "search"; it needs to sense which information is relevant to who you are now, needs to evolve itself as you use it, needs to maintain coherence across projects and timelines. Karpathy proved with a hand-rolled solution that this direction is right. But he also proved firsthand that you can't go far with just file systems and prompts. Memory needs to be infrastructure, not a collection of scripts. [Translated from EN to English]

→ View original post on X — @elliotchen100, 2026-04-05 04:31 UTC