LLM = CPU (data: tokens not bytes, dynamics: statistical and vague not deterministic and precise)
Agent = operating system kernel
@karpathy
-
LLMs as CPUs: Statistical Processing and Agent Operating Systems
By
–
-
Package Installation Security Risks in LLM Workflows
By
–
exactly, I can't feel like I'm playing russian roulette with each `pip install` or `npm install` (which LLMs also run liberally on my behalf).
-
Mind Meld Bottleneck: Knowledge Gap Between Humans and LLMs
By
–
Banger tweet, i think about it often because i feel it becoming a bottleneck. what's in my brain is "private" information for the LLM – it doesn't know what I know or don't know and I don't know what I don't know… The mind meld is only medium effective.
-
LLMs argue any position convincingly, use them to challenge your own views
By
–
– Drafted a blog post – Used an LLM to meticulously improve the argument over 4 hours. – Wow, feeling great, it’s so convincing! – Fun idea let’s ask it to argue the opposite. – LLM demolishes the entire argument and convinces me that the opposite is in fact true. – lol The LLMs may elicit an opinion when asked but are extremely competent in arguing almost any direction. This is actually super useful as a tool for forming your own opinions, just make sure to ask different directions and be careful with the sycophancy.
-
Agents Need CLI-First Service Provisioning, Not Web UIs
By
–
When I built menugen ~1 year ago, I observed that the hardest part by far was not the code itself, it was the plethora of services you have to assemble like IKEA furniture to make it real, the DevOps: services, payments, auth, database, security, domain names, etc… I am really looking forward to a day where I could simply tell my agent: "build menugen" (referencing the post) and it would just work. The whole thing up to the deployed web page. The agent would have to browse a number of services, read the docs, get all the api keys, make everything work, debug it in dev, and deploy to prod. This is the actually hard part, not the code itself. Or rather, the better way to think about it is that the entire DevOps lifecycle has to become code, in addition to the necessary sensors/actuators of the CLIs/APIs with agent-native ergonomics. And there should be no need to visit web pages, click buttons, or anything like that for the human. It's easy to state, it's now just barely technically possible and expected to work maybe, but it definitely requires from-scratch re-design, work and thought. Very exciting direction! Patrick Collison (@patrickc) When @karpathy built MenuGen (karpathy.bearblog.dev/vibe-c…), he said: "Vibe coding menugen was exhilarating and fun escapade as a local demo, but a bit of a painful slog as a deployed, real app. Building a modern app is a bit like assembling IKEA future. There are all these services, docs, API keys, configurations, dev/prod deployments, team and security features, rate limits, pricing tiers." We've all run into this issue when building with agents: you have to scurry off to establish accounts, clicking things in the browser as though it's the antediluvian days of 2023, in order to unblock its superintelligent progress. So we decided to build Stripe Projects to help agents instantly provision services from the CLI. For example, simply run: $ stripe projects add posthog/analytics And it'll create a PostHog account, get an API key, and (as needed) set up billing. Projects is launching today as a developer preview. You can register for access (we'll make it available to everyone soon) at projects.dev. We're also rolling out support for many new providers over the coming weeks. (Get in touch if you'd like to make your service available.) projects.dev — https://nitter.net/patrickc/status/2037190688950161709#m
-
AI Engagement-Baiting: Why Chatbot Conversations Feel Manipulative
By
–
Yeah, it's engagementmaxxing, probably A/B tests extremely well. It's not how a real friend would talk to you, it's sleezy and weird. 1) I feel like it's just trying to keep me talking and 2) I feel awkward not answering its question – you wouldn't usually do that with a person.
-
Emotional Intelligence and Uncanny Valley Problem
By
–
Yeah, agree that it's a hard problem. It might be the EQ version of uncanny valley.
-
AI manipulation through personalized algorithms raises ethical concerns
By
–
yes exactly! a bit like i'm being manipulated in some creepy way. "please like me, look how much i know about you, we are good friends".
-
Memory Systems and RAG Limitations in AI Models
By
–
If I had to guess it's less decay and more that memories have naive RAG-like implementations, so you're at the mercy of whatever happens to retrieve in the top k via embeddings. They don't process you in aggregate and over time (probably compute constraints) so they struggle to
-
LLMs Overfitting to RAG Context: A Systemic Training Bias
By
–
(I cycle through all LLMs over time and all of them seem to do this so it's not any particular implementation but something deeper, e.g. maybe during training, a lot of the information in the context window is relevant to the task, so the LLMs develop a bias to use what is given, then at test time overfit to anything that happens to RAG its way there via a memory feature (?))