SGD is a ResNet too (the blocks of it are fwd+bwd), the residual stream is the weights so… We're not taking the Attention is All You Need part literally enough? 😀
@karpathy
-
C Compiler to LLM Weights and Logarithmic Attention Innovation
By
–
Wait this is so awesome!! Both 1) the C compiler to LLM weights and 2) the logarithmic complexity hard-max attention and its potential generalizations. Inspiring!
-
Obsidian Extension Automates Research Loop from Blog Content
By
–
Ty, I used your blog post, exported with obsidian ext into markdown, used it to enqueue ideas into my autoresearch loop
-
LLM Job Exposure Scoring and Employment Impact Complexity
By
–
The "exposure" was scored by an LLM based on how digital the job is. This has no baring on what actually happens to these occupations, which has to do with demand elasticity and a lot more. People are sensationalizing the visualization tool and putting words in my mouth.
-
Weekend BLS Dataset Visualization Project for Data Exploration
By
–
This was a saturday morning 2 hour vibe coded project inspired by a book I’m reading. I thought the code/data might be helpful to others to explore the BLS dataset visually, or color it in different ways or with different prompts or add their own visualizations. It’s been wildly
-
Questioning If This Content Was Generative AI
By
–
Wow. I was sure this was AI. (I mean generative AI.)
-
AI Research Labs Affected by OAuth Outage, Intelligence Brownouts Concern
By
–
My autoresearch labs got wiped out in the oauth outage. Have to think through failovers. Intelligence brownouts will be interesting – the planet losing IQ points when frontier AI stutters.
-
Agentic Organizations Can Be Forked Through IDE Code
By
–
All of these patterns as an example are just matters of “org code”. The IDE helps you build, run, manage them. You can’t fork classical orgs (eg Microsoft) but you’ll be able to fork agentic orgs.
-
Monitor Shortage: Increased Demand for Larger Displays
By
–
I wanted to buy a bigger monitor and discovered that others had the same idea
-
IDE Evolution: From Files to AI Agents Programming
By
–
Expectation: the age of the IDE is over
Reality: we’re going to need a bigger IDE
(imo). It just looks very different because humans now move upwards and program at a higher level – the basic unit of interest is not one file but one agent. It’s still programming.