Very interesting, will have to try! Seems like the perfect fit as a very latency sensitive, very high value application.
Currently (and awkwardly) half of my open code windows are VS Code and half are Cursor, still evaluating.
@karpathy
-
Evaluating Cursor as VS Code Alternative for Low Latency Applications
By
–
-
Focusing on Transformers Over ResNets
By
–
So I’ve heard too, not sure.
(fwiw I’d be inclined to focus on the Transformer and ignore ResNets, too) -
Tinygrad’s Public Quest to Build Commodity Petaflop Compute
By
–
Follow along the @__tinygrad__ saga, who are (very publicly!) trying to build your commodity ~petaflop compute node. tinybox specs: https://
tinygrad.org
the youtube videos form @realGeorgeHotz are actually quite great and entertaining, featuring the signature blend of -
Defining AGI: Surpassing Humans in Economically Valuable Tasks
By
–
The def I'm familiar with is: AGI = "an autonomous system that surpasses human in most economically valuable tasks." For "economically valuable tasks" I like to look at the U.S. Bureau of Labor Statistics index of occupations: https://
bls.gov/ooh/a-z-index.
htm
… You'd have to imagine -
Language Model Advantages and Training Data Optimization Strategies
By
–
It's a very amusing thought for sure! Big advantage to languages that have more training data, in both programming (Python / C), and in spoken language (English) 🙂
I'm cautiously optimistic though. Either by dumping docs into contexts, or via synthetic data means. -
A Decade Gap Between Perfect Demo and Commercial Autonomous Driving
By
–
Actually I am very sympathetic to this. Back to my example of self-driving, my first demo drive in an early Waymo was 2014, and it was already great. It took me around for a 20min drive. From that ~perfect demo it was one decade before I could pay for a drive in a Waymo.
-
Automating Software Engineering: Levels of Autonomy and Abstraction
By
–
# automating software engineering
— Andrej Karpathy (@karpathy) 12 mars 2024
In my mind, automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like:
1. first the human performs all driving actions… https://t.co/u2EXfxcV6e# automating software engineering In my mind, automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like: 1. first the human performs all driving actions
-
Silent Bugs in Fine-tuning: Attention to Detail Matters
By
–
Beautiful work / attention to detail trying to get Gemma to finetune correctly. There are so many foot guns here to be super careful with. All of these issues don't throw any errors, they silently make your network worse. A great example of what I wrote about in my "A Recipe for
-
Training LLMs at Scale: Hidden Infrastructure Challenges and Hardware Health
By
–
Nice read on the rarely-discussed-in-the-open difficulties of training LLMs. Mature companies have dedicated teams maintaining the clusters. At scale, clusters leave the realm of engineering and become a lot more biological, hence e.g. teams dedicated to "hardware health". It