Let's keep in mind these are still super simple "task" evals. Little queries served on a platter, even if increasingly difficult. Which are super helpful, but when people talk about AGI they usually have an autonomous agent swarm in mind performing long-running jobs across
@karpathy
-
Reproducing Software Build Issues: Testing Consistency Analysis
By
–
Hah yeah I can reproduce it and got something similar 5/5 attempts. I wonder what build they sent him earlier
-
Why Humor in LLMs Remains Difficult to Achieve Through Post-Training
By
–
Great question right? I'd love to know, I don't think I fully understand this either. But considering that noone has (to my knowledge) figured out a way to post-train an LLM to be funny, I am prepared to believe humor is really difficult and requires more underlying capability?
-
Grok 3 Early Access: State-of-the-Art Thinking Model Review
By
–
I was given early access to Grok 3 earlier today, making me I think one of the first few who could run a quick vibe check. Thinking First, Grok 3 clearly has an around state of the art thinking model ("Think" button) and did great out of the box on my Settler's of Catan
-
Organizing and Sharing Algorithm Controls in a Marketplace
By
–
Would be interesting if you could organize them into groups, turn them on and off as groups, and share them, vote them etc. Would then basically be a lite version similar to controlling and the algorithm in a marketplace that @jack has been thinking about.
-
Karpathy praises ChatGPT 4o’s new conversational personality improvements
By
–
Actually I quite like the new ChatGPT 4o personality, whatever they did. – it's a lot more chill / conversational, feels a bit more like talking to a friend and a lot less like to your HR partner
– now has a pinch of sassy, may defend itself e.g. when accused of lying
– a lot of -
Tic Tac Toe Failure Analysis in Recent AI Models
By
–
Great collection! Agree that failure to play tic tac toe is most interesting. Has someone looked at the recent models more thoroughly
-
Apps Should Offer Native Export for Prompt Feature
By
–
More apps should natively offer this. “Export for prompt” button
-
Prompt Injection Attempts with Invisible Bytes in AI Models
By
–
I'm able to do basic prompt injections with the invisible bytes but I can't get it to work without explicit decoding hints. https://
chatgpt.com/share/67acd3ba
-d234-8007-ad44-ba9d4dfc2920
… The thinking models actually feel a bit more susceptible because they love puzzles and they notice the added bytes and get very