The code was written at layers 22-30 and is stored in the value activations you just can’t read it. I think you owe the LLM an apology.
@karpathy
-
Books as synthetic data generation prompts, not static information
By
–
Most people misunderstand books as data for pertaining when it’s more a set of prompts for synthetic data generation.
-
GSM8K Paper Insights on Mathematical Reasoning in AI
By
–
reminded of this paragraph from gsm8k paper, 2021 🙂
-
Apple’s iPhone Strategy: Continuous Innovation in Consumer Technology
By
–
Actually jason, every iPhone is the best iPhone they have made yet. And this is something only Apple can do.
-
Consumer Demand for Simpler Lighter Smartphone Alternatives Growing
By
–
The phones have been getting ever bigger, heavier, more full-featured. There are dozens of us that want the opposite. Dozens!
-
Model Evaluation: Testing AI Bias in Self-Assessment Rankings
By
–
I love doing this actually :). I think it's a pretty powerful eval too. Have all models generate something, then put it all together and give it back to all of them and ask them to rank all outputs. I thought models might have a bias to prefer their own outputs, but this doesn't
-
GPT-5 Pro outperforms competitors in code editing tasks
By
–
not yet but planning to! I rotate through a lot of what exists in few day intervals. Actually ~3 weeks ago I found codex worse and less polished than CC right now for the more basic edits and overall experience. It's just 5 Pro specifically feels SOTA by a good margin atm.
-
GPT-5 Pro Outperforms Claude on Complex Code Generation
By
–
I think congrats again to OpenAI for cooking with GPT-5 Pro. This is the third time I've struggled on something complex/gnarly for an hour on and off with CC, then 5 Pro goes off for 10 minutes and comes back with code that works out of the box. I had CC read the 5 Pro version
-
LLM Security Risk: Shell Command Injection via Tilde Expansion
By
–
I almost had this happen yesterday, but my LLM suggested the right thing as `rm -rf '~'`. (with 'quotes'). Still very scary. Example of how this happens, e.g. if something like this sneaks into your .bashrc: export WANDB_DIR="~/.cache/wandb" The ~ isn't expanded inside "quotes"
-
Algorithm convergence trap: viral content as optimization limit
By
–
in the limit of revenue/growth maxxing via A/B tests doesn't it all just converge to funny panda videos? it's the supermasive black hole at the center of the universe and no matter the initial conditions you eventually get sucked in. reels. youtube shorts. substack notes.