Ty I try to keep a high bias on my tokens quality discriminator
@karpathy
-
LLM Speak: Reframing Common Descriptions in AI Terminology
By
–
LLM speak :
– You didn't find some material boring. It had low quality tokens.
– You didn't describe a task to someone. You prompted them zero-shot.
– You didn't say something non-sensical. You sampled at a high temperature.
– The person is not bad/evil, they are unaligned.
– -
Internet Text Data Loss: Missing Draft Revisions for GPT Training
By
–
Yep! The interesting part is that most of the text on the internet is the "final" text, after you've revised it for a bit. All of that "latent structure" of your drafts, edits, going back and forth etc. is sadly lost. This would make for ideal data for GPTs so they can learn the
-
GPT-4 Potential: Prompt Design and Meta-Cognition Strategies
By
–
Good example of us not seeing max GPT-4 capability yet, imo. Prompt design, tool use, meta cognition strategies (eg idea of attempt, critique, retry, capabilities model, etc) are very likely to go a long way.
-
SVM Dual Solver Implementation and Convergence Proof
By
–
That time I wrote a solver for an SVM in the dual, proved it’s convergence and felt pretty swole 😀
-
AI and AGI once considered disreputable and fringe ideas
By
–
Oh AI was a very dirty word. And even worse – AGI? That’s crackpot territory
-
Infrastructure Layer for GPT APIs and PDF Interaction Tools
By
–
"How to chat with a 56-page PDF"
— Andrej Karpathy (@karpathy) 24 mars 2023
Good developer-focused YouTube explainer: https://t.co/gNUQ7MhNpp
Very excited about the growing layer of software infrastructure on top of GPT APIs, and all of the possible extensions here. https://t.co/jR057wxHei"How to chat with a 56-page PDF"
Good developer-focused YouTube explainer: https://
youtube.com/watch?v=ih9PBG
VVOO4
…
Very excited about the growing layer of software infrastructure on top of GPT APIs, and all of the possible extensions here. -
The Evolution of AI: From 2008 Workshops to Today’s Landscape
By
–
The vibes when I joined AI in ~2008:
– workshops w 50 ppl musing on whether deep learning will ever work
– papers w cute toy problems
– fun poster sessions
– this experiment I ran in MATLAB
– high-level panels on paths to AI
– neuroscience guest lectures
Today is *not* the same. -
GPT: A New Computer Architecture Running on Text
By
–
GPT is a new kind of computer architecture that runs on text. Yes it can talk to us, but also to much of our existing software infrastructure. First via apps on top of APIs, now inside ChatGPT via plugins.
What a time right now… https://
openai.com/blog/chatgpt-p
lugins
… -
John Connor as Prompt Engineer Instead of Soldier
By
–
Plot twist John Connor is not a soldier but a prompt engineer