1 GPT call is a bit like 1 thought. Stringing them together in loops creates agents that can perceive, think, and act, their goals defined in English in prompts. For feedback / learning, one path is to have a "reflect" phase that evaluates outcomes, saves rollouts to memory,
@karpathy
-
AutoGPTs: The Next Frontier of Prompt Engineering and AI Systems
By
–
Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run(). https://t.co/EKy84pa5bB
— Andrej Karpathy (@karpathy) 2 avril 2023Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run().
-
Children as Base Models Before Societal Reinforcement Learning
By
–
I feel like children are the base model before society RLHFs us
-
Comments as Code: A Cleaner Approach to Programming
By
–
Tired: write comments to prompt copilot to write code.
Wired: just write comments. it's cleaner 😀 -
High bias on token quality discrimination approach
By
–
Ty I try to keep a high bias on my tokens quality discriminator
-
LLM Speak: Reframing Common Descriptions in AI Terminology
By
–
LLM speak :
– You didn't find some material boring. It had low quality tokens.
– You didn't describe a task to someone. You prompted them zero-shot.
– You didn't say something non-sensical. You sampled at a high temperature.
– The person is not bad/evil, they are unaligned.
– -
Internet Text Data Loss: Missing Draft Revisions for GPT Training
By
–
Yep! The interesting part is that most of the text on the internet is the "final" text, after you've revised it for a bit. All of that "latent structure" of your drafts, edits, going back and forth etc. is sadly lost. This would make for ideal data for GPTs so they can learn the
-
GPT-4 Potential: Prompt Design and Meta-Cognition Strategies
By
–
Good example of us not seeing max GPT-4 capability yet, imo. Prompt design, tool use, meta cognition strategies (eg idea of attempt, critique, retry, capabilities model, etc) are very likely to go a long way.
-
SVM Dual Solver Implementation and Convergence Proof
By
–
That time I wrote a solver for an SVM in the dual, proved it’s convergence and felt pretty swole 😀
-
AI and AGI once considered disreputable and fringe ideas
By
–
Oh AI was a very dirty word. And even worse – AGI? That’s crackpot territory