@karpathy

–

04 April 2023 17h39

MiniWoB!!! I remember building that 😀 very cool

4 April 2023

Confidence in LLM Development and Future Perspectives

By

–

03 April 2023 18h30

Not exactly, there are and will be others who fab LLMs. But I have increasing confidence in the last paragraph of this post https://
karpathy.github.io/2022/03/14/lec
un1989/
…

Deep Learning Skills Shift: From Training to Prompt Engineering

By

–

03 April 2023 18h00

Expectation: I need more deep learning engineers to train better models
Reality: You need prompt engineers and LLM Ops (not sure what to call it (?), post-LLM above-API infra, langchain & friends)
– training is centralizing into megamodels – not fully played out yet but trending

State of the Art Image Generation Results from 5 Years Ago

By

–

03 April 2023 2h15

Around 5 years ago we were very proud of these state of the art results in image generation, trained on 32×32 "images" of CIFAR-10. You can kind of make out little wheel shapes, car/plane parts, and organic structures and textures. Pretty cool right

Von Neumann’s Hypothetical Transformer Architecture Hyperparameters

By

–

03 April 2023 2h01

I wonder if von Neumann had a large d_model, n_layer, head_size or block_size, or kv cache. All of these hyperparams might manifest slightly different.

AutoGPTs Will Organize into Specialized Autonomous Organizations

By

–

02 April 2023 21h30

All of that is just one agent/thread. People coalesce into organizations so they can specialize and parallelize work towards shared goals. Imo this is likely to happen to AutoGPTs and for the same reasons, strung into AutoOrgs, with AutoCEO, AutoCFO, AutoICs, etc.

2 April 2023

GPT Calls as Thoughts: Building Agents with Reflection and Memory

By

–

02 April 2023 21h19

1 GPT call is a bit like 1 thought. Stringing them together in loops creates agents that can perceive, think, and act, their goals defined in English in prompts. For feedback / learning, one path is to have a "reflect" phase that evaluates outcomes, saves rollouts to memory,

2 April 2023

AutoGPTs: The Next Frontier of Prompt Engineering and AI Systems

By

–

02 April 2023 20h44

Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run(). https://t.co/EKy84pa5bB
— Andrej Karpathy (@karpathy) 2 avril 2023

Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run().