We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name – system prompt learning? Pretraining is for knowledge.
Finetuning (SL/RL) is for habitual behavior. Both of these involve a change in parameters but a lot of human

→ View original post on X — @karpathy,