There's a chance that LoRA finetunes work so well that it dramatically alters the finetuning vs. retrieval + few-shot prompting power dynamic in favor of the former for many applications.

PEFT (Parameter Efficient Finetuning, LoRA included) are emerging techniques that make it… https://t.co/u3Udqpk3dz

— Andrej Karpathy (@karpathy) 20 avril 2023

There's a chance that LoRA finetunes work so well that it dramatically alters the finetuning vs. retrieval + few-shot prompting power dynamic in favor of the former for many applications. PEFT (Parameter Efficient Finetuning, LoRA included) are emerging techniques that make it

→ View original post on X — @karpathy,