It's no mystery why LLMs can learn in context. They're just doing nearest neighbor on the manifold learned by pretraining. See:
LLMs Learn In Context Through Pretraining Manifold Nearest Neighbor
By
–
By
–
It's no mystery why LLMs can learn in context. They're just doing nearest neighbor on the manifold learned by pretraining. See: