It's no mystery why LLMs can learn in context. They're just doing nearest neighbor on the manifold learned by pretraining. See: https://
arxiv.org/abs/2012.00152
LLMs Learn In Context via Nearest Neighbor on Learned Manifold
By
–
By
–
It's no mystery why LLMs can learn in context. They're just doing nearest neighbor on the manifold learned by pretraining. See: https://
arxiv.org/abs/2012.00152