“The Truth Lies Somewhere in the Middle of the Generated Tokens” LLMs don’t store the meaning of a prompt in one hidden state. As they generate, the meaning gets spread across many token embeddings. So this paper propose a mean-pool over generated token embeddings instead of
Improving LLM Embedding Representations via Mean-Pooling of Generated Tokens
By
–
