language models encode information in their *weights* while embeddings encode information in their *activations* this distinction is important, possibly somewhat profound
Language Models Weights vs Embeddings Activations Information Encoding
By
–
By
–
language models encode information in their *weights* while embeddings encode information in their *activations* this distinction is important, possibly somewhat profound