That's true, but you still have positional embeddings, and Q and K weights that are learned
Positional Embeddings and Learned Query-Key Weights in Transformers
By
–
By
–
That's true, but you still have positional embeddings, and Q and K weights that are learned