Each one still has the softmax issue though and it all gets turned into one vector at the end. Learning to do addition in that projection feels hard, but I don’t know enough to say it can’t happen
Softmax projection and difficulty of learning additions in embeddings
By
–