3/ A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention – studies the theoretical understanding of learning with attention layers by exploring the interplay between positional and semantic attention.
Phase Transition in Dot-Product Attention: Positional vs Semantic Learning
By
–
