Why do AI models sometimes obsess over useless words? Researchers from Tsinghua University, The University of Hong Kong, and Meituan LongCat Team present the first comprehensive survey on Attention Sink. The problem: Transformers often waste attention on a few meaningless

→ View original post on X — @jiqizhixin,