This isn't claiming attention is dead. But it proves attention isn't fundamental. "What one fundamentally needs is not attention itself, but a sufficiently expressive geometric evolution mechanism for hidden representations." The era of exploring geometric alternatives to
Attention not fundamental; geometric evolution needed for hidden representations
By
–