7/ Hyena Hierarchy – a subquadratic drop-in replacement for attention; it interleaves implicit long convolutions and data-controlled gating and can learn on sequences 10x longer and up to 100x faster than optimized attention.

→ View original post on X — @dair_ai