This one looks like a big deal. It's not a pure transformer model – it's a combination of both Mamba and transformer, which appears to address different limitations present in each architecture

→ View original post on X — @simonw