I think Olmo hybrid also uses Gated DeltaNet though right? Jamba is a good point, I think that’s kind of like a predecessor of Nemotron 3 (with mamba 1 instead of 2) if I recall correctly. It’s been a while

→ View original post on X — @rasbt,