these models don’t use RoPE but you’re probably directionally correct

→ View original post on X — @jxmnop