I think the problem is that we don't have ablation studies: how would the same architecture and training run do with RoPE vs NoPE?
That being said, I'd say Kimi Linear would be an example where NoPE worked well.

→ View original post on X — @rasbt