It actually turned out the particular spectrum of the chosen waves often is pretty horrible, but it also turned out positional embeddings weren't actually that important anyway so no-one really noticed for years.

→ View original post on X — @jeremyphoward