i guess ROPE and some positional embeddings use sine and cosine as well

→ View original post on X — @jxmnop