Remember how HF precalculated positional embeddings in bf16 for a long time and the impact was small enough that it slipped through unnoticed? :O

→ View original post on X — @jeremyphoward