we also compute capacity in bf16 and it drops a bit, to 3.5ish. but that's a relative increase in bitwise usage (11% of bits or so to 22% of bits) (my first thought was that transformers are doing a bad job of using params efficiently, but now im not sure. it's not *that* bad)