Even my laptop can generate text at a much higher rate than me. It's not clear how much the compute costs of the necessary improvements in inference can be compensated by better algorithms, but it may well turn out that H100s are seriously superhuman in practical applications?

→ View original post on X — @plinz,