Sorry can you expand? Are you saying this might be due to the flash attention inside cuDNN?

→ View original post on X — @karpathy