Yes, it's an optional feature. It uses FlashAttention by default, but you can optionally use FlashAttention 2 via pip install 'flash-attn>=2.0.0.post1' –no-build-isolation

→ View original post on X — @rasbt