Yes, it's an optional feature. It uses FlashAttention by default, but you can optionally use FlashAttention 2 via pip install 'flash-attn>=2.0.0.post1' –no-build-isolation
Global AI News Aggregator
By
–
Yes, it's an optional feature. It uses FlashAttention by default, but you can optionally use FlashAttention 2 via pip install 'flash-attn>=2.0.0.post1' –no-build-isolation
Leave a Reply