Understood, testing now! I got your previous version working too and that seemed to be much faster on 2×3090 (fp8 part disabled) than the code above, not sure why yet…
GPU Optimization Performance Comparison FP8 Quantization Testing
By
–
By
–
Understood, testing now! I got your previous version working too and that seemed to be much faster on 2×3090 (fp8 part disabled) than the code above, not sure why yet…