How confident are you with respect to the output quality given the 2-bit quantization and reducing experts from 10 to 4? Did you have a mechanism for measuring that?
Quantization Impact on Model Quality and Expert Reduction
By
–
By
–
How confident are you with respect to the output quality given the 2-bit quantization and reducing experts from 10 to 4? Did you have a mechanism for measuring that?