Not sure how it will play out, but it means that generations will be very fast for such a big model, as only a small portion of params will be active at a time.
Fast Generation in Large Models with Sparse Parameter Activation
By
–
By
–
Not sure how it will play out, but it means that generations will be very fast for such a big model, as only a small portion of params will be active at a time.