For a large enough batch size on a given expert, you'll either saturate the SMs or run out of memory
Batch Size Constraints: GPU SM Saturation and Memory Limits
By
–
By
–
For a large enough batch size on a given expert, you'll either saturate the SMs or run out of memory