the issue is that unified memory would still be faster for loading MoEs that are larger than the largest single GPU in terms of Memory
Unified Memory Faster for Loading Large MoE Models
By
–
By
–
the issue is that unified memory would still be faster for loading MoEs that are larger than the largest single GPU in terms of Memory