Not sure if there's a way to do that with model sharding — which is necessary if your model is too big to fit on the GPU.
Model Sharding Challenges for Large GPU Deployment
By
–
By
–
Not sure if there's a way to do that with model sharding — which is necessary if your model is too big to fit on the GPU.