The Ultra-Scale Playbook (the large-scale LLM training guide from the @huggingface science team) is out now! It is a 246-page, very nicely designed PDF that walks you through learning how to train your own DeepSeek-V3 model using:
• 5D parallelism, • ZeRO,
• fast kernels,
Ultra-Scale Playbook: Training DeepSeek-V3 with Advanced Parallelism
By
–
