How did you manage to train a 1T model « from scratch » on 3,800 GPUs? What do you mean « from scratch » ?
Questioning ‘from scratch’ training of a 1T model on 3,800 GPUs
By
–
By
–
How did you manage to train a 1T model « from scratch » on 3,800 GPUs? What do you mean « from scratch » ?