DistBelief is the distributed training system used for that paper & 1000s of other things. Your statement is equivalent to implementing an unsupervised algorithm in PyTorch, seeing modest results (like 70% relative improvement in SoTA on Imagenet 20k) & declaring PyTorch a dead

→ View original post on X — @jeffdean