Distbelief was used internally for training 10,000+ distinct models of all kinds of architectures (convnets, RNNs, LSTMs, feed forward networks, sparse MoEs, etc), with all kinds of different training objectives (supervised, unsupervised, RL, etc).

→ View original post on X — @jeffdean