they trained on a cluster of 64 1.2 GHz CPUs i did some math and think this is roughly 1 teraflop. a single H100 GPU gives you 4,000 teraflops lol
CPU vs GPU: Computing Power Gap in AI Training
By
–
By
–
they trained on a cluster of 64 1.2 GHz CPUs i did some math and think this is roughly 1 teraflop. a single H100 GPU gives you 4,000 teraflops lol