In hindsight, I would have been much better off with a more conventional system with PCI A100 cards — I still haven’t done anything dramatic with the extra NVLink bandwidth, and it could have had twice the GPUs.

→ View original post on X — @id_aa_carmack,