A fully optimized system might wind up with persistent kernels running continuously on all GPUs, but I want the ability to debug a single process using all GPUs, as well as launch parallel experiments directly from one host without needing a cluster layer.

→ View original post on X — @id_aa_carmack,