Can you comment on my original question — can a process on one grace CPU launch kernels and operate on all 72 GPUs, or do you need to treat it as 18 separate hosts that just happen to have great NCCL connectivity?

→ View original post on X — @id_aa_carmack,