You can always make a batch large enough that the GPU stays busier than the CPU, but if you want to do hundreds or thousands of steps a second, CUDA graphs are really helpful. You should never be rebuilding graphs — the trick is to make sure the one graph you build at startup

→ View original post on X — @id_aa_carmack,