No. That's just false.
Whenever you can use it, gradient-based optimization is incomparably more efficient than gradient-free, regardless of hardware.
There are situations where you have no choice but use gradient free, you should make that as rare as possible.
When you do, you

→ View original post on X — @ylecun,