Yeah except it wasn’t actually that bad. I mean it was a bit annoying, code bloating and you’d have to run grad check, but it also didn’t feel like a major impediment after a bit of practice. I don’t recall spending significant portion of my time deriving/writing backward pass.

→ View original post on X — @karpathy,