This post became popular; Few more thoughts / pointers on the topic for the interested reader. Example of the complexity involved: @cHHillee has a great post "Making Deep Learning Go Brrrr From First Principles" https://
horace.io/brrr_intro.html
I was always struck by this diagram from

→ View original post on X — @karpathy