All of the modern NN work that is accomplishing things is with dense matrix ops — 1 mat mul forward and 2 back. There are plenty of things Cray vector computers weren’t good at, but dense matmuls was their strong point, and no PC could touch them for nearly two decades. The XMP

→ View original post on X — @id_aa_carmack,