the optimization dynamics of a two-layer perceptron are barely understood. that's just two matmuls and one pointwise non-linear function in between

→ View original post on X — @soumithchintala,