I follow the summations and network connectivity to understand resilience to information loss.
When each output is the sum over a large set of input computations, which themselves densely cross-connect in previous layers — I just expect information to be really resilient to

→ View original post on X — @soumithchintala,