Pretty amazing that we can get such good results even without batchnorm — and not using a proper resnet init like Fixup either! Maybe I should try lsuv with batchnorm-free resnet sometime…

→ View original post on X — @jeremyphoward,