to pretrain is to learn to compress the universe. and as byproduct, learning to simulate all natural processes gather all the data you can. compress. backpropagate you don't see elegance in that?

→ View original post on X — @jxmnop,