You mean, assuming that both an encoder-decoder and a decoder-only architecture, the encoder-decoder is easier to train because you make better use of the data due to the masking pretraining tasks?

→ View original post on X — @rasbt