We've updated the UL2 paper with details. http://
arxiv.org/abs/2205.05131. UL2 was only trained w 512 inputs, but this extends UL2 20B to 2048 inputs. While we found mode switching to help, it was admittedly v inconvenient. This update also removes the need to use mode tokens 🙂

→ View original post on X — @yitayml,