9). Mixture of Transformers – introduce Mixture-of-Transformers (MoT), a new sparse multi-modal transformer architecture that matches the performance of traditional models while using only about half the computational resources for text and image processing.

→ View original post on X — @dair_ai