New method “grows” smaller language models that were already trained into larger ones. The technique performs better than similar methods, saving about 50% of the computational cost required to train a large model: https://
bit.ly/3JX3hw6

→ View original post on X — @mit_csail