We recently released Meta Lingua, a lightweight and self-contained codebase designed to train language models at scale. Lingua is designed for research and uses easy-to-modify @PyTorch components in order to try new architectures, losses, data and more.

→ View original post on X — @aiatmeta