We might be hitting limits of the c4 pretraining that the base model uses as well.

→ View original post on X — @yitayml