Training t5 on lm objective for 100k more steps. There are lm adapted ckpts of t5 on the internet iirc.
Training T5 Model on Language Modeling Objective
By
–
By
–
Training t5 on lm objective for 100k more steps. There are lm adapted ckpts of t5 on the internet iirc.