Language modelling has wildly succeeded due to current objective functions(most notably, next token prediction) and evidently powerful, efficient and scalable models(ie. Transformer) . With next token prediction, you have both input text and labels from same text(input is 1 word

→ View original post on X — @jeande_d