First you’d teach the model quite a bit from recent content (w/ tons of LLM outputs in it it’s likely, if we were to train this in normally, the LLM will likely get stuck in that default LLM ‘mode’ we know so well + make it harder to break out of this w/ post-training) So

→ View original post on X — @mattshumer_