This created a self-reinforcing feedback loop. The more in-distribution tokens ChatGPT was getting in its chat history, the more strongly the auto-regressive model was pushed to stay in that distribution. ChatGPT memory made this even worse, letting it happen across chats.

→ View original post on X — @jeremyphoward,