what are some explanations for constant codeswitching? 1. OpenAI has figured out RL. the models no longer speak english
2. data corruption issues via OCR or synthetic training
3. somehow i forced the model to output too many tokens and they gradually shift out of distribution
Understanding Constant Codeswitching in Language Models
By
–