Yeah context rot is about bad tokens (mistakes etc) getting into the context and causing poor performance later on – what Theo is describing is more models straight up wasting tokens

→ View original post on X — @simonw