What I'm disputing is the specific, but I think common, misconception that of course any LLM could count *tokens* easily; it's just that tokenization makes letters unnecessarily hard. Without CoT or tools, LLMs are just bad at counting.
LLMs struggle counting tokens without Chain-of-Thought or tools
By
–