I saw something like this before. Curious why Chinese characters are multiple tokens, and how it would be different if they were single tokens.

→ View original post on X — @yoheinakajima