The tokenizer being open is a big deal. More than people realize. Tokenization often breaks downstream use cases, this fixes that.

→ View original post on X — @godofprompt