It's possible but switching to a new tokenizer is actually very cheap and easy, so it'd be a big waste of compute

→ View original post on X — @maximelabonne,