TLDR: surprised that it works, skeptical whether this will replace BPE-style tokenizers, delighted about this refreshing approach
New Tokenization Approach Challenges BPE-Style Methods
By
–
By
–
TLDR: surprised that it works, skeptical whether this will replace BPE-style tokenizers, delighted about this refreshing approach