I don't think I've talked about going beyond tokenization? Still seems like a sensible approach to me – Llama 3 upped their token vocabulary from 32,000 to 128,000 which gave them some optimization benefits

→ View original post on X — @simonw