New Submission On NanoGPT Speed-Run! Improved training token efficiency by ~40% to reach target HellaSwag score by switching the vocabulary and tokenizer only. It now takes 1050 steps instead of 1750 steps; current record on 11/24/24 labelled in the graph as "TikToken
NanoGPT Achieves 40% Training Efficiency Boost Via Tokenizer
By
–
