btw nanoGPT is meant for education, possibly have a look at some of the slightly bit more "prod" repos i link to it in the readme, e.g. litgpt or tinyllama. When you look at the code it will look quite nanoGPT-like and recognizable, but possibly a bit more battle-tested.

→ View original post on X — @karpathy