Late to the party but "GPT in 60 Lines of NumPy" / picoGPT is nicely done: https://
jaykmody.com/blog/gpt-from-
scratch/
…
– good supporting links/pointers
– flexes some of the benefits of JAX: 1) trivial to port numpy -> jax.numpy, 2) get gradients, 3) batch with jax.vmap
– inferences gpt-2 checkpoints

→ View original post on X — @karpathy,