Have to think a bit about how to best visualize it, but if you are interested, I have a working from-scratch code implementation of Gemma 4 E2B in the meantime to see how per-layer embeddings are implemented: https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch05/17_gemma4/standalone-gemma4.ipynb
…

→ View original post on X — @rasbt,