nice, exactly! 🙂
(except you're swapping q's and k's – q is the query, the "what am i looking for", k is the key, the "what do i have", and in encoder-decoder the key,value from come from side. admittedly confusing because in dictionaries the _key_ is the "lookup" information.)

→ View original post on X — @karpathy,