In this short note, we explore thinking of the traditional idea of "distributed representations" as two distinct phenomena: "composition" and "superposition". We walk through toy examples from Thorpe (1989), discussing them from this lens. https://
transformer-circuits.pub/2023/superposi
tion-composition/index.html
…

→ View original post on X — @anthropicai