We replicated Gould et al.'s “successor heads” paper, and found that independent component analysis can further decompose these heads into a "successor" component, induction component, and category components. https://
transformer-circuits.pub/2024/september
-update/index.html#successor-heads
…

→ View original post on X — @anthropicai