Recent work has observed that some dimensions in the Transformer residual stream consistently have values that are 20x larger than others. Today we're releasing a short technical note exploring this phenomenon and trying to identify the cause. https://
transformer-circuits.pub/2023/privilege
d-basis/index.html
…

→ View original post on X — @anthropicai,