nah that’s not the same thing at all; logit lens is pretty poorly conditioned esp at early layers and certainly doesn’t capture all of the hidden state

→ View original post on X — @jxmnop