"Steered LLM Activations are Non-Surjective" Activation steering can make an LLM behave very differently, but that doesn't mean any prompt could have caused the same internal state. This paper shows steering pushes activations off the prompt-reachable manifold into

→ View original post on X — @askalphaxiv