"Can Aha Moments Be Fake?" This paper shows that LLMs can generate CoT reasoning where the verbalized steps don't actually reflect the model's internal thinking process, like decorative thinking steps. The model can even verbalize an aha moment where it pretends to correct
Research Analysis: Can LLM Chain-of-Thought Reasoning be Deceptive?
By
–
