turns out we're all biased toward familiar text when rating AI outputs. RLHF (Reinforcement Learning from Human Feedback) learned this preference, sharpened it, and now every model collapses into repetition. the fix? ask for probability distributions instead of single answers.