Much of the AI industry is caught in a particularly toxic feedback loop rn. Blindly chasing better human preference scores is to LLMs what chasing total watch time is to a social media algo. It's a recipe for manipulating users instead of providing genuine value to them.

→ View original post on X — @alexalbert__