"Iterative Finetuning is Mostly Idempotent" If a model has a behavioral tendency and you train the next generation on its outputs, does the tendency snowball? This paper actually finds that it's mostly no. So they seed models with traits like sycophancy and misalignment, then

→ View original post on X — @askalphaxiv