Not necessarily. I thought this way for a while, but at this point I've seen so many "wow, this did incredible on an eval, let's put it in prod! oh, shit, our users hate this"… + "this scored terribly, but whatever, let's give it to 1% of users, oh my god, it converts so well!"

→ View original post on X — @mattshumer_