We need a Consumer Reports or Underwriters Lab for AI testing. All the public benchmarks are game-able and mostly not useful measures of things LLMs do. We need secret test batteries for subject areas (coding, reasoning, human conversation, writing) and secret red team tests, too

→ View original post on X — @emollick