If there’s anything that needs to be closed source and secret and run only by a few trusted capable people, it’s good high quality evals. The closed nature is primarily to ensure training datasets don’t get contaminated. The “few” is to ensure we don’t have noise and low quality

→ View original post on X — @aravsrinivas