For capability evaluation, our approach is hybrid again. We believe in automated methods for fast identification of performance, but most researchers do not trust fully automated evaluations for full assessments of model quality. Expert evals are a necessity for accurate evals.
