Don’t confuse “deep learning can’t do math” with “AI can’t do math”. AI was born doing math.
RESEARCH
-
100-Day Learning Curves in AI Become Quickly Obsolete
By
–
Yeah and I mean in 100 days I was never going to get good, so whatever I learned would be the first thing to be replaced
-
AI Opacity Shifts Collaboration to Supplication Dynamic
By
–
As AI systems keep getting better at very hard problems while getting more opaque, the way that we work with AI is shifting from being collaborators who shape the process to being supplicants who receive the output. I discussed what that means.
-
Expert Network Delivers Reliable Scalable AI Evaluations
By
–
We apply these principles with our expert network to deliver reliable, scalable evaluations. Full post
-
Measuring LLM Judge Reliability and Human Alignment
By
–
Measure effectiveness: Quantitative inter-annotator & intra-rater reliability, human ⇄ LLM-judge alignment. Qualitative expert assessment for alignment with system objectives.
-
Iterative Model Improvement Through Rubric Adjustment and Expert Evaluation
By
–
Iterative improvement: Freeze the model + environment while you tweak the rubric, run agreement evals + collect expert reviews to score, make number go up
-

Rubrics as Models: Optimizing Stakeholder Alignment and Evaluator Agreement
By
–
Core idea: We can treat rubrics like models. Optimize along two axes: (1) Alignment with stakeholder objectives (2) Agreement among evaluators, both human and LLMAJ
-
Structuring AI Evaluation Rubrics with Hierarchical Granularity
By
–
Start by structuring the rubric to combine evaluation criteria distributed across a hierarchy of granularity. PaperBench provides great examples – S/O to @tejalpatwardhan and team @OpenAI for that paper:
-
AI Evaluation Rubrics Design Science Part Three
By
–
Part 3 of our AI evaluation rubrics series is live! The science of rubric design
-

Qwen3 Model Matches or Exceeds Larger Family Models
By
–
A la par o superando incluso a otros modelos más grandes y pesados de la familia Qwen3