we have updated the Long Context and the Image models reading lists in the wake of 4o and gemini's chosen benchmarks today.
RESEARCH
-

Balancing GenAI Evaluation: Automation Meets SME Validation
By
–
GenAI eval fails when: Too much automation → lacks nuance Too much SME input → too slow The best teams balance both for 7x faster evaluations & 90% SME agreement. How? We break it down here: https://
snorkel.ai/blog/why-genai
-evaluation-requires-sme-in-the-loop-for-validation-and-trust/
… #GenAI #AIevaluation #SnorkelAI -
Stanford Research Reveals Critical Flaws in AI Benchmarks
By
–
A recent Stanford paper reveals that many popular AI benchmarks are fundamentally flawed: They can be outdated, easily gamed, or inaccurate. @StanfordHAI Graduate Fellow @AnkaReuel talks about how researchers are rethinking AI benchmarks via @MorningBrew
: https://
emergingtechbrew.com/stories/2025/0
3/24/researchers-ai-intelligence-benchmarks
… -
China’s Leading AI Research: DeepSeek, Kling, and Alibaba Qwen
By
–
This is a brilliant podcast. Highly recommended by @arkitus and I.
— Nando de Freitas (@NandoDF) 25 mars 2025
I liked the points on China. I’ll go farther. I think some of the best AI research is now happening in China. No doubt @deepseek_ai, @Kling_ai and @Alibaba_Qwen models are among the best. It seems rather futile… https://t.co/iUZTfMhFIVThis is a brilliant podcast. Highly recommended by @arkitus and I. I liked the points on China. I’ll go farther. I think some of the best AI research is now happening in China. No doubt @deepseek_ai
, @Kling_ai and @Alibaba_Qwen models are among the best. It seems rather futile -
1 billion open source methods with LoRAs
By
–
You have 1 billion methods to do that in open source with LoRAs. Gone are the days when OpenAI released features never seen before.
-
GPT-4 Turbo’s Record Score Jump Over Claude-1
By
–
Very kind of you to say this, but this isn't the largest score jump ever. I did some vibe coding to put the data https://
huggingface.co/spaces/lmarena
-ai/chatbot-arena-leaderboard/tree/main
… together, but it was a pain. "The largest lead was 69.30 points when GPT-4 Turbo took over from Claude-1 on November 16, 2023" -

Gemini 2.5 Pro Exp Tops SEAL Leaderboards Across Multiple Benchmarks
By
–
Gemini 2.5 Pro Exp dropped and it's now #1 across SEAL leaderboards: Humanity’s Last Exam VISTA (multimodal) (tie) Tool Use (tie) MultiChallenge (multi-turn) (tie) Enigma (puzzles) Congrats to @demishassabis @sundarpichai & team! https://
scale.com/leaderboard -
Google Announces Gemini 2.5 Pro Model and Performance Metrics
By
–
Read more about Gemini 2.5 Pro and see the metrics here:
-
Which tasks will it outperform other models the most?
By
–
Which tasks will it outclass other models the most?
-
Gemini 2.5 Pro Tops LMArena with Major Multimodal and Coding Improvements
By
–
Gemini 2.5 Pro is an awesome state-of-the-art model, no.1 on LMArena by a whopping +39 ELO points, with significant improvements across the board in multimodal reasoning, coding & STEM. You can try it out now in AI Studio https://t.co/lLpF8ToTVJ & @GeminiApp with Gemini Advanced https://t.co/bgjabz8O1u
— Demis Hassabis (@demishassabis) 25 mars 2025Gemini 2.5 Pro is an awesome state-of-the-art model, no.1 on LMArena by a whopping +39 ELO points, with significant improvements across the board in multimodal reasoning, coding & STEM. You can try it out now in AI Studio http://
ai.dev & @GeminiApp with Gemini Advanced
