Good question. I think the challenge here is that the evaluation is more challenging because of a) it's relative to all the other models
b) it's expensive to run the other models PS: Fun fact, I just wrote a tutorial on Elo ratings and constructing a leaderboard last week:
MACHINE LEARNING
-
Challenges in Model Evaluation and Elo Rating Leaderboards
By
–
-
Automatic Language Detection and Title Generation for arXiv Articles
By
–
https://alphaxiv.org/pdf/2509.22358
-
Meta’s Stochastic Activation Achieves 90% Sparsity LLM Speedup
By
–
First ever activation function swapping strategy just dropped for LLMs! Meta’s new Stochastic Activation proposes a new method which randomly selects between non-linear functions (ReLU & SiLU) in the FFN of LLM, achieving an activation sparsity of 90% with 1.65x CPU speedups!
-
Engaging podcast moments on AI topics discussed
By
–
awkwardly worded. ~31:00 – 32:47 are two of the coolest minutes of a podcast I've ever heard. there we go.
-
AI practical examples and real-world applications
By
–
Re: ai, I have yet to see that be the case. Any good examples?
-
TRM motivation and specialized model design considerations
By
–
Thanks for sharing. Points a 1 and 2 are actually what motivated the work on TRM as mentioned in the paper. I think point 3 makes sense as this is a small, specialized model, not a large generalist model.
Point number 4 also makes sense, this has always been true for ML and DL. -
Multimodal Edge AI: Vision Audio Integration for Speaker Identification
By
–
Combining vision + audio on edge AI to identify who's speaking. A huge number of great applications across all kinds of industries and use cases. Real world engineering at its best Demo: https://
eu1.hubs.ly/H0nKK6B0 -
300 Hours on GPT5: Everything I Learned
By
–
I've spent over 300h on #GPT5… Here's everything I've learned → https://
youtu.be/jJe5DHTLBvM #ChatGPT #ChatGPT5 -
AI Evaluation Method Using Embeddings and Likert Ratings
By
–
And before anyone complains, "which another AI rates" is a bit of an oversimplification (character limits!): as explained in the diagram, the free text response from the AI acting like a consumer is converted into embeddings & compared to reference statements with Likert ratings