INNOVATION

Is the fundamental scaling law guiding large language model development broken? Researchers from Tsinghua University have found the answer. They've decomposed cross-entropy loss into three components: Error-Entropy, Self-Alignment, and Confidence, finding that only Error-Entropy truly scales with model size. This new "Error-Entropy scaling law" provides a far more accurate guide for LLM development, outperforming the traditional cross-entropy law, especially for the largest models. Crucial for future AI design. What Scales in Cross-Entropy Scaling Law? Paper: arxiv.org/abs/2510.04067 Code: github.com/yanjx2021/Rethink… Our report: mp.weixin.qq.com/s/ngn6YY6Aj… 📬 #PapersAccepted by Jiqizhixin

→ View original post on X — @jiqizhixin, 2026-04-04 08:49 UTC