@xeophon On the topic of swe-rebench and lower scores, another data point for you: my own analysis suggests open models are overfitting to popular patterns/benchmarks while failing to get better at logical reasoning / problem solving:
MACHINE LEARNING
-
Logic Reasoning Benchmark: Frontier vs Open Weight Models
By
–
@scaling01 Before your pivot to Star Wars memes, I remember you used to be interested in LLMs! I just built a logic reasoning / problem solving benchmark where frontier models one-shot solutions, but the open weights models really struggle:
-
Open Weights Model Performance: MoE Sparsity Challenges
By
–
I will add the remaining two (?) models rumoured for release early next week and finalize it with a blog post… In the meantime, if you have any theories why open weights struggle (my theory is that it's MoE/sparsity induced) — let me know!
-
Open Weights Models Lag Behind Frontier on Logical Reasoning
By
–
PREVIEW: The Joy Of Benchmarks (Q1'26) My new #AI benchmark on out-of-domain programming languages (joy) suggests that open weights models are qualitatively *far* behind the frontier on logical reasoning and problem solving… The newest models: GLM-5, Minimax M2.5, and Kimi
-
LLMs Transform Data Cleaning From Manual to Automated Preparation
By
–
Could LLMs finally end the nightmare of manual data cleaning? Researchers from SJTU, Tsinghua, Microsoft Research, MIT, and Alibaba present a comprehensive survey on the future of application-ready data preparation. They detail a massive paradigm shift from rigid, rule-based
-
Learn Neural Network Fundamentals from ‘Neural Smithing’
By
–
Learn Neural Networks fundamentals in this classic #MachineLearning book: “Neural Smithing — Supervised Learning…” http://
amzn.to/2RX40V2
——————
#DataScience #ML #DeepLearning #AI #Algorithms #Mathematics #DataScientist -
Modern Computer Vision with PyTorch — 2nd Edition Book
By
–
2nd Edition, 746 pages, massive! Modern Computer Vision with #PyTorch #DeepLearning — from practical fundamentals to advanced applications and Generative AI: http://
amzn.to/3xAkB7X v/ @PacktDataML ——
#DataScience #MachineLearning #ML #GenAI #DataScientist
——
𝓚𝓮𝔂 -
Book: Unlocking Data with Generative AI and RAG
By
–
New updated 2nd Edition, by @keithbourne v/ @PacktDataML "Unlocking Data with Generative AI and RAG — Learn AI Agent Fundamentals with RAG-powered Memory, Graph-based RAG, and Intelligent Recall" Get the book here: http://
amzn.to/49zsIkb -
LLM Engineer’s Handbook — Build and Train Large Language Models
By
–
LLM Engineer's Handbook — Master the art of engineering Large Language Models #LLMs from concept to production: http://
amzn.to/4dUQrv6 v/ @PacktDataML Implement robust data pipelines and manage LLM training cycles Create your own LLM and refine with the help of hands-on -
Human-AI Interaction Framework for Mathematics Assistance
By
–
Yes, we provided 3 things for AI-assisted math:
* Human-AI interaction (HAI) card (photo), inspired by model cards
* Full transcripts https://
github.com/google-deepmin
d/superhuman/tree/main/aletheia
…
* A label for novelty-autonomy, inspired by SAE Levels of autonomy, see #Aletheia paper https://
arxiv.org/abs/2602.10177