I am pretty sure next models like o2 or 3 will have much much better results
RESEARCH
-
MathFrontier Benchmark: New Challenge for LLM Capabilities
By
–
Very exciting article about the new MathFrontier benchmark, which they say LLMs are currently still failing at. The tasks are so difficult, so specialized, that perhaps PhDs from the special field can solve a task, but certainly not in general. It will be very exciting to see
-
Anthropomorphizing trap but wrong CoT outperforms non-CoT
By
–
I mostly agree — anthropomorphizing is a trap — but the choice of “Let’s think step by step” isn’t arbitrary either. Wrong CoT working better than non-CoT still says something
-
LLM Development Limits: Orion and Gemini Model Improvements
By
–
Another interesting perspective on the development of Orion. As I said, though, I believe that this refers to the fact that regular LLMs are slowly reaching their limits. That's why Google is having increasing trouble improving their Gemini models. However, since we can
-
Continuous Optimization Over Discrete Search in AI
By
–
Continuous optimization. Not discrete search.
-
Fine-tuning LLMs to learn from tools and adaptive problem-solving
By
–
9). Adapting while Learning – proposes a two-part fine-tuning approach that first helps LLMs learn from tool-generated solutions and then trains them to determine when to solve problems directly versus when to use tools; testing on math, climate science, and epidemiology
-
Personalization Framework for Large Language Models
By
–
10). Personalization of LLMs – presents a comprehensive framework for understanding personalized LLMs; introduces taxonomies for different aspects of personalization and unifying existing research across personalized text generation and downstream applications.
-
LLM Numerical Understanding: Fine-tuning Improves Processing Ability
By
–
7). Number Understanding of LLMs – provides a comprehensive analysis of the numerical understanding and processing ability (NUPA) of LLMs; finds that naive finetuning can improve NUPA a lot on many but not all tasks…
-
WebRL Self-Evolving Framework Boosts Open LLM Web Agents
By
–
8). WebRL – proposes a self-evolving online curriculum RL framework to bridge the gap between open and proprietary LLM-based web agents; it improves the success rate of Llama-3.1-8B from 4.8% to 42.4%, and from 6.1% to 43% for GLM4-9B; the open models significantly surpass the
-
Vision-Language Agents Vulnerable to Pop-up Adversarial Attacks
By
–
5). Attacking Vision-Language Agents via Pop-ups – shows that integrating adversarial pop-ups into existing agent testing environments leads to an attack success rate of 86%; this decreases the agents' task success rate by 47%.