9. MCPEval MCPEval is an open-source framework that automates end-to-end evaluation of LLM agents using a standardized Model Context Protocol, eliminating manual benchmarking.
@dair_ai
-
Inverse Scaling in Large Reasoning Models Test-Time Compute
By
–
6. Inverse Scaling in Test-Time Compute Presents a systematic study of inverse scaling in large reasoning models, where increasing the test-time compute (i.e., reasoning length) harms rather than helps model performance.
-
Compute-Optimal Strategies for Many-Shot In-Context Learning
By
–
7. Towards Compute-Optimal Many-Shot In-Context Learning Proposes practical strategies for reducing the cost of many-shot in-context learning while preserving or improving performance. https://
arxiv.org/abs/2507.16217 -
In-Context Learning Without Weight Updates in LLMs
By
–
5. Learning without Training This paper provides a theoretical and empirical explanation for how LLMs exhibit in-context learning, the ability to learn from examples in a prompt without weight updates.
-
Routine: Structured Planning Format for LLM Agent Tool-Calling
By
–
4. Structural Planning for LLM Agents This paper introduces Routine, a structured planning format designed to improve the stability and accuracy of LLM agents executing multi-step tool-calling tasks in enterprise settings.
-
Context Engineering Survey: Optimizing LLM Information Systems
By
–
8. A Survey of Context Engineering for LLMs This survey defines Context Engineering as a formal discipline for optimizing information given to LLMs, outlining its core components, retrieval, processing, and management, and their integration in systems like RAG, memory, and
-
Q-Chunking: Reinforcement Learning for Long-Horizon Sparse-Reward Tasks
By
–
9. Q-Chunking Q-chunking is a reinforcement learning approach that uses action chunking to improve offline-to-online learning in long-horizon, sparse-reward tasks.
-
LLMs in AIOps: Survey of 183 Papers and Methods
By
–
10. A Survey of AIOps This survey analyzes 183 papers to evaluate how LLMs are being used in AIOps, focusing on data sources, task evolution, applied methods, and evaluation practices.
-
Scaling Reinforcement Learning for Enhanced Reasoning in Small Models
By
–
6. Scaling up RL This paper investigates how prolonged RL can enhance reasoning abilities in small language models across diverse domains.
-
Machine Bullshit: LLMs and Indifference to Truth
By
–
7. Machine Bullshit This paper introduces the concept of machine bullshit, extending Harry Frankfurt’s definition, discourse made with indifference to truth, to LLMs.