Standard RAG falls apart on aggregative queries. Things like "average ARR for companies with >1k employees" require reasoning across hundreds of docs, and vector retrieval just can't handle that reliably. We built Structured-RAG to fix this:
– induces a schema from your
@ai21labs
-
Structured-RAG: Solving Aggregative Queries in RAG Systems
By
–
-
Intelligent Compute Allocation for Agent Test-Time Scaling
By
–
Test-time compute scaling for agents isn’t just running multiple full trajectories and picking a winner. Duplicate a 50–150 step workflow and you multiply tool calls, latency, and cost.
— AI21 Labs (@AI21Labs) 5 mars 2026
Our take: allocate compute at high-uncertainty steps + use strong reducers to close the… pic.twitter.com/IG4Q240iTrTest-time compute scaling for agents isn’t just running multiple full trajectories and picking a winner. Duplicate a 50–150 step workflow and you multiply tool calls, latency, and cost. Our take: allocate compute at high-uncertainty steps + use strong reducers to close the
-
Discussion on Production AI Infrastructure Development
By
–
Always great to see more discussion around production AI infra.
-
Modular Agent Architecture with Conversational and Investigatory Routing
By
–
In a modular agent setup, a router first classifies the request as conversational or investigatory. If investigatory, a retrieval module fetches relevant docs [eval: grounding], and a synthesis module integrates the evidence into a response [eval: faithfulness, coverage].
-
AI Agents Need Sequential Modularity for Better Evaluation
By
–
2/4 Modern AI systems plan, retrieve across sources, reason, take actions, and update state. But most evals capture only the end result of that entire process. Agents need sequential modularity, like human language production, to enable evaluation per module and support
-
Agent Architectures and Psycholinguistics: Lessons in Modularity
By
–
1/4 Our Head of Human Alignment and Standards @JulieFadlon, PhD, breaks down what agent architectures have to learn from Psycholinguistics. Hint: modularity.
-
Agent Architectures and Psycholinguistics: Lessons in Modularity
By
–
1/4 Our Head of Human Alignment and Standards @JulieFadlon, PhD, breaks down what agent architectures have to learn from Psycholinguistics. Hint: modularity.
-
AI21 Labs focuses on frontier model capabilities failure taxonomy and agent validation
By
–
Thanks for this research & for building practical measures of frontier model capabilities. We're focusing these days on the failure taxonomy you highlighted in the paper – testing how elements like structured programmatic plans and in-flow validators & reducers can help agents
-
Model-Agnostic Approach Recovers 90% Sequence Packing Gains
By
–
5/5 Because this approach is model-agnostic, it applies to any architecture. Even on transformers (like Qwen2.5-7B by @alibaba_cloud
, this method recovers ~90% of the gains of sequence packing, without relying on specific attention implementations. Full breakdown + -
Model-Agnostic Approach: Minimizing Padding with Truncation and Batching
By
–
4/5 So we took a different, model agnostic path: minimize padding before it reaches the model using truncation combined with padding-aware batching.