5/5 Ensemble: Run multiple distinct agent configs in parallel → select the best trajectory. Leverages success of the portfolio vs single variants (see also: @/karpathy’s LLM Council). Can outperform best-of-N on accuracy but at lower cost & latency. Here’s how each method did
@ai21labs
-
Best-of-N : optimisation parallèle d’agents IA
By
–
4/5 Best-of-N: Run one agent config N times in parallel → select the best trajectory. Leverages LLMs’ non-determinism – but hinges on a good eval mechanism (we use an LLM-as-a-Judge). Can increase accuracy without linearly increasing latency – but parallel runs can add up $$.
-
Agent Scaffolding: Building the Infrastructure
By
–
3/5 Agent scaffolding: Adjust prompts, integrate tools/skills, set execution policies, retries, memory, etc. → create the operational framework. Useful for long-horizon or multi-step tasks.
-
Simple model testing for prompt tuning
By
–
2/5 Simple model testing: Tune a prompt → evaluate across models → pick the best operating point. Good for single-call or low-complexity agents.
-
4 agent optimization methods tested on BrowseComp-Plus
By
–
1/5 We’re seeing 4 common agent optimization methods for hitting the right accuracy-cost or accuracy-latency tradeoff. We tried them all out on BrowseComp-Plus (results at end of the thread).
-
YAAP: Harness Engineering and Why Agents Fail in Production
By
–
New #YAAP episode out now @yuvalinthedeep sits down with @mikegchambers from @awsdevelopers to unpack harness engineering and why it's the reason most agents never make it to production. Listen/Watch it now: https://
ai21.com/yaap/everythin
g-but-the-model-harness-engineering/?utm_source=org-twitter
… -
Webinar: Bridging the Gap Between Demo Agents and Production Systems
By
–
Join @YuvalinTheDeep
, Senior Developer Advocate at @AI21Labs
, for a live webinar in partnership with @DataCamp
: The Four Gaps Between Demo Agents and Production Systems. If you're shipping agents to prod, this is where prototypes break and real systems begin. May 6 · 11am ET -
Maestro Maps Accuracy-Cost-Latency Tradeoff Surface Automatically
By
–
5/5 Maestro automatically searches across this space (model ensembles, scaling strategies, execution policies), maps the Pareto frontier, and surfaces the full accuracy–cost–latency tradeoff surface. Read the full methodology here:
-
Sequential vs Batched Execution Trade-offs in LLM Ensembles
By
–
3/5 Execution policies help, too: you can see how sequential execution (vs. batched execution) saves spend but drives up latency for the same GPT-5 + MiniMax ensemble.
-
Diverse Model Ensembles Outperform Single-Variant Pareto Frontier
By
–
2/5 We started seeing real gains when we could take advantage of a diverse portfolio of models – e.g. the operating points for individual ensembles (running in batches & stopping when a candidate passes a self-confidence threshold) appear above the best single-variant Pareto