2/5 When sequence lengths vary a lot, as in RL training, padding can consume a large fraction of the tokens. This wastes compute and memory and drives up training time and cost.
@ai21labs
-
AI21 Labs Cuts LLM Online-RL Training Time by 70% with Padding Minimization
By
–
1/5 As part of our work on improving the efficiency of our LLM online-RL training pipelines, we cut policy update step time by ~70% by introducing a model-agnostic padding minimization method.
-
Optimizing vLLM Deployments: Workload Tuning and Metrics Scaling
By
–
5/5 The takeaway: know your workload, tune your config, scale on metrics that reflect client experience. These lessons apply beyond GRPO – any high-throughput vLLM deployment facing variable load can benefit. Full blog post:
-
Autoscaling by Queue Depth: Beyond GPU Utilization Metrics
By
–
4/5 The horizontal fix: autoscale on queue depth, not GPU utilization 100% GPU = efficiency, not overload. We use vllm:num_requests_waiting to trigger scale-up when the deployment can't keep up with incoming requests.
-
Auto-Tune vLLM Config Doubles Throughput and Halves Latency
By
–
3/5 The vertical fix: auto-tune vLLM config Our old settings were way too conservative. After tuning with Auto-Tune vLLM + GuideLLM we got: ~2× throughput, 2× lower latency, same GPU budget @VLLM @Openshift
-
JLM Deployment Sharing Across Multiple Training Jobs for GPU Optimization
By
–
2/5 The setup: JLMs are used only during the reward phase of Online RL, sitting idle the rest of each step. To avoid GPU waste, we shared JLM deployments across multiple training jobs. This improved utilization but exposed JLMs to unpredictable traffic bursts from multiple
-
Scaling vLLM: Doubling Throughput and Halving Latency
By
–
1/5 Go Big or Go OOM: The Art of Scaling vLLM .
We doubled throughput and cut latency in half-same GPUs, just better vLLM config then added smart autoscaling to handle traffic bursts. Here's what we learned optimizing LLM-as-a-Judge for GRPO training. -
Boring AI Agents Maintain Reliability Across Enterprise Workflows
By
–
@AnthropicAI: “The longer models spend reasoning and taking actions, the more incoherent they become.”
— AI21 Labs (@AI21Labs) 8 février 2026
Our boring agents: "Nope. We do not."
Enterprise workflows need reliability across dozens of actions.
👉 Learn why Boring AI is a necessity, not a nice-to-have:… pic.twitter.com/aWobgU590w@AnthropicAI
: “The longer models spend reasoning and taking actions, the more incoherent they become.” Our boring agents: "Nope. We do not." Enterprise workflows need reliability across dozens of actions. Learn why Boring AI is a necessity, not a nice-to-have: -
Boring AI Agents: Reliability Over Extended Reasoning for Enterprise
By
–
@AnthropicAI: “The longer models spend reasoning and taking actions, the more incoherent they become.”
— AI21 Labs (@AI21Labs) 5 février 2026
Our boring agents: "Nope. We do not."
Enterprise workflows need reliability across dozens of actions.
👉 Learn why Boring AI is a necessity, not a nice-to-have:… pic.twitter.com/KVf3pYCCVx@AnthropicAI
: “The longer models spend reasoning and taking actions, the more incoherent they become.” Our boring agents: "Nope. We do not." Enterprise workflows need reliability across dozens of actions. Learn why Boring AI is a necessity, not a nice-to-have: -
The Real Impact of AI: Boring AI Making Business Operations Better
By
–
Well said.
The real impact of AI shows up when it’s applied to the unglamorous parts of the business and quietly makes them work better.
We’ve been calling that Boring AI for a reason.