LLMS - AI Dynamics

Autoscaling by Queue Depth: Beyond GPU Utilization Metrics

By

–

09 February 2026 10h10

4/5 The horizontal fix: autoscale on queue depth, not GPU utilization 100% GPU = efficiency, not overload. We use vllm:num_requests_waiting to trigger scale-up when the deployment can't keep up with incoming requests.

→ View original post on X — @ai21labs

9 February 2026

Auto-Tune vLLM Config Doubles Throughput and Halves Latency

By

@ai21labs

–

09 February 2026 10h10

3/5 The vertical fix: auto-tune vLLM config Our old settings were way too conservative. After tuning with Auto-Tune vLLM + GuideLLM we got: ~2× throughput, 2× lower latency, same GPU budget @VLLM @Openshift

→ View original post on X — @ai21labs

9 February 2026

JLM Deployment Sharing Across Multiple Training Jobs for GPU Optimization

By

@ai21labs

–

09 February 2026 10h10

2/5 The setup: JLMs are used only during the reward phase of Online RL, sitting idle the rest of each step. To avoid GPU waste, we shared JLM deployments across multiple training jobs. This improved utilization but exposed JLMs to unpredictable traffic bursts from multiple

→ View original post on X — @ai21labs

9 February 2026

Scaling vLLM: Doubling Throughput and Halving Latency

By

@ai21labs

–

09 February 2026 10h10

1/5 Go Big or Go OOM: The Art of Scaling vLLM .
We doubled throughput and cut latency in half-same GPUs, just better vLLM config then added smart autoscaling to handle traffic bursts. Here's what we learned optimizing LLM-as-a-Judge for GRPO training.

→ View original post on X — @ai21labs

9 February 2026

Kling 3.0: Removing technical barriers for cinematic scene creation in one pass.

By

@aibreakfast

–

09 February 2026 4h41

The goal of Kling 3.0 is simple:

Remove the technical ceiling between an idea and a finished scene. Direction, coverage, pacing, and shot logic now happen in one pass.

You describe intent once and the system executes with real cinematic structure.

Also Kling 3.0 now supports… pic.twitter.com/Z41fanoe8e
— AI Breakfast (@AiBreakfast) 9 février 2026

The goal of Kling 3.0 is simple: Remove the technical ceiling between an idea and a finished scene. Direction, coverage, pacing, and shot logic now happen in one pass. You describe intent once and the system executes with real cinematic structure. Also Kling 3.0 now supports

→ View original post on X — @aibreakfast

9 February 2026