PRO TIP vLLM telling you to use `–enforce-eager` to avoid OOM because CUDA Graphs “don’t have enough VRAM”? Don’t jump straight to eager mode Try this first: – lower `–max-model-len`, ex: 4k – let CUDA Graph compile (which will be cached by torch.compile) – restart, then
vLLM CUDA Graphs optimization: avoid OOM without eager mode
By
–
