INNOVATION

What if your LLM could dramatically reduce memory usage without sacrificing quality, even in tough situations?

Researchers at University of Science and Technology of China & Data Darkness Lab present DefensiveKV.

They tackle the fragility of LLM Key-Value cache eviction by… pic.twitter.com/iH2znFX5Ni

— 机器之心 JIQIZHIXIN (@jiqizhixin) 7 avril 2026

What if your LLM could dramatically reduce memory usage without sacrificing quality, even in tough situations? Researchers at University of Science and Technology of China & Data Darkness Lab present DefensiveKV. They tackle the fragility of LLM Key-Value cache eviction by using a smart, two-step "defensive aggregation strategy." This approach proactively controls worst-case risks, preventing performance drops in extreme scenarios with negligible computational cost. Layer-DefensiveKV extends this with intelligent layer-wise budget allocation. Their methods slash generation quality loss by an impressive 2.3x and 4.3x respectively against the strongest baselines, across seven task domains and 18 datasets, even when cache is cut to just 20%. This breakthrough sets new performance benchmarks for efficient LLM inference! DefensiveKV: Taming the Fragility of KV Cache Eviction in LLM Inference Paper: openreview.net/forum?id=nJgS… Code: github.com/FFY0/DefensiveKV/… Our report: mp.weixin.qq.com/s/81wiPTdye… 📬 #PapersAccepted by Jiqizhixin

→ View original post on X — @jiqizhixin