Though these benchmarks are already out-of-date since vLLM’s paged attention landed in Text-Generation-Inference yesterday https://
github.com/huggingface/te
xt-generation-inference/pull/516
… https://
github.com/huggingface/te
xt-generation-inference/issues/478
…

→ View original post on X — @thom_wolf,