1/5 Caching relies on a deterministic assumption: same input, same output. This breaks down with LLMs, particularly with agents using parallel rollout strategies like best-of-N sampling.

→ View original post on X — @ai21labs,