“Speculative Speculative Decoding” Even though speculative decoding speed up LLM inference, it still has a hidden “stop-and-wait” step, where the draft model can’t start the next guess until the big model finishes verifying the last one. This paper fixes that by making drafting
