RIFT proposes a dynamic KV cache compression strategy to mitigate the issue of decoding staleness and improve speculative decoding quality.
On October 24, 2026, arXiv published the paper arXiv:2610.31098v1, introducing RIFT—a dynamic KV cache compression method for drafter models in speculative decoding. By balancing pre-filled information with newly generated tokens and incorporating compressibility training, RIFT enables the Qwen3.5-4B drafter model to maintain over 98% of its full cached receptive length when performing long-context generation at the scale of Qwen3.5-122B, all within an 8K-token KV cache budget.
Why it matters：This study directly addresses the trade-off between hardware costs and accuracy degradation caused by KV cache bloat in long‑sequence generation. It is the first to formally define and quantify the phenomenon of “decode staleness,” and proposes a practical refresh mechanism along with a training–inference co‑optimization scheme. The experimental setup is well specified (32K input / 4K output / 8K cache), and the results offer valuable engineering insights. The paper is an arXiv preprint, with a publication date (2026-10-24) that matches the system’s current date (2026-10-25), ensuring strong timeliness.