The study proposes Delta N-grams: a forward-only, continuous-writing method for native N-gram tables in language models.
On October 24, 2026, arXiv published the paper arXiv:2610.31125v1, introducing Delta N-grams—a novel mechanism that does not modify the pre-trained N-gram table T⁰, but instead writes sparse incremental patches Δ via forward propagation, enabling efficient and reversible continual memory updates.
Why it matters：This work addresses the issue of large perturbations to a model’s base parameters in continual learning by proposing a memory-enhanced pathway that is lightweight in structure (4.1M parameters), computationally efficient (purely forward pass), and fully invertible (resetting Δ restores the original model). Its superior document-specificity and robustness against interference have been validated on models such as Qwen3.8-Flash-Next, making it a cutting-edge contribution to trustworthy AI systems research.