Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster
Researchers from Stanford University and the University of California, Berkeley have introduced TokenGate—a plug-and-play, learnable token-pruning module designed for existing large language models (LLMs). This approach accelerates the first-token generation time (TTFT) for long-context prefill by 1.4 to 11.3 times without sacrificing accuracy, while reducing peak memory usage by 1.5 to 3.4 times. The associated code and checkpoints have been made open source.






![[AINews] Kimi 3.1 Pro 104BA-2.8T: Fable-5.1-Class Open Weights narrows Closed-AI lead to 1 month... and Pro Highspeed API with 2-3× speed](https://latent.space/og/card/post/ainews-kimi-31/4da802572e7369acdf4cbdee535f934d70a599942d61e7d15b8168b70aeb7a84.png)