The latest AI news, in focus.

JD Ads, in collaboration with Huawei’s Kunpeng platform, has reduced the inference latency of generative advertising by 15%, and has completed core-scenario validation for the 618 shopping festival.

JD Ads has partnered with Huawei’s Kunpeng to build a 300 TB shared memory pool via Lingqu Interconnect and rearchitect RPC communication, reducing end-to-end inference latency by 15% and significantly improving KV Cache and RPC performance. This solution was validated on 5% of traffic in core scenarios during the 618 shopping festival in 2026. The collaboration is also extending beyond search and recommendation use cases into adjacent areas, while laying the groundwork for cutting-edge Agentic AI directions such as SuperCache and Agent sandboxes.

Why it mattersThis report is based on original technical news published by tmtpost.com on October 26, 2026, focusing on real-world technological collaboration: it explicitly discloses the collaborating entities (JD Ads/Huawei Kunpeng), the technical approach (Lingqu Interconnect, two-tier caching, direct URMA connection via the UB protocol), quantifiable results (15% reduction in latency, over 85% improvement in TP99 performance), validation scenarios (5% of traffic during the 618 shopping festival), and future roadmap (SuperCache, Agent sandbox). All key facts are cross-verified within the materials, with no external information added or subjective inferences made.

Tsinghua University and The Chinese University of Hong Kong jointly open-source AKV: an engineering service system for KV caching tailored to LLM agents.

On October 26, 2026, researchers from Tsinghua University and the Chinese University of Hong Kong open-sourced AKV—a KV cache engineering server built on SGLang and specifically designed for large language model (LLM) agents. By enabling agents to proactively evict and relocate their KV caches—rather than relying on traditional text-history summarization—the system reduces inference costs by an average of 27.7% across four agent benchmarks and improves task scores by 1.7 points on average. The code is licensed under the Apache-2.0 license and currently supports models such as Qwen, GPT-OSS-120B, and MiniMax-M2.7.

Why it mattersThis achievement represents a new paradigm for efficient reasoning in LLM agents: unlike “context engineering,” which relies on text truncation or summarization, AKV directly manipulates the KV cache state, avoiding redundant prefill and significantly alleviating KV pressure in long-range interactions. Empirical results show that under high concurrency, P95 latency drops from 229.9 seconds to 30.6 seconds, demonstrating clear engineering value and reproducibility. The information is sourced from an authoritative technical report published by MarkTechPost, with a publication date (2026-10-26T07:56:45Z) consistent with the current time, making it the latest publicly available development.

llama.cpp releases version b11871: adds a limit on JSON nesting depth to prevent service crashes

llama.cpp introduced a critical security fix in version b11871, imposing a hard limit of 128 nested JSON levels on requests to prevent HTTP thread stack overflows and service crashes caused by deep recursive parsing (triggering SIGBUS on macOS and SIGSEGV on Linux). This policy covers multiple input scenarios for the chat interface and uniformly returns an HTTP 400 error.

Underdog Saluki 27B: Technical Analysis of the 2-bit Qwen3.8-27B Tool-Calling Model Based on GSQ-RCO

This article introduces Saluki 27B, a model released by Underdog—a 7.9 GB, 2-bit quantized variant of Qwen3.8-27B—built using the GSQ-RCO method from ISTA-DASLab. It claims to outperform the original 54 GB full-precision model on the BFCL v4 benchmark across 120 tasks; however, all performance conclusions are based solely on the developers’ assertions and have not been independently verified.

llama.cpp has released version b11870, officially supporting the DeepSeek-V4.1-Flash model.

llama.cpp has added support for the DeepSeek-V4.1-Flash (deepseek41) model in version b11870, and implemented several low-level optimizations tailored to its Engram architecture, including simplification of KV cache computation, replacement of MXFP8 with BF16 precision, and in-graph row gather operations; it also provides multi-platform binary builds covering macOS, Linux, Windows, Android, iOS, and openEuler.

llama.cpp releases version b11869: fixes Jinja template function argument handling and adds multi-platform binary support.

The llama.cpp project has released build version b11869, which primarily fixes the parameter-parsing logic for the select and selectattr functions in Jinja templates and provides precompiled binaries that support dozens of hardware–backend combinations across macOS, Linux, Windows, Android, iOS, and openEuler.

llama.cpp releases version b11868: enhanced Hexagon NPU multi-core Flash Attention support, with cross-platform compilation and build capabilities.

llama.cpp v2.45.0 (tag b11868) has been officially released, with the core update being the introduction of a KV-head splitting mechanism in the Hexagon backend to optimize multi-core HMX Flash Attention performance and unify the FA splitting control flag. Binary packages are simultaneously available for all major platforms, including macOS, Linux, Windows, Android, iOS, and openEuler, supporting multiple backends such as CUDA 12/13, Vulkan, ROCm, OpenVINO, SYCL, and Snapdragon (Adreno + Hexagon).

llama.cpp releases version b11867: server-side entry point refactored, with full support for multiple platforms and heterogeneous acceleration backends.

llama.cpp has officially released version b11867 on GitHub. Key changes include a refactored server entry point (PR #31113) and the addition of the server_role mechanism. Additionally, more than 40 precompiled binaries are now available, covering macOS, Linux, Windows, Android, iOS, and openEuler, with support for diverse hardware backends such as CPU, CUDA (12.4/12.8/13.4), Vulkan, ROCm 10.0, OpenVINO, SYCL (FP16/FP32), and Snapdragon (Adreno + Hexagon).

llama.cpp releases version b11866: removes redundant AMAX scaling logic from the Metal backend and streamlines the MoE computation path.

llama.cpp v.b11866 has removed the outdated src1 power-of-two AMAX scaling mechanism from the Metal backend, instead relying on a more granular per-expert-token amax scaling within the computation graph, ensuring that MoE projection activation values remain within the f16 representation range; concurrently, it also deleted the associated two-stage reduction kernels, F32-precision scheduling, and out-of-bounds test cases.

llama.cpp releases version b11865: adds RDMA receive boundary checking and supports multiple platform backends, including CUDA 13.4, ROCm 10.0, and Snapdragon NPU.

The llama.cpp project has released build version b11865, with core changes including the addition of memory-boundary checks in the RDMA receive logic (PR #30399) and a significant expansion of hardware-acceleration support, covering CUDA 13.4, ROCm 10.0, Vulkan, OpenVINO, SYCL, and Qualcomm Snapdragon—providing full-stack heterogeneous computing backends for CPU, Adreno, and Hexagon. macOS KleidiAI support and openEuler builds have been explicitly marked as DISABLED.

llama.cpp releases version b11861: fixes an inference exception caused by the minimax-m3 model’s quantized scaling tensor not being propagated.

llama.cpp v.b11861 fixes the weight distortion issue in the minimax-m3 model that occurred under NVFP4/MXFP8 quantization due to the scale tensor not being injected into the computation graph, thereby preventing “token soup” and residual vanishing; it also provides multi-platform binary builds covering macOS, Linux, Windows, Android, iOS, and openEuler.

AnyLanguageModel 1.0 released: a unified Swift interface supporting both local and hosted models, compatible with the entire Apple platform family of operating systems 27

Hugging Face has officially released AnyLanguageModel 1.0, an open-source Swift framework fully compatible with the Apple Foundation Models API. It supports seamless switching between local models (MLX, llama.cpp, Core ML) and hosted models (OpenAI, Anthropic, Gemini, Hugging Face Inference Providers, etc.), and introduces new features such as image input, tool invocation, guided generation, and automatic adaptation of inference chunks.

Transformers now runs llama.cpp quants

Transformers now runs llama.cpp quants

Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

Native-speed vLLM transformers modeling backend

Native-speed vLLM transformers modeling backend

Run a vLLM Server on HF Jobs in One Command

Run a vLLM Server on HF Jobs in One Command

Heaps do lie: debugging a memory leak in vLLM.

Heaps do lie: debugging a memory leak in vLLM.