Search AI news

b11899

mtmd: add mtmd_helper_get_capabilities (#31183) * mtmd: add mtmd_helper_get_capabilities * cap audio_stream * add back old api for compat * audio_gen_spk_ref, text_gen **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11899/llama-b11899-

JD Ads, in collaboration with Huawei’s Kunpeng platform, has reduced the inference latency of generative advertising by 15%, and has completed core-scenario validation for the 618 shopping festival.

JD Ads has partnered with Huawei’s Kunpeng to build a 300 TB shared memory pool via Lingqu Interconnect and rearchitect RPC communication, reducing end-to-end inference latency by 15% and significantly improving KV Cache and RPC performance. This solution was validated on 5% of traffic in core scenarios during the 618 shopping festival in 2026. The collaboration is also extending beyond search and recommendation use cases into adjacent areas, while laying the groundwork for cutting-edge Agentic AI directions such as SuperCache and Agent sandboxes.

Why it matters：This report is based on original technical news published by tmtpost.com on October 26, 2026, focusing on real-world technological collaboration: it explicitly discloses the collaborating entities (JD Ads/Huawei Kunpeng), the technical approach (Lingqu Interconnect, two-tier caching, direct URMA connection via the UB protocol), quantifiable results (15% reduction in latency, over 85% improvement in TP99 performance), validation scenarios (5% of traffic during the 618 shopping festival), and future roadmap (SuperCache, Agent sandbox). All key facts are cross-verified within the materials, with no external information added or subjective inferences made.

b11898

ggml-cpu : fix unused variables in arm arch (#31180) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-b11898-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11898/llama-

b11897

tests : refactor test-llama-archs to not link internal libllama symbols (#30880) test-llama-archs included src/llama-arch.h and src/llama-model-saver.h, which prevented building on Windows with shared libraries since the used symbols are not exported from the DLL. - declare llama_arch_* wrappers around the internal llm_ arch query functions in the llama-ext.h staging header, together with llama_arch_kv_name() and llama_arch_supports_save() - slim llama_model_saver down to the private implementation of llama_model_save_to_file(): the fixture-writer half used by the test moves into tests/test-llama-archs.cpp as a local kv_writer - replace the roundtrip saver usage with the public llama_model_save_to_file() writing to a temp path (also removes the tmpfile() fd leak and its Windows workaround)

b11896

ci : fix race in webgpu memset (#31162) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11896/llama-b11896-bin-ub

b11895

metal : stop overflow from happening in large matrix output offsets (#30072) * Metal : stop overflow from happening in large matrix output offsets Assisted-by: Codex * metal : widen batch offsets and MUL_MAT_ID token indices Assisted-by: Codex --------- Co-authored-by: feni6 <270037094+feni6@users.noreply.github.com> **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11895/llama-b11895-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11895/llama-b11895-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/dow

b11890

deepseekv4.1: follow-up review fixes (#31137) * kv-cache : use ggml_backend_tensor_copy to refill the DSV4 K mirrors * llama : move the Engram pad id to llama_hparams and the token map to llama_model The saver no longer needs the deepseek41 model class. * deepseek41 : drop the F32 src1 request on the HC mixes matmul * ggml : add softplus TODOs in the backends that still use log(1 + exp(x)) * convert : make the DSV4/DSV4.1 modify_tensors generators, drop the redundant vision filter * deepseek41 : clean up comments * deepseek41 : drop the F32 src1 requests on the router and indexer matmuls * convert : use format_tensor_name directly in DeepseekV4Model * s/comp_kv_src/comp_il_src/g * cont : fix TODOs for log1p --------- Co-authored-by: Georgi Gerganov **Website:** - **Attestations:** - **macO

Tsinghua University and The Chinese University of Hong Kong jointly open-source AKV: an engineering service system for KV caching tailored to LLM agents.

On October 26, 2026, researchers from Tsinghua University and the Chinese University of Hong Kong open-sourced AKV—a KV cache engineering server built on SGLang and specifically designed for large language model (LLM) agents. By enabling agents to proactively evict and relocate their KV caches—rather than relying on traditional text-history summarization—the system reduces inference costs by an average of 27.7% across four agent benchmarks and improves task scores by 1.7 points on average. The code is licensed under the Apache-2.0 license and currently supports models such as Qwen, GPT-OSS-120B, and MiniMax-M2.7.

Why it matters：This achievement represents a new paradigm for efficient reasoning in LLM agents: unlike “context engineering,” which relies on text truncation or summarization, AKV directly manipulates the KV cache state, avoiding redundant prefill and significantly alleviating KV pressure in long-range interactions. Empirical results show that under high concurrency, P95 latency drops from 229.9 seconds to 30.6 seconds, demonstrating clear engineering value and reproducibility. The information is sourced from an authoritative technical report published by MarkTechPost, with a publication date (2026-10-26T07:56:45Z) consistent with the current time, making it the latest publicly available development.

b11889

cuda : use std::string instead of VLA for archName (#30463) * cuda : use std::string instead of VLA for archName Modernize the code (old C style) and fix the following warnings: ``` ggml/src/ggml-cuda/ggml-cuda.cu:180:19: warning: variable length arrays in C++ are a Clang extension [-Wvla-cxx-extension] 180 | char archName[archLen + 1]; | ^~~~~~~~~~~ ggml/src/ggml-cuda/ggml-cuda.cu:180:19: note: read of non-const variable 'archLen' is not allowed in a constant expression ggml/src/ggml-cuda/ggml-cuda.cu:179:9: note: declared here 179 | int archLen = strlen(devName); | ^ ``` Signed-off-by: Adrien Gallouët * Add a comment about archName Signed-off-by: Adrien Gallouët --------- Signed-off-by: Adrien Gallouët Signed-off-by: Adrien Gallouët **Website:** - **Attestations:** - **macOS/iOS:** - [ma

b11888

common, grammar: add recursive depth limit, make it a requirement (#31153) * common, grammar: add recursive depth limit, make it a requirement * catch more recursive functions * common : tidy up peg parser changes * grammar : reject candidates in a loop, without a depth limit The depth limit in llama_grammar_reject_candidates_for_stack counted the code points of a vocab token, so any token of 256 chars or more was always rejected under a grammar, such as the 512-space token of GLM-4.7-Flash or the long space tokens of Laguna-S-2.1. The rejection now walks the code points with its own stack of levels instead of recursing: there is no limit and no stack risk, whatever the length of a token from the GGUF, and the masks match master. * grammar : track rejected candidates as indices into the in

b11887

tests : adjust default threads for test-llama-context (#31157) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11887/llama-b11887-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11

b11886

qwen: merge tensors on convert (#31028) * qwen: merge tensors on convert * convert: use generator functions * qwen4exp: remove conts and pre-scale **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11886/llama-b11886-bin-ubuntu-x64.tar.gz

b11884

llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31035) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11884/llama-b11884-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/d

b11883

llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection (#30820) * llama-quant : fix quantizing 4D tensors nrows_total was ne[1]*ne[2], so for a 4D tensor only the first ne[2] slices were quantized and the rest of the rows were dropped. Use ggml_nrows(tensor) to count all rows, including ne[3]. Fixes #29916 * llama-quant : detect expert tensors by name and fix imatrix size for 4D tensors The MXFP4_MOE path used ne[2] > 1 to detect MoE expert tensors. This is wrong for non-expert tensors with 3 or more dims, e.g. MLA projections (BailingMoE3) and 4D conv kernels. Use the tensor name instead: all tensors used by GGML_OP_MUL_MAT_ID are named *_exps or *_chexps. This also removes the BailingMoE3 special case. The imatrix holds one vector of ne[0] values per

b11882

grammar: limit recursion depth to prevent stack overflow (#29495) (#29755) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11882/llama-b11882-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/

b11881

llama : respect no_alloc in all memory caches (#31087) * llama : respect no_alloc in memory caches Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : TODO: move no_alloc out of llama_hparams Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : TODO: move no_alloc out of llama_hparams Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11881/llama-b11881-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS

b11880

tools : reject truncated GGUF tensor data in merge (#31044) Co-authored-by: Shreya Agrawal **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11880/llama-b11880-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/lla

b11879

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948) * cuda: fuse FFN gate/up and GLU in MMQ with interleaved gate/up rows For prompt processing, run the gate and up matmuls of a dense FFN as one mul_mat_q launch and apply the GLU in the write-back. - The loaders take the gate weights as a second pointer and fill the weight tile with up and gate rows interleaved in chunks of 8, so each thread holds an up value and the matching gate value (rows r and r + 8 of an MMA tile). - The MMA loop is unchanged: same tile, warp layout, vec_dot and one sum[] per thread. The fused write-back computes the GLU from registers and writes half the rows. - The fused path uses the regular tiling (no stream-k) with the column tiles innermost so that neighbouring blocks share weig

b11878

ggml-cuda: fuse qwen MRoPE + set-rows (#31025) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b11878/llama-b11878

b11877

vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920) * vulkan: Add a larger tile size for conv3d (coopmat2 only) This adds a larger tile size, in practice it is only for coopmat2 where we don't need shared memory to store the output (would use more than 64KB). Add some additional representative backend tests. At least, this is what codex came up with from looking at sd.cpp Wan VAE. * only apply large tile size for newer driver versions **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11877/llama-b11877-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama