AI NEWS.ccAI is happening. The future is updating.
English
中文English日本語한국어
EXPLORE · DISCOVER · CONNECT

AI News

llama.cpp · b12032

sycl: Try PkgConfig for LevelZero includes / libs (#31076) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12032/llama-b12032-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12032/llama-b12032-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12032/llama-b12032-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12032/llama-b12032-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12032/

llama.cpp · b12031

opencl: add initial iq3 support (#31338) * opencl: add iq3_s AOS mv Assisted-by: Codex * opencl: add iq3_s SOA mv and tiled mm Assisted-by: Codex * opencl: add iq3_s noshuffle gemm/gemv for adreno Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: add iq3_s dp4a for adreno Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: optimize iq3_s adreno decode and narrow batches Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: add iq3_xxs AOS mv Assisted-by: Codex * opencl: add iq3_xxs SOA mv and tiled mm Assisted-by: Codex * opencl: add iq3_xxs noshuffle gemm/gemv for adreno Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: add iq3_xxs dp4a for adreno Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: optimize iq3_xxs adreno decode and narrow batches Co-authored-by: Sha

llama.cpp · b12029

mtmd : fix InternVL CLS token order (#30886) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12029/llama-b12029-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12029/llama-b12029-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12029/llama-b12029-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12029/llama-b12029-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12029/llama-b12029-b

llama.cpp · b12028

tests : enable Gemma 4 in `test-llama-archs` (#31401) * tests : enable Gemma 4 in `test-llama-archs` * tests : use GQA for Gemma 4 in `test-llama-archs` **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12028/llama-b12028-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12028/llama-b12028-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12028/llama-b12028-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12028/llama-b12028-bin-ubuntu-x64.

llama.cpp · b12027

llama : enable tensor split for LFM3 (#31431) Assisted-by: Claude Opus 5.5 **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12027/llama-b12027-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12027/llama-b12027-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12027/llama-b12027-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12027/llama-b12027-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/

llama.cpp · b12024

reshaping so mm on 2d instead of 3d tensor for lfm sc (#30831) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12024/llama-b12024-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12024/llama-b12024-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12024/llama-b12024-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12024/llama-b12024-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12

llama.cpp · b12022

server : fix /v1/responses endpoint to accept extra_params (#30959) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12022/llama-b12022-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12022/llama-b12022-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12022/llama-b12022-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12022/llama-b12022-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/downloa

llama.cpp · b12021

mtmd : resize Gemma 4 images without letterbox padding (#30922) * mtmd : resize Gemma 4 images without letterbox padding * Apply suggestion from @ngxson --------- Co-authored-by: Xuan-Son Nguyen **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12021/llama-b12021-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12021/llama-b12021-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12021/llama-b12021-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/do

llama.cpp · b12020

opencl: add fwht (#31434) Co-authored-by: Li He **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12020/llama-b12020-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12020/llama-b12020-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12020/llama-b12020-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12020/llama-b12020-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12020/llama-b1202

llama.cpp · b12019

chat : strip the think block in the qwen3-coder with reasoning_format none (#31380) * chat : accept the think block in the qwen3-coder response format with reasoning_format none Assisted-by: Claude Opus 5.5 * chat : define both qwen3-coder think block variants up front **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12019/llama-b12019-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12019/llama-b12019-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12019/llama-b12019-xcframework.zip) **Linu

llama.cpp · b12018

sycl: fix f32 to q2_0/q4_0/q5_0 rounding in CPY (#31120) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12018/llama-b12018-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12018/llama-b12018-bin-macos-x64.tar.gz) - [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b12018/llama-b12018-xcframework.zip) **Linux:** - [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12018/llama-b12018-bin-ubuntu-x64.tar.gz) - [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b12018/ll

b12007

kv-cache : keep rotation constants on the GPU (#31073) * kv-cache : keep rotation constants on the GPU Assisted-by: Codex * tests : verify resident KV rotation buffers Assisted-by: Codex * kv-cache : document remaining multi-device rotation work Assisted-by: Codex * kv-cache : support per-layer rotation buffers Assisted-by: Codex * kv-cache : fix hybrid SWA rotation inputs and mixed placement check Assisted-by: Codex * models : use per-layer rotation in mistral-large4 and dflash * tests: drop the rotation buffer tests --------- Co-authored-by: Pascal **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12007/llama-b12007-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](ht

b12005

opencl: add initial q2_0 support (#31340) * opencl: add q2_0 AOS mv Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: add q2_0 SOA mv and tiled mm Co-authored-by: Shaofei Qi Assisted-by: Codex * opencl: add q2_0 noshuffle gemm/gemv for adreno Assisted-by: Codex * opencl: add q2_0 dp4a for adreno * opencl: optimize q2_0 adreno decode and narrow batches --------- Co-authored-by: Li He **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b12005/llama-b12005-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780) - [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b12005/llama-b12005-bin-macos-x64.tar.g

b12002

ggml : add batched tensor copy API (#30881) * ggml : add generic batched tensor copy API Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : use batched tensor copy API in context io Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * ggml : add batched tensor copy API Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * llama : batch device io tensor copies Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * ggml : add CUDA support for batched tensor APIs Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * ggml : remove async tensor_n APIs a

b12000

Quant: OCP FP8 E4M3 support (#27512) * WIP OCP FP8 E4M3 support * Fix python type-checker * Materialize input_scales also for FP8 when converting * Fix conversion of https://huggingface.co/RedHatAI/Qwen3.8-27B-NVFP4 1. Per row FP8 wrongly fell to NVFP4 conversion, as NVFP4 checked on dims alone. Also check on dtype for FP8 2. Need to reshape FP8 QKV projections scales in the same way that weights are reshaped * Allow remaining FP8 tensors to be dequantized during conversion * Allow tensor-strategy for FP8-only compressed-tensor ckpts * Fix and test gguf-py behavior for FP8_E4M3 * Disallow QKV-fusion for FP8 temporarily * Fix conversion of fused qkvz & FP8 for Qwen3-Next * Avoid fusing FP8 for --fuse-gate-up-exps as we may have different ws * Emit warnings when not fusing FP8 during convers

b11986

opencl: initial support for iq2 (#31267) * opencl: add iq2_s AOS mv * opencl: add iq2_s SOA mv and tiled mm * opencl: add iq2_s noshuffle gemm/gemv for adreno * opencl: add iq2_s dp4a for adreno * opencl: optimize iq2_s adreno decode and narrow batches Co-authored-by: Hongqiang Wang * opencl: add iq2_xs AOS mv * opencl: add iq2_xs SOA mv and tiled mm * opencl: add iq2_xs noshuffle gemm/gemv for adreno * opencl: add iq2_xs dp4a for adreno * opencl: optimize iq2_xs adreno decode and narrow batches Co-authored-by: Hongqiang Wang * opencl: add iq2_xxs AOS mv * opencl: add iq2_xxs SOA mv and tiled mm * opencl: add iq2_xxs noshuffle gemm/gemv for adreno * opencl: add iq2_xxs dp4a for adreno * opencl: optimize iq2_xxs adreno decode and narrow batches Co-authored-by: Hongqiang Wang --------- Co-au

b11985

metal : faster MXFP4 mat-vec, full threadgroups for unary/binary/GLU (#31075) * metal : faster MXFP4 mat-vec kernel Decode the fp4 values with the fp16 bit trick instead of a threadgroup lookup table, and load the packed nibbles as aligned 32-bit words. The kernel no longer uses threadgroup memory. * metal : launch contiguous unary and binary ops with full threadgroups The contiguous unary path and the row-broadcast binary path used one thread per threadgroup. Use up to 256 threads per threadgroup and bound-check the tail. * metal : split GLU rows across threadgroups A GLU row ran on one threadgroup, so single-row decode used a single threadgroup for the whole op. Split each row over threadgroups of up to 256 threads. **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon