Commit Graph
12471 Commits
Author SHA1 Message Date
Ruben Ortlam 8f647f1fdb vulkan: disable flip Flash Attention shader on AMD Windows due to a driver bug (#32415) 2026-11-17 10:55:28 +02:00
Neo Zhang dde9de82cb sycl : Fix FA mkl issue: a mask broadcast over heads or batches (#31536)
* fix the SYCL fa with mkl error of a mask broadcast over heads or batches; add argument cache-ram to start-svr.sh

* add argument cache-ram to win-start-svr.bat

* fix format issue, remove --cache-ram in script: move to another PR
2026-11-17 10:50:32 +02:00
Souleyman e0a9118fe9 metal : add implicit GEMM kernel for CONV_2D (#28069) 2026-11-17 09:46:24 +01:00
Nikhil Jain 140a279874 webgpu : Implement timestep embedding (#31695)
* Only run webgpu CI on my fork

* Add webgpu only workflow

* resolve merge conflicts

* save state

* save kernel

* add kernel back in

* fix merge issues with kernels

* updates

* update tests

* change invocation count and workgroup to be u32

* clang format

* clang format

* update docs

* format

* remove extra file
2026-11-17 10:41:37 +02:00
Takamichi Ohara c145db2728 webgpu : use f32 while dequantizing to avoid FTZ (#32140) 2026-11-17 10:40:23 +02:00
lhez ede6106780 opencl: support f16 and non-contig f32 silu (#32319)
* opencl: support non-contig silu

Assisted-by: Claude Opus 5.5

* opencl: support f16 silu

Assisted-by: Claude Opus 5.5
2026-11-17 10:38:07 +02:00
9214bdb554 sycl: use fast division in bin-bcast kernels (#32213)
Replace the per-element modulo of i0 with fast_div_modulo in the ravel and
unravel bin-bcast kernels, drop the now-unused ne10/ne20 kernel arguments,
and reject divisors that do not fit in 32 bits when the magic values are
computed, as the CUDA implementation does.

Co-authored-by: zhaoyshine <zhaoyshine@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-11-17 15:17:47 +08:00
shaofeiqi d3c3e242c1 opencl: add A7 Q8_0 non-MoE binary kernel (#31003) 2026-11-16 20:15:56 -08:00
Sidhartha Reddy Potu 9581154a22 mtmd : fix Muse Glimmer vision embedding order (#32433)
Assisted-by: Claude
2026-11-17 00:29:02 +01:00
Gabe GoodhartandSigbjørn Skjæret 1ef8c5d684 convert : respect max_position_embeddings for granite hybrid (#32436)
* fix(conversion): Respect max_position_embeddings for granite hybrid

Branch: GraniteHybridContextLengthConversion
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* style: shorter comment w/ HF reference

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-16 23:49:57 +01:00
Pascal e8fbcad855 tests: print the exception message in test-llama-context (#32428)
A case that throws only reported "unhandled exception", which hides
the cause of the intermittent failures on the Vulkan Apple runner.
2026-11-16 21:53:32 +01:00
Xuan-Son Nguyen 95e837aaf1 skills: explicitly specify paths to README-dev.md (#32431) 2026-11-16 21:52:12 +01:00
Bartowski d998c676ff convert: Fix inkling 2 mmproj conversion (#32430) 2026-11-16 20:39:32 +01:00
Xuan-Son NguyenandPascal 92192d31e4 mtmd: support YuE2-3B (#32369)
* mtmd: support YuE2-3B

* improve speed

* add CFG

* fix streaming

* add cfg control from api

* check ctx size

* fix n_predict

* add flow_steps config

* nits

* fix batch api

* nits

* adapt

* mtmd: yue2 samples from its own saved logits

The context logits belong to the last decode of any slot, so with
several server slots the score and the unguided codes were sampled
from another sequence. Keep the logits of each own decode in
save_logits(), which also does the guidance mix.

* mtmd: let the yue2 snake chain fuse

The beta reshape landed between sqr and mul, so the mul, sin, sqr,
mul, add chain was not consecutive and the backends never fused it.
Expand both reshapes first, like qwen3tts does.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-16 20:22:23 +01:00
shaofeiqi ad5cbfa911 opencl: add A7 Q4_K non-MoE binary kernel (#30948) 2026-11-16 10:58:29 -08:00
Hongqiang Wang d261e49e7e opencl: fix three defects of the bin flash-attention prefill path (#32110)
* opencl: release the WMM flash-attention mask fallback image

The fallback image is created on every call of the bin flash-attention
kernel over a sub-buffer of the K cache and was never released, so each
call kept a K-cache sub-buffer alive. Every call with more than one query
row takes this path, including each speculative-decoding verify step, so
the objects pile up and the server slows with every request.

* opencl: keep q8_0 K/V at head size 512 off the WMM flash-attention kernel

With q8_0 K/V at DK=512 the bin flash-attention kernel gives wrong results
whenever V holds different values from K (test-backend-ops FLASH_ATTN_EXT
hsk=512 q8_0, max error 2.04 on X2-90); f16 K/V at DK=512 is correct. The
backend's own flash-attention kernels already decline q8_0 at DK=512, so the
shape is now handled the same way with or without the kernel library.

* opencl: decline WMM flash-attention prefill beyond the device's 3D image limits

The WMM path repacks Q/K/V into 3D images; V is n_kv wide, so past the device's
maximum 3D image width (about 16k tokens of context on X2-90) clCreateImage
failed and the call asserted. Such shapes now take the regular flash-attention
path.
2026-11-16 10:57:48 -08:00
Aldehir Rojas 4ddf2b9470 chat : implement inkling parser (#32381) 2026-11-16 19:48:24 +01:00
Xuan-Son Nguyen 4303cd94ae contributing: mark agents.md and code-review as required (#32416) 2026-11-16 19:26:10 +01:00
Beinsezii c7518dbec5 MMVQ: Tune ne11 Gate for RDNA3_5 (#32308) 2026-11-17 00:56:20 +08:00
Xuan-Son Nguyen a68c6136bb mtmd: support fish audio s2 (#32330)
* add fisk-audio-s2 support

* add voice cloning

* cont merge master

* nits

* nits

* clean up

* nits

* use new batch api
2026-11-16 17:32:27 +01:00
Aleksander Grygier 3a7660371d ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)
* ui: move tools/ui to ui/apps/chat

The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's package @ggml-org/llama-chat

It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: call the built-in web UI Llama Chat in docs, help text and comments

The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.

Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name tools/ui as a former path of Llama Chat in the code-review skill

Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's path in the security policy's exclusions

Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-16 17:23:36 +01:00
Shubham RoyandDeveloper e8698ac163 fix:- Misc. bug: Settings/Tools is lying about tool picks inside a ch… (#32223)
* fix:- Misc. bug: Settings/Tools is lying about tool picks inside a chat #32214

* checks run lint error

---------

Co-authored-by: Developer <dev@example.com>
2026-11-16 17:15:16 +01:00
Rudransh Shekhar 7804be0634 arg : allow up to llama_max_devices() configs for --tensor-split and --fit-target (#13293) (#31435) 2026-11-16 17:07:45 +01:00
Xuan-Son Nguyen ef6bc6f367 jinja: impl scope-chain key lookup (#32396)
* jinja: impl scope-chain key lookup

* add test for call_statement lambda
2026-11-16 15:48:46 +01:00
Niklas Wenzel a06044b934 llama : deprecate old batch API (#32306) 2026-11-16 15:16:13 +01:00
Niklas Wenzel 62a6d3a302 llama : migrate to llama_batch_ext (#32356)
* llama : migrate to llama_batch_ext

* mtmd : migrate zonos3-mini code

Added in https://github.com/ggml-org/llama.cpp/pull/32020
2026-11-16 15:16:12 +01:00
Jeff Bolz ef3848f9d7 vulkan: Support split_k for conv3d. (#29929)
This tries to use the 256x128 shape (coopmat2-only) with split_k when
it appears profitable. It reuses the mul_mat split_k reduce shader.
2026-11-16 15:25:12 +02:00
uvos 47fe133318 HIP/CUDA: enable GDN mma kernel on CDNA (#31327) 2026-11-16 15:24:19 +02:00
Ruben Ortlam 301db63370 vulkan: fix Nvidia fp32 FA shmem overflow, disable shmem staging if it takes too much memory (#32244) 2026-11-16 15:22:57 +02:00
BlivionIaG a001af915e hip: add gfx1013 to rdna1 (#32236) 2026-11-16 15:22:35 +02:00
Pascal 164ab018f9 webgpu: support per-head flash attention masks (#32394)
The flash attention shaders indexed the mask without its head
dimension, so masks with ne[2] > 1 gave wrong results. The mask offset
and the vec block states now include the head, broadcast as
head % mask->ne[2] like the other backends. The params buffer grows to
256 bytes to fit the two new mask fields.

The shared memory estimate dropped kv_shmem when only one of K and V
is read directly, while the shaders keep it unless both are, which made
the K f16 / V q8_0 cases exceed the workgroup storage limit. The vec kv
tile now takes both flags: the memory estimate drops kv_shmem only when
both are direct, and the tile still divides the KV padding when either
is. get_alloc_size and supports_op use the same rule as the pipelines,
and supports_op accounts for the register tile fallback.
2026-11-16 14:20:18 +01:00
Georgi Gerganov 5cb9516798 readme : fix line endings 2026-11-16 14:03:30 +02:00
8bfd64b5ac readme : update (#32382)
* readme : update

* authors : update

* readme : minor

* cont : minor

* cont : typos

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Niklas Wenzel <dev@nikwen.de>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Niklas Wenzel <dev@nikwen.de>
2026-11-16 13:28:31 +02:00
Xuan-Son Nguyen e1925b3695 jinja: support json sort_keys=true (#32366) 2026-11-16 12:28:19 +01:00
Georgi Gerganov 95db855bce ci : use vX.Y.Z tags as docker tags (#32388)
* ci : use vX.Y.Z tags as docker tags

When the Docker workflow is manually dispatched from a vX.Y.Z tag, use that tag verbatim for the image tags instead of appending the build number and commit hash.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* ci : require a tag for manual docker runs

Manual dispatches of the Docker workflow must now use a tag ref, so the image tags are derived from the selected tag. The check can be skipped with the new skip_ref_check input. Scheduled nightly builds are unchanged.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-16 13:27:10 +02:00
miyan 1d2e880bad vulkan : fix 32-bit compilation (#32275) 2026-11-16 11:50:31 +01:00
PascalandRuben Ortlam adf554d239 vulkan: make the radix top-k tie selection deterministic (#31931)
* vulkan: make the radix top-k tie selection deterministic

The radix select gathers the elements above the threshold and the ties
with an atomic counter, so with more ties than remaining slots the kept
columns depend on the thread scheduling.

Each subgroup now owns a contiguous chunk of the row with the same
column mapping in every pass, counts its kept elements, and writes them
at ranks from a scan of the subgroup totals; overflowing ties are ranked
in column order with a ballot, so the lowest tied columns win, as on
CUDA and Metal. The radix pipelines require subgroup arithmetic, ballot
and full subgroups. Enable the deterministic top-k check on Vulkan.

* optimize shader for AMD, Intel and Nvidia

---------

Co-authored-by: Ruben Ortlam <rortlam@redhat.com>
2026-11-16 11:41:44 +01:00
Ruixiang Wang 6b01a4767f CI: trigger spec dec testing for relevant PRs (#32281)
* ci: trigger spec dec testing after refactoring

* add options to run spec dec ci nightly manually
2026-11-16 11:14:07 +01:00
liminfei-amd f477c1b90a cuda : fix MUL_MAT_ID abort with odd src0 columns on AMD (#32385)
On AMD devices MUL_MAT_ID with an F32, F16 or BF16 src0 and at most 8 tokens always used MMVF.
MMVF reads src0 in pairs of values and asserts an even ne[0], so a rank 1 LoRA on MoE expert weights (lora_b has ne[0] == 1) aborted.
Use MMVF there only for an even ne[0]; other shapes take the existing fallback.

Fixes #30576

Assisted-by: GitHub Copilot

Signed-off-by: AMD ROCm Contributor <91481003+liminfei-amd@users.noreply.github.com>
2026-11-16 17:53:06 +08:00
Titaniumtown 21fa9c043f kv-cells: avoid an O(n_cells) scan per sequence removal (#32380) 2026-11-16 10:52:09 +01:00
Pascal a946eb541a server: answer reasoning_effort levels the template rejects with 400 (#32332)
The reasoning effort probe already renders every candidate level, but
it dropped the ones the template raises on, so a request with such a
level (e.g. max on Qwen3.8) reached the template and came back as a
500 on /v1/chat/completions, /v1/responses and /v1/messages.

Keep the raising levels in reasoning_effort_rejected and reject them
in oaicompat_chat_params_parse with an invalid_argument, which the
server answers with a 400 listing the supported levels. Aliases and
levels a template ignores are not rejected, so high on Qwen3.8
still renders as xhigh.

Also trim the output_config.effort test to a single level.
2026-11-16 06:55:56 +01:00
thelittlefireman 05db09a4d3 ggml-cuda: fix MoE regression on HIP with #29948 (#32344) 2026-11-16 06:37:42 +01:00
BeinseziiandSigbjørn Skjæret 9fba51d2c9 convert: fix ripple dflash2 NoPE metadata (#32363)
* convert: fix ripple dflash2 NoPE metadata

was wrong

https://huggingface.co/ricursiveAI/Ripple-65B-Preview-DFlash2/blob/743c4669f7daa11d3ca350b9a057c059481b1afd/config.json#L45-L47

* Update conversion/base.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-16 00:47:27 +01:00
Xuan-Son NguyenandAldehir Rojas b4c0e00a4f jinja: add 4 missing features (#32352)
* jinja: add 4 missing features

* apply suggestion

Co-authored-by: Aldehir Rojas <hello@alde.dev>

* fix error msg

---------

Co-authored-by: Aldehir Rojas <hello@alde.dev>
2026-11-16 00:40:44 +01:00
Evan Huus 1719dbb0ee server : skip MCP discovery for router children (#32359) 2026-11-15 23:55:33 +01:00
Xuan-Son Nguyen fc504f91d8 mtmd: support higgs v3 STT and vibevoice-ASR (#32329)
* mtmd: support higgs v3 and vibevoice-ASR

* use proper mapping

* address review comment

* nits

* rm hacky get_audio_length, add mtmd_helper_get_media_meta
2026-11-15 21:25:34 +01:00
Seungjun Bae 18b74fa41d model : read qwen3vl output head size from the tensor (#32349)
Allows Qwen3-ForcedAligner, whose output head (5000 timestamp classes) is smaller than the vocab.
2026-11-15 19:45:17 +01:00
Xuan-Son Nguyen 7ae3c6b231 mtmd: add zonos3-mini TTS model (#32020)
* zonos3 init

* add audio gen pipeline

* fix string_format

* add default voice as gguf metadata

* better ref

* support voice clone

* add delivery params

* optimize speed

* nits

* nits 2

* add text normalization

* nits
2026-11-15 19:44:44 +01:00
Xuan Son Nguyen f4d0673917 Revert "jinja: add 4 missing features"
This reverts commit a681a53b60.
2026-11-15 19:35:31 +01:00
Xuan Son Nguyen a681a53b60 jinja: add 4 missing features 2026-11-15 19:33:06 +01:00