* opencl: release the WMM flash-attention mask fallback image
The fallback image is created on every call of the bin flash-attention
kernel over a sub-buffer of the K cache and was never released, so each
call kept a K-cache sub-buffer alive. Every call with more than one query
row takes this path, including each speculative-decoding verify step, so
the objects pile up and the server slows with every request.
* opencl: keep q8_0 K/V at head size 512 off the WMM flash-attention kernel
With q8_0 K/V at DK=512 the bin flash-attention kernel gives wrong results
whenever V holds different values from K (test-backend-ops FLASH_ATTN_EXT
hsk=512 q8_0, max error 2.04 on X2-90); f16 K/V at DK=512 is correct. The
backend's own flash-attention kernels already decline q8_0 at DK=512, so the
shape is now handled the same way with or without the kernel library.
* opencl: decline WMM flash-attention prefill beyond the device's 3D image limits
The WMM path repacks Q/K/V into 3D images; V is n_kv wide, so past the device's
maximum 3D image width (about 16k tokens of context on X2-90) clCreateImage
failed and the call asserted. Such shapes now take the regular flash-attention
path.
* ui: move tools/ui to ui/apps/chat
The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name Llama Chat's package @ggml-org/llama-chat
It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: call the built-in web UI Llama Chat in docs, help text and comments
The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.
Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name tools/ui as a former path of Llama Chat in the code-review skill
Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name Llama Chat's path in the security policy's exclusions
Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.
Assisted-by: Claude (Opus 5.5 Medium)
The flash attention shaders indexed the mask without its head
dimension, so masks with ne[2] > 1 gave wrong results. The mask offset
and the vec block states now include the head, broadcast as
head % mask->ne[2] like the other backends. The params buffer grows to
256 bytes to fit the two new mask fields.
The shared memory estimate dropped kv_shmem when only one of K and V
is read directly, while the shaders keep it unless both are, which made
the K f16 / V q8_0 cases exceed the workgroup storage limit. The vec kv
tile now takes both flags: the memory estimate drops kv_shmem only when
both are direct, and the tile still divides the KV padding when either
is. get_alloc_size and supports_op use the same rule as the pipelines,
and supports_op accounts for the register tile fallback.
* ci : use vX.Y.Z tags as docker tags
When the Docker workflow is manually dispatched from a vX.Y.Z tag, use that tag verbatim for the image tags instead of appending the build number and commit hash.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
* ci : require a tag for manual docker runs
Manual dispatches of the Docker workflow must now use a tag ref, so the image tags are derived from the selected tag. The check can be skipped with the new skip_ref_check input. Scheduled nightly builds are unchanged.
Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
* vulkan: make the radix top-k tie selection deterministic
The radix select gathers the elements above the threshold and the ties
with an atomic counter, so with more ties than remaining slots the kept
columns depend on the thread scheduling.
Each subgroup now owns a contiguous chunk of the row with the same
column mapping in every pass, counts its kept elements, and writes them
at ranks from a scan of the subgroup totals; overflowing ties are ranked
in column order with a ballot, so the lowest tied columns win, as on
CUDA and Metal. The radix pipelines require subgroup arithmetic, ballot
and full subgroups. Enable the deterministic top-k check on Vulkan.
* optimize shader for AMD, Intel and Nvidia
---------
Co-authored-by: Ruben Ortlam <rortlam@redhat.com>
On AMD devices MUL_MAT_ID with an F32, F16 or BF16 src0 and at most 8 tokens always used MMVF.
MMVF reads src0 in pairs of values and asserts an even ne[0], so a rank 1 LoRA on MoE expert weights (lora_b has ne[0] == 1) aborted.
Use MMVF there only for an even ne[0]; other shapes take the existing fallback.
Fixes#30576
Assisted-by: GitHub Copilot
Signed-off-by: AMD ROCm Contributor <91481003+liminfei-amd@users.noreply.github.com>
The reasoning effort probe already renders every candidate level, but
it dropped the ones the template raises on, so a request with such a
level (e.g. max on Qwen3.8) reached the template and came back as a
500 on /v1/chat/completions, /v1/responses and /v1/messages.
Keep the raising levels in reasoning_effort_rejected and reject them
in oaicompat_chat_params_parse with an invalid_argument, which the
server answers with a 400 listing the supported levels. Aliases and
levels a template ignores are not rejected, so high on Qwen3.8
still renders as xhigh.
Also trim the output_config.effort test to a single level.
* ggml-cpu: use VNNI for Q2_0 dot products
Keep the upstream AVX2 unpacking, accumulation, and fallback. Use VNNI for the integer dot product on supported x86 CPUs.
Assisted-by: Codex
* ggml-cpu: use VBMI for Q2_0 unpacking
Use multishift to unpack Q2_0 weights on CPUs with AVX-512 VBMI and VL. Retain the existing shift-and-mask unpacking on other CPUs.
Assisted-by: Codex
* Update ggml/src/ggml-cpu/arch/x86/quants.c
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* fixed support for DFlash2 for Ripple 65B
* spec: Add NoPE handling to DFlash
(cherry picked from commit 89ea8e3e77e3359d17902a2db136022d5d419d88)
* spec: Fix NoPE not being picked up by DFlash
(cherry picked from commit 43393b95fda2ce7e7d243e2650bf7b1cadf1c732)
---------
Co-authored-by: James Pickering <james_pic@hotmail.com>
* Fail tests on missing or unsupported type
* Fix output
* Remove whitespace
* Fail if no tests ran regardless of backend support
* Revert unnecessary cosmetic change