Commit Graph
12197 Commits
Author SHA1 Message Date
Aparna M PandMax Krasnyansky 76a67730ff hexagon: support for automatic loading of libhtp without legacy env vars (#31781)
* hexagon: use absolute skel path

* hexagon: support non-ascii dir and long path in skel-abs path

* hex-libdir: add support for FASTRPC_MANAGE_DSP_LIBRARY_PATH and GGML_HEXAGON_LIBHTP_DIR on all platforms

* hex-libdir: keep extra fwd slash in abs path on Android and Linux

* hex-libpath: always start with absolute path then fallback to session ctl then env

* hex-libdir: add CDSP1,2,3 to the fallback, needed for the IQ10

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-11-07 16:25:20 -08:00
Jesse Symons 12e9084dc1 sycl: exact rounding in CPY to Q4_1, Q5_1 and Q8_0 (#31465) 2026-11-07 18:47:26 -05:00
KyleHagy 818ac6b078 sycl: refuse device buffers that would exceed VRAM (#31285) 2026-11-07 18:13:04 -05:00
Max Krasnyansky e9a2a44fe9 hexagon: n_hvx and n_threads cleanup (#31857)
* hex-nhvx: apply nhvx updates to the latest master

* hex-nhvx: set N_HVX and N_THREADS based on the arch to avoid overallocation

* hex-build: add support for enabling HTP_DEBUG via build.py

* hex-trace: move tracing state/helpers into new htp-trace.h header

* hex-dcache: cleanup dcache #defines

* hex-hnvx: minimize the diff by reflowing the macros

* hex-ci: fix python type/linter warnings

* hex-nhvx: consitent logging
2026-11-07 15:03:31 -08:00
Xuan-Son Nguyen a2654ec431 mtmd: move log-mel computation to cgraph (#31808)
* mtmd: move log-mel computation to cgraph

* rm stray file

* also include source in mtmd_audio_track

* rm stray file

* rm mtmd_audio_stft_op

* add test
2026-11-07 23:53:11 +01:00
Xuan-Son NguyenandPascal 9f0e5098a5 mtmd: support MageVL-Cowork model (#31819)
* mtmd: support MageVL-Cowork model

* mtmd: keep magevl rope_freqs at model level and check its shape

The rope factors are shared by every layer, so they live in clip_model
instead of a pointer copied into each clip_layer. The loader now checks
that v.rope_freqs is F32 with d_head elements and that d_head splits
into the 4:6:6 pairs the graph expects.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-07 19:36:44 +01:00
e95426f169 llama_moe_cache: run missed experts on CPU, add tuning script (#30966)
* moe-cache : compute part of the misses on the CPU

--moe-cache-cpu-share F computes a share F of the MoE cache misses on the CPU
from the host experts instead of uploading them. The graph builds the expert
FFN twice: on the device from the cache slots, where the misses left to the CPU
read zero slots, and on the CPU from the host experts, where the experts
computed on the device have id -1. The two results are added before the down
bias.

- ggml-cpu: MUL_MAT_ID skips negative ids and writes zero rows
- sched: the copy callback is called for all the inputs of a split, and before
  the host sync when there are no events
- the slot lookup callback reads the inputs of the CPU part together with the
  selected experts, so the CPU part does not wait for the device, and its
  result is copied back asynchronously
- each token uses distinct zero slots, mul_mat_id does not support an expert
  selected twice by a token
- LLAMA_MOE_CACHE_TRACE writes the selected experts of each lookup, for a
  simulator of the cache

* scripts : add moe-cache-tune.py to pick the MoE cache arguments

The script wraps a llama-server command: it starts the server at two cache
sizes and three CPU shares, fits the upload and CPU speeds of the machine, and
replays the routing of the runs through a copy of the cache logic to predict
other sizes and shares. It prints the recommended --moe-cache-mib and
--moe-cache-cpu-share.

docs/moe-cache.md describes the MoE cache arguments and the script.

* add am17an as codeowner for MoE cache

* moe-cache : upload all the misses until the cache is full

With a high --moe-cache-cpu-share, few misses are uploaded, so an empty cache
fills slowly and more misses than needed go to the CPU. The CPU share now
starts once all the slots of a cache group hold an expert. moe-cache-tune.py
replays the same rule.

* add backend sync for async transfer queue

* moe-cache : let the scheduler copy the results of the CPU part

* Fix MSVC compilation

* remove sync change

* llama-graph : extract the CPU share condition into can_cpu_share

* llama-graph : make llm_graph_params::moe_cache const

The graph construction no longer updates the MoE cache directly: the CPU
parts of the layers are accumulated in the graph result and applied to
the cache in llama_context::process_ubatch, after the graph inputs are
set and before the graph is computed.

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (Mac Studio, M5 Ultra, 256GB)

* scripts : replay the ClassLRU policy in moe-cache-tune.py

* moe-cache : pick CPU misses by CRF instead of random (#31853)

* moe-cache : remove the backend logic of the CPU part from the graph

The CPU part reads the host experts, so the scheduler places it on the
CPU without set_tensor_backend. The cache gives no CPU map for batches
that a device would offload. Graphs with a CPU part look up their own
slot map, so the graph does not register the part with the cache.

ggml-backend : the inputs of the next split that are already computed
are copied to host memory before the current split runs, so a CPU split
does not wait for the device split before it. This replaces the stash
of the cache, and the copy callback is again called for host weights
only.

* llama-graph : return the MoE cache slots and CPU ids in a struct

The scheduler prefetch no longer checks that the next split runs on
another backend: the inputs of a split are never on its own backend,
so the input loop already skips that case.

---------

Co-authored-by: Gaurav Garg <gaugarg@nvidia.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Alexey Kopytko <alexey@kopytko.com>
2026-11-08 00:20:58 +08:00
Michael Taylor f7591a6a01 sycl: add f16 support for SWIGLU_OAI (#30857) 2026-11-07 09:02:58 -05:00
Georgi Gerganov 80d2c0c2c5 metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs (#31852)
The kernel_group_norm_f32 kernel and its host dispatch ignored the
batch dimension (ne03) of the source tensor: only ngrp threadgroups
were dispatched and each normalized the flat range
[tgpig*gs, (tgpig+1)*gs) with gs = ne00*ne01*ceil(ne02/ngrp). For
tensors with ne03 > 1, all but the first ne00*ne01*ne02 elements were
left unwritten (exposed by the new GROUP_NORM test with
ne=[3,1,1,65537], num_groups=1).

Dispatch one threadgroup per (group, i03) pair (ngrp in x, ne03 in z,
same work distribution as the Vulkan backend) and compute per-batch
statistics, matching the CPU reference. Also clamp the group element
count for the last group when ne02 is not divisible by ngrp and use
int64_t indexing.

The kernel now only requires ggml_is_contiguous_rows(op->src[0])
instead of full contiguity: it addresses src0 and dst with their own
strides (nb03 and the dst strides are added to the kernel args - dst
can be a strided view of src0 with ggml_group_norm_inplace) and falls
back from the flat fast path to a strided path that decodes the linear
element index when rows or channels are padded. The flat/strided
selection is done on the host in
ggml_metal_library_get_pipeline_group_norm and compiled into one of
two specialized pipeline variants via a FC_group_norm_flat function
constant, so the dead path is eliminated at pipeline creation.
GROUP_NORM support is also gated on F32 src and dst in supports_op -
the kernel only implements f32 and an f16 input would silently be read
as f32.

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-07 13:04:13 +02:00
Harkanwal BediandPascal 0956684529 server: visit parent slots before child slots (#30779)
* server: visit parent slots before child slots
A parent slot wakes its waiting child slots once it finishes the shared
prompt prefix, while slots are visited in id order. When a child task
lands on a lower slot id than its parent, the child is visited before
the parent wakes it and waits an extra batch to process its suffix.
Non-child slots are now visited before active child slots, so every
child woken by its parent joins the same batch.

* server: check systemone test events through test tags

The systemone tests matched free-form debug messages, including the
formatted f_keep value of the prompt cache. The server now emits
__TEST_TAG_WAIT_PARENT_SLOT__ when a child slot waits for its parent,
__TEST_TAG_PROMPT_CACHE_RESTORE__ on every prompt cache restore and
__TEST_TAG_PROMPT_CACHE_FULL_RESTORE__ when the whole cached prompt is
a prefix of the new one, compared on token counts.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-07 13:03:38 +02:00
Nikhil Jain cf659f240b webgpu : Implement group norm kernel (#31490)
* Only run webgpu CI on my fork

* Add webgpu only workflow

* Add WebGPU group_norm kernel

* clean up

* resolve merge conflicts

* parameterize data type; clean up cmake changes

* add nullptr check

* clean up webkit experiments

* more cleanup

* more cleanup

* clang format

* Remove unused feature check for WebGPU

* clean up supports_op condition and update documentation

* add perf tests for group_norm

* clean up doc with script and undo change in test-backend-ops

* Do 2d dispatch of workgroups

* Do 2d dispatch of workgroups

* Unstage unrelated changes

* undo docs churn

* add new test case

* undo small change

* undo small change
2026-11-07 10:01:27 +02:00
Jeff Bolz 1938152648 vulkan: Add 'readonly' to decodeBuf declarations. (#31833)
In a future driver version I hope to be able to get some perf gain out of this.
2026-11-07 10:00:40 +02:00
Ali F. Al-Ramadan 5d7d4aec16 CUDA: fix shared memory bank conflicts in the lightning indexer (#31803) 2026-11-07 09:59:26 +02:00
Adrien Gallouët c0ab6e5c0b llama : add missing link to pthread (#31785)
* llama : add missing link to pthread

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Link Threads::Threads privately

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-07 08:43:50 +02:00
shaofeiqiandLi He 31424b7cf1 opencl: add arange (#31829)
Assisted-by: Claude Opus 5.5

Co-authored-by: Li He <lih@qti.qualcomm.com>
2026-11-06 21:44:05 -08:00
Yonghye Kwon fd27b20349 meta : raise the compute view limit for buffers with few tensors (#31447)
* ggml-backend-meta : minimum number of compute views per buffer

Assisted-by: Claude Opus 5.5

* tests : revert the deepseek41 test model to 6 layers

Assisted-by: Claude Opus 5.5
2026-11-07 11:05:24 +08:00
Ehsan BateniandMax Krasnyansky ca89968a23 hexagon: fuse post-norm residual add into RMS_NORM_MUL (#31822)
* hexagon: fuse post-norm residual add into RMS_NORM_MUL

* hex-fusion: refactor RMS_NORM fusion and add dedicated RMS_NORM_MUL_ADD opcode

Improves performance by keeping things in HVX registers.
Also simplifies overall fusion logic and logging.

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-11-06 16:06:59 -08:00
usrlocalben bcb0152876 model: add Kimi-K3.1-Pro text model (#31779)
* model: add Kimi-K3.1-Pro text model

K3 with the full-attention MLA layers replaced by DSA: on this checkpoint 24
of the 93 layers carry a lightning indexer (32 heads x 128, top_k 2048, no
rope). The KDA (linear-attention) layers and the rest of the K3 skeleton -
cross-layer residuals, latent MoE, situ, MLA output gate, full-rank KDA gate -
are unchanged, so this reuses the K3 graph and converter and adds the indexer.

Indexer:
  - the selection is an additive mask over the whole cache, not a gather of
    the top-k rows, so it reuses the existing sparse flash attention path;
    off that path the mask stays correct but still costs a kq over the cache
  - the indexer key LayerNorm takes its eps from the model's rms_norm_eps,
    because dsa_config declares no eps of its own

Graph and cache:
  - new arch with four indexer tensors per DSA layer, and a hybrid memory
    that keeps a K-only MLA cache for those layers, recurrent state for the
    KDA layers, and the indexer key cache
  - the graph node budget shares the K3 entry

Conversion:
  - KimiK31Model(KimiK3Model), picked by KimiK31ForConditionalGeneration
  - dsa_config becomes attention.indexer.{head_count,key_length,top_k,types}
    plus the indexer LayerNorm eps; the layer list is 1-indexed like
    linear_attn_config's, and the converter asserts that it covers exactly
    the full-attention layers
  - the routed experts stay MXFP4 and are repacked as in K3, not dequantized
  - the vision tower and mm_projector are skipped: text only for now
  - the arch key follows the checkpoint's own model_type ("kimi_k31") and the
    loader registers KimiK31ForConditionalGeneration; the release name carries
    a -Pro suffix that nothing in the config or the tensor names uses

Quantisation: the indexer's q_b, k and proj are kept out of the quant
overrides, since the top-k selection is sensitive to their precision.

Verified on an 8-layer slice against the HF checkpoint: 38 tensors checked
exactly, the indexer metadata against the config, and the selection against
full attention - identical PPL at top_k beyond the context length, divergent
once top_k is forced below it. Arch test fixture and the full arch suite pass.

* converter: avoid clutter in the k3 base

* narrow comment to match model

* update metal fusion expectations

* restore

* sync with PR 31733 refactoring
2026-11-06 23:42:46 +01:00
669204ee2b hexagon: conv_2d optimizations and support for conv_3d (#31814)
* hexagon: Generalization of CONV_2D implementation for CONV_3D

- New op HTP_OP_CONV_3D is wired through htp-ops.h, main.c (shares op_conv_2d) and the op remap in htp-kparams.cpp.
- conv_3d runs on the existing HMX tiled (implicit GEMM) conv2d path. KD and KH are folded into one virtual kernel row (KD*KH), and output rows are counted as N*OD*OH.
- htp_kparams_solve_conv3d checks the conv_3d layout and calls the tiled solver htp_kparams_conv2d_tiled, which takes an is3d flag.
- The tiled kernel takes the sizes and strides that differ between conv_2d and conv_3d from a struct htp_conv_geom. Only the input stage unfolds depth.
- 256-entry DMA queue can no longer overflow

Assisted-by: Claude Code

* hexagon: modify checking to eliminate divide-by-zero hazards

* hexagon: conv2d tiled: compute the conv_3d unfold divisors on the host

The KH and OH fastdiv values used to unfold conv_3d kh into (kd, kh) and
oh into (od, oh) were built per op on the device, each with a 64-bit
divide. Pass them in the kernel params instead; they fit in the space the
window variant already reserves, so the struct size does not change.

Assisted-by: Claude Code

* hexagon : preserve fold scheduling in conv2d for small size

hexagon : support arbitrary positive conv2d strides
Support positive horizontal strides for WHCN and CWHN inputs. Preserve
load/shuffle paths for strides 1/2 and use gather for larger strides.

hexagon : improve conv2d tile size selection

Prefer the largest feasible output-channel block and derive spatial
chunk capacity from VTCM usage. Compare capacity and thread-aligned
candidates using per-thread work and chunk overhead.

Co-authored-by: Yiwei Shao <yiwei@aizip.ai>

* hex-conv: fix compiler warning

---------

Co-authored-by: Yiwei Shao <yiwei@aizip.ai>
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-11-06 11:51:57 -08:00
Xuan-Son Nguyen 954de248ac docker: add explicit /usr/local/nvidia search path (#31806) 2026-11-06 18:53:04 +01:00
Sofian Elmotiem 6c01da6278 cli : fix crash on invalid UTF-8 in prompt or files (#31563)
Assisted-by: Claude
2026-11-06 13:54:48 +01:00
Georgi Gerganov 8820b088de tests : qualify isnan and signbit with std:: (#31791)
* tests : qualify isnan and signbit with std::

The unqualified calls rely on the C math.h names also being present in the
global namespace, which is not the case for the NetBSD 11.99.9/x86_64 gcc
14.4.0 build reported in the issue.  Prefix the calls with std:: to select
the standard C++ overloads declared by <cmath>.

ref: https://github.com/ggml-org/ggml/issues/1681

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

* tests : qualify C math calls with std::

Follow-up to the isnan / signbit fix: make the remaining math calls in
test-quantize-fns.cpp consistent with the std:: isfinite / max / numeric_limits
already used in the same file.

The float variants are overloads in C++, not distinct names, so the mapping is
fabsf -> std::fabs, copysignf -> std::copysign, nextafterf -> std::nextafter -
not std::fabsf & friends, which do not exist in libstdc++ and would break the
gcc build.  NAN and INFINITY are macros and are left unqualified.

Only behavioural difference: array_rmse() fed a double accumulator to sqrtf(),
now std::sqrt() keeps the computation in double.  The result is more accurate
and differs from the previous value by at most 1.4e-7 relative, three orders of
magnitude below the tightest tolerance in this test.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
2026-11-06 14:53:04 +02:00
Georgi Gerganov 9a1d03c7ca tests : override hardware_concurrency in CI (#31797)
The self-hosted CPU runners are containers with 8 CPUs, but
std::thread::hardware_concurrency reports the cores of the host. The
table mode of test-llama-context sizes its pool from that count, so it
runs 4 models x 8 threads on an 8-CPU job and hangs.

Add TEST_ARG_HARDWARE_CONCURRENCY and TEST_ARG_PARALLEL_WORKERS as the
defaults for the auto-selected counts of test-llama-context and of
test-backend-ops, and set them to 8 and 4 for cpu-x64-high-perf.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)
2026-11-06 14:52:26 +02:00
Adrien Gallouët 18ab62a8ec cmake : generate RC file directly without win-app.rc.in (#31718)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-06 11:02:47 +01:00
lhez 52c9bad946 opencl: support repeat for f16, i32, i16, bf16 (#31782)
* opencl: add f16 repeat

Assisted-by: Codex

* opencl: support repeat for i32, i16, bf16

Assisted-by: Codex

* opencl: update ops

Assisted-by: Codex
2026-11-06 11:36:27 +02:00
Masashi Yoshimura 5d3ed2ae7e webgpu: add batched tensor API (get_tensor_n and set_tensor_n) (#31473)
* add set_tensor_n and get_tensor_n

* fix for editor-config checker

* remove the outdated comment
2026-11-06 11:35:47 +02:00
Ruben Ortlam 80088fa60b vulkan: detect ReBAR support, avoid BAR allocations if unavailable (#31658) 2026-11-06 11:35:03 +02:00
Alexey Kopytko c4fb579695 moe-cache : LRFU/MQ-like hybrid cache, ClassLRU (#31233)
* moe-cache : LRFU/MQ-inspired hybrid cache

* make the clock advance by N tokens: use the real token count (ne[1] is always 1, ids have n_expert_used entries per token)

* sel is a RESHAPE node, the orig input is in src[0], so sel->src[0]->ne[1] is the count

* c for the class index, last_use for last use time

* explain the roles of these

* compute the key once

* use the key directly instead of re-reading

* show how the score is computed from the age

* remove the branch and clarify exp2 and log2 relationship

* reword the header comment and elaborate on unused slots and sorting

* not a copy, not just skip

* std::array maybe

* Revert "std::array maybe"

This reverts commit 7649b1f172db4941a854ad115bec0c7e062b65b9.

* half life it is

* add key and iteration helpers

* extract decayed_crf

* bump classes to 8, document the formula, display the cache size in token worth count

* assert the computed number of classes matches the half life
2026-11-06 11:32:54 +02:00
Samriddha Sinha 3ed7284f2d ci : bump CUDA 13 release builds to 13.5 (#31788)
* ci : bump CUDA 13 release builds to 13.5

* docker : bump cuda13 images to 13.5.0
2026-11-06 09:52:47 +01:00
Alessandro de Oliveira Faria (A.K.A.CABELO) d9cf1870be vendor: update BoringSSL to 0.20261105.0 (#31757)
[no ci]
2026-11-06 09:52:16 +01:00
Johannes Gäßler 240d68fcfe CUDA: fix Maxwell build (#31787) 2026-11-06 09:06:27 +01:00
Adrien Gallouët bbc2e4b657 common : restrict HF token to huggingface.co hosts (#31730)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-06 08:43:33 +01:00
Shawn GuandLi He 4b77dee052 opencl: add q6_k multi column tiled GEMV for speculative verification and refactoring (#31758)
* opencl: use the multi-column q6_K vocab head GEMV for 2..4 tokens on X2E

Co-authored-by: Shangqing Gu <shawngu@qti.qualcomm.com>
Assisted-by: Claude Opus 5.5

* opencl: rename GGML_OPENCL_Q6K_MC3 to GGML_OPENCL_Q6_K_MC3

Co-authored-by: Shangqing Gu <shawngu@qti.qualcomm.com>
Assisted-by: Claude Opus 5.5

* opencl: rename GGML_OPENCL_Q40/Q41/Q4K_MC3 to GGML_OPENCL_Q4_0/Q4_1/Q4_K_MC3

GGML_OPENCL_Q4K_MC3_DQ and GGML_OPENCL_Q4K_MC3_LDS become GGML_OPENCL_Q4_K_MC3_DQ
and GGML_OPENCL_Q4_K_MC3_LDS.

Co-authored-by: Shangqing Gu <shawngu@qti.qualcomm.com>
Assisted-by: Claude Opus 5.5

* opencl: remove unnecessary debug probe

Co-authored-by: Shangqing Gu <shawngu@qti.qualcomm.com>
Assisted-by: Claude Opus 5.5

* opencl: clean up

---------

Co-authored-by: Li He <lih@qti.qualcomm.com>
2026-11-05 21:11:07 -08:00
Max Krasnyansky 67107ac34f hexagon: Implement set_tensor_n and improve repacking/tiling robustness (#31744)
* hex-repack: move repacking/tiling into graph-optimize

* hex-tiling: simplify tiling rules and remove obsolete helpers

* hex-ops: use a hash for de-duplicating tensor descriptors

* hex-bufs: improve tensor hash

* hex-ops: accept empty (zero-elements) ops

* hex-bufs: enable hostbuf by default and reject tensors that do not fit into max_vmem

* hex-bufs: improve error message and update readme

* hex-bufs: make sure to sync flush opbatch in all sync set_tensor paths
2026-11-05 18:15:58 -08:00
Xuan-Son Nguyen e14a779719 mtmd: enable cmake unity build (#31760) 2026-11-06 00:16:13 +01:00
Sigbjørn Skjæret d38bea3422 tests : drop u8path() (#31755)
[no ci]
2026-11-05 23:06:51 +01:00
Eve 186eab1cb8 ci: fix build-cmake-pkg ccache (#31705) 2026-11-05 21:16:56 +00:00
Xuan-Son Nguyen 6023692b1e llama: fix kv shifting, add tests, add per-model build_rope (#31683)
* llama: fix kv shifting and add tests

* llama: add graph build_rope (#31693)

* llama: add graph build_rope

* improve

* migrate new models

* update review skill

* fix ci

* add --kv-shift to CI
2026-11-05 21:10:36 +01:00
Xuan-Son Nguyen b1c902f026 llama, mtmd: (chatterbox) add use_classifier_free_guidance as gguf metadata (#31698)
* llama, mtmd: add use_cfg as gguf metadata

* use_classifier_free_guidance
2026-11-05 20:09:33 +01:00
Sigbjørn Skjæret e8df20d238 models : recursive fix (#31749) 2026-11-05 20:06:55 +01:00
fecdc81eec model : add Ripple (ricursiveai) (#31692)
* feat : added ricursiveai ripple support

* fix : mandatiry SWA, scale through helper, comments back

* fix : metal

* model : make ricursive RoPE/NoPE explicit and validate the layer pattern

Write attention.rope_pattern and read it as optional, falling back to the
sliding window pattern so existing GGUFs keep loading. Refuse a
sliding_window_pattern that does not cover every layer, a model with no
sliding layer, and a gate or value residual present in only some layers.
The converter derives layer_types when the config omits it, ignores a stale
hybrid_layer_pattern and drops tensors of features the config disables.

Assisted-by: Claude Opus 5.5

* model : make ricursive swiglu_clamp per-layer

Store swiglu_clamp as a per-layer array, read it with get_key_or_arr so the
scalar in existing GGUFs still applies to every layer, and refuse a negative
or non-finite value. Add a test-llama-archs fixture with mixed per-layer
clamps and no rope_pattern, so the saver roundtrip checks both.

Assisted-by: Claude Opus 5.5

* model : use the ricursive per-tensor scales and RoPE parameters

Pass the attn_gate and output weight scales to build_lora_mm, take
ext_factor and attn_factor from the graph instead of hardcoding them, and
copy rope_freq_scale_train to the SWA layers instead of pinning it to 1.
Cite the reference lines for the value residual and the NoPE layers.

Assisted-by: Claude Opus 5.5

* convert : drop the ricursive chat_template.jinja override

The precedence between tokenizer_config.json and chat_template.jinja is to
be changed in SpecialVocab for all models instead. Until then a new
conversion stores the short tokenizer_config.json template; GGUFs that
already carry the tool-aware template are not affected.

Assisted-by: Claude Opus 5.5

* fix : remove NVFP4 checkpoint quantization block

* match #31733 refactor

---------

Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-05 19:55:15 +01:00
45b6604b20 model: Add support for the xing4_0 (#31589)
* model: add support for Xing4.0

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* fix: rename model file to xing4-0.cpp to match CI expectations

* model: rename xing4_0.cpp to xing4-0.cpp

* lint: fix E128 continuation indent in conversion/xing.py

* tests: add xing4_0 fusion baseline for Metal

* reduce comments

---------

Co-authored-by: shixq7 <shixq7@chinatelecom.cn>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-05 17:47:27 +01:00
Nick Towle e30ad80fd7 ui: prevent a hidden sidebar from intercepting clicks (#31712)
* ui: prevent a hidden sidebar from intercepting clicks

* ui: also fix desktop sidebar
2026-11-05 15:44:34 +01:00
Xuan-Son Nguyen e0f96a1a12 llama: refactoring: move model to class member of llm_graph_context (#31733) 2026-11-05 15:41:03 +01:00
Adrien Gallouët 468fde90e8 server : simplify windows MCP command resolution (#31728)
* server : simplify windows MCP command resolution

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* common : remove utf8_to_wstring()

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-05 15:34:54 +01:00
Adrien Gallouët 1416f76aa1 vendor : update cpp-httplib to 0.65.0 (#31731)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-05 15:34:18 +01:00
Zheyuan Chen 10d4846a57 ggml-cuda: sanitize post-bias NaNs in MoE selection (#31005)
Fixes #28370
2026-11-05 13:50:16 +01:00
Jeff Bolz 61d4472344 vulkan: Implement test-fusion interface (#30769) 2026-11-05 13:31:44 +01:00
ManuelPDMandDukeNukem-dev 016c1d914c CUDA: use highest compiled arch in get_mmvq_mmid_max_batch (#31679)
Co-authored-by: DukeNukem-dev <54203477+DukeNukem-dev@users.noreply.github.com>
2026-11-05 20:18:38 +08:00
Alex-JP-93 da3416d807 vulkan: RDNA3/4 Large-Tile Split-K Optimization (#30280)
* RDNA3/4 Vulkan Large-Tile Split-K Optimization

* Add single-operator performance test cases

* Update the policy to use driver-reported LDS usage

* Add comments

* Use the simpler split_k rule from review

Assisted-by: GitHub Copilot

* tests : drop the split_k test cases

* vulkan : simplify the RDNA3/4 split_k condition
2026-11-05 13:11:07 +01:00