Commit Graph
11514 Commits
Author SHA1 Message Date
bri-prism de7fa0a3c6 Musa FWHT fix (#30167) 2026-10-08 21:56:54 +02:00
71ad0590f4 CUDA: improve top-k algorithm selection (#28713)
* CUDA: radix top-k for large row counts

Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select,
gated on GGML_CUDA_TOPK_RADIX_MIN_ROWS. On qwen4exp at 34,816 tokens this cuts
top-k from 1,671,253 launches / 5,761.8 ms to 2,329 / 941.8 ms.

* CUDA: select the TOP_K implementation by shape

Replace the nrows/ncols special case with the decision boundary from #28547
(as implemented in #29278): bitonic for short rows, radix select for several
long rows, and DeviceTopK or CUB argsort for a single long row. The
thresholds stay overridable at build time.

Two refinements on top of that boundary:
- bitonic stays in use for rows up to a padded 1024 while the rows fit in one
  wave of blocks (nrows <= number of SMs); radix select pays a fixed cost of
  about a dozen launches that only amortizes over more rows
- with DeviceTopK available, it handles up to two rows

Radix select now processes rows in chunks so its scratch memory stays bounded,
and the bitonic path keeps its chunking. HIP and MUSA keep their previous
thresholds.

Add perf cases around the bitonic/radix crossover to test-backend-ops.

* CUDA: make top-k comments less verbose

* CUDA: remove the TOP_K width limit from supports_op

* CUDA: use DeviceTopK for single-row TOP_K if available

* CUDA: avoid ncols overflow in the TOP_K bitonic check

* CUDA: share the row chunking helper between argsort and top-k

* CUDA: do the TOP_K radix blocks_per_row math in int64_t

* CUDA: rename GGML_CUDA_TOP_K_NROWS_THRESHOLD_DEVICETOPK to GGML_CUDA_TOP_K_NROWS_THRESHOLD

* CUDA: share one sort helper between the bitonic and CUB TOP_K paths

* CUDA: update the TOP_K TODO, threshold and chunking comments

* tests: add TOP_K cases that span several row chunks

* CUDA: use int64_t col in the TOP_K radix loops, fix threshold comment

* CUDA: limit TOP_K and ARGSORT support to ne[0] <= INT_MAX

---------

Co-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>
Co-authored-by: Pranesh Gonegandla <pgonegandla@nvidia.com>
2026-10-08 18:34:27 +02:00
Hrishith Thadicherla a11f57ba93 model : fix DFlash output head sharing (#30111)
* llama : fix DFlash output head sharing

Assisted-by: Codex

* dflash : read tied output weights from GGUF metadata

Assisted-by: Codex

* llama : share tied word embedding metadata

Assisted-by: Codex

* llama : remove DFlash embedding head fallback

Assisted-by: Codex
2026-10-08 17:42:28 +02:00
Johannes Gäßler fc9ce6b9d5 CUDA: fix MMQ out-of-bounds reads (#29953) 2026-10-08 16:41:49 +02:00
Prabhsimran Singh c35b66744f CUDA : looped PAD kernel for more than 65535 rows or slices (#30147) 2026-10-08 15:36:53 +02:00
hey-gmandOliver Simons 1167d3f42c CUDA: fix CCCL version guard breaking on major version rollover (#29453)
* CUDA: fix CCCL version guard breaking on major version rollover

The guard compared the major and minor components independently:

    CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 1

Minor resets to 0 whenever a new major series is cut, so on CCCL 4.x
this evaluates as 4 >= 3 && 0 >= 1, i.e. false. STRIDED_ITERATOR_AVAILABLE
stops being defined and argsort silently falls back to the
init_offsets path. Nothing warns and the build still succeeds, so the
regression is a quiet performance loss rather than a compile error.

CCCL already exposes the version as a single packed integer in
MMMmmmpp form, which is what its own version header uses:

    CCCL_VERSION = MAJOR * 1000000 + MINOR * 1000 + PATCH

so 3.4.3 is 3004003 and ">= 3.1" is a plain ">= 3001000". One
comparison, with no component arithmetic left to get wrong.

Checked against a hand-written "version >= 3.1" reference over 2.9.9,
3.0.0, 3.1.0, 3.1.99, 3.2.0, 3.4.3, 3.9.9, 3.99.99, 4.0.0, 4.2.7 and
5.0.0: no divergences. The old guard disagreed at 4.0.0 and 5.0.0.

Verified on RTX 4070 (sm_89), CUDA 13.4, CCCL 3.4.3:

  - cmake --build build --config Release: exit 0
  - test-backend-ops test -o ARGSORT -b CUDA0: 98/98 passed, CUDA0 OK

Note that a passing regression test does not on its own prove the guard
is still taken, since the fallback path passes too. Preprocessing the
real translation unit confirms the strided-iterator branch is the one
compiled in: counting_iterator is present, init_offsets is not.

Signed-off-by: Heitor <heitorgm@outlook.com>

* Update ggml/src/ggml-cuda/argsort.cu

* Apply suggestion from @ORippler

---------

Signed-off-by: Heitor <heitorgm@outlook.com>
Co-authored-by: Oliver Simons <osimons@nvidia.com>
2026-10-08 15:29:34 +02:00
Simon Sudarushkin 4f92965a7b ui: apply ui_settings on first visit in router mode (#29668) 2026-10-08 15:08:22 +02:00
Aman Gupta c811cb8f0a llama: support MoE cache over multiple GPUs (#30112) 2026-10-08 15:50:49 +05:30
Leebr Data ConsultingandIgor Okulist 033df86b69 server : preserve context checkpoints across slot save/restore (#26004)
* server : preserve context checkpoints across slot save/restore

Append the checkpoints after the packed server_tokens payload added in #26640
and count them in n_written / n_read, so a restored slot can still roll back to
a checkpoint instead of re-processing the whole prompt.

* server : drop draft checkpoint data that does not match the draft context

Restoring a slot saved with a different draft KV cache type aborted in
load_dft(). Test-load one draft checkpoint on restore and drop the draft
data if it does not fit, instead of crashing. Adds a regression test.

Co-authored-by: Igor Okulist <okigan@gmail.com>

* server : harden the checkpoint appendix of slot save files

Bound each blob size by the bytes left in the file before allocating, open the
file with UTF-8 paths on Windows like the llama state payload, fall back to full
prompt re-processing when a checkpoint restored from a slot file fails to load,
and replace the 1024 count cap by keeping the last n_ctx_checkpoints while reading.

* server : report an incomplete checkpoint appendix as a failed slot save

Return an error to the client when the appendix cannot be written, like a
failed payload write, and make the oversized-blob test declare a size that
cannot be allocated, so an unbounded allocation fails the test.

* server : reject an empty target state in the checkpoint appendix

A saved checkpoint always holds a target state, an empty blob would roll back
without restoring anything. Also log with the slot id, and load the draft test
model from the HF cache instead of a second download.

* common : return bool from checkpoint load_tgt / load_dft

A checkpoint restored from a slot file falls back to full prompt re-processing
when it fails to load, a checkpoint created in memory still aborts.

---------

Co-authored-by: Igor Okulist <okigan@gmail.com>
2026-10-08 13:18:20 +03:00
gianni-cor ff5888f999 vulkan : fix TOP_K for +inf/NaN inputs and k = 1 on negative values (#30107)
The bucket search in topk_nary_search.comp started from the range
[0, 0xFF800000), which ends just below the ordered-uint mapping of +inf,
so +inf and NaN were never counted. A workgroup block with fewer than k
countable values left the ballot empty and the shader read uninitialized
shared state (hang/device lost on NVIDIA, wrong indices on AMD), and a few
+inf in a block were selected without being counted, dropping real top
values.

Map NaN to -inf on input, start from [0, 0xFFFFFFFF) so every value is
counted, and clamp the top bucket's end (2^32) instead of wrapping to 0.

The k = 1 path compared float bits as signed integers, which orders
negative values backwards; compare floats instead.

Add test_top_k_inf to test-backend-ops: negative values, fewer than k
+inf and many -inf, for k = 1, 10, 40.

Assisted-by: Claude Opus 5.5
2026-10-08 12:14:35 +02:00
Jeff Bolz dac3087394 vulkan: extend sparse FA support to coopmat2 (#30003) 2026-10-08 11:36:20 +02:00
Adrien Gallouët 03aa006acb vendor : update cpp-httplib to 0.60.1 (#30134)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-08 10:57:47 +02:00
Muhammad SaadandJohannes Gäßler 08246a28f6 cuda : support arbitrary striding for unary ops on f16, f32, and bf16 (#29781)
* cuda : support arbitrary striding for unary ops on f16, f32, and bf16

* Remove added newline

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2026-10-08 09:21:42 +02:00
bri-prism 46baf1f1fe sycl: FWHT optimizations (#29605) 2026-10-08 02:50:22 -04:00
Max Krasnyansky 097f5b5332 hex-mmadd: do not assume aligned read/write when bias-add is fused (#30133) 2026-10-08 09:48:06 +03:00
Bertay Eren d888016041 cuda : Use byte strides for roll to allow non-contiguous ROLL operations (#29547) 2026-10-08 09:46:38 +03:00
Eden fda1866613 CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits (#28175) 2026-10-08 09:46:21 +03:00
Titaniumtown ff30363a0e sycl: fuse the delta-net alpha gate (add + unary + mul) (#29687)
* sycl: fuse the delta-net alpha gate (add + unary + mul)

* tests: cover the fused add + unary + mul chain

* sycl: give the fused alpha gate a flat path and pin the node skip
2026-10-08 09:45:21 +03:00
cwriterandcwriter 000bee54a5 sycl: stage bulk uploads (model loading) through a pinned ring buffer (#29608)
Co-authored-by: cwriter <cwriter@localhost>
2026-10-08 02:30:00 -04:00
bosh 37ac634566 model : support classifier_activation for rerankers (#29692)
* model : support classifier_activation for rerankers

Assisted-by: Claude Opus 5.5

* model : map classifier gelu to gelu_erf and accept tanh

Assisted-by: Claude Opus 5.5

* model : default act_cls to tanh, ModernBERT falls back to gelu_erf

Assisted-by: Claude Opus 5.5
2026-10-08 09:29:20 +03:00
SXX 24e41838e0 ggml-cuda: assign four GDN state columns per warp (#30087)
* ggml-cuda: assign two GDN state columns per warp

* ggml-cuda: use 4 GDN state columns per warp at S_v=128

* ggml-cuda: default cols_per_warp=4

* ggml-cuda: address GDN review nits
2026-10-08 11:37:34 +05:30
cwriterandcwriter 847f447c31 sycl: add grouped MoE XMX GEMM (#29245)
Co-authored-by: cwriter <cwriter@localhost>
2026-10-08 01:41:13 -04:00
Sam Malayek 75118a3a59 convert : support Qwen3.5 embedding models (#27920)
* Update convert_hf_to_gguf: support Qwen3.5 embedding models

* Behavior-preserving refactor for conventions.

* conversion: simplify pooling comment
2026-10-08 08:40:01 +03:00
Titaniumtown 9b4ed0ca57 sycl: remove duplicate block-size defines from op headers (#29507)
All were duplicates of the values in ggml/src/ggml-sycl/presets.hpp
2026-10-08 08:39:25 +03:00
Max KrasnyanskyandJhen-Jie Hong 9c2e0e491a hexagon: enable alloc_buffer_n (#30126)
* hex-bufs: add support for alloc_buffer_n

* hex-bufs: add support for splitting large tensors into separate buffers

* hex-bufs: update GGML_HEXAGON_MBUF to accept three values dyn,static,total

* hex-bufs: bump dyn. default to 512MB since 128MB causes perf regressions with big MOEs

* hex-run: add --no-embd-offload option to simplify command lines on devices that need it

* Update scripts/snapdragon/run.py

Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>

---------

Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>
2026-10-07 18:30:06 -07:00
kurquhar 06cad0b9e7 hexagon: Q6_K weight dequant speedup (#30121)
* hexagon: Q6_K weight dequant speedup

Assisted-by: OpenCode

* unroll by another factor of 2

Assisted-by: OpenCode
2026-10-07 17:13:21 -07:00
Tarek Dakhran a657f7e981 model : add LiquidAI/d1-omni-600M decision model (#30114)
* model : add LiquidAI/d1-omni-600M decision model

Assisted-by: Claude Opus 5.5

* mtmd : keep conformer GLU sigmoid on CUDA

Assisted-by: Claude Opus 5.5

* server : take d1omni audio through images and input_audio, scope memory-less lfm2 to non-causal

Assisted-by: Claude Opus 5.5

* common : rename decision type d1omni to lfm2-d1-omni, server : make images an alias of files

Assisted-by: Claude Opus 5.5
2026-10-08 02:07:49 +02:00
kurquhar aa5e0092fd hexagon: support tiled Q4_K and Q6_K GET_ROWS (#30115)
* hexagon: support tiled Q4_K GET_ROWS

Assisted-by: OpenCode

* properly reject Q4_K views

Assisted-by: OpenCode

* hexagon: support tiled Q6_K GET_ROWS

Assisted-by: OpenCode
2026-10-07 17:04:45 -07:00
kurquhar 70815103c8 hexagon: improved GELU accuracy (#30104)
Assisted-by: OpenCode
2026-10-07 16:57:09 -07:00
Nicolas Budyn bd4eeaa047 chat: fix jinja parser for TranslateGemma (#30096)
* chat: fix jinja parser for translategemma

* log if missing soruce_lang_code or target_lang_code
2026-10-07 22:48:06 +02:00
Frost-54andAlde Rojas 5de733437b chat : name tool and argument parser rules by index (#30088)
* bugfix: infinite recursion caused by a tool named 'call'(#29967)

* chat : use index for schema and argument rules

* tests : remove tests

* tests : add expect_rules to peg test parser

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
2026-10-07 22:37:51 +02:00
Tarek Dakhran 88dcc460d6 model : add LiquidAI/d1-3B decision model (#30110)
* model : add LiquidAI/d1-3b decision model

mtmd : read LFM2 image resize algo from GGUF

Assisted-by: Claude Opus 5.5

* common : rename decision type d1 to lfm2-d1

Assisted-by: Claude Opus 5.5
2026-10-07 22:04:39 +02:00
bri-prism b86d2f0754 cuda: FWHT kernels for block widths above 512 (#29100)
The CUDA FWHT covers widths 64 to 512. It runs one row per warp and keeps N/32
values per lane, so wider blocks need more registers per lane than that layout
allows.

fwht_cuda_block runs one row per thread block with 256 threads, so each thread
keeps N/256 values. Stages below the warp width still shuffle, those up to the
block width go through shared memory, and the rest stay in registers. Same
butterfly and sign convention as the warp kernel.

Widths 64 to 512 keep the warp kernel. 1024 through 8192 use the new one, for
both F32 and F16 sources. ggml_cuda_op_mul_mat_use_fwht (the shared
supports_op/dispatch predicate added in #29096) does not check width, so it
needed no change here: any width it admits that ggml_cuda_op_fwht can't serve
already falls through correctly to the cuBLAS path.

Rebased onto current master with #29096's F16 commit underneath it, since this
depends on the same F16 template infrastructure; that commit applied cleanly,
the only conflict was in test-backend-ops.cpp where an unrelated intervening
commit's own test additions landed near this block.

test-backend-ops on an A10 (lambdalabs): MUL_MAT 1297/1297, including all
FWHT/Hadamard cases (18 existing, 4 new F32 wide, 4 new F16 wide, 2 new
many-rows, 1 too-big boundary moved to 16384).
2026-10-07 20:56:18 +02:00
Terrence Zhao 50a6c5cf7c mtmd: add cohere2 vision support (#30062)
* cohere2 vision model

* address comments

* remove redundant mapping

* follow existing patterns

* fused linear_1
2026-10-07 20:07:50 +02:00
Aman Gupta d6cf9acb25 llama : add a GPU cache for MoE experts kept in host memory (#29887)
* llama : add a GPU cache for MoE experts kept in host memory

Assisted-by: Claude

* use llama_moe_cache_ptr
2026-10-07 21:07:45 +03:00
ShobhitandGeorgi Gerganov 42c787e8c1 cuda: update uncoalesced memory reads in pool2d (#29425)
* cuda: update uncoalesced memory reads in pool2d

* cuda: Added perf and __restrict__ in pool2d

* Update POOL2D_WARP_KERNEL_MIN_WINDOW macro

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* cuda: Fix compiler bugs

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-07 19:53:43 +02:00
Aldehir Rojas 18b5f8b186 server : accumulate generated text and tokens as parse input (#29876)
* server : collect text and token input

* common : add a tokenize helper that aligns tokens with bytes

* common : simplify tokenize logic
2026-10-07 11:49:17 -05:00
Pascal 448147d42a llama: share the nextn tensor flags between models (#30097)
* llama: share the nextn tensor flags between models

Follow-up of the TODO in glm5-next: move the trunk-only and MTP-only
detection that each model copied into a nextn_flags helper of
llama_model_base. It probes the first trunk layer and the first NextN
layer, and adds TENSOR_SKIP when MTP is not loaded. qwen4exp probes
hc_attn_norm since it has no attn_norm.

deepseek4, nemotron-h, qwen35, qwen35moe, qwen3next and qwen4exp now
also accept a trunk-only file, like the other models.

* llama: avoid capturing structured bindings in the nextn flags

Lambdas that capture structured bindings need C++20, and GCC 15
rejects them under -Werror, so the models read the trunk and MTP
flags into plain variables.
2026-10-07 16:30:00 +02:00
pratiknarola-t 988190680d metal : few-row MMA mat-mul for the remaining src0 types (#30065)
The generic few-row MMA kernel works for any type with a 16-weight
dequantizer, so it now also takes BF16, Q1_0, Q2_0, MXFP4, Q2_K, Q3_K,
TQ2_0 and the IQ types. Each type starts at the row count where it beats
the current kernels on an M3 Ultra: 5 rows for TQ2_0, 4 for BF16, 3
for MXFP4, Q2_0, Q2_K and IQ4_NL, and 2 for the others.

test-backend-ops perf -o MUL_MAT, m=4096, k=14336, M3 Ultra, time of this
change over master (mean of two interleaved runs each): 0.23 to 0.98 from
the threshold to 8 rows, 0.24 to 0.33 at 9 to 16 rows, and 0.99 to 1.01 at
1 and 512 rows.
2026-10-07 16:49:37 +03:00
Niko MaroulisandGeorgi Gerganov 7e8324f5fe metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (#30100)
* metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT

ggml_metal_op_mul_mat_mma picks the residual of a fused MUL_MAT+ADD as
"the ADD operand whose op is not MUL_MAT". When both operands of the ADD
are mat-mul outputs (x = W1 @ u + W2 @ v), that test is true for both, so
the residual resolves to the fused mat-mul's own, never-written output and
the kernel adds whatever that buffer holds.

The fusion check (ggml_metal_mul_mat_add_operand) already selects the
operand by identity; make the encoder do the same.

Clef decision models hit this in their head (proj_option_context @ ctx +
proj_option_lexical @ lex, 9 option rows): on Metal, /v1/systemone
probabilities collapse toward uniform (billing 0.28 where the CPU backend
gives 0.977, Cloudflare_clef-flash Q8_0), deterministic per memory layout,
correct with GGML_METAL_FUSION_DISABLE=1. Not a quantization issue: the
same file is right on CPU.

Add a MUL_MAT_ADD mode to test-backend-ops where the residual is a second
mat-mul; on Metal it fails 27 of 28 cases before this change (the one pass
is f16 n=2, under the MMA row threshold, so nothing fuses).

* Update tests/test-backend-ops.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-07 16:41:50 +03:00
b9acf138a1 feat: add GLM5Next MTP, optimize (#29928)
* llama : add GLM5-Next NextN (MTP) graph

Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block
embeds enorm(tok)+hnorm(h) through eh_proj, runs one plain DSA layer and the
shared lm_head, reusing the trunk's builders through the no_build tag ctor.
llama_memory_recurrent also tolerates a partial seq_rm when the context holds
no recurrent layers, which is what the MTP draft context needs.

Assisted-by: Claude

* llama : glm5-next: skip dead compute in headless NextN forwards

A NextN forward with no output rows (the MTP catch-up and the draft-context
prefill) persists only through its cache writes, so the headless graph keeps
the MLA latent, indexer key|gate and pooled-key writes and drops the query
path, the indexer selection, the attention body, the FFN and the LM head. The
4-token catch-up falls from 6.9 ms to 0.33 ms of kernels; the greedy output
hashes and the draft acceptance are unchanged.

Assisted-by: Claude

* llama : glm5-next: fix NextN extraction contracts and shared-tail rollback

Three fixes from the architectural review. The headless graph prune now also
requires that no unmasked nextn extraction is live, because that mode reads
n_tokens hidden rows regardless of the logits flags. Masked extraction
publishes the hidden rows gathered by the output ids, so a batch whose output
flags are not a prefix exports the right rows. A partial recurrent rollback
whose tail cell is shared with another sequence is now rejected instead of
silently moving that sequence's tail.

Assisted-by: Claude

* llama : glm5-next: tidy comments in the MTP changes

Assisted-by: Claude

* llama : glm5-next: crop the MTP graph to the output rows instead of pruning it

Replace the headless NextN prune with the crop pattern the other MTP
graphs use: gather the attention output and the block input at the
output ids before the position-wise FFN and the shared head. A NextN
forward with no output rows (the MTP catch-up and the draft-context
prefill) then runs the FFN and the head over zero rows. The 4-token
catch-up falls from 6.9 ms to 2.9 ms of kernels; greedy output hashes
are unchanged.

Assisted-by: Claude

* glm5-next: use the nextn crop helpers in the MTP graph

Replace the local crop condition and the masked select of t_h_nextn
with crop_before_nextn and crop_after_nextn, so the MTP graph narrows
its rows the same way as the main graph and the other models.

Describe the shared cell and empty filter branches of the recurrent
partial rollback.

* glm5-next: load MTP-only and trunk-only GGUF files

Make the trunk tensors optional when the file only holds the NextN
layer, and the NextN tensors optional when the file only holds the
trunk, so the split MTP GGUF loads as a draft model.

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-07 13:16:53 +02:00
SIDDARTHA REDDY 48499d2e1c qwen3tts : guard speaker_encoder_config patch for CustomVoice variant (#29179)
* Fix KeyError converting Qwen3-TTS CustomVoice variant without speaker_encoder_config (fixes #29088)

Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>

* Address review feedback: drop tests/test-convert-qwen3tts.py

Per reviewer feedback on PR #29179, remove the regression test file.
The fix itself is unchanged.

Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>

---------

Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
2026-10-07 12:57:48 +02:00
Hrishith ThadicherlaandGeorgi Gerganov d0b490f25e sampling : use greedy selection for eligible temperature-zero chains (#29797)
* sampling : use greedy selection for eligible temperature-zero chains

Assisted-by: OpenAI Codex

* Apply suggestion from @ggerganov

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* sampling: simplify zero-temperature greedy eligibility

Allow the same greedy selection on CPU and grammar/reasoning-budget paths.
Keep distribution sampling for dynamic temperature and requested probabilities.
Cover the common sampler selection and probability behavior in the existing sampler tests.

Assisted-by: OpenAI Codex

* sampling : use greedy selection after final top-k with k=1

Assisted-by: OpenAI Codex

* cont : clean-up

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-07 13:34:18 +03:00
Sigbjørn Skjæret 42b021b4dc vocab : add plamo fim tokens (#30090) 2026-10-07 11:40:35 +02:00
Toki Nasin 7481354a17 convert : fix token configuration for PLaMo-3 (#29843)
* convert : Fix token configuration for PLaMo-3

The reasoning and tool calling tags in PLaMo-3 consist of three tokens
each. For example, for reasoning:

* reasoning start: `<|plamo:begin_`, `think`, `:plamo|>`
* reasoning end: `<|plamo:end_`, `think`, `:plamo|>`

Registering `<|plamo:begin_`, `<|plamo:end_`, and `:plamo|>` as
`USER_DEFINED` so that they are parsed correctly.

Also PLaMo-3 models use <|plamo:tag|> as EOT, while PLaMo-2 models
use <|plamo:op|>.

Take EOT token as a parameter and look it up so that PLaMo-2 and
PLaMo-3 can use their appropriate ones.

* use NORMAL instead of USER_DEFINED
2026-10-07 11:34:16 +02:00
R0CKSTAR ad21565331 musa: use the tile lightning indexer kernel (#30080)
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
2026-10-07 11:22:56 +02:00
Alessandro de Oliveira Faria (A.K.A.CABELO) b7dafa01e5 vendor : update cpp-httplib to 0.60.0 (#30081) 2026-10-07 11:18:33 +02:00
yarikandyarik 26908739bc imatrix : include clocale for std::setlocale (#30079)
Co-authored-by: yarik <dazzywi@github.com>
2026-10-07 10:58:22 +02:00
Masashi Yoshimura 36a73916ee ggml-webgpu: fix flash_attn supports_op check for overlapping KV (#28205) 2026-10-07 09:56:53 +02:00
Evan Huus fa3c2fab36 tests : retain the anchor when testing recurrent rollback (#29923) 2026-10-07 10:44:10 +03:00