llama: create new fused op for QSA mask (#30640)

* llama: create new fused op for QSA scoring

* enroll glm5.3-next too

* tests : initialize the remaining tensors in test_kpool_mask

The harness sentinels were left uninitialized, so leftover memory could read as NaN and fail the NaN check.

* ggml: generalize the k-pool mask op into ggml_top_k_mask

The k-pool mask op only covered the pooled QSA selection of qwen4exp
and glm5-next, while the DSA models and the DeepSeek V4 CSA build the
same top-k mask by hand with fill, set_rows and add.

ggml_top_k_mask(mask, top_k, score, blocks, extra) keeps the mask
cells of the live picks and the extra cells, and sets the rest to
-inf. Every row tensor is 4D in the layout of top_k, and the mask is
broadcast to those rows, so a selection can also be made per head.
score, blocks and extra are optional: without score every pick is
live, without blocks a pick is a cell, and extra holds the cells a
row keeps without a pick.

llm_graph_context::build_fused_top_k_mask views the row tensors in
the layout of kq_mask and registers the fused node. The k-pool
models, the DSA attention of build_attn and the DeepSeek V4 CSA mask
use it when the probe enables the op, and keep their current path on
backends without it.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Aman GuptaandPascal authored and GitHub committed 2026-10-22 01:02:22 +08:00
1 parent 0756e83fc1
commit d454a204ef
18 files changed
+592 -31

No files matched your search

+7
View File
@@ -683,6 +683,13 @@ ggml_tensor * llama_model_deepseek4::graph::build_top_k_mask(
GGML_ASSERT(kq_mask);
GGML_ASSERT(top_k);
if (cparams.fused_top_k_mask) {
ggml_tensor * kq_mask_top_k = build_fused_top_k_mask(kq_mask, top_k, nullptr, nullptr, nullptr, il);
cb(kq_mask_top_k, name, il);
return kq_mask_top_k;
}
ggml_tensor * kq_mask_all = ggml_fill(ctx0, kq_mask, -INFINITY);
kq_mask_all = ggml_view_4d(ctx0, kq_mask_all, 1, kq_mask_all->ne[0], kq_mask_all->ne[1], kq_mask_all->ne[3],
kq_mask_all->nb[0], kq_mask_all->nb[1], kq_mask_all->nb[2], 0);