qwen4exp : build the QSA selection without the kq_mask (#30965)

* qwen4exp : build the QSA selection without the kq_mask

* clean

* lang

* qwen4exp : reshape the fused QSA selection to the kq_mask streams

ggml_top_k_mask with a single keep row returns [n_kv, n_tokens], while
build_attn_qsa expects the kq_mask layout [n_kv, n_tokens/n_stream, 1, n_stream].
With several streams the shape assert failed at graph reserve.

* test the QSA optimization

* reword

* update build_fused_top_k_mask instead of handwiring

* simplify

* keep il at the end for consistency
This commit is contained in:
Alexey Kopytko authored and GitHub committed 2026-10-23 19:49:08 +02:00
1 parent da2ee1e7a6
commit 5c31142682
6 files changed
+28 -14

No files matched your search

+1 -1
View File
@@ -975,7 +975,7 @@ ggml_tensor * llama_model_glm5_next::graph::build_kpool_select(
ggml_tensor * tail = hparams.indexer_kpool_select_tail ? inp_kpool->tail_idxs : nullptr;
GGML_ASSERT(kpool*n_top_pool + (tail ? tail->ne[0] : 0) == (int64_t) inp_kpool->n_sel);
ggml_tensor * sel = build_fused_top_k_mask(kq_mask, top_k, score, inp_kpool->pool_idxs, tail, il);
ggml_tensor * sel = build_fused_top_k_mask(kq_mask, top_k, score, inp_kpool->pool_idxs, tail, nullptr, il);
cb(sel, "indexer_sel", il);
return sel;