kv-cache : keep rotation constants on the GPU (#31073)

* kv-cache : keep rotation constants on the GPU

Assisted-by: Codex

* tests : verify resident KV rotation buffers

Assisted-by: Codex

* kv-cache : document remaining multi-device rotation work

Assisted-by: Codex

* kv-cache : support per-layer rotation buffers

Assisted-by: Codex

* kv-cache : fix hybrid SWA rotation inputs and mixed placement check

Assisted-by: Codex

* models : use per-layer rotation in mistral-large4 and dflash

* tests: drop the rotation buffer tests

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Anton IandPascal authored and GitHub committed 2026-10-30 10:15:06 +02:00
1 parent ab3768cdc2
commit fb5997274e
15 files changed
+179 -246

No files matched your search

+1 -1
View File
@@ -632,7 +632,7 @@ ggml_tensor * llama_model_deepseek41::graph::build_attention41(
GGML_ASSERT(top_k && "DeepSeek-V4.1: top-k source layer produced no top-k");
// a quantized K cache stores the window Hadamard-rotated (as build_raw_attention)
ggml_tensor * k_rot = inp_attn->self_k_rot;
ggml_tensor * k_rot = inp_attn->self_k_rot[il];
if (k_rot) {
q = llama_mul_mat_hadamard(ctx0, q, k_rot);
kv = llama_mul_mat_hadamard(ctx0, kv, k_rot);