Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

vulkan : dequant q8_0 KV once in coopmat1 (#25494)

Browse Source 
* vulkan : dequant q8_0 KV once in coopmat1

Assisted-by: Claude (Opus 4.8)

* vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBufferRange

* vulkan : require KV-cache layout in FA dequant path

Assisted-by: Claude (Opus 4.8)

* vulkan : skip FA dequant path on coopmat2

Assisted-by: Claude (Opus 4.8)

* tests : add contiguously-allocated quant K/V FA tests

Assisted-by: Claude (Opus 4.8)

* vulkan : trim comments

* vulkan : tighten permutation checks for FA path

* vulkan : set prealloc_x_need_sync after the FA dispatch

* vulkan : exclude Intel Xe1 from FA dequant path
This commit is contained in:
Nathanw1014
2026-08-19 17:44:15 +02:00
committed by GitHub
parent b95502ba9a
commit dc72703fc6
4 changed files with 101 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
+4
View File
@@ -780,6 +780,10 @@ void process_shaders() {
if (tname != "f16" && tname != "bf16") {
string_to_spv("dequant_" + tname, "dequant_" + tname + ".comp", merge_maps(base_dict, {{data_a_key, "1"}, {"D_TYPE", "float16_t"}}));
}
// Fused dequant+transpose variant for FA quant-KV (per-head-contiguous f16 scratch).
if (tname == "q8_0") {
string_to_spv("dequant_" + tname + "_transpose", "dequant_" + tname + ".comp", merge_maps(base_dict, {{data_a_key, "1"}, {"D_TYPE", "float16_t"}, {"DEQUANT_TRANSPOSE", "1"}}));
}
shader = (tname == "f32" || tname == "f16" || tname == "bf16") ? "get_rows.comp" : "get_rows_quant.comp";