vulkan : dequant q8_0 KV once in coopmat1 (#25494)

* vulkan : dequant q8_0 KV once in coopmat1

Assisted-by: Claude (Opus 4.8)

* vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBufferRange

* vulkan : require KV-cache layout in FA dequant path

Assisted-by: Claude (Opus 4.8)

* vulkan : skip FA dequant path on coopmat2

Assisted-by: Claude (Opus 4.8)

* tests : add contiguously-allocated quant K/V FA tests

Assisted-by: Claude (Opus 4.8)

* vulkan : trim comments

* vulkan : tighten permutation checks for FA path

* vulkan : set prealloc_x_need_sync after the FA dispatch

* vulkan : exclude Intel Xe1 from FA dequant path
This commit is contained in:
Nathanw1014
2026-08-19 17:44:15 +02:00
committed by GitHub
parent b95502ba9a
commit dc72703fc6
4 changed files with 101 additions and 10 deletions
@@ -18,7 +18,18 @@ void main() {
return;
}
#ifdef DEQUANT_TRANSPOSE
// read [HS, NH, KV, NS], write [HS, KV, NH, NS]
const uint HS = p.M, NH = p.K, KVn = p.stride_a;
const uint e0 = ib * 32;
const uint b_idx = (e0 % HS)
+ ((e0 / (HS * NH)) % KVn) * HS
+ ((e0 / HS) % NH) * (HS * KVn)
+ (e0 / (HS * NH * KVn)) * (HS * KVn * NH)
+ 16 * il;
#else
const uint b_idx = 1024*i + 32*ir + 16*il;
#endif
const float d = float(data_a[ib].d);
@@ -780,6 +780,10 @@ void process_shaders() {
if (tname != "f16" && tname != "bf16") {
string_to_spv("dequant_" + tname, "dequant_" + tname + ".comp", merge_maps(base_dict, {{data_a_key, "1"}, {"D_TYPE", "float16_t"}}));
}
// Fused dequant+transpose variant for FA quant-KV (per-head-contiguous f16 scratch).
if (tname == "q8_0") {
string_to_spv("dequant_" + tname + "_transpose", "dequant_" + tname + ".comp", merge_maps(base_dict, {{data_a_key, "1"}, {"D_TYPE", "float16_t"}, {"DEQUANT_TRANSPOSE", "1"}}));
}
shader = (tname == "f32" || tname == "f16" || tname == "bf16") ? "get_rows.comp" : "get_rows_quant.comp";