vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm (#30829)

With head sizes above 128 and KV >= 1024, scalar flash attention uses
16 rows per workgroup. On the Qualcomm Adreno driver this variant
spills registers to scratch memory (264 bytes per invocation on the
X1-85, none at 8 rows) and runs about 40x slower. The largest
test-backend-ops case then becomes a single dispatch of several
seconds, past the Windows GPU timeout, and the test never completes on
the X1 Elite.

Use at most 8 rows per workgroup on Qualcomm when hsk or hsv > 128.
Smaller heads keep 16 rows, which is faster for them on the X2 Elite.

Adreno X1-85, FLASH_ATTN_EXT hsk=256 kv=4096: nb=64 711 ms -> 17 ms,
nb=512 no result -> 125 ms; test-backend-ops completes. Adreno X2-90:
Gemma 4 E4B pp512 at depth 4096 with flash attention 81-90 -> 117 t/s.
This commit is contained in:
Hongqiang Wang authored and GitHub committed 2026-11-10 08:00:55 +01:00
1 parent 58e38bdf65
commit 78ffbaa185
1 file changed
+6
+6
View File
@@ -1260,6 +1260,12 @@ static vk_fa_tuning_params get_fa_tuning_params_scalar(const vk_device& device,
result.block_rows /= 2;
}
// With head sizes above 128, 16 rows per workgroup spill registers on Qualcomm and run about
// 40x slower; at large KV a single dispatch can then exceed the Windows GPU timeout.
if (device->vendor_id == VK_VENDOR_ID_QUALCOMM && (hsk > 128 || hsv > 128)) {
result.block_rows = std::min(result.block_rows, 8u);
}
// On AMD RDNA, for small head sizes and big batch size the shader uses few registers, so too many subgroups get scheduled
// at once and end up thrashing the cache. Fix this by setting a large (unused) shmem buffer that reduces occupancy.
// This targets an occupancy of 4 subgroups per SIMD.