Commit Graph
3 Commits
Author SHA1 Message Date
Jeff Bolz 8161641005 vulkan: add iq4_nl support back to FA (#24585)
* vulkan: add iq4_nl support back to FA

I was originally concerned about wasting shared memory on the LUT, but it's small
and unlikely to matter in practice.

Also support q1_0 for non-coopmat2.

Fixes #23681

* remove q1_0 FA support
2026-07-28 07:06:03 -05:00
Ruben Ortlam 6e093b80ea vulkan: add Flash Attention support for BFloat16 KV cache (#23420)
* vulkan: add flash attention bf16 kv support

* vulkan: bf16 FA coopmat1 support

* vulkan: bf16 FA coopmat2 support

* fix FA bf16 f32 fallback

* fix FA bf16 coopmat1 shader

* fix FA bf16 coopmat2 shader

* code cleanup

* cleanup comment change

* address feedback

* add O_TYPE for cm2 FA

* use O_TYPE for gqaStore function

* reduce BFLOAT16 ifdefs
2026-05-30 10:39:31 +02:00
Jeff Bolz dd9280a664 vulkan: Support asymmetric FA in scalar/mmq/coopmat1 paths (#22589) 2026-05-11 12:49:03 +02:00