Commit Graph
3 Commits
Author SHA1 Message Date
Jeff Bolz c9ca51c1f6 vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526)
* vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs

* remove one more fastdiv
2026-08-29 10:59:48 +03:00
ravel7524andJeff Bolz 90c26fcd4b Vulkan: add hoisting support for row IDs and expert count in shaders (#26686)
* vulkan: add hoisting support for row IDs and expert count in shaders

* use hoisted row ids in coopmat2

* vulkan: address review feedback on count_experts
- use vk_op_count_experts_push_constants instead of a raw uint vector
- apply the fastdiv trick to the ne00 div/mod in count_experts
- compute the per-expert offsets with subgroupExclusiveAdd when the
  device supports it, keeping the serial path as fallback
- document the data_d layout and the hoisted_row_id_words bound
- drop a leftover debug print in ggml_vk_matmul_id

* vulkan: use init_pushconst_fastdiv for count_experts push constants

* vulkan: refine comments for row ID hoisting and data layout in count_experts shader

* Whitespace

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
2026-08-28 16:52:49 +02:00
Jeff Bolz c9ced4910b vulkan: preprocess mul_mat_id experts and discard workgroups more quickly (#18352)
Run a preprocess to count how many times each expert is used, and use this to
quickly discard workgroups that aren't needed.
2025-12-26 16:12:58 -06:00