Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (#27449)

Browse Source 
* vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec when NUM_COLS > 4. 5x perf at n=8

Assisted-by: Claude Opus 5

* adds 2 cases per quant type at `k=16*256` to the `all_types` mat-vec sweep

---------

Co-authored-by: Marshall <assistant@llama.cpp>
This commit is contained in:
Laurent Zuijdwijk
2026-09-02 09:14:52 +03:00
committed by GitHub
co-authored by Marshall
parent 56dd8150cc
commit ba8818cbf3
2 changed files with 22 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test-backend-ops.cpp
+4
View File
@@ -9395,6 +9395,10 @@ static std::vector<std::unique_ptr<test_case>> make_test_cases_eval() {
//test_cases.emplace_back(new test_mul_mat(type_a, GGML_TYPE_F32, 18, i, 32*256, { 1, 1}, {8, 1}));
//test_cases.emplace_back(new test_mul_mat(type_a, GGML_TYPE_F32, 19, i, 33*256, { 1, 1}, {1, 1}));
}
// mat-vec shaders split k across lanes and loop over the blocks in strides. k must be
// long enough that the loop wraps, else the stride is never exercised
test_cases.emplace_back(new test_mul_mat(type_a, GGML_TYPE_F32, 16, 1, 16*256, { 1, 1}, {1, 1}));
test_cases.emplace_back(new test_mul_mat(type_a, GGML_TYPE_F32, 16, 8, 16*256, { 1, 1}, {1, 1}));
}
test_cases.emplace_back(new test_mul_mat(GGML_TYPE_Q4_0, GGML_TYPE_F32, 2880, 32, 2880, {1, 1}, {1, 1}));