Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Vulkan: route large matmuls to medium tile on Adreno (#24877)

Browse Source 
* [Vulkan] Fixes llama-cli breaking over longer promts sizes

The llama-cli was breaking for longer promts sizes for q4_0 quantized networks. Causing due to insufficient shared memory.

* Removed the un-used Adreno device

* Updated matmul for small pipeline.
This commit is contained in:
Raman Shinde
2026-07-11 10:28:29 +02:00
committed by GitHub
parent 1d1d9a9ed7
commit 76f2798059
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-vulkan/ggml-vulkan.cpp
+8
View File
@@ -6501,6 +6501,14 @@ static vk_device ggml_vk_get_device(size_t idx) {
device->mul_mat_id_m[i] = true; device->mul_mat_id_m[i] = true;
device->mul_mat_id_s[i] = false; device->mul_mat_id_s[i] = false;
break; break;
case VK_VENDOR_ID_QUALCOMM:
device->mul_mat_l[i] = false;
device->mul_mat_m[i] = true;
device->mul_mat_s[i] = true;
device->mul_mat_id_l[i] = false;
device->mul_mat_id_m[i] = true;
device->mul_mat_id_s[i] = true;
break;
#endif #endif
default: default:
device->mul_mat_l[i] = true; device->mul_mat_l[i] = true;