Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

metal : add TQ2_0 support (#26980)

Browse Source 
* metal: add TQ2_0 support

Add support for the GGML_TYPE_TQ2_0 (ternary, 2 bits per element) type in
the Metal backend.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* cont : optimize mul_mv kernel

- float ops over integer ops
- precalculate sums
- hoist coef out of the inner loop
- contiguous y loads

llama.cpp:DeepSeek-v4-Flash-0731
This commit is contained in:
Georgi Gerganov
2026-08-13 14:33:53 +03:00
committed by GitHub
parent f65e568fd8
commit 4a84b0ad10
5 changed files with 232 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-metal/ggml-metal-device.m
+3
View File
@@ -1407,6 +1407,7 @@ bool ggml_metal_device_supports_op(ggml_metal_device_t dev, const struct ggml_te
case GGML_TYPE_Q5_0:
case GGML_TYPE_Q5_1:
case GGML_TYPE_IQ4_NL:
case GGML_TYPE_TQ2_0:
case GGML_TYPE_I32:
return true;
default:
@@ -1435,6 +1436,7 @@ bool ggml_metal_device_supports_op(ggml_metal_device_t dev, const struct ggml_te
case GGML_TYPE_Q5_0:
case GGML_TYPE_Q5_1:
case GGML_TYPE_Q8_0:
case GGML_TYPE_TQ2_0:
switch (op->type) {
case GGML_TYPE_F32:
case GGML_TYPE_F16:
@@ -1470,6 +1472,7 @@ bool ggml_metal_device_supports_op(ggml_metal_device_t dev, const struct ggml_te
case GGML_TYPE_Q5_0:
case GGML_TYPE_Q5_1:
case GGML_TYPE_IQ4_NL:
case GGML_TYPE_TQ2_0:
return true;
default:
return false;