metal : add TQ2_0 support (#26980)

* metal: add TQ2_0 support

Add support for the GGML_TYPE_TQ2_0 (ternary, 2 bits per element) type in
the Metal backend.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* cont : optimize mul_mv kernel

- float ops over integer ops
- precalculate sums
- hoist coef out of the inner loop
- contiguous y loads

llama.cpp:DeepSeek-v4-Flash-0731
This commit is contained in:
Georgi Gerganov
2026-08-13 14:33:53 +03:00
committed by GitHub
parent f65e568fd8
commit 4a84b0ad10
5 changed files with 232 additions and 1 deletions
+10
View File
@@ -953,6 +953,11 @@ ggml_metal_pipeline_with_params ggml_metal_library_get_pipeline_mul_mv(ggml_meta
nr0 = N_R0_IQ4_XS;
smem = 32*sizeof(float);
} break;
case GGML_TYPE_TQ2_0:
{
nsg = N_SG_TQ2_0;
nr0 = N_R0_TQ2_0;
} break;
default:
{
GGML_LOG_ERROR("Asserting on type %d\n", (int) tsrc0);
@@ -1182,6 +1187,11 @@ ggml_metal_pipeline_with_params ggml_metal_library_get_pipeline_mul_mv_id(ggml_m
nr0 = N_R0_IQ4_XS;
smem = 32*sizeof(float);
} break;
case GGML_TYPE_TQ2_0:
{
nsg = N_SG_TQ2_0;
nr0 = N_R0_TQ2_0;
} break;
default:
{
GGML_LOG_ERROR("Asserting on type %d\n", (int)op->src[2]->type);