Files
llama.cpp/docs/ops.md
T
Georgi Gerganovandlosewayy 29dbbad2b1 ggml : prune (#29801)
* ggml : prune deprecated API

* cont : bump RPC version

* cont : minor

* ggml : remove outdated documentation from ggml.h

The 172-line comment block at the top of the header has not been updated in a
substantive way since the backend v2 split (Nov 2023) and now documents APIs
that no longer exist:

- `ggml_set_f32()`, `ggml_get_f32_1d()`, `ggml_set_f32_1d()` were removed
- `ggml_conv_1d_1s()`, `ggml_conv_1d_2s()` ops no longer exist
- `ggml_set_param()` no longer takes a context argument
- `ggml_graph_compute_with_ctx()` moved to the CPU backend header
- the optimization API now lives in `ggml-opt.h`

The block also ended with 5 empty `TODO` sections.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

* ggml : remove DIAG_MASK_INF and DIAG_MASK_ZERO (#31946)

* rpc : bump RPC protocol major version

The tensor op enum is sent over the wire, so removing GGML_OP_ADD1,
GGML_OP_DIAG_MASK_INF and GGML_OP_DIAG_MASK_ZERO shifts the value of every
op that follows them and peers built before and after this change would
misinterpret each other's ops.

The handshake compares the major version for exact equality and ignores the
patch version, so bumping the major version is what rejects incompatible
peers instead of silently misbehaving.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

---------

Co-authored-by: losewayy <1134285612@qq.com>
2026-11-12 20:58:24 +02:00

14 KiB

GGML Operations

List of GGML operations and backend support status.

How to add a backend to this table:

  1. Run test-backend-ops support --output csv with your backend name and redirect output to a csv file in docs/ops/ (e.g., docs/ops/CUDA.csv)
  2. Regenerate /docs/ops.md via ./scripts/create_ops_docs.py

Legend:

  • ✅ Fully supported by this backend
  • 🟡 Partially supported by this backend
  • ❌ Not supported by this backend
Operation BLAS CANN CPU CUDA ET HTP MTL OpenCL SYCL Vulkan WebGPU ZenDNN zDNN
ABS ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
ACC ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
ADD ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
ADD1 ❌ ✅ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌
ADD_ID ❌ ❌ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
ARANGE ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ✅ ❌ ❌
ARGMAX ❌ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
ARGSORT ❌ ✅ ✅ ✅ ❌ ✅ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
CEIL ❌ ❌ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
CLAMP ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
COL2IM_1D ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
CONCAT ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ 🟡 ❌ ❌
CONT ❌ 🟡 ✅ ✅ 🟡 🟡 ✅ 🟡 ✅ ✅ 🟡 ❌ ❌
CONV_1D_DW ❌ ❌ ❌ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌ ✅
CONV_2D ❌ ❌ 🟡 ✅ ❌ ❌ ✅ ✅ ✅ 🟡 ✅ ❌ ❌
CONV_2D_DW ❌ ❌ ✅ ✅ ❌ ❌ ❌ 🟡 ✅ ✅ ✅ ❌ ❌
CONV_3D ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
CONV_TRANSPOSE_1D ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
CONV_TRANSPOSE_2D ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
COS ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ✅ ❌ ❌
COUNT_EQUAL ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ✅ ❌ ❌
CPY ❌ 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 ❌ ❌
CROSS_ENTROPY_LOSS ❌ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
CROSS_ENTROPY_LOSS_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
CUMSUM ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
DIAG ❌ ❌ ✅ ✅ 🟡 ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
DIV ❌ ✅ ✅ ✅ ❌ 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
DSV4_HC_COMB ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ❌
DSV4_HC_POST ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ❌
DSV4_HC_PRE ❌ ❌ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌ ❌
DUP ❌ ✅ ✅ ✅ ❌ 🟡 🟡 🟡 ✅ ✅ ❌ ❌ ❌
ELU ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
EXP ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
EXPM1 ❌ ❌ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
FILL ❌ ❌ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
FLASH_ATTN_EXT ❌ 🟡 ✅ 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 ❌ ❌
FLOOR ❌ ❌ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
GATED_DELTA_NET ❌ ❌ ✅ ❌ ✅ ✅ 🟡 ✅ ✅ ✅ ✅ ❌ ❌
GATED_LINEAR_ATTN ❌ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
GEGLU ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
GEGLU_ERF ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
GEGLU_QUICK ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
GELU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
GELU_ERF ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
GELU_QUICK ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
GET_ROWS ❌ 🟡 ✅ 🟡 🟡 🟡 🟡 🟡 🟡 ✅ 🟡 ❌ ❌
GET_ROWS_BACK ❌ ❌ 🟡 🟡 ❌ ❌ ❌ ❌ 🟡 🟡 ❌ ❌ ❌
GROUP_NORM ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌
HARDSIGMOID ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
HARDSWISH ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
IM2COL ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
IM2COL_3D ❌ ❌ ✅ ✅ ❌ 🟡 ❌ ✅ ✅ ✅ ❌ ❌ ❌
L2_NORM ❌ ✅ ✅ ✅ 🟡 🟡 ✅ 🟡 ✅ ✅ 🟡 ❌ ❌
LEAKY_RELU ❌ ✅ ✅ ✅ ❌ 🟡 🟡 ❌ ✅ ✅ ❌ ❌ ❌
LIGHTNING_INDEXER ❌ ❌ ✅ ❌ ❌ ❌ ❌ 🟡 ✅ ❌ ❌ ❌ ❌
LOG ❌ ✅ ✅ ✅ ❌ ✅ ✅ ❌ ✅ ✅ ✅ ❌ ❌
MEAN ❌ ✅ ✅ ✅ ✅ ❌ ✅ ✅ 🟡 ✅ ✅ ❌ ❌
MUL ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
MUL_MAT 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡
MUL_MAT_HADAMARD ❌ ❌ ✅ ❌ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ❌ 🟡
MUL_MAT_ID ❌ 🟡 ✅ ✅ 🟡 🟡 🟡 🟡 🟡 ✅ 🟡 🟡 ❌
MUL_MAT_ID_W4A4 ❌ ❌ ✅ ❌ ❌ 🟡 ❌ 🟡 ✅ ❌ ❌ ❌ ❌
MUL_MAT_ID_W4A8 ❌ ❌ ✅ ❌ ❌ 🟡 ❌ 🟡 ✅ ❌ ❌ ❌ ❌
MUL_MAT_W4A4 ❌ ❌ ✅ ❌ ❌ 🟡 ❌ 🟡 ✅ ❌ ❌ ❌ ❌
MUL_MAT_W4A8 ❌ ❌ ✅ ❌ ❌ 🟡 ❌ 🟡 ✅ ❌ ❌ ❌ ❌
NEG ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
NORM ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ✅ 🟡 ❌ ❌
OPT_STEP_ADAMW ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
OPT_STEP_SGD ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
OUT_PROD 🟡 🟡 🟡 🟡 ❌ ❌ ❌ ❌ 🟡 🟡 ❌ ❌ ❌
PAD ❌ 🟡 ✅ 🟡 ❌ ✅ 🟡 🟡 🟡 ✅ ✅ ❌ ❌
PAD_REFLECT_1D ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ❌ ✅ ❌ ❌
POOL_1D ❌ ❌ ✅ ❌ ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ ❌
POOL_2D ❌ 🟡 ✅ ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ ❌
REGLU ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
RELU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
REPEAT ❌ ✅ ✅ 🟡 🟡 🟡 ✅ ✅ ✅ ✅ 🟡 ❌ ❌
REPEAT_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
RMS_NORM ❌ ✅ ✅ ✅ 🟡 ✅ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
RMS_NORM_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
ROLL ❌ ❌ ✅ ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ ❌
ROPE ❌ ✅ ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ✅ ✅ ❌ ❌
ROPE_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ✅ ✅ ❌ ❌ ❌
ROUND ❌ ❌ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
RWKV_WKV6 ❌ ❌ ✅ ✅ ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
RWKV_WKV7 ❌ ❌ ✅ ✅ ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌
SCALE ❌ 🟡 ✅ ✅ ✅ 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SET ❌ ❌ ✅ ✅ ❌ ❌ ✅ ✅ 🟡 ✅ ✅ ❌ ❌
SET_ROWS ❌ 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 🟡 ❌ ❌
SGN ❌ ✅ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SIGMOID ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SILU ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SILU_BACK ❌ ❌ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌ ❌
SIN ❌ ✅ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ✅ ❌ ❌
SOFTPLUS ❌ ❌ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SOFT_MAX ❌ 🟡 ✅ ✅ 🟡 🟡 ✅ ✅ ✅ ✅ ✅ ❌ ❌
SOFT_MAX_BACK ❌ ❌ 🟡 🟡 ❌ ❌ ❌ ❌ 🟡 ✅ ❌ ❌ ❌
SOLVE_TRI ❌ ❌ ✅ 🟡 🟡 ✅ ✅ ✅ 🟡 ✅ ✅ ❌ ❌
SQR ❌ ✅ ✅ ✅ 🟡 ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
SQRT ❌ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
SSM_CONV ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
SSM_SCAN ❌ ❌ ✅ ✅ ✅ ❌ ✅ 🟡 🟡 🟡 ✅ ❌ ❌
STEP ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
SUB ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
SUM ❌ 🟡 ✅ 🟡 ❌ 🟡 🟡 ❌ 🟡 🟡 🟡 ❌ ❌
SUM_ROWS ❌ ✅ ✅ 🟡 ❌ 🟡 ✅ 🟡 🟡 ✅ ✅ ❌ ❌
SWIGLU ❌ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ❌ ❌
SWIGLU_CLAMP ❌ ❌ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ❌ ❌ ❌ ❌
SWIGLU_OAI ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
TANH ❌ ✅ ✅ 🟡 🟡 🟡 ✅ 🟡 ✅ ✅ ✅ ❌ ❌
TIMESTEP_EMBEDDING ❌ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ✅ ❌ ❌ ❌
TOP_K ❌ ❌ ✅ ❌ ❌ ✅ ✅ ✅ ✅ 🟡 ✅ ❌ ❌
TOP_K_MASK ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌ ❌
TRI ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌
TRUNC ❌ ❌ ✅ 🟡 🟡 ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
UPSCALE ❌ 🟡 ✅ ✅ ❌ ❌ ✅ 🟡 ✅ ✅ ✅ ❌ ❌
XIELU ❌ ❌ ✅ ✅ ❌ ❌ ✅ ❌ ✅ ✅ ✅ ❌ ❌