Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

sycl: fuse UNARY(silu|sigmoid|softplus) + MUL (#26411)

Browse Source 
Measured on Arc Pro B70 (Battlemage), Qwen3.6-27B Q4_K_M, -fa on, f16 KV,
-b 2048 -ub 2048, llama-bench -r 3, three interleaved A/B rounds:

  pp2048        1014.70 -> 1018.56 t/s   (+0.38%, within run-to-run spread)
  tg128         23.73 -> 23.86 t/s       (+0.57%)
  tg128 @ d4096 22.71 -> 22.86 t/s       (+0.62%)
This commit is contained in:
Titaniumtown
2026-08-13 11:41:38 +03:00
committed by GitHub
parent 8efbf65dbd
commit 1ee1cd9bc6
6 changed files with 280 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-sycl/fusion.hpp
+4 -2
View File
@@ -6,10 +6,12 @@
#include "common.hpp"
// Backend-side fusability test. `ops` names a candidate op sequence starting at cgraph node
// `node_idx`; the result is true only if ggml considers that subgraph fusable *and* the SYCL
// `node_idx`, and `unary_ops` the GGML_UNARY_OP each GGML_OP_UNARY in `ops` must carry, in
// order; the result is true only if ggml considers that subgraph fusable *and* the SYCL
// kernel which would service it accepts the tensors involved (types, shapes, contiguity).
//
// Lives in its own translation unit because it grows a branch per supported op sequence.
bool ggml_sycl_can_fuse(const ggml_cgraph * cgraph, int node_idx, std::initializer_list<enum ggml_op> ops);
bool ggml_sycl_can_fuse(const ggml_cgraph * cgraph, int node_idx, std::initializer_list<enum ggml_op> ops,
std::initializer_list<enum ggml_unary_op> unary_ops);
#endif // GGML_SYCL_FUSION_HPP