Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

sycl: fuse the gated-delta-net state writeback cpy (#26643)

Browse Source 
Port of https://github.com/ggml-org/llama.cpp/pull/23940.

Arc Pro B70, Qwen 3.6 27B Q4_K - Medium (48 of its 64 blocks run
gated_delta_net), -ngl 99 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 2048,
interleaved A/B passes of r=3:

  tg128           23.91 / 23.90 / 23.90 -> 24.19 / 24.17 / 24.20   +1.2%
  tg128 (rebuild) 23.81 / 23.81         -> 24.09 / 24.10           +1.2%
  pp2048        1050.8  / 1053.9        -> 1053.8 / 1054.5         flat
  2 seqs, tg128    32.73 / 32.75        ->  33.11 / 33.10          +1.1%
This commit is contained in:
Titaniumtown
2026-08-14 09:00:36 +03:00
committed by GitHub
parent 2bacf9ea5c
commit 3d93885352
3 changed files with 143 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-sycl/gated_delta_net.hpp
+10
View File
@@ -5,5 +5,15 @@
#include "common.hpp"
#include "ggml.h"
// fused-kernel recurrent-state output; strides in elements (per-seq stride is always D, set in-kernel)
struct ggml_sycl_gated_delta_net_fused_cache {
float * data; // rollback slot 0
int64_t slot_stride; // between rollback slots (0 when K==1)
};
void ggml_sycl_op_gated_delta_net(ggml_backend_sycl_context & ctx, ggml_tensor * dst);
void ggml_sycl_gated_delta_net(ggml_backend_sycl_context & ctx, ggml_tensor * dst);
// same op, but writes the snapshot(s) into the cache instead of dst (see ggml_sycl_try_gdn_cache_fusion)
void ggml_sycl_op_gated_delta_net_fused_cache(ggml_backend_sycl_context & ctx, ggml_tensor * dst,
ggml_sycl_gated_delta_net_fused_cache cache);