Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

sycl: honor GGML_HINT_SRC0_IS_HADAMARD (#27298)

Browse Source 
Kernel is a port of `ggml-cuda/fwht.cu`

(us/run, median):
```
m x n x k           GEMM      FWHT   speedup
64 x    1 x  64     10.20     2.93     3.48x
64 x 2048 x  64     10.75     2.71     3.97x
128 x    1 x 128    10.33     2.88     3.59x
128 x   32 x 128     9.20     2.77     3.33x
128 x 2048 x 128    16.46     2.76     5.95x
256 x    1 x 256    10.19     2.77     3.68x
256 x 2048 x 256    16.69     3.41     4.89x
512 x 2048 x 512    54.16    12.89     4.20x
```
This commit is contained in:
Titaniumtown
2026-08-18 21:21:25 +08:00
committed by GitHub
parent afd439df1f
commit 0882c7bc89
3 changed files with 144 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-sycl/ggml-sycl.cpp
+13
View File
@@ -58,6 +58,7 @@
#include "ggml-sycl/backend.hpp"
#include "ggml-sycl/common.hpp"
#include "ggml-sycl/element_wise.hpp"
#include "ggml-sycl/fwht.hpp"
#include "ggml-sycl/gemm.hpp"
#include "ggml-sycl/getrows.hpp"
#include "ggml-sycl/norm.hpp"
@@ -4473,6 +4474,18 @@ static bool can_use_mul_mat_vec_q(const ggml_tensor * src0, const ggml_tensor *
static void ggml_sycl_mul_mat(ggml_backend_sycl_context & ctx, const ggml_tensor * src0, const ggml_tensor * src1, ggml_tensor * dst) {
scope_op_debug_print scope_dbg_print(__func__, dst, /*num_src=*/2);
// Handle HADAMARAD hint given from further up the pipeline and pass it to the correct
// kernel.
//
// The op check is not redundant: this backend also routes MUL_MAT_ID through here with a
// stack copy of dst, which carries MUL_MAT_ID's own op_params. ggml_mul_mat_set_hint()
// asserts GGML_OP_MUL_MAT for the same reason.
if (dst->op == GGML_OP_MUL_MAT && ggml_get_op_params_i32(dst, 1) == GGML_HINT_SRC0_IS_HADAMARD &&
ggml_sycl_op_fwht(ctx, src1, dst)) {
return;
}
const bool split = ggml_backend_buffer_is_sycl_split(src0->buffer);
int64_t min_compute_capability = INT_MAX;