Commit Graph
8 Commits
Author SHA1 Message Date
Titaniumtown 1ee1cd9bc6 sycl: fuse UNARY(silu|sigmoid|softplus) + MUL (#26411)
Measured on Arc Pro B70 (Battlemage), Qwen3.6-27B Q4_K_M, -fa on, f16 KV,
-b 2048 -ub 2048, llama-bench -r 3, three interleaved A/B rounds:

  pp2048        1014.70 -> 1018.56 t/s   (+0.38%, within run-to-run spread)
  tg128         23.73 -> 23.86 t/s       (+0.57%)
  tg128 @ d4096 22.71 -> 22.86 t/s       (+0.62%)
2026-08-13 11:41:38 +03:00
Titaniumtown f8e30266d2 sycl: coalesce the ssm_conv window loads (#26612)
test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against
master, 6 reps, us/run:

  ne_a=[515,3328,1,1] ne_b=[4,3328,1,1]   n_t=512     97.68 -> 52.95   1.85x
  ne_a=[937,8192,1,1] ne_b=[4,8192,1,1]   n_t=934    516.16 -> 276.13  1.87x
  ne_a=[4,3328,1,1]   ne_b=[4,3328,1,1]   n_t=1        2.73 -> 2.71    flat

llama-bench on qwen35 27B Q4_K - Medium (48 of its 64 blocks run ssm_conv),
-ngl 99 -fa 1 -ctk f16 -ctv f16, interleaved passes of r=3:

  -b 2048 -ub 2048  pp2048  1045.1 / 1043.5 / 1043.7 -> 1069.5 / 1066.3 / 1065.9  +2.2%
  -b 2048 -ub 512   pp2048   771.8 /  772.7          ->  785.5 /  786.6           +1.8%
  -b 2048 -ub 512   tg128     23.81 /  23.88         ->   23.87 /  23.86          flat
2026-08-07 21:09:32 +03:00
Titaniumtown 6b5c2efb4e sycl: *glu flat path (#26354)
* tests: add SWIGLU perf cases

perf mode had no GLU coverage. Adds SWIGLU at 17408 columns, 512 and
2048 tokens, f16 and f32, with the operands both fused and split.

* sycl: consolidate fused-GLU kernels

They differed only in which op_* they called, so take the op as an argument and share a common launcher.
Their block sizes were all 256, so launch geometry is unchanged;
SYCL_GELU_BLOCK_SIZE and SYCL_SILU_BLOCK_SIZE lose their last users so are dropped.

* sycl: contiguous fast path for the fused GLU ops

o0 == n and o1 == n collapse the de-interleave index math to the
identity, so dispatch a flat kernel in that case. It fires for
ggml_glu_split with packed operands; a fused [gate|up] tensor keeps the
strided path. test-backend-ops perf -o SWIGLU on an Arc Pro B70: split
+14% f16 and +4% f32, fused unchanged.
2026-08-07 08:24:40 +03:00
Titaniumtown 6c8dcaa7ae sycl: parallelize the non-contiguous concat kernel (#25852)
* sycl: parallelize the non-contiguous concat kernel

Launch geometry only: the non-contiguous concat kernel launched a single-lane
work-group (1, 1, 1), now it will launch a (1, 1, SYCL_CONCAT_BLOCK_SIZE) one.

SYCL_CONCAT_BLOCK_SIZE is defined in `ggml/src/ggml-sycl/presets.hpp`.

llama-bench (Arc Pro B70, Qwen3.6-27B-UD-Q4_K_XL, -fa on, q8_0 KV),
on top of upstream master: pp2048 920 -> 1006 t/s (+9.4%)

* sycl: cap non-contiguous concat block at ne0

* sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)

* Revert "sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)"

This reverts commit 2709909e790d298cc0746950caff16ad8ecf9d6f.
2026-08-04 10:08:05 +08:00
Titaniumtown 1553725965 sycl: fuse RMS_NORM + MUL (#26015) 2026-07-31 09:17:53 +03:00
Titaniumtown 11b068d066 sycl: contiguous fast path + 32-bit index math for unary elementwise ops (#25946)
* sycl: contiguous fast path + 32-bit index math for unary elementwise ops

* sycl: use fastdiv for elementwise index math
2026-07-29 15:16:57 +03:00
Titaniumtown 8e8681e0e2 sycl(build): parallelize ocloc invocations (#25903) 2026-07-27 15:33:11 +03:00
Titaniumtown c1063ac9d7 sycl: set fattn_vec_nthreads to 256 for Battlemage (#25205)
Currently detects lunarlake + battlemage / xe2 and
sets the value to 256.

Keeps default at 128, Intel's ARC Alchemist's prefered value.
2026-07-14 12:00:00 +03:00