 Titaniumtown
|
6509138622
|
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779)
Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two
interleaved rounds, tg128:
qwen2.5-3B-Instruct Q4_K_M 154.18 -> 158.53 t/s +2.8%
gemma-2-2b-it Q4_K_M 162.45 -> 165.62 t/s +2.0%
llama-batched-bench on qwen2.5-3B, S_TG by batch size:
B=1 142.72 -> 147.57 t/s +3.4%
B=2 243.72 -> 268.26 t/s +10.1%
B=4 359.58 -> 398.02 t/s +10.7%
B=8 449.75 -> 505.63 t/s +12.4%
|
2026-08-14 02:26:23 -04:00 |
|
|
Titaniumtown
|
1ee1cd9bc6
|
sycl: fuse UNARY(silu|sigmoid|softplus) + MUL (#26411)
Measured on Arc Pro B70 (Battlemage), Qwen3.6-27B Q4_K_M, -fa on, f16 KV,
-b 2048 -ub 2048, llama-bench -r 3, three interleaved A/B rounds:
pp2048 1014.70 -> 1018.56 t/s (+0.38%, within run-to-run spread)
tg128 23.73 -> 23.86 t/s (+0.57%)
tg128 @ d4096 22.71 -> 22.86 t/s (+0.62%)
|
2026-08-13 11:41:38 +03:00 |
|