32b741c336
* [SYCL] F16 (default) Flash Attention with XMX engine via oneDNN graph API; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80k * [SYCL] Address review on FA oneDNN path. Result: llama-bench---pp512; 32% increase with fa1; llama-perplexity---0.11% difference; tested model: mradermacher/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf * PR-25222 revision v2: addressed audits * [SYCL] flash-attn oneDNN SDPA KV F16 rev 3.0: add BMG gate + multi-device sync. Narrow the scrope of this PR to Battlemage only (bmg; Xe2). Other archs (e.g., alchemist) fall back to existing FA kernel. When device_count >1, apply stream -> wait_and_throw(), validated working path for multi-gpu sync fix by @maxious. Co-authored-by: maxious <81432+maxious@users.noreply.github.com> * updated comment on bmg gate, noted the issue --------- Co-authored-by: scientist3 <scientist.3@users.noreply.github.com> Co-authored-by: hmscider <hmscider@users.noreply.github.com> Co-authored-by: maxious <81432+maxious@users.noreply.github.com>
15 lines
603 B
C++
15 lines
603 B
C++
#ifndef GGML_SYCL_FATTN_ONEDNN_HPP
|
|
#define GGML_SYCL_FATTN_ONEDNN_HPP
|
|
|
|
#include "common.hpp"
|
|
|
|
// Static-only check: fused-XMX oneDNN Graph SDPA path==flash-attn op
|
|
// (f16 KV, no softcap/ALiBi, single stream, tuned head_dim, prefill-sized q.)
|
|
bool ggml_sycl_flash_attn_ext_onednn_supported(const ggml_tensor * dst);
|
|
|
|
// Run flash attention through oneDNN's fused xmx SDPA
|
|
// execute the cached SDPA partition, write the f32 dst. Falls back to the TILE kernel on any failure.
|
|
void ggml_sycl_flash_attn_ext_onednn(ggml_backend_sycl_context & ctx, ggml_tensor * dst);
|
|
|
|
#endif // GGML_SYCL_FATTN_ONEDNN_HPP
|