Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[SYCL] supprt Flash Attention for fp32/fp16/Q4/Q5/Q8 (#20190)

Browse Source 
* support flash-attention for fp32/fp16/Q4/Q5/Q8

* rm warining

* update for JIT
This commit is contained in:
Neo Zhang
2026-03-08 12:00:07 +08:00
committed by GitHub
parent c5a778891b
commit 213c4a0b81
65 changed files with 20091 additions and 8593 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-sycl/template-instances/fattn-vec-instance-q5_1-q5_1.cpp
+7
View File
@@ -0,0 +1,7 @@
// This file has been autogenerated by generate_cu_files.py, do not edit manually.
#include "../fattn-vec.hpp"
DECL_FATTN_VEC_CASE( 64, GGML_TYPE_Q5_1, GGML_TYPE_Q5_1);
DECL_FATTN_VEC_CASE(128, GGML_TYPE_Q5_1, GGML_TYPE_Q5_1);
DECL_FATTN_VEC_CASE(256, GGML_TYPE_Q5_1, GGML_TYPE_Q5_1);