Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) (#25545)

Browse Source 
* cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel)

* chore : remove indentation of #pragma unroll

* cuda : remove unnecessary kernel template declarations

* cuda : add WARPS_PER_BLOCK and K_VECS_PER_BLOCK template parameters in lightning indexer kernels to avoid duplication of constants.

* cuda : relax MMA architecture requirements to Turing in lightning indexer implementation

* chore : renamed variables

* chore : rename ggml_cuda_op_lightning_indexer() to ggml_cuda_lightning_indexer()

* chore : TODO for AMD rocWMMA

* chore : whitespace formatting

* chore : another variable rename to fix problems caused by shadowing

* chore : yet another rename, this time uppercased all constants

* cuda : added alignment checks for Q and K tensors in lightning indexer implementation

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
This commit is contained in:
fairydreaming
2026-07-15 19:57:52 +02:00
committed by GitHub
co-authored by Stanisław Szymczyk
parent aff6eb6e75
commit 3b53219361
3 changed files with 598 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-cuda/lightning-indexer.cuh
+4
View File
@@ -0,0 +1,4 @@
#include "common.cuh"
void ggml_cuda_lightning_indexer(ggml_backend_cuda_context & ctx, ggml_tensor * dst);
bool ggml_cuda_lightning_indexer_supported(int device, const ggml_tensor * dst);