Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ggml-cuda: provide static workspace for cuBLAS handles (#26574)

Browse Source 
* provide static workspace for cuBLAS handles

* account for concurrent streams when using GGML_CUDA_GRAPH_OPT

* drop cublas_handle overloads and remove direct cublasSetStream calls

* Update ggml/src/ggml-cuda/common.cuh

---------

Co-authored-by: Oliver Simons <osimons@nvidia.com>
This commit is contained in:
Alexander Heisler
2026-08-20 10:27:51 +03:00
committed by GitHub
co-authored by Oliver Simons
parent 929d47a391
commit d9b6be07d0
5 changed files with 32 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-cuda/ssm-scan.cu
-1
View File
@@ -632,7 +632,6 @@ static void ssm_scan_ssd_f32_cuda(
// Step 3: chunked SSD loop
// Per chunk: pre_matmul (incl. M) + 4 cuBLAS (CB, Y, S@C, state update) + scale_state
cublasHandle_t handle = ctx.cublas_handle();
CUBLAS_CHECK(cublasSetStream(handle, stream));
const float alpha_one = 1.0f;
const float beta_zero = 0.0f;
const float beta_one = 1.0f;