Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

cuda : relax tensor contiguity requirements for quantized concat (#25678)

Browse Source 
* cuda : relax tensor contiguity requirements for quantized concat

* tests : add test cases for non-contiguous quantized concat

* ggml : relax contiguity requirements for quantized concat

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
This commit is contained in:
fairydreaming
2026-07-15 13:36:32 +02:00
committed by GitHub
co-authored by Stanisław Szymczyk
parent c81029373d
commit a3e5b96ac5
4 changed files with 55 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-cuda/ggml-cuda.cu
+12 -2
View File
@@ -4816,13 +4816,23 @@ static bool ggml_backend_cuda_device_supports_op(ggml_backend_dev_t dev, const g
{
ggml_type src0_type = op->src[0]->type;
ggml_type src1_type = op->src[1]->type;
const int32_t dim = op->op_params[0];
return src0_type == src1_type &&
src0_type == op->type &&
(
(
ggml_is_quantized(src0_type) &&
ggml_is_contiguous(op->src[0]) &&
ggml_is_contiguous(op->src[1]) &&
(
(
dim == 3 &&
ggml_is_contiguous(op->src[0]) &&
ggml_is_contiguous(op->src[1])
) || (
dim != 3 &&
ggml_is_contiguous_to_3(op->src[0]) &&
ggml_is_contiguous_to_3(op->src[1])
)
) &&
op->src[0]->ne[0] % ggml_blck_size(src0_type) == 0 &&
op->src[1]->ne[0] % ggml_blck_size(src0_type) == 0
) || (