vulkan: skip maxStorageBufferRange pre-check for non-compute tensors (#31114)

* vulkan: skip maxStorageBufferRange pre-check for non-compute tensors

Pre-allocated leaf tensors and views (GGML_OP_NONE / GGML_OP_VIEW) are never
executed by the backend, but were still rejected by the descriptor-size
pre-check in ggml_backend_vk_device_supports_op when larger than
maxStorageBufferRange on devices without shader_64b_indexing. The scheduler
then aborts with 'pre-allocated tensor (...) in a buffer (Vulkan0) that cannot
run the operation (NONE)' - e.g. the MoE expert cache banks (llama-moe-cache)
of a 256-expert model with --moe-cache-mib on GCN GPUs, where a whole bank
(2.4 GB) exceeds the limit while the consuming ops only access small
per-step views.

Skip the size pre-check for non-compute ops (NONE and VIEW); the buffer type
allocation limit still bounds the actual allocation, and compute ops keep
validating their sources.

Tested on AMD Radeon Pro VII (gfx906, no shader_64b_indexing):
- before: GGML_ABORT 'pre-allocated tensor (moe_cache.0.blk.0.ffn_down_exps.weight) ... cannot run the operation (NONE)' with llama-cli -ngl 99 --cpu-moe --moe-cache-mib 6144 (Cyber-Tiel-Coder-35B-A3B, qwen35moe, 256 experts per layer)
- after: loads and generates at 8.1 t/s (vs 0.6 t/s without the cache)
- regression: -ncmoe 16 path unchanged (19.2 t/s), qwen3moe/qwen35moe without cache unchanged

Fixes #30697

* vulkan: use ggml_op_is_empty for the non-compute size-check skip

---------

Co-authored-by: FamousMonsterr <FamousMonsterr@users.noreply.github.com>
This commit is contained in:
FamousMonsterrandFamousMonsterr authored and GitHub committed 2026-11-03 08:53:04 +01:00
1 parent b74298c109
commit fac3a894fc
1 file changed
+4 -3
+4 -3
View File
@@ -15550,13 +15550,14 @@ static bool ggml_backend_vk_device_supports_op(ggml_backend_dev_t dev, const ggm
}
return true;
};
// reject any tensors larger than the max buffer size
// non-compute ops are never executed here; their size is bounded by the buffer allocation limit
const bool non_compute = ggml_op_is_empty(op->op);
for (int i = 0; i < GGML_MAX_SRC; i++) {
if (op->src[i] && !tensor_size_supported(ggml_nbytes(op->src[i]))) {
if (op->src[i] && !non_compute && !tensor_size_supported(ggml_nbytes(op->src[i]))) {
return false;
}
}
if (!tensor_size_supported(ggml_nbytes(op))) {
if (!non_compute && !tensor_size_supported(ggml_nbytes(op))) {
return false;
}