ggml-vulkan: skip the concat transpose fast path below one tile

The tiled-transpose fast path in ggml_vk_concat is a large prefill win but a
loss at decode. On a hybrid model like Qwen3.6-35B (gated delta-net on 30 of
40 layers) build_conv_state emits a transposed concat per recurrent layer, so
the fast path runs 30 times per decoded token.

At one token the transposed source is a single column, so there is no
uncoalesced stride left to fix, but the path still costs two dispatches and an
unconditional ggml_vk_sync_buffers - a full pipeline barrier that serializes
the command stream. Measured -17% tg256 on RX 580; the barrier, not the
half-empty transpose dispatch, is nearly all of it.

Require the transposed source to be at least one TILE_DIM wide. That source is
qkv_mixed transposed, so ne[0] is the ubatch token count: prefill keeps the
fast path, decode and small speculative drafts take the generic one.

pp8192 +11% retained, tg256 back to parity, generation byte-identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
This commit is contained in:
2026-09-10 17:06:01 +02:00
co-authored by Claude Opus 5
parent f9d41c711f
commit c233ce9b51
+2
View File
@@ -13579,6 +13579,8 @@ static void ggml_vk_concat(ggml_backend_vk_context * ctx, vk_context& subctx, co
const uint32_t ts = ggml_type_size(s->type);
return s->nb[1] == ts // dim1 innermost
&& s->nb[0] == (size_t) s->ne[1] * ts // consistent 2D transpose
&& s->ne[0] >= 32 // at least one full transpose tile (TILE_DIM); at 1 token the
// tiled path is pure overhead, the generic copy is faster
&& s->ne[2] == 1 && s->ne[3] == 1;
};
const uint32_t dst_ts = ggml_type_size(dst->type);