llama: env knob LLAMA_LOOP_FULL_COPY=0 to skip upstream's inkling2_loop full-copy guard (#32807)

Upstream #32807 makes sched_copy_experts return false for inkling2_loop,
so the scheduler copies every expert of a host weight it shares across the
loop passes (the fix for #32755). With the F13 per-pass views each pass
has its own device copy and its own ids, so the partial copy (routing
readback below the F1 skip threshold, 32-63-token ubatches on
Inkling-2-Mini) is correct again and the guard only costs uploads.
LLAMA_LOOP_FULL_COPY=0 skips the guard for an A/B; the default keeps
upstream's behaviour. With LLAMA_LOOP_HOST_VIEWS=0 the knob brings #32755
back, do not combine them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
This commit is contained in:
LumpiastyandClaude Opus 5.5 committed 2026-11-22 20:41:04 +01:00
1 parent e7e907b7c3
commit 5f88124b38
1 file changed
+8 -1
+8 -1
View File
@@ -3232,7 +3232,14 @@ bool llama_context::sched_copy_experts(ggml_backend_t backend, const ggml_tensor
// Later loop passes can select different experts from the same device copy.
// TODO: copy only the experts needed by each pass (see #32807 and #32809).
if (lctx->model.arch == LLM_ARCH_INKLING2_LOOP) {
// [fork F13] with the per-pass host-weight views every pass has its own copy, so the partial
// copy is correct again; LLAMA_LOOP_FULL_COPY=0 skips this guard (A/B knob, only correct with
// the views on, LLAMA_LOOP_HOST_VIEWS unset or 1)
static const bool loop_full_copy = [] {
const char * env = getenv("LLAMA_LOOP_FULL_COPY");
return env == nullptr || strcmp(env, "0") != 0;
}();
if (loop_full_copy && lctx->model.arch == LLM_ARCH_INKLING2_LOOP) {
return false;
}