llama: env knob LLAMA_LOOP_FULL_COPY=0 to skip upstream's inkling2_loop full-copy guard (#32807)
Upstream #32807 makes sched_copy_experts return false for inkling2_loop, so the scheduler copies every expert of a host weight it shares across the loop passes (the fix for #32755). With the F13 per-pass views each pass has its own device copy and its own ids, so the partial copy (routing readback below the F1 skip threshold, 32-63-token ubatches on Inkling-2-Mini) is correct again and the guard only costs uploads. LLAMA_LOOP_FULL_COPY=0 skips the guard for an A/B; the default keeps upstream's behaviour. With LLAMA_LOOP_HOST_VIEWS=0 the knob brings #32755 back, do not combine them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
This commit is contained in:
1 parent
e7e907b7c3
commit
5f88124b38
1 file changed
+8
-1
@@ -3232,7 +3232,14 @@ bool llama_context::sched_copy_experts(ggml_backend_t backend, const ggml_tensor
|
||||
|
||||
// Later loop passes can select different experts from the same device copy.
|
||||
// TODO: copy only the experts needed by each pass (see #32807 and #32809).
|
||||
if (lctx->model.arch == LLM_ARCH_INKLING2_LOOP) {
|
||||
// [fork F13] with the per-pass host-weight views every pass has its own copy, so the partial
|
||||
// copy is correct again; LLAMA_LOOP_FULL_COPY=0 skips this guard (A/B knob, only correct with
|
||||
// the views on, LLAMA_LOOP_HOST_VIEWS unset or 1)
|
||||
static const bool loop_full_copy = [] {
|
||||
const char * env = getenv("LLAMA_LOOP_FULL_COPY");
|
||||
return env == nullptr || strcmp(env, "0") != 0;
|
||||
}();
|
||||
if (loop_full_copy && lctx->model.arch == LLM_ARCH_INKLING2_LOOP) {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user