llama: env knob for the routing-ids readback skip

LLAMA_MOE_IDS_SKIP=0 forces the ids readback in sched_copy_experts (the
upstream path) and =log reports each skip on stderr, so validation can show
that the skip engages at prefill and that output is byte-identical with and
without it in one build. Read once (function-local statics); unset changes
nothing. Not set in production.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
LumpiastyandClaude Opus 5.5 committed 2026-10-25 00:31:25 +02:00
1 parent f5c0d8f28a
commit f664b0f20e
1 file changed
+11 -1
+11 -1
View File
@@ -2889,9 +2889,19 @@ bool llama_context::sched_copy_experts(ggml_backend_t backend, const ggml_tensor
// range. Copying an expert that no id selects cannot change the result:
// mul_mat_id only reads the rows the ids point at, so this stays exact.
// With the MoE cache on, cached experts are still copied device-to-device.
const bool all_experts_used = ggml_nelements(ids) >= 4*n_expert;
// validation/A-B knob: LLAMA_MOE_IDS_SKIP=0 always reads the ids back (the upstream path),
// =log reports every skip on stderr
static const char * skip_env = getenv("LLAMA_MOE_IDS_SKIP");
static const bool skip_off = skip_env != nullptr && strcmp(skip_env, "0") == 0;
static const bool skip_log = skip_env != nullptr && strcmp(skip_env, "log") == 0;
const bool all_experts_used = !skip_off && ggml_nelements(ids) >= 4*n_expert;
if (all_experts_used) {
if (skip_log) {
fprintf(stderr, "sched_copy_experts: ids readback skipped, n_ids=%" PRId64 " n_expert=%" PRId64 "\n",
ggml_nelements(ids), n_expert);
}
st.ids_data.clear();
st.used.assign(n_expert, true);
} else {