models : consolidate nextn row cropping into shared helpers (#30017)

* mimo2 : always emit h_nextn

the other nextn-capable models set it unconditionally

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD

* models : consolidate nextn row cropping into shared helpers

- replace the duplicated crop conditions and the per-model flags (narrow_early,
  crop_before_ffn, crop_last_layer, emit_h_nextn) with two helpers on llm_graph_context:
  crop_before_nextn() / crop_after_nextn()
- models that only tested embeddings_nextn_masked now share the same condition, so they
  crop the last layer before the nextn capture whenever extraction is off
- t_h_nextn is now set unconditionally in mimo2, qwen4exp and deepseek4 (as in the other
  nextn-capable models); host-side reads stay gated by cparams.embeddings_nextn

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD
This commit is contained in:
Georgi Gerganov authored and GitHub committed 2026-10-06 14:08:54 +03:00
1 parent d7a695ef67
commit f0c41e0168
20 files changed
+59 -62

No files matched your search

+7 -10
View File
@@ -101,9 +101,7 @@ llama_model_mimo2::graph::graph(const llama_model & model, const llm_graph_param
ggml_tensor * inp_out_ids = build_inp_out_ids();
const float v_scale = hparams.f_attn_value_scale;
const bool emit_h_nextn = cparams.embeddings_nextn;
const bool extract_final_inp = (size_t) n_layer < cparams.embeddings_layer_inp.size() && cparams.embeddings_layer_inp[n_layer];
const bool crop_last_layer = inp_out_ids && (!emit_h_nextn || cparams.embeddings_nextn_masked) && !extract_final_inp;
for (int il = 0; il < n_layer; ++il) {
res->t_layer_inp[il] = inpL;
@@ -185,7 +183,7 @@ llama_model_mimo2::graph::graph(const llama_model & model, const llm_graph_param
}
}
if (il == n_layer - 1 && crop_last_layer) {
if (il == n_layer - 1 && crop_before_nextn(inp_out_ids) && !extract_final_inp) {
cur = ggml_get_rows(ctx0, cur, inp_out_ids);
inpSA = ggml_get_rows(ctx0, inpSA, inp_out_ids);
}
@@ -236,18 +234,17 @@ llama_model_mimo2::graph::graph(const llama_model & model, const llm_graph_param
cur = inpL;
if (extract_final_inp) {
res->t_layer_inp[n_layer] = cur;
if (inp_out_ids && (!emit_h_nextn || cparams.embeddings_nextn_masked)) {
if (crop_before_nextn(inp_out_ids)) {
cur = ggml_get_rows(ctx0, cur, inp_out_ids);
}
}
if (emit_h_nextn) {
cb(cur, "h_nextn", -1);
res->t_h_nextn = cur;
// pre-norm state for MTP; set even when extraction is off
cb(cur, "h_nextn", -1);
res->t_h_nextn = cur;
if (!cparams.embeddings_nextn_masked && inp_out_ids) {
cur = ggml_get_rows(ctx0, cur, inp_out_ids);
}
if (crop_after_nextn(inp_out_ids)) {
cur = ggml_get_rows(ctx0, cur, inp_out_ids);
}
cur = build_norm(cur,