model : drop the redundant precision overrides in the Kolibri-1 router (#30865)
* model : drop the redundant precision overrides in the Kolibri-1 router Follow-up to the review in #30673: the router matmul works with the default precision, so both ggml_prec_set_* calls can go away. Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * cont : fix comment Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
This commit is contained in:
1 parent
b38960f520
commit
9df0b73c70
1 file changed
+1
-4
@@ -131,11 +131,8 @@ llama_model_kolibri1::graph::graph(const llama_model & model, const llm_graph_pa
|
||||
cur = build_norm(ffn_inp, layer.ffn_norm, NULL, LLM_NORM_RMS, il);
|
||||
cb(cur, "ffn_norm", il);
|
||||
|
||||
// The router runs in f32: the selection adds the bias to the logits, the weights
|
||||
// are the sigmoid of the unbiased logits, so the same logits feed both.
|
||||
// The selection adds the bias to the logits, the weights are the sigmoid of the unbiased logits, so the same logits feed both.
|
||||
ggml_tensor * logits = build_lora_mm(layer.ffn_gate_inp, cur);
|
||||
ggml_prec_set_acc(logits, GGML_PREC_F32);
|
||||
ggml_prec_set_src(logits, GGML_PREC_F32, 1);
|
||||
cb(logits, "ffn_moe_logits", il);
|
||||
|
||||
ggml_tensor * selection = ggml_add(ctx0, logits, layer.ffn_exp_probs_b);
|
||||
|
||||
Reference in new issue
Block a user