model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

* model: add Hy3 (hy_v3) architecture support

Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).

The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX, src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).

Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.

Co-Authored-By: charlie12345 <charlie12345@users.noreply.github.com>
Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Assisted-by: Claude Fable 5
This commit is contained in:
Satinder Grewal
2026-07-14 00:31:04 +02:00
committed by GitHub
co-authored by charlie12345 Piotr Wilkin
parent 6eddde06a4
commit 2969d6d15d
16 changed files with 882 additions and 4 deletions
+3
View File
@@ -1944,6 +1944,9 @@ static void test_role_markers_all_templates(testing & t) {
// MiniMax M2: ]~b]{user|ai}
{ "MiniMax-M2.jinja", "]~b]user", "]~b]ai" },
// HunYuan V3: <hy_User:opensource> / <hy_Assistant:opensource>
{ "tencent-Hy3.jinja", "<hy_User:opensource>", "<hy_Assistant:opensource>" },
// Nemotron Nano v2: <SPECIAL_11>{User|Assistant}; assistant marker
// is followed by a prefilled <think> block that gets included.
{ "NVIDIA-Nemotron-Nano-v2.jinja", "<SPECIAL_11>User", "<SPECIAL_11>Assistant" },