Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

Browse Source 
* model: add Hy3 (hy_v3) architecture support

Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).

The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX, src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).

Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.

Co-Authored-By: charlie12345 <charlie12345@users.noreply.github.com>
Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Assisted-by: Claude Fable 5
This commit is contained in:
Satinder Grewal
2026-07-14 00:31:04 +02:00
committed by GitHub
co-authored by charlie12345 Piotr Wilkin
parent 6eddde06a4
commit 2969d6d15d
16 changed files with 882 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama-model.cpp
+4 -1
View File
@@ -262,6 +262,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params
return new llama_model_hunyuan_vl(params);
case LLM_ARCH_HUNYUAN_DENSE:
return new llama_model_hunyuan_dense(params);
case LLM_ARCH_HY_V3:
return new llama_model_hy_v3(params);
case LLM_ARCH_SMOLLM3:
return new llama_model_smollm3(params);
case LLM_ARCH_OPENAI_MOE:
@@ -2169,7 +2171,7 @@ llama_memory_i * llama_model::create_memory(const llama_memory_params & params,
filter = [&](uint32_t il) { return il >= hparams.n_layer(); };
}
if (arch == LLM_ARCH_STEP35 && hparams.n_layer_nextn > 0) {
if ((arch == LLM_ARCH_STEP35 || arch == LLM_ARCH_HY_V3) && hparams.n_layer_nextn > 0) {
if (params.ctx_type == LLAMA_CONTEXT_TYPE_MTP) {
filter = [&](uint32_t il) { return il >= hparams.n_layer(); };
} else {
@@ -2525,6 +2527,7 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {
case LLM_ARCH_JAIS2:
case LLM_ARCH_OPENAI_MOE:
case LLM_ARCH_HUNYUAN_DENSE:
case LLM_ARCH_HY_V3:
case LLM_ARCH_LFM2:
case LLM_ARCH_LFM2MOE:
case LLM_ARCH_SMALLTHINKER: