Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

Browse Source 
* model: add Hy3 (hy_v3) architecture support

Adds Tencent Hunyuan 3 (HF architecture HYV3ForCausalLM, GGUF arch
hy_v3): a MoE decoder stack with per-head Q/K RMSNorm, a sigmoid
router with expert selection bias, an always-active ungated shared
expert, and leading dense block(s) (first_k_dense_replace).

The base implementation is ported from charlie12345's fork
(https://github.com/charlie12345/ROCmFPX, src/models/hyv3.cpp),
adapted to current mainline APIs (hparams.n_layer(), build_qkv,
build_moe_ffn with fused gate_up + scale tensors, output_s).

Note: blk.N.exp_probs_b is stored without a .bias suffix for
compatibility with existing hy_v3 GGUFs produced by that fork.

Co-Authored-By: charlie12345 <charlie12345@users.noreply.github.com>
Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Assisted-by: Claude Fable 5
This commit is contained in:
Satinder Grewal
2026-07-14 00:31:04 +02:00
committed by GitHub
co-authored by charlie12345 Piotr Wilkin
parent 6eddde06a4
commit 2969d6d15d
16 changed files with 882 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/chat-auto-parser.h
+4 -3
View File
@@ -192,9 +192,10 @@ struct tool_format_analysis {
};
struct tool_function_analysis {
std::string name_prefix; // e.g., "<function=", "\"name\": \"", "functions."
std::string name_suffix; // e.g., ">", "\"", ":0"
std::string close; // e.g., "</function>", "" (for tag-based)
std::string name_prefix; // e.g., "<function=", "\"name\": \"", "functions."
std::string name_suffix; // e.g., ">", "\"", ":0"
std::string args_separator; // e.g., "<tool_sep>" (marker between function name and arguments)
std::string close; // e.g., "</function>", "" (for tag-based)
};
struct tool_arguments_analysis {