model: DSV4.1 flash (#31086)
* model : add DeepSeek-V4.1-Flash (deepseek41) * deepseek41 : remove MXFP8, convert the Engram table to BF16 * deepseek41 : compute the compressed-KV read layers once in the cache * deepseek41 : use get_cells() for the shared-history check, drop llama_kv_cache::seq_pos_tok_le * deepseek41 : gather the Engram rows in-graph like qwen4exp, drop the host gather * deepseek41 : keep the Engram hash constants in hparams.ple_*, one module per Engram layer * deepseek41 : drop the LLAMA_DSV41_KV_MIRROR toggle * deepseek41 : keep il last in build_hc_mixes * deepseek41 : sort the top-k rows before the sparse gather * tests : add deepseek41 to the Metal fusion baseline --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This commit is contained in:
1 parent
d327bf1f07
commit
74bca39500
39 files changed
+2798
-104
No files matched your search
@@ -316,6 +316,9 @@ static bool tensor_allows_quantization(const llama_model_quantize_params * param
|
||||
// do not quantize the i32 token-id -> expert-id routing table (DeepSeek-V4)
|
||||
quantize &= name.find("ffn_gate_tid2eid.weight") == std::string::npos;
|
||||
|
||||
// Engram gate is an elementwise weight (ggml_mul), not a matmul operand
|
||||
quantize &= name.find("engram_gate.weight") == std::string::npos;
|
||||
|
||||
// these are very small (e.g. 4x4)
|
||||
quantize &= name.find("altup") == std::string::npos;
|
||||
quantize &= name.find("laurel") == std::string::npos;
|
||||
|
||||
Reference in new issue
Block a user