model : add Ripple (ricursiveai) (#31692)
* feat : added ricursiveai ripple support * fix : mandatiry SWA, scale through helper, comments back * fix : metal * model : make ricursive RoPE/NoPE explicit and validate the layer pattern Write attention.rope_pattern and read it as optional, falling back to the sliding window pattern so existing GGUFs keep loading. Refuse a sliding_window_pattern that does not cover every layer, a model with no sliding layer, and a gate or value residual present in only some layers. The converter derives layer_types when the config omits it, ignores a stale hybrid_layer_pattern and drops tensors of features the config disables. Assisted-by: Claude Opus 5.5 * model : make ricursive swiglu_clamp per-layer Store swiglu_clamp as a per-layer array, read it with get_key_or_arr so the scalar in existing GGUFs still applies to every layer, and refuse a negative or non-finite value. Add a test-llama-archs fixture with mixed per-layer clamps and no rope_pattern, so the saver roundtrip checks both. Assisted-by: Claude Opus 5.5 * model : use the ricursive per-tensor scales and RoPE parameters Pass the attn_gate and output weight scales to build_lora_mm, take ext_factor and attn_factor from the graph instead of hardcoding them, and copy rope_freq_scale_train to the SWA layers instead of pinning it to 1. Cite the reference lines for the value residual and the NoPE layers. Assisted-by: Claude Opus 5.5 * convert : drop the ricursive chat_template.jinja override The precedence between tokenizer_config.json and chat_template.jinja is to be changed in SpecialVocab for all models instead. Until then a new conversion stores the short tokenizer_config.json template; GGUFs that already carry the tool-aware template are not affected. Assisted-by: Claude Opus 5.5 * fix : remove NVFP4 checkpoint quantization block * match #31733 refactor --------- Co-authored-by: Piotr Wilkin <ilintar@gmail.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
This commit is contained in:
15 files changed
+406
-3
No files matched your search
@@ -294,6 +294,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params
|
||||
return new llama_model_afmoe(params);
|
||||
case LLM_ARCH_LAGUNA:
|
||||
return new llama_model_laguna(params);
|
||||
case LLM_ARCH_RICURSIVE:
|
||||
return new llama_model_ricursive(params);
|
||||
case LLM_ARCH_ERNIE4_5:
|
||||
return new llama_model_ernie4_5(params);
|
||||
case LLM_ARCH_ERNIE4_5_MOE:
|
||||
@@ -1548,6 +1550,7 @@ void llama_model_base::load_hparams(llama_model_loader & ml) {
|
||||
|
||||
std::fill(hparams.swiglu_clamp_exp.begin(), hparams.swiglu_clamp_exp.end(), 0.0f);
|
||||
std::fill(hparams.swiglu_clamp_shexp.begin(), hparams.swiglu_clamp_shexp.end(), 0.0f);
|
||||
std::fill(hparams.swiglu_clamp.begin(), hparams.swiglu_clamp.end(), 0.0f);
|
||||
|
||||
ml.get_key_or_arr(LLM_KV_FEED_FORWARD_LENGTH, hparams.n_ff_arr, hparams.n_layer_all, false);
|
||||
ml.get_key_or_arr(LLM_KV_ATTENTION_HEAD_COUNT, hparams.n_head_arr, hparams.n_layer_all, false);
|
||||
@@ -3709,6 +3712,7 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {
|
||||
case LLM_ARCH_QWEN3MOE:
|
||||
case LLM_ARCH_LLADA_MOE:
|
||||
case LLM_ARCH_RND1:
|
||||
case LLM_ARCH_RICURSIVE:
|
||||
case LLM_ARCH_OLMO2:
|
||||
case LLM_ARCH_OLMOE:
|
||||
case LLM_ARCH_PHI2:
|
||||
|
||||
Reference in new issue
Block a user