Commit Graph
96 Commits
Author SHA1 Message Date
Xuan-Son Nguyen 351025ec53 mtmd: add kimi K3.1 vision (#31980)
* mtmd: support kimi k3.1 vision

* fix preproc (both k2.5 and k3.1) and projector mlp

* small renaming
2026-11-10 11:28:47 +01:00
Xuan-Son NguyenandPascal 9f0e5098a5 mtmd: support MageVL-Cowork model (#31819)
* mtmd: support MageVL-Cowork model

* mtmd: keep magevl rope_freqs at model level and check its shape

The rope factors are shared by every layer, so they live in clip_model
instead of a pointer copied into each clip_layer. The loader now checks
that v.rope_freqs is F32 with d_head elements and that d_head splits
into the 4:6:6 pairs the graph expects.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-07 19:36:44 +01:00
usrlocalben bcb0152876 model: add Kimi-K3.1-Pro text model (#31779)
* model: add Kimi-K3.1-Pro text model

K3 with the full-attention MLA layers replaced by DSA: on this checkpoint 24
of the 93 layers carry a lightning indexer (32 heads x 128, top_k 2048, no
rope). The KDA (linear-attention) layers and the rest of the K3 skeleton -
cross-layer residuals, latent MoE, situ, MLA output gate, full-rank KDA gate -
are unchanged, so this reuses the K3 graph and converter and adds the indexer.

Indexer:
  - the selection is an additive mask over the whole cache, not a gather of
    the top-k rows, so it reuses the existing sparse flash attention path;
    off that path the mask stays correct but still costs a kq over the cache
  - the indexer key LayerNorm takes its eps from the model's rms_norm_eps,
    because dsa_config declares no eps of its own

Graph and cache:
  - new arch with four indexer tensors per DSA layer, and a hybrid memory
    that keeps a K-only MLA cache for those layers, recurrent state for the
    KDA layers, and the indexer key cache
  - the graph node budget shares the K3 entry

Conversion:
  - KimiK31Model(KimiK3Model), picked by KimiK31ForConditionalGeneration
  - dsa_config becomes attention.indexer.{head_count,key_length,top_k,types}
    plus the indexer LayerNorm eps; the layer list is 1-indexed like
    linear_attn_config's, and the converter asserts that it covers exactly
    the full-attention layers
  - the routed experts stay MXFP4 and are repacked as in K3, not dequantized
  - the vision tower and mm_projector are skipped: text only for now
  - the arch key follows the checkpoint's own model_type ("kimi_k31") and the
    loader registers KimiK31ForConditionalGeneration; the release name carries
    a -Pro suffix that nothing in the config or the tensor names uses

Quantisation: the indexer's q_b, k and proj are kept out of the quant
overrides, since the top-k selection is sensitive to their precision.

Verified on an 8-layer slice against the HF checkpoint: 38 tensors checked
exactly, the indexer metadata against the config, and the selection against
full attention - identical PPL at top_k beyond the context length, divergent
once top_k is forced below it. Arch test fixture and the full arch suite pass.

* converter: avoid clutter in the k3 base

* narrow comment to match model

* update metal fusion expectations

* restore

* sync with PR 31733 refactoring
2026-11-06 23:42:46 +01:00
fecdc81eec model : add Ripple (ricursiveai) (#31692)
* feat : added ricursiveai ripple support

* fix : mandatiry SWA, scale through helper, comments back

* fix : metal

* model : make ricursive RoPE/NoPE explicit and validate the layer pattern

Write attention.rope_pattern and read it as optional, falling back to the
sliding window pattern so existing GGUFs keep loading. Refuse a
sliding_window_pattern that does not cover every layer, a model with no
sliding layer, and a gate or value residual present in only some layers.
The converter derives layer_types when the config omits it, ignores a stale
hybrid_layer_pattern and drops tensors of features the config disables.

Assisted-by: Claude Opus 5.5

* model : make ricursive swiglu_clamp per-layer

Store swiglu_clamp as a per-layer array, read it with get_key_or_arr so the
scalar in existing GGUFs still applies to every layer, and refuse a negative
or non-finite value. Add a test-llama-archs fixture with mixed per-layer
clamps and no rope_pattern, so the saver roundtrip checks both.

Assisted-by: Claude Opus 5.5

* model : use the ricursive per-tensor scales and RoPE parameters

Pass the attn_gate and output weight scales to build_lora_mm, take
ext_factor and attn_factor from the graph instead of hardcoding them, and
copy rope_freq_scale_train to the SWA layers instead of pinning it to 1.
Cite the reference lines for the value residual and the NoPE layers.

Assisted-by: Claude Opus 5.5

* convert : drop the ricursive chat_template.jinja override

The precedence between tokenizer_config.json and chat_template.jinja is to
be changed in SpecialVocab for all models instead. Until then a new
conversion stores the short tokenizer_config.json template; GGUFs that
already carry the tool-aware template are not affected.

Assisted-by: Claude Opus 5.5

* fix : remove NVFP4 checkpoint quantization block

* match #31733 refactor

---------

Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-05 19:55:15 +01:00
45b6604b20 model: Add support for the xing4_0 (#31589)
* model: add support for Xing4.0

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/xing4_0.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* fix: rename model file to xing4-0.cpp to match CI expectations

* model: rename xing4_0.cpp to xing4-0.cpp

* lint: fix E128 continuation indent in conversion/xing.py

* tests: add xing4_0 fusion baseline for Metal

* reduce comments

---------

Co-authored-by: shixq7 <shixq7@chinatelecom.cn>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-05 17:47:27 +01:00
Xuan-Son NguyenandPascal 5851a8e505 model, mtmd: drama-3 tts support (#31613)
* model, mtmd: drama-3 tts support

* clean up

* mtmd: drama3 helper stops at the codebook tower limit

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-04 15:15:09 +01:00
b97627967c model, mtmd: add chatterbox TTS support (#30993)
* add chatterbox TTS support

Co-authored-by: Pascal <admin@serveurperso.com>

* add built-in voice support

* rm redundant code

* clean up

* clean up 2

* Apply batched suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* improve

---------

Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-03 23:47:11 +01:00
Giulianno 9309bf1bcf model : Add support for Metis-1.6 models (#31134)
Assisted-by: Claude
2026-11-03 16:24:45 +01:00
Prabhsimran Singh 28b812d6cb mtmd : add Nemotron 3.5 Transcribe support (#31471)
* mtmd : add Nemotron 3.5 Transcribe support

Assisted-by: Claude

* convert : fix vocab of draft models for Nemotron-H targets

Assisted-by: Claude

* convert : reject non-BF16 Transcribe checkpoints

Assisted-by: Claude

* mtmd : make speaker FIFO predictions local

Assisted-by: Claude

* mtmd : normalize the mel in encode_windows

Assisted-by: Claude

* convert : map the generated speaker kernel

Assisted-by: Claude

* mtmd : validate Nemotron Transcribe mmproj shapes

Assisted-by: Claude
2026-11-03 14:27:05 +01:00
Maphist0andSigbjørn Skjæret e4b6c8ca6a model: add Clover support (#31306)
* model : add AR-only Clover MoE support

Add GGUF conversion, model loading, and causal hybrid attention for Clover, reusing shared attention, recurrent-state, MoE, and LoRA helpers.

* Update tests/test-llama-archs.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* convert : filter Clover tensors using filter_tensors

* clover : use the standard ssm_a tensor name

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-02 18:37:27 +01:00
Xuan-Son Nguyen 78feec062d model: mistral le chonk (#31376)
* llama: extend DSA and sdpark to new model

* fix ci
2026-10-29 15:43:33 +01:00
Giuseppe Scrivano abb76ab380 model : Support Limite 1B - Violetto (#29433)
* gguf-py : add Limite arch

Assisted-by: Claude Opus 5.5
Signed-off-by: Giuseppe Scrivano <giuseppe@scrivano.org>

* convert : add Limite support

Assisted-by: Claude Opus 5.5
Signed-off-by: Giuseppe Scrivano <giuseppe@scrivano.org>

* llama : add Limite support

Assisted-by: Claude Opus 5.5
Signed-off-by: Giuseppe Scrivano <giuseppe@scrivano.org>

---------

Signed-off-by: Giuseppe Scrivano <giuseppe@scrivano.org>
2026-10-29 10:13:41 +02:00
Tarek DakhranandXuan Son Nguyen d05706b4f3 model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275)
* model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B

Assisted-by: Claude Opus 5.5

* convert : keep the speech task templates out of the d1 GGUFs

Assisted-by: Claude Opus 5.5

* small refactoring: no_batching

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-27 23:36:32 +01:00
Tarek DakhranandXuan Son Nguyen 19ff817fb2 model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257)
* model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5

* convert : read the LFM3-Audio config.json as released

A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys).

Assisted-by: Claude Opus 5.5

* mtmd : disable flash attention for the LFM3-Audio depthformer

Assisted-by: Claude Opus 5.5

* tts : add a voice option for LFM3-Audio and LFM3-Omni

The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts.

Assisted-by: Claude Opus 5.5

* clean up template handling

* convert : detect LFM3-Audio and LFM3-Omni by model_type

Assisted-by: Claude Opus 5.5

* move banned tokens --> logits bias --> mtmd_helper

* leave mtmd-cli unchanged

* no more template_asr_system

* left over

* llama : keep the LFM3-Audio encoder convs and codec unquantized

Assisted-by: Claude Opus 5.5

* common, mtmd : shorten comments

Assisted-by: Claude Opus 5.5

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-27 18:47:14 +01:00
19cce69199 model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252)
* model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5

* tests : set the SWA rope base of the LFM3 test model

Assisted-by: Claude Opus 5.5

* llama : enable recurrent state rollback for LFM3

Assisted-by: Claude Opus 5.5

* Apply batched suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Tarek Dakhran <github@dakhran.com>

* gguf-py : align the lfm3vl mapping comments

Assisted-by: Claude Opus 5.5

* llama : fix CRLF line endings from the review suggestions

Assisted-by: Claude Opus 5.5

* tests : add the LFM3 Metal fusion baseline

Assisted-by: Claude Opus 5.5

* llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing

Assisted-by: Claude Opus 5.5

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
2026-10-27 17:06:40 +01:00
tc-mbandXuan Son Nguyen 7eed6b7685 model: Add MiniCPM-OCR-Jev (#30980)
* model : add MiniCPM-OCR-Jev support

The model is a Qwen3.5-VL trunk with a value head read at the last token: it
scores an OCR text against an image rendered from it, it does not generate.

Add the openbmb-minicpm-ocr-jev decision type, served by POST /v1/systemone.
The class scores are the pooled output of the sequence, read with last pooling,
so the conversion turns score.weight into cls.output and sets
embedding_length_out. The checkpoint has no LM head, so text generation is
rejected instead of using the output weights duplicated from the token
embeddings.

The task is fixed, so its prompt is entirely in the systemone template, the
same way h2o-lightning keeps its default there. The processor of the checkpoint
allows up to 16384 merged tokens per image, more than the Qwen-VL default of
mtmd, so the mmproj keeps the image budget of the preprocessor config. The
checkpoint declares MiniCPMOCRForSequenceClassification in its config.json, as
other models do.

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix code

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* use gguf meta classifier_output_labels

* remove redudant code changes

---------

Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-27 11:49:12 +01:00
Aman GuptaandDaniel Han 74bca39500 model: DSV4.1 flash (#31086)
* model : add DeepSeek-V4.1-Flash (deepseek41)

* deepseek41 : remove MXFP8, convert the Engram table to BF16

* deepseek41 : compute the compressed-KV read layers once in the cache

* deepseek41 : use get_cells() for the shared-history check, drop llama_kv_cache::seq_pos_tok_le

* deepseek41 : gather the Engram rows in-graph like qwen4exp, drop the host gather

* deepseek41 : keep the Engram hash constants in hparams.ple_*, one module per Engram layer

* deepseek41 : drop the LLAMA_DSV41_KV_MIRROR toggle

* deepseek41 : keep il last in build_hc_mixes

* deepseek41 : sort the top-k rows before the sparse gather

* tests : add deepseek41 to the Metal fusion baseline

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-10-25 09:43:51 +08:00
Hans 9ae73e68f5 mtmd : add Soprano TTS support (#28607)
Add Soprano-80M and Soprano-1.1-80M (https://huggingface.co/ekwek/Soprano-1.1-80M)
as an audio generation model: a Qwen3 backbone plus a Vocos decoder that runs
as a gen-only mmproj.

- convert: detect Soprano checkpoints and export the backbone as qwen3 and the
  decoder.pth Vocos weights as the mmproj
- vocab: add the soprano pre-tokenizer (lowercase, whitespace collapse,
  per-digit split, [UNK] fallback)
- mtmd: add the Vocos graph and ISTFT, allow loading a gen-only mmproj for
  Soprano and include the gen-audio context in mtmd_get_memory_usage
- mtmd-helper: add the Soprano pipeline; the decoder runs over the whole
  utterance, so stream mode sends the audio in one chunk after flush()
- tts: document Soprano usage

Assisted-by: Codex
Assisted-by: Claude Opus 5.5
2026-10-24 01:09:50 +02:00
tc-mb a08ab75ae7 mtmd : add MiniCPM-OCR 1.0 support (#30979)
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
2026-10-22 19:22:15 +02:00
alexey-rsiandSigbjørn Skjæret a37cc46104 models : add Flint model (FlintForCausalLM) (#30928)
* Add Flint model (FlintForCausalLM). Assisted by Codex

* cleanup

* one more

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-21 23:26:22 +02:00
Maphist0andSigbjørn Skjæret 7775ffb7e4 model: Add support for SenseNova U1 model (#28919)
* model : add SenseNova U1 understanding support

Add support for the dense SenseNova U1 understanding branch, including
the U1.5-8B-MoT model.

- Add HF-to-GGUF conversion for NEOChatModel, excluding MoT generation
  tensors and concatenating temporal and spatial Q/K normalization
  weights.
- Register the sensenova_u1 GGUF architecture and custom spatial RoPE
  metadata.
- Implement temporal/spatial Q/K normalization with 4-axis RoPE.
- Keep U1-specific text position handling in the model implementation.
- Add synthetic architecture coverage for sensenova_u1.

Only the dense understanding branch is supported; generation
branches are intentionally rejected during conversion.

* conversion : treat one-expert SenseNova U1 configs as dense

* mtmd : add SenseNova U1 vision support

Add conversion support for the SenseNova U1 vision encoder and implement
its CPU mtmd projector graph.

The graph applies patch embedding, GELU, adjacent-pair 2D RoPE, spatial
downsampling, and dense projection for dynamic-resolution image inputs.

* mtmd : fix SenseNova U1 vision token layout

* model : clean up SenseNova U1 RoPE metadata

Replace the hard-coded spatial RoPE key with shared GGUF and LLM_KV
definitions used by conversion, model loading, and architecture tests.

Clarify why SenseNova U1 retains its custom text-position input: U1 uses
[t, 0, 0, 0], while the generic M-RoPE input uses [t, t, t, 0].

* mtmd : simplify SenseNova U1 projector integration

Map the SenseNova U1 dense projection to the standard mm.0 weight and
bias tensors and reuse the shared projector fields in the loader and graph.

Read vision parameters from the checkpoint configuration and remove the
U1-specific tensor names and model fields while preserving its strided
projection, bias, and row-major token layout.

* mtmd : fix SenseNova U1 image positions and attention

* fix spacing

* mtmd : restore generic test input chunks

* mtmd : restore SenseNova U1 integration after upstream merge

* Update conversion/sensenova_u1.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/models/sensenova-u1.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* model : remove SenseNova U1 BF16 activation casts

Use standard backend precision handling for QKV, FFN, and output
projections instead of casting inputs based on the output tensor type.

Assisted-by: Codex

* Update src/models/sensenova-u1.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-21 21:54:16 +02:00
Takamichi Ohara 52ffa02745 model : support Nemotron-3(.5)-Embed (#30877) 2026-10-20 22:48:45 +02:00
7e3eeee048 model : add H2O-Lightning-4B decision model (#30361)
* model : add H2O-Lightning-4B decision model

Adds the h2o-lightning decision type for h2oai/h2o-lightning-4b
(Qwen3.5-4B with a merged decision LoRA).

None of the existing types match this model. It answers with " A"
rather than "A", uses up to 255 labels, and was trained on Python
json.dumps text, so floats like 12.0 must render as 12.0.

The converter writes the decision metadata and the systemone template
from the model's serve_config.json and refuses a config that doesn't
match. Tests skip unless the tiny test model is available locally.

Assisted-by: Claude Opus 5.5

* rm redundant code

* put labels to gguf

---------

Co-authored-by: Jonathan C. McKinney <jon.mckinney@h2o.ai>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-20 18:54:27 +02:00
Xue ZhenliangandSigbjørn Skjæret 62318cd57d model : add Kolibri-1 (Aleph-Alpha) support (#30673)
* model : add Kolibri-1 (Aleph-Alpha) support

Kolibri-1 is a bilingual German/English MoE reasoning model with 78B
parameters and 3.46B active per token: 50 layers, 384 routed experts plus
one shared expert.

Attention interleaves four sliding layers at a 513 window with one full
layer. The sliding layers rotate with NeoX RoPE, the full layers are NoPE.
GQA runs 48 query heads against 4 KV heads at head_dim 128 with a
QK-RMSNorm, and every block carries four RMSNorms, so the attention and the
MLP output are normalized before they rejoin the residual stream. Routing
picks the experts from logits + expert_bias and weights them with the
sigmoid of the unbiased logits, without renorm.

The converter reuses AfmoeModel for the per-expert checkpoint layout and
the expert selection bias, and writes the shared expert and the per layer
sliding window pattern. The released FP8 checkpoint needs no special code,
it goes through the generic weight_scale_inv path. Three tensor names are
added to the mapping (post_attn_norm, post_ffn_norm,
moe.router.expert_bias) and the 128k vocabulary registers the qwen2
pre-tokenizer by hash. The arch is registered in test-llama-archs, with GQA
and the sliding window pattern.

Assisted-by: DeepSeek Harness (DeepSeek V4.1 Flash)

* Update src/models/kolibri1.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Add fusion records for kolibri1

* Update src/models/kolibri1.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-17 13:00:17 +02:00
Daniel Han 01fc3aae84 model: add Beam (Reflection) (#30670) 2026-10-17 00:07:55 +02:00
Xuan-Son Nguyen 2851b3bcbf mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629)
* mtmd: add mtmd_audio_stream + support voxtral-realtime

* address review
2026-10-16 01:46:42 +02:00
ba3c05f71d model: add Muse Spark Open (text, vision and audio) (#30616)
* model: add Muse Spark Open (text, vision and audio)

Muse Spark Open is a latent space MoE: the residual stream stays at 8192
while the 256 experts live in 4096, with a projection down and up around
them, top-8 sigmoid routing with a correction bias. Attention interleaves
three sliding layers at a 2048 window with rope for one global layer
without rope, with a sigmoid output gate and a weightless QK-norm.
Residuals are norm preserving, their gate is folded at conversion time.

Vision reuses the Muse Glimmer tower. Audio is a causal convolutional
pyramid with no attention and no spectrogram, four stages folding 16, 6,
5 and 4 samples into the channel dim, one embedding per 80 ms of 24 kHz.

The quantizer keeps the latent projections and the attention gate at
Q8_0, a little over 1% of the weights, since the whole residual stream
passes through them and the low-bit types turned the output into a
repeating loop. The model saver writes the post norm epsilon, and the
arch is registered in test-llama-archs.

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>

* Muse Spark: drop the redundant forward expand

* Muse Spark: drop the redundant forward expand and tighten the comments

* Update src/llama-quant.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/llama-graph.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update conversion/muse_spark.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update conversion/muse_spark.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update conversion/muse_spark.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update conversion/muse_spark.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-15 23:27:12 +02:00
Kartik Gulia cd0a5b6a18 models : add Nemotron Super 3.5 VL and Nemotron Bolt 74B A9B support (#30520)
* add support

* remove unneeded bos fix

* add video flag

* fix flake error

* remove video-specific budgeting

* convert : fix lilicorr-dflash bid clash

* spec : enforce lilicorr n_tokens condition

* spec : fix lilicorr walk

[no release]
2026-10-15 16:34:56 +03:00
Kartik GuliaandGeorgi Gerganov 843e7dcc15 spec : add lilicorr support (#30506)
* spec : add lilicorr support

* convert : use convention for hparam assignment

* add todo for hparam use in llama-batch.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* typo fix

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-10-15 05:49:25 +02:00
Xuan-Son Nguyen efe3626b11 model: support vllm d3 models (#30383)
* models: support pplx-decider

* vllm d2

* update

* clarify naming rule

* stricter validation

* d3
2026-10-12 15:01:52 +02:00
Xuan-Son Nguyen 097cf33d94 model: support vllm decision 2 model (#30378)
* models: support pplx-decider

* vllm d2

* update

* clarify naming rule

* stricter validation

* fix ci

* rm redundant filter_tensors
2026-10-12 13:39:55 +02:00
69f201a205 model : support MiniCPM-V 4.7 (#29416)
* mtmd : add MiniCPM-V 4.7 support

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* model : allow mrope time from an extra position slot

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* Update conversion/minicpm.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Slim down comments

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix for "do not hand-wrap comments"

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix ci

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* rm 3d repo for pr one

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* gguf: add rope.section_order metadata

* fix comments

* handle grid layout

* allow compat

---------

Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-10 20:41:59 +02:00
Sam Malayek 75118a3a59 convert : support Qwen3.5 embedding models (#27920)
* Update convert_hf_to_gguf: support Qwen3.5 embedding models

* Behavior-preserving refactor for conventions.

* conversion: simplify pooling comment
2026-10-08 08:40:01 +03:00
Tarek Dakhran a657f7e981 model : add LiquidAI/d1-omni-600M decision model (#30114)
* model : add LiquidAI/d1-omni-600M decision model

Assisted-by: Claude Opus 5.5

* mtmd : keep conformer GLU sigmoid on CUDA

Assisted-by: Claude Opus 5.5

* server : take d1omni audio through images and input_audio, scope memory-less lfm2 to non-causal

Assisted-by: Claude Opus 5.5

* common : rename decision type d1omni to lfm2-d1-omni, server : make images an alias of files

Assisted-by: Claude Opus 5.5
2026-10-08 02:07:49 +02:00
Tarek Dakhran 88dcc460d6 model : add LiquidAI/d1-3B decision model (#30110)
* model : add LiquidAI/d1-3b decision model

mtmd : read LFM2 image resize algo from GGUF

Assisted-by: Claude Opus 5.5

* common : rename decision type d1 to lfm2-d1

Assisted-by: Claude Opus 5.5
2026-10-07 22:04:39 +02:00
Terrence Zhao 50a6c5cf7c mtmd: add cohere2 vision support (#30062)
* cohere2 vision model

* address comments

* remove redundant mapping

* follow existing patterns

* fused linear_1
2026-10-07 20:07:50 +02:00
4625240437 model : add K2 Horizon dense and MoVA support (#29535)
* model: K2 Horizon gguf conversion code

* model: loading hparams and tensors in k2-horizon.cpp

* model: K2 Horizon compute graph

* model: K2 Horizon compute graph adjustment and registering tokenizers

* model: K2 Horizon chat template and accomodate safetensors naming

* unicode : add the K2-Horizon pre-tokenizer splitter

The K2-Horizon regex had no arm in unicode_regex_split_custom and fell through to the
general std::regex fallback, which fails two ways.

On MSVC std::regex rejects \p{...}, so no K2-Horizon GGUF loads on Windows at all:
llama-quantize, llama-imatrix and llama-perplexity all abort with
regex_error(error_escape) before a token is produced.

Where the fallback does compile it is still wrong. unicode_regex_split collapses each
codepoint to a single byte naming its Unicode category before matching, and U+200C/U+200D
are category Control, which has no entry in k_ucat_cpt, so both become the 0xD0 fallback
byte. The literal ‌ and ‍ alternatives in K2's regex can then never match and
every ZWNJ or ZWJ ends a letter run.

The splitter is the existing llama3 one with a single rule widened, since K2's regex
differs from llama3's only in that a letter run also takes marks, ZWNJ and ZWJ.

tests/test-unicode.cpp gains a case for this: it fails before the change with
[Amy] [ZWNJ khaham] and passes after with the run intact.

* tests: expand K2 Horizon unicode splitter coverage

* unicode: handle K2 Horizon case folding and empty input

Assisted-by: Codex

* jinja : support sequence indices in selectattr and rejectattr

Assisted-by: Codex

* model : add K2 Horizon dense and MoVA support

Includes the K2 Horizon implementation from ifm-ai/llama.cpp with converter, tensor-parallel and model save/reload fixes.

Assisted-by: Codex

* chat : support K2 Horizon reasoning and tool calls

Assisted-by: Codex

* conversion: remove obsolete K2 Aurora alias

Assisted-by: Codex

* k2-horizon: enforce response schemas and load YaRN betas

Constrain final JSON after reasoning, accept flexible JSON tool envelopes,
enforce XML dialects, and handle repeated or alternate thinking markers.
Load YaRN beta metadata instead of retaining the default values.

Add schema, streaming, continuation, and model reload regressions. Validate
CUDA and CPU builds and 0.9B, 4B, and MoVA conversation/tool round trips.

Assisted-by: Codex

* renaming template fixture

* adressing cisc follows ups

* desloppify the parser / adress aldehir comments

* clean test-chat

* remove fallback : model trained mostly on high anyway

* fix k2 attn_v_exp tn splitting and metal fusion baseline

* k2-horizon : forward expand views before sums

* k2-horizon: copy embds before group norm to fix TP

* disable tesnor parallelism

---------

Co-authored-by: Ryandito Diandaru <ryandito.diandaru@mbzuai.ac.ae>
Co-authored-by: WestWaters <mario.papaleo2013@gmail.com>
Co-authored-by: Natani L. Mayday <71436458+TaskPuppyNatani@users.noreply.github.com>
Co-authored-by: West <100190545+WestWaters@users.noreply.github.com>
Co-authored-by: aaryamonvikram <aaryamonvikram@gmail.com>
Co-authored-by: aaryamonvikram <96529820+aaryamonvikram@users.noreply.github.com>
2026-10-06 18:42:24 +02:00
Xuan-Son Nguyen 4fbc76dec5 model: support embeddinggemma2 (text+vision+audio) (#30054) 2026-10-06 18:20:31 +02:00
Xuan-Son Nguyen da263e7275 models: support pplx-decider (#30044) 2026-10-06 16:07:46 +02:00
Tarek Dakhran cb7934c52c model : Add LFM2.5-Encoder-350M and LFM2.5-Encoder-230M (#29862)
Register `Lfm2BidirectionalForMaskedLM` architecture for LFM2.5-Encoder
models.
2026-10-03 08:44:45 +02:00
Xuan-Son NguyenandSigbjørn Skjæret 99b95488ca model: add support for clef decision model (text-only) (#29831)
* init support for clef (text only)

* more static graph

* clean up

* nits

* nits 2

* Update gguf-py/gguf/constants.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-03 02:50:48 +02:00
Xuan-Son Nguyen 46ca246de9 model: support nimble decision model (#29844) 2026-10-02 14:56:53 +02:00
Xuan-Son Nguyen a4cb4c61fd llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) (#29818)
* init conversion

* convert: ok

* model loaded

* add server code

* improve conversion script

* support shared prompt prefix

* add docs, imorove UX a bit

* add vision support

* add openjev tiny model for testing

* add dev docs

* support lev & kev

* clean up

* fix lev noul

* fix py lint

* nits docs

* clarify about not supporting date_facts
2026-10-02 11:56:04 +02:00
649dcb1036 add GLM-5.3-Flash (GLM5-Next) support (#27773)
* Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx

* Add initial MTP support

* Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements.

* Review driven changes, remove env vars, protect tensors

* Strip MTP for initial PR

* Clean up after mtp strip

* Clean up after mtp strip

* Update speculative.cpp

* Update llama-context.h

* Clean up after mtp strip

* Fix tokenizer ignore merges

* Improve quantization protection selection

* Refactor mhc helpers, graph base

* Lint Fixes

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Skip glm5-next in model saver, fix CRLF

* Skip glm5-next in sweep

* Remove T4 fallback

* Review cleanup

* Review suggestions

* Defer separate MTP gguf handling to MTP PR, drop filter

* Repad n_head_kv

* kpool init apply

* Order by descending score

* Drop guard

* read kpool from hparams, clarify kpool cache flags, remove kpool_build_state(nullptr)

* Add glm5-next support to model saver and add arch test fixture

* Review cleanup

* Kpool pooled caching clarify

* Add multi stream support

* Finish Rebase

* Sparse FA fir DSA prefill

* Const

* Update llama-model.cpp to fix rebase error

* gguf-py : merge tensor map entries for HC tensors

* model : use build_gdn_l2_norm in GLM5_NEXT implementation

* chore : remove trailing whitespace

* model : use new OP precision setting API in GLM5_NEXT implementation

* mtmd : use ggml_swiglu_clamp in GLM5V and apply the image token limit

The two clamps around swiglu_split are what ggml_swiglu_clamp already does,
so the clamp bounds collapse back to one value. GLM5V also never called
set_limit_image_tokens(), so --image-max-tokens had no effect.

Assisted-by: Claude Opus 5
(cherry picked from commit 46d18e12d422be4cc04a70e4a9a9e0168bb3d5b7)

* llama : keep the GLM5-Next k-pool layout across ubatches

The layout was rebuilt from a full cell scan on every ubatch. Pools are fixed
by the positions relative to the sequence's first one, so the layout now lives
on the memory and a ubatch only appends to it.

A sequence edit no longer stales every pooled key either, only the ones at or
after the edited position, which makes a tail seq_rm free. The pooling subgraph
is built unconditionally so the graph shape no longer changes every kpool
tokens, and the pool axis is folded into rows before soft_max, which otherwise
exceeds the CUDA gridDim.y limit past n_kv 262144.

Assisted-by: Claude Opus 5
(cherry picked from commit 5d1c40b93e17fddbf73b785efe43e0d02ccb3977)

* model : write the GLM5-Next recurrent rollback checkpoints

The conv state and the delta net state were only written to the live row, so a
rollback restored whatever the checkpoint rows happened to hold. Take the same
route as kimi-k3: build_recurrent_attn for the state, and write all K_rs conv
groups. That also drops a state view that assumed contiguous rows.

Enroll the arch in test-recurrent-state-rollback, which catches this under its
garbage-filled cache pass.

Assisted-by: Claude Opus 5
(cherry picked from commit 5ace37e86d5d448e83ef5dde5632c748185b18cd)

* llama: fix PR #27773 test-save-load-state restore failure

Clear the attention and indexer cache data after a failed hybrid state restore so restored NaNs cannot affect a later sequence.

Assisted-by: Codex

* llama: fix PR #27773 gpu-rocm graph reallocation

Reserve the full GLM5-Next pool capacity and dirty pool count. The gpu-rocm Test step aborts when n_new grows while the graph node count stays fixed; CUDA, Vulkan, Metal, and WebGPU checks report the same error.

Assisted-by: Codex

* llama : fix GLM5-Next k-pool layout staleness after edits and shared teardown

Two defects in the cross-ubatch k-pool layout added by the k-pool commit:

1. Wrong results. An edited sequence only rebuilt its pool layout when its cell
   count changed, so if the first ubatch after an edit added back exactly as many
   cells as were removed, the stale position-to-cell list survived. With a unified
   cache and more than one sequence, where another sequence takes the freed cells,
   the reused layout points at the wrong cells (CPU: large logit drift, CUDA: NaN).
   Rebuild whenever the sequence is stale, not only on a size mismatch.

2. Slowdown. "shared" mode was assumed to end only with an edit that forces a
   rebuild, but sharing also ends when the other sequence is removed. The survivor
   kept shared = true, pinning cache_safe off and re-pooling every pool on every
   ubatch (server trigger: n>1 completions with -kvu, via the seq_cp in
   copy_state_to). In seq_rm, if the layout has shared cells, stale every sequence
   so one rebuild re-derives sharing and cache_safe returns to 1.

Assisted-by: Claude Opus 5

* llama : fix build_attn_mha stream stride for non-contiguous q

build_attn_mha split the batch into streams with a stream stride of
q->nb[3]/n_stream. That only equals one stream's span, (ne[2]/n_stream)*nb[2],
when q is contiguous. GLM5-Next is nope-only, so it does not concat a rope part
and passes the permuted q_absorbed straight in, where nb[3] != ne[2]*nb[2]; the
stride was then n_head times too large and every stream s >= 1 read another
head's queries. Split-KV (-np N without --kv-unified) multi-stream prefill was
wrong for every stream past the first. Unified KV and decode were unaffected
(n_stream == 1, and decode takes the gather path). Other MLA models concat rope
so q is contiguous and the computed value is unchanged for them.

Compute the stride from the token dimension, which is identical for a
contiguous q.

Assisted-by: Claude Opus 5

* llama : re-derive GLM5-Next k-pool sharing on state_read/state_drop

The shared-cell teardown added to seq_rm (stale every sequence when the layout
has shared cells, so a survivor does not keep shared = true and pin cache_safe
off) was missing from the other paths that can free shared cells: state_read
and state_drop staled only the one sequence. Apply the same re-derivation there
and correct the comment that claimed sharing ends only via an edit or seq_rm.

Assisted-by: Claude Opus 5

* quant : drop duplicate GLM5-Next hc_ filter

The hc_ name filter was listed twice in the GLM5_NEXT protection block.

Assisted-by: Claude Opus 5

* glm5-next: scope K-pool cache access to indexed operations

* glm5-next: keep K-pool access in hybrid index memory

* glm5-next: keep mHC graph builders model-local

* glm5-next: mark only touched pools per ubatch

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Co-authored-by: Piotr Wilkin <ilintar@gmail.com>
2026-09-30 14:20:32 +08:00
Tobyandaetherbird f830688e91 model : add Ling 3.0 VL support (#29151)
* model : fold Ling 3.0 VL into the BailingMoeV3 architecture

Assisted-by: Scout

* model : keep shared NORM rope list intact when gating bailingmoe3 on mrope sections

---------

Co-authored-by: aetherbird <aetherbird@users.noreply.github.com>
2026-09-24 08:57:31 +02:00
Hrishith Thadicherla 633733d0ae model : support Gemma4 DSpark draft backbone (#29226)
* dspark: add Gemma 4 draft support

Add GGUF conversion and runtime support for full-attention and SWA Gemma 4
DSpark drafts, including tied output weights and boolean backbone metadata.

Assisted-by: Codex

* dflash: infer Gemma draft features from metadata
2026-09-23 13:34:09 +03:00
GeorgeandSigbjørn Skjæret 7d6f5d02bb model : add support for HrmTextForCausalLM (DFM Mimir 1B) (#27625)
* model : add support for HrmTextForCausalLM (DFM Mimir 1B)

HRM-Text runs two transformer stacks (low, high) in an alternating cycle over the same token stream. The low-cycle state z_l starts from a learned [n_embd] tensor and is broadcast over positions.

- conversion: new writer for the fused gqkv projection (order gate,q,k,v) remapped to llama.cpp q/k/v plus a separate sigmoid gate tensor
- loader: block_count = lps * h_cycles * (l_cycles + 1) cache slots aliasing 2*lps physical blocks via struct copies
- graph: looped build with sigmoid-gated attention, SwiGLU FFN and parameterless RMS norms; learned embedding_scale applied in build_inp_embd
- saver: pointer-deduplicated layer loop (looped archs alias tensors)
- tests: hrm_text fixture (lps 1, h 2, l 3) in test-llama-archs

Limitations:
causal attention only - the upstream prefix-LM mode is not implemented (the prefix_lm GGUF key round-trips unused).
The KV cache holds one entry per pass: 128 layers for Mimir 1B, i.e. 4x a same-width 32-layer model - about 3072 MiB at ctx 4096 in F16 (halves with q8_0 KV + FA).
Every token runs all 128 block passes, so decode cost is roughly 4x a dense model of equal width (2.65 t/s BF16, 8-thread desktop CPU).

Verified against the HF reference: identical argmax at 334/334 positions across 20 prompts (BF16 GGUF vs FP32 golden).
q8_0 requant: 95.8% top-1, all remaining misses inside the HF top-5 (accumulated error over 128 sequential blocks).

AI usage disclosure: YES
Used GLM-5.3 for the majority of code AI-generated under my direction, all gates verified locally.
All in all I could say that I have written less than 20% of the code and most of the heavy lifting has been done by the model. As such, this should be considered experimental.

* Update conversion/hrm_text.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/llama-arch.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* convert : add gguf_writer methods for hrm_text metadata

replace raw add_uint32/add_bool calls with dedicated GGUFWriter methods, following the add_embedding_scale pattern

Assisted-by: GLM-5.3

* convert : map regular hrm_text tensors via tensor_mapping

delegate unfused checkpoints to the base tensor mapping; training-style attn. names are renamed to self_attn. so the patterns match

Assisted-by: GLM-5.3

* model : format hrm-text build_* calls as in other models

one argument group per line, matching sibling model files

Assisted-by: GLM-5.3

* llama : move hrm z_l_init table entries out of the nemotron group

place the name and tensor-info entries with the other global input tensors

Assisted-by: GLM-5.3

* convert : slim down hrm_text comments

Assisted-by: GLM-5.3

* convert : build hrm_text block tensor names from the {bid} template

The tensor map holds concrete per-block names, so format the template
with the computed layer index before handing it to super().

* llama : name hrm metadata keys in their own hrm. namespace

The four keys are arch-independent, unlike the arch-substituted
Keys.LLM entries, so group them under Keys.HRM (like Keys.Split) and
rename the llm_kv entries to LLM_KV_HRM_*. Only our own GGUFs carry
the old hrm_text.* keys; they are regenerated.

* Update src/llama-model-saver.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* llama : keep hrm metadata keys arch-substituted

Per review: the GGUF keys stay "{arch}.h_cycles" style, so the Python
members drop the LLM_KV_HRM_ prefix and keep arch templates; C++ keeps
the LLM_KV_HRM_* enums. GGUF output is unchanged - existing files and
HF uploads stay valid.

* Update gguf-py/gguf/constants.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/llama-arch.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/llama-arch.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* convert : rename hrm writer methods to add_hrm_*

Generic names like add_h_cycles/add_prefix_lm are too broad on the
shared GGUFWriter; prefix them with hrm_ like the metadata keys.

* model : fix meta-split lookup for archs with aliased cache slots

Cache tensors of archs that alias physical blocks across looped slots
(hrm_text, nanbeige with num_loops > 1) can reference block indices
without weight tensor names. Take the output projection from the layer
array instead of asserting; all other lookups are unchanged.

* model : replicate hrm_text tensors on meta devices instead of splitting

The aliased cache slots rotate split states differently from their
physical weights, so the meta-split execution invariants (set_rows
requires the cache state to match the token indices) cannot hold for
any device count. Replicate all hrm_text tensors on every meta device
instead; single-device and non-meta paths are unchanged.

Assisted-by: Claude Sonnet

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-09-16 15:18:45 +02:00
Alex 3d10bcd197 llama: add Maple 20B-A1B ternary MoE architecture (CPU) (#27000)
* gguf-py: add Maple tensor constants

Add MODEL_ARCH.MAPLE, its "maple" name, and the tensor list for the
Maple 20B-A1B ternary MoE architecture: token embeddings, output,
attention with Q/K RMS norms, and per-expert FFN tensors.

* convert: add Maple HF->GGUF converter

Register MapleForCausalLM in the HF architecture map and add the
converter for the Maple 20B-A1B ternary MoE model: 24 layers, 256
experts with 8 active, sliding-window attention (SWA-512) interleaved
with global attention at a 3:1 ratio, partial rotary factor 0.5, and
per-expert weight stacking into merged 3D tensors.

* llama: add Maple architecture (20B-A1B ternary MoE)

Add the Maple 20B-A1B ternary MoE architecture: 24 layers, 256
experts with 8 active, sliding-window attention (SWA-512) interleaved
with global attention at a 3:1 ratio, and ternary TQ1_0/TQ2_0
quantization support.

- register LLM_ARCH_MAPLE between MAMBA2 and JAMBA
- implement llama_model_maple: Q/K RMS norms after projection (GEMMA4
  style), rope applied only on SWA layers (nope_on_global_attention),
  ISWA KV cache, and MoE FFN with swiglu gate clamp at +7 (DEEPSEEK4
  style)
- mark MAPLE as unsupported by the model saver (roundtrip skipped)

* tests: mark Maple as MoE-mandatory

Maple is always-MoE: the model throws when n_expert == 0, so the
test harness must only run the MoE config for LLM_ARCH_MAPLE.

* maple: apply review feedback (n_ff_exp_arr, get_arr, rope params)

- load_arch_hparams: use n_ff_exp_arr + n_ff_exp() accessor (upstream
  changed these from a scalar member during the rebase)
- sliding_window_pattern: get_arr, the pattern is mandatory for this arch
- partial_rotary_factor: read only from rope_parameters (base.py mirrors
  the top-level key automatically)
- document why TOKEN_EMBD/OUTPUT are forced to F16 (they are the two
  dense tensors in Maple, and the reference GGUFs ship them as F16)
- add @ModelBase.example("deepgrove/maple-preview")

* tests: add Maple to the SWA pattern array list

get_arr for maple.attention.sliding_window_pattern requires an array, but
the harness only emitted a per-layer array for the arches in its list, so
test-llama-archs -a maple failed to load the model.

Assisted-by: DeepSeek Harness

* maple: move swiglu_clamp_exp to the converter

The loader prefilled 7.0 and read the key optionally. The converter now
writes it and the loader reads it as required, because llama-graph.cpp
skips the clamp when the limit is 0 and an optional read would silently
run unclamped. The test harness provides the key for the same reason.

Also drops tensor_force_quant: base.py already forces FFN_GATE_INP to F32
and TOKEN_EMBD/OUTPUT to F16 for ternary file types.

Assisted-by: DeepSeek Harness

* convert: fix the LazyBase func signature in the Maple converter

ty flagged the stack() closure: it takes no argument, while LazyBase is
annotated with func: Callable[[Any], Any]. Pass the tensor list through
args instead of closing over it, the same way kimi_k3 does, so the
callable shape matches.

Assisted-by: DeepSeek Harness
2026-09-14 14:04:05 +03:00
3ad1ba7336 [Model] Support for Spark2_5ForCausalLM implementation (#27868)
* Add Spark3 Model
* rename spark3 -> spark2_5

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: dongjiang <dongjiang2010@gmail.com>
2026-09-06 17:43:58 +02:00
49c0dc82b8 model : add Tencent Hy 4 (hy_v4) preview architecture support (#28127)
* model: add Tencent Hy 4 (hy_v4) preview architecture support

Adds support for the Tencent Hy 4 model (Hugging Face architecture
HYV4ForCausalLM, GGUF arch hy_v4):

Add HF -> GGUF conversion script (conversion/hy_v4.py) and wire it into the conversion registry
Register hy_v4 GGUF constants, arch enum, and writer support
Implement the hy-v4 model graph, hparams, vocab and context changes
Register the new arch in llama-arch and models registry
Extend arch tests to cover hy_v4

Assisted by Claude Opus 5

* Update convert_hf_to_gguf_update.py

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* Update conversion/base.py

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* convert : move hy_v4 entry to the same place as in convert_hf_to_gguf_update.py

* model : apply changes related to n_ff_exp becoming per-layer in Hy4-preview

* n_layer_all

---------

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-09-04 14:31:36 +02:00