Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: add GLM 5.2 Indexer support (#25407)

Browse Source 
* Start building graph - reuse deepseek32

* Enable kv cache and rotation for glm_dsa architecture

Just follow Deepseek 3.2 for now.

* Reuse prev_top_k for "shared" indexer layers

* GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer.

This is transformers' `apply_rotary_pos_emb_interleave`

* Default indexer types to GLM pattern

Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26.

Note that conversion is not saving this key yet.

* Save indexer types to gguf, restore on load

* Use ggml_lightning_indexer when cparams.fused_lid

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* GLM 5 and 5.1 use full indexers

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* Fix indentation

* Ensure array is zero-filled

* Prefer explicit std::fill

* Assert prev_top_k exists for shared indexer

---------

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
This commit is contained in:
Pedro Cuenca
2026-07-24 20:55:56 +02:00
committed by GitHub
co-authored by fairydreaming
parent 95a923a64c
commit 88bfee1429
12 changed files with 426 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama-model-saver.cpp
+1
View File
@@ -281,6 +281,7 @@ void llama_model_saver::add_kv_from_model() {
add_kv(LLM_KV_ATTENTION_INDEXER_HEAD_COUNT, hparams.indexer_n_head);
add_kv(LLM_KV_ATTENTION_INDEXER_KEY_LENGTH, hparams.indexer_head_size);
add_kv(LLM_KV_ATTENTION_INDEXER_TOP_K, hparams.indexer_top_k);
add_kv(LLM_KV_ATTENTION_INDEXER_TYPES, hparams.is_indexer_full_impl, true);
add_kv(LLM_KV_ATTENTION_RECURRENT_LAYERS, hparams.is_recr_impl, true);
const float rope_scaling_factor = hparams.rope_freq_scale_train == 1.0f ? 0.0f : 1.0f/hparams.rope_freq_scale_train;