Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: add GLM 5.2 Indexer support (#25407)

Browse Source 
* Start building graph - reuse deepseek32

* Enable kv cache and rotation for glm_dsa architecture

Just follow Deepseek 3.2 for now.

* Reuse prev_top_k for "shared" indexer layers

* GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer.

This is transformers' `apply_rotary_pos_emb_interleave`

* Default indexer types to GLM pattern

Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26.

Note that conversion is not saving this key yet.

* Save indexer types to gguf, restore on load

* Use ggml_lightning_indexer when cparams.fused_lid

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* GLM 5 and 5.1 use full indexers

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* Fix indentation

* Ensure array is zero-filled

* Prefer explicit std::fill

* Assert prev_top_k exists for shared indexer

---------

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
This commit is contained in:
Pedro Cuenca
2026-07-24 20:55:56 +02:00
committed by GitHub
co-authored by fairydreaming
parent 95a923a64c
commit 88bfee1429
12 changed files with 426 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama-kv-cache.cpp
+1 -1
View File
@@ -323,7 +323,7 @@ llama_kv_cache::llama_kv_cache(
hparams.n_embd_head_k() % 64 == 0;
// always create Hadamard rotation tensors for DeepSeek lightning indexers
if ((model.arch == LLM_ARCH_DEEPSEEK32 || model.arch == LLM_ARCH_DEEPSEEK4) &&
if ((model.arch == LLM_ARCH_DEEPSEEK32 || model.arch == LLM_ARCH_DEEPSEEK4 || model.arch == LLM_ARCH_GLM_DSA) &&
hparams.n_embd_head_k_full == hparams.indexer_head_size) {
attn_rot_k = true;
}