Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: add GLM 5.2 Indexer support (#25407)

Browse Source 
* Start building graph - reuse deepseek32

* Enable kv cache and rotation for glm_dsa architecture

Just follow Deepseek 3.2 for now.

* Reuse prev_top_k for "shared" indexer layers

* GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer.

This is transformers' `apply_rotary_pos_emb_interleave`

* Default indexer types to GLM pattern

Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26.

Note that conversion is not saving this key yet.

* Save indexer types to gguf, restore on load

* Use ggml_lightning_indexer when cparams.fused_lid

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* GLM 5 and 5.1 use full indexers

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* Fix indentation

* Ensure array is zero-filled

* Prefer explicit std::fill

* Assert prev_top_k exists for shared indexer

---------

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
This commit is contained in:
Pedro Cuenca
2026-07-24 20:55:56 +02:00
committed by GitHub
co-authored by fairydreaming
parent 95a923a64c
commit 88bfee1429
12 changed files with 426 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama-model.cpp
+2
View File
@@ -1129,6 +1129,7 @@ void llama_model_base::load_hparams(llama_model_loader & ml) {
std::fill(hparams.rope_sections.begin(), hparams.rope_sections.end(), 0);
std::fill(hparams.is_swa_impl.begin(), hparams.is_swa_impl.end(), 0);
std::fill(hparams.is_recr_impl.begin(), hparams.is_recr_impl.end(), llm_arch_is_recurrent(ml.get_arch()) ? 1 : 0);
std::fill(hparams.is_indexer_full_impl.begin(), hparams.is_indexer_full_impl.end(), 0);
std::fill(hparams.xielu_alpha_n.begin(), hparams.xielu_alpha_n.end(), 0.0f);
std::fill(hparams.xielu_alpha_p.begin(), hparams.xielu_alpha_p.end(), 0.0f);
@@ -2065,6 +2066,7 @@ llama_memory_i * llama_model::create_memory(const llama_memory_params & params,
res = nullptr;
} break;
case LLM_ARCH_DEEPSEEK32:
case LLM_ARCH_GLM_DSA:
{
res = new llama_kv_cache_dsa(
*this,