model: add GLM 5.2 Indexer support (#25407)
* Start building graph - reuse deepseek32 * Enable kv cache and rotation for glm_dsa architecture Just follow Deepseek 3.2 for now. * Reuse prev_top_k for "shared" indexer layers * GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer. This is transformers' `apply_rotary_pos_emb_interleave` * Default indexer types to GLM pattern Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26. Note that conversion is not saving this key yet. * Save indexer types to gguf, restore on load * Use ggml_lightning_indexer when cparams.fused_lid Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * GLM 5 and 5.1 use full indexers Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * Fix indentation * Ensure array is zero-filled * Prefer explicit std::fill * Assert prev_top_k exists for shared indexer --------- Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
This commit is contained in:
co-authored by
fairydreaming
parent
95a923a64c
commit
88bfee1429
@@ -237,6 +237,9 @@ class GlmMoeDsaModel(DeepseekV2Model):
|
||||
self.gguf_writer.add_indexer_head_count(self.hparams["index_n_heads"])
|
||||
self.gguf_writer.add_indexer_key_length(self.hparams["index_head_dim"])
|
||||
self.gguf_writer.add_indexer_top_k(self.hparams["index_topk"])
|
||||
if (indexer_types := self.hparams.get("indexer_types")) is not None:
|
||||
indexer_types = [t == "full" for t in indexer_types]
|
||||
self.gguf_writer.add_indexer_types(indexer_types)
|
||||
|
||||
|
||||
@ModelBase.register("SolarOpenForCausalLM")
|
||||
|
||||
Reference in New Issue
Block a user