model: add GLM 5.2 Indexer support (#25407)
* Start building graph - reuse deepseek32 * Enable kv cache and rotation for glm_dsa architecture Just follow Deepseek 3.2 for now. * Reuse prev_top_k for "shared" indexer layers * GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer. This is transformers' `apply_rotary_pos_emb_interleave` * Default indexer types to GLM pattern Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26. Note that conversion is not saving this key yet. * Save indexer types to gguf, restore on load * Use ggml_lightning_indexer when cparams.fused_lid Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * GLM 5 and 5.1 use full indexers Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * Fix indentation * Ensure array is zero-filled * Prefer explicit std::fill * Assert prev_top_k exists for shared indexer --------- Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
This commit is contained in:
co-authored by
fairydreaming
parent
95a923a64c
commit
88bfee1429
@@ -200,6 +200,7 @@ class Keys:
|
||||
HEAD_COUNT = "{arch}.attention.indexer.head_count"
|
||||
KEY_LENGTH = "{arch}.attention.indexer.key_length"
|
||||
TOP_K = "{arch}.attention.indexer.top_k"
|
||||
TYPES = "{arch}.attention.indexer.types"
|
||||
|
||||
class HyperConnection:
|
||||
COUNT = "{arch}.hyper_connection.count"
|
||||
|
||||
@@ -793,6 +793,10 @@ class GGUFWriter:
|
||||
def add_indexer_top_k(self, top_k: int) -> None:
|
||||
self.add_uint32(Keys.Attention.Indexer.TOP_K.format(arch=self.arch), top_k)
|
||||
|
||||
def add_indexer_types(self, value: Sequence[bool]) -> None:
|
||||
key = Keys.Attention.Indexer.TYPES.format(arch=self.arch)
|
||||
self.add_array(key, value)
|
||||
|
||||
def add_max_alibi_bias(self, bias: float) -> None:
|
||||
self.add_float32(Keys.Attention.MAX_ALIBI_BIAS.format(arch=self.arch), bias)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user