model : support granite multilingual embeddings R2 (ibm-granite/granite-embedding-{97,311}m-multilingual-r2) (#22716)
* Add support for the ibm-granite/granite-embedding-{97m,311m}-multilingual-r2 embedding models:
* Added a version of the gpt4o tokenizer that has a fixed regex (better handling of marks), and different token merging setting for the 97m model
* Reused gemma4 tokenizer for the 311m model
* granite-embedding-*-multilingual-r2 : add support SwiGLU FFN for Granite Embedding Multilingual R2
* added new GGUF key <arch>.hidden_activation (LLM_KV_HIDDEN_ACT) + writer
* added a forward declaration of llm_ffn_op_type to llama-hparams.h
* added llm_ffn_op in hparams
* added LLM_FFN_NONE = 0 sentinel to llm_ffn_op_type (value-initialization), modern-bert: explicitly assigns LLM_FFN_GEGLU before reading GGUF (unchanged).
* centralized hidden_act mapping in llama-model.cpp, added llm_ffn_op_type_from_string() helper, mirroring rope_scaling_type/llama_rope_scaling_type_from_string()
* modern-bert reads the GGUF key (when present) and uses the resulting op in its FFN graph
* Added granite-embedding-{97m,311m}-multilingual-r2 to the converter code
* Added the hashes for the granite embedding multilingual R2 models
* Set the hidden_activation in the GGUF if the field is present in config.json (such as for the granite embedding models)
This commit is contained in:
@@ -23,6 +23,9 @@ enum llama_swa_type {
|
||||
LLAMA_SWA_TYPE_SYMMETRIC = 3,
|
||||
};
|
||||
|
||||
// forward declaration; full definition in llama-graph.h
|
||||
enum llm_ffn_op_type : int;
|
||||
|
||||
struct llama_hparams_posnet {
|
||||
uint32_t n_embd;
|
||||
uint32_t n_layer;
|
||||
@@ -227,6 +230,14 @@ struct llama_hparams {
|
||||
enum llama_rope_scaling_type rope_scaling_type_train = LLAMA_ROPE_SCALING_TYPE_NONE;
|
||||
|
||||
|
||||
// Resolved FFN gated activation flavor for archs that read
|
||||
// `<arch>.hidden_activation` from the GGUF (e.g. ModernBert derivatives).
|
||||
// Defaults to LLM_FFN_NONE (sentinel = 0); the mapping from the GGUF
|
||||
// string to a real op is done at hparam-load time via
|
||||
// llm_ffn_op_type_from_string() in llama-model.cpp, mirroring how
|
||||
// rope_scaling_type_train is handled.
|
||||
enum llm_ffn_op_type llm_ffn_op;
|
||||
|
||||
// Step35: optional per-layer clamps for (Swi)GLU
|
||||
std::array<float, LLAMA_MAX_LAYERS> swiglu_clamp_exp; // clamping for expert FFN
|
||||
std::array<float, LLAMA_MAX_LAYERS> swiglu_clamp_shexp; // shared expert
|
||||
|
||||
Reference in New Issue
Block a user