Commit Graph
9 Commits
Author SHA1 Message Date
Xuan-Son Nguyen 6023692b1e llama: fix kv shifting, add tests, add per-model build_rope (#31683)
* llama: fix kv shifting and add tests

* llama: add graph build_rope (#31693)

* llama: add graph build_rope

* improve

* migrate new models

* update review skill

* fix ci

* add --kv-shift to CI
2026-11-05 21:10:36 +01:00
Xuan-Son Nguyen 78feec062d model: mistral le chonk (#31376)
* llama: extend DSA and sdpark to new model

* fix ci
2026-10-29 15:43:33 +01:00
Georgi Gerganov 7a2e5aba90 llama : remove obsolete n_pad from memory instances (#31200)
Every caller passes n_pad = 1 and the only two users clamp it to 256.
Drop the parameter, the members and the trivially true kv_size assert.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
2026-10-27 07:53:44 +02:00
Georgi Gerganov a4699400e3 llama : move no_alloc out of llama_hparams (#31141)
* llama : move no_alloc out of llama_hparams

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))

* cont : clean-up

* cont : clean-up

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))

* tests : add a no_alloc case to test-llama-context

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
2026-10-26 16:21:25 +02:00
Chipmunk 08618ff8e7 llama : fix K/V and recurrent state cleanup after failed restores (#27530)
* llama : add discard for deferred state writes

* llama : add tensor zeroing helper for backends without tensor memset

* llama : clear K/V data after failed sequence restore

* llama : clear recurrent state data after failed sequence restore

* llama : simplify discard and restore cleanup

* llama : report error when abnormal cell count is found in state_read_meta

* llama : clear attention state on hybrid restore failure

* tests : cover failed state restore cleanup

* llama : clear MLA state on dsa restore failure

* tests : update test for rebased test suite

* llama : clarify comment in llama_memory_recurrent::state_read
2026-09-26 10:23:03 +03:00
fairydreamingandStanisław Szymczyk 563dec81c1 llama : allocate indexer cache only in "full" indexer layers (#26474)
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
2026-08-03 14:56:30 +02:00
Aman Gupta 230ea9d214 llama-batch: add n_keep_tail in split_equal for recurrent models (#25278) 2026-07-08 15:55:19 +08:00
Aman Gupta 04eb4c446d llama : add Gemma4 MTP (#23398) 2026-06-07 20:50:54 +08:00
1f0aa2a696 model : support for DeepseekV32ForCausalLM with generic DeepSeek Sparse Attention (DSA) implementation (#23346)
* llama : support DeepSeek V3.2 model family (with DSA lightning indexer)

* convert : handle DeepseekV32ForCausalLM architecture

* ggml : support for f16 GGML_OP_FILL

* memory : separate hparams argument in llama_kv_cache constructor

* memory : add llama_kv_cache_dsa memory (KV cache + lightning indexer cache)

* llama : support for LLM_ARCH_DEEPSEEK32

* model : llama_model_deepseek32 implementation

* model : merge two scale operations into one in DSA lightning indexer implementation

* chore : remove unused code

* model : support NVFP4 in DeepSeek V3.2

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* memory : refactoring TODO

Co-authored-by: ggerganov <ggerganov@users.noreply.github.com>

---------

Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com>
2026-05-29 10:15:17 +02:00