Xuan-Son Nguyen
6023692b1e
llama: fix kv shifting, add tests, add per-model build_rope ( #31683 )
...
* llama: fix kv shifting and add tests
* llama: add graph build_rope (#31693 )
* llama: add graph build_rope
* improve
* migrate new models
* update review skill
* fix ci
* add --kv-shift to CI
2026-11-05 21:10:36 +01:00
Xuan-Son Nguyen
78feec062d
model: mistral le chonk ( #31376 )
...
* llama: extend DSA and sdpark to new model
* fix ci
2026-10-29 15:43:33 +01:00
Georgi Gerganov
7a2e5aba90
llama : remove obsolete n_pad from memory instances ( #31200 )
...
Every caller passes n_pad = 1 and the only two users clamp it to 256.
Drop the parameter, the members and the trivially true kv_size assert.
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
2026-10-27 07:53:44 +02:00
Georgi Gerganov
a4699400e3
llama : move no_alloc out of llama_hparams ( #31141 )
...
* llama : move no_alloc out of llama_hparams
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
* cont : clean-up
* cont : clean-up
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
* tests : add a no_alloc case to test-llama-context
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
2026-10-26 16:21:25 +02:00
Chipmunk
08618ff8e7
llama : fix K/V and recurrent state cleanup after failed restores ( #27530 )
...
* llama : add discard for deferred state writes
* llama : add tensor zeroing helper for backends without tensor memset
* llama : clear K/V data after failed sequence restore
* llama : clear recurrent state data after failed sequence restore
* llama : simplify discard and restore cleanup
* llama : report error when abnormal cell count is found in state_read_meta
* llama : clear attention state on hybrid restore failure
* tests : cover failed state restore cleanup
* llama : clear MLA state on dsa restore failure
* tests : update test for rebased test suite
* llama : clarify comment in llama_memory_recurrent::state_read
2026-09-26 10:23:03 +03:00
fairydreaming and Stanisław Szymczyk
563dec81c1
llama : allocate indexer cache only in "full" indexer layers ( #26474 )
...
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com >
2026-08-03 14:56:30 +02:00
Aman Gupta
230ea9d214
llama-batch: add n_keep_tail in split_equal for recurrent models ( #25278 )
2026-07-08 15:55:19 +08:00
Aman Gupta
04eb4c446d
llama : add Gemma4 MTP ( #23398 )
2026-06-07 20:50:54 +08:00
1f0aa2a696
model : support for DeepseekV32ForCausalLM with generic DeepSeek Sparse Attention (DSA) implementation ( #23346 )
...
* llama : support DeepSeek V3.2 model family (with DSA lightning indexer)
* convert : handle DeepseekV32ForCausalLM architecture
* ggml : support for f16 GGML_OP_FILL
* memory : separate hparams argument in llama_kv_cache constructor
* memory : add llama_kv_cache_dsa memory (KV cache + lightning indexer cache)
* llama : support for LLM_ARCH_DEEPSEEK32
* model : llama_model_deepseek32 implementation
* model : merge two scale operations into one in DSA lightning indexer implementation
* chore : remove unused code
* model : support NVFP4 in DeepSeek V3.2
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* memory : refactoring TODO
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com >
---------
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com >
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
Co-authored-by: ggerganov <ggerganov@users.noreply.github.com >
2026-05-29 10:15:17 +02:00