llama: evict recurrent/SSM state on device release

The recurrent (SSM/conv) state of hybrid models (e.g. Qwen3.5) was left
resident when a model's device buffers were released for on-demand VRAM
sharing - llama_memory_recurrent::release_device_buffers() was a no-op
default. Implement it (and restore_device_buffers) with the same
capture-host-shadow / free / reallocate pattern as llama_kv_cache, so
llama_memory_hybrid now evicts both its attention KV and its recurrent
state. The state is read-write, so its shadow is recaptured on every
release.

Assisted-by: Claude
This commit is contained in:
2026-09-09 00:36:41 +02:00
parent 0d68afc4c1
commit ec691cd027
3 changed files with 95 additions and 3 deletions
+1 -1
View File
@@ -202,7 +202,7 @@ void llama_memory_hybrid::state_read(llama_io_read_i & io, llama_seq_id seq_id,
}
void llama_memory_hybrid::release_device_buffers() {
// evict the attention KV (grows with context); the recurrent state uses the no-op default
// evict both the attention KV (grows with context) and the recurrent/SSM state
mem_attn->release_device_buffers();
mem_recr->release_device_buffers();
}