Files
llama.cpp/tools
Lumpiasty e457c2fd63 server: restore weights/KV on wake, not at decode, so KV eviction is safe
With LLAMA_SLEEP_EVICT_KV the KV cache device buffers are freed when a model
goes cold. update_slots() touches the KV before the decode-time vram_ensure_warm
(e.g. SWA models create a checkpoint that reads the KV via ggml_backend_tensor_get),
so the KV must already be resident by then. Move the restore to the sleep-wake
handler (handle_sleeping_state(false)), which runs before update_slots, fixing a
GGML_ASSERT(buffer) crash on iSWA models (gemma) when KV eviction is enabled.

Assisted-by: Claude
2026-11-22 20:38:31 +01:00
..
2026-11-12 20:59:09 +02:00
2026-11-04 10:10:22 +02:00
2026-11-12 20:59:09 +02:00
2026-11-12 20:59:09 +02:00