After llama_context_release_device(ctx, true) the KV cache and the recurrent
state live only in host shadows, and their tensors have no buffer, so
llama_state_seq_get_data cannot read them. The disk prompt cache must save
a cold server's slots at exit, and a shutting-down server must not restore
to the device (it may not hold the GPU token, F7).
llama_context_restore_memory_host() makes the state readable without the
device. restore_device_buffers(to_host = true) wraps each shadow as a
CPU_Mapped buffer (ggml_backend_cpu_buffer_from_ptr) and points the tensors
into it: there is no copy and no allocation. The first version copied into a
CPU buffer instead, which needs the full KV allocation a second time, about
2.7 GB for the 35B at ctx 131072; the node has had an OOM-controller kill
before. The buffer base is rounded down to the 32-byte alignment that
cpu_buffer_from_ptr asserts (a large std::vector is only 16-byte aligned);
those leading bytes are never accessed. With to_host the shadows are kept,
because the buffers point into them.
It sets device_discarded, so restore_device() aborts afterwards. Exit only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>