llama: do not release CPU extra buffers (CPU_REPACK) on device release

A model with weights on the CPU device (-dev none / -ngl 0) segfaulted in
llama_model::release_device_weights() the first time the VRAM arbiter put
it to sleep: CPU_REPACK buffers are not host buffers, so load_tensors marked
them releasable, and capturing the shadow called their NULL get_tensor.
Reproduced with the production build 2fc9f9c35 (Qwen3.5-0.8B, -dev none
-ngl 0, LLAMA_SLEEP_VRAM_ONLY=1 LLAMA_SLEEP_EVICT_KV=1, force a yield with a
waiter on want.lock): exit 139 at the release, before any SIGTERM. The
faulting call (rip 0) returned into release_device_weights.

Only buffers on a non-CPU device are releasable now. These buffers are in RAM
already, so there is nothing to free. Production is not affected: every
arbiter model there has its weights on the GPU, and a releasable CPU_REPACK
buffer would have crashed it. Found by the --diskcache check, which runs
its arbiter cases CPU only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
LumpiastyandClaude Opus 5.5 committed 2026-10-25 18:55:57 +01:00
1 parent 71d4cbe195
commit f24c3582c2
1 file changed
+3 -1
+3 -1
View File
@@ -2026,7 +2026,9 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
auto & last_bufs = pimpl->ctxs_bufs.back().second;
if (!host_ptr_path && !ml.no_alloc && last_bufs.size() == 1) {
ggml_backend_buffer_t b = last_bufs[0].get();
if (b != nullptr && !ggml_backend_buffer_is_host(b)) {
// CPU extra buffers (CPU_REPACK) are not host buffers, but they are in RAM and have no get_tensor
ggml_backend_dev_t buft_dev = ggml_backend_buft_get_device(buft);
if (b != nullptr && !ggml_backend_buffer_is_host(b) && (buft_dev == nullptr || ggml_backend_dev_type(buft_dev) != GGML_BACKEND_DEVICE_TYPE_CPU)) {
slot.releasable = true;
slot.buft = buft;
}