llama: guard buffer iteration against released device buffers

release_device_weights()/release_device_buffers() leave a null buffer in
ctxs_bufs while the device memory is released for on-demand VRAM sharing.
The memory-breakdown / total_size / clear paths iterated these and called
ggml_backend_buffer_get_size()/get_type()/clear() on the null buffer, tripping
GGML_ASSERT(buffer) and aborting on shutdown of a cold model. Skip null buffers
in llama_model::memory_breakdown and llama_kv_cache::{memory_breakdown,
total_size,clear}.

Assisted-by: Claude
This commit is contained in:
2026-07-26 21:58:43 +02:00
parent 2af8f5ae6e
commit f707a2430d
2 changed files with 14 additions and 2 deletions
+5
View File
@@ -1838,6 +1838,11 @@ std::map<ggml_backend_buffer_type_t, size_t> llama_model::memory_breakdown() con
ret[buft] += ggml_backend_alloc_ctx_tensors_from_buft_size(ctx.get(), buft);
} else {
for (const auto & buf : bufs) {
// buf may be null if the device weights were released for on-demand VRAM sharing
// (see release_device_weights); skip it in the breakdown
if (!buf) {
continue;
}
// GGML_ASSERT(ggml_backend_buffer_get_base(buf.get()) != nullptr); // multi_buffer does not have a defined base
ret[ggml_backend_buffer_get_type(buf.get())] += ggml_backend_buffer_get_size(buf.get());
}