From f2bb4c7f32a453d7f8a37f6acfe5d55493bd5b01 Mon Sep 17 00:00:00 2001 From: Lumpiasty Date: Fri, 11 Sep 2026 03:26:15 +0200 Subject: [PATCH] llama: skip the compute-buffer size check when the context is cold release_device(evict_kv) drops the scheduler, so a context evicted by the VRAM arbiter and then destroyed reaches the destructor with a null sched. The upstream "compute buffer size matches expectation" loop calls ggml_backend_sched_get_buffer_size() unconditionally, whose GGML_ASSERT(sched) then aborts: the server exits 134 instead of 0 on every shutdown taken while cold, which under llama-swap makes an ordinary stop look like a crashed child and leaves a ggml backtrace in the log each time. It also skipped the rest of the destructor, so the cold teardown path had never actually run to completion. Guard at the call site rather than relaxing the assert - the assert is right, and every other caller reserves the scheduler first. Same shape as the null guards in synchronize(), memory_breakdown() and the released-buffer iteration. The loop is a diagnostic size comparison, already reported by sched_reserve() at load, so skipping it when cold loses nothing. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ --- src/llama-context.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/llama-context.cpp b/src/llama-context.cpp index f4c094271..94999da78 100644 --- a/src/llama-context.cpp +++ b/src/llama-context.cpp @@ -483,7 +483,8 @@ llama_context::~llama_context() { synchronize(); // when training, ggml_opt allocates extra buffers through the scheduler, so the sizes no longer match the expectation - if (!model.hparams.no_alloc && !opt_ctx) { + // the scheduler is also gone if the context is destroyed while cold (on-demand VRAM eviction, see release_device) + if (sched && !model.hparams.no_alloc && !opt_ctx) { for (size_t i = 0; i < backend_ptrs.size(); ++i) { ggml_backend_t backend = backend_ptrs[i]; ggml_backend_buffer_type_t buft = backend_buft[i];