server: on-demand VRAM sharing to time-share one GPU between models

Add release/restore of a model's GPU weight buffers (keeping a host shadow
and the KV cache) so several always-loaded llama-server processes can
time-share a single GPU without reloading or losing the prompt cache.

- llama-model: release_device_weights()/restore_device_weights() capture a
  compact host shadow (stable iteration order, view-skipping) and free then
  realloc the device weight buffers; weights_resident() query.
- llama-context: release_device()/restore_device() wrappers; decode() auto-
  restores; public C API llama_context_release_device/restore_device.
- server: LLAMA_SLEEP_VRAM_ONLY makes idle-sleep release only the VRAM weights
  (not a full unload/reload). A cross-process flock token in LLAMA_VRAM_ARENA
  enforces "resident iff holds token"; an inotify doorbell forces the holder
  to release on contention. The warden thread only touches the task queue, so
  releases run on the loop thread and never race a decode.

Validated on RX 580 (Vulkan): two models share 8GB, never both resident,
correct output under contention, KV cache preserved (no re-prefill).

Assisted-by: Claude

src/llama-context.cpp

@@ -728,6 +728,26 @@ void llama_context::synchronize() {
     t_compute_start_us = 0;
 }
 
+void llama_context::release_device() {
+    if (!model.weights_resident()) {
+        return;
+    }
+    // ensure no compute is in flight before freeing the device buffers
+    synchronize();
+    model.release_device_weights();
+}
+
+void llama_context::restore_device() {
+    if (model.weights_resident()) {
+        return;
+    }
+    model.restore_device_weights();
+    // make sure all weight uploads have completed before any compute reads them
+    if (sched) {
+        ggml_backend_sched_synchronize(sched.get());
+    }
+}
+
 const llama_model & llama_context::get_model() const {
     return model;
 }
@@ -1705,6 +1725,12 @@ int llama_context::decode(const llama_batch & batch_inp) {
         return -1;
     }
 
+    // on-demand VRAM: if the weights were released while this model was idle, bring them back
+    // to the device before building the compute graph
+    if (!model.weights_resident()) {
+        restore_device();
+    }
+
     const auto & vocab   = model.vocab;
     const auto & hparams = model.hparams;
 
@@ -3674,6 +3700,18 @@ void llama_synchronize(llama_context * ctx) {
     ctx->synchronize();
 }
 
+void llama_context_release_device(llama_context * ctx) {
+    ctx->release_device();
+}
+
+void llama_context_restore_device(llama_context * ctx) {
+    ctx->restore_device();
+}
+
+bool llama_context_weights_resident(const llama_context * ctx) {
+    return ctx->get_model().weights_resident();
+}
+
 float * llama_get_logits(llama_context * ctx) {
     ctx->synchronize();