Files
llama.cpp/tools
Lumpiasty f5c6f04b4e mtmd: release/restore the encoder weights from VRAM on demand
Add clip_release_device/clip_restore_device (and mtmd_release_device/
mtmd_restore_device wrappers over the vision + audio contexts) that free
the multimodal encoder's device weight buffer to a read-only host shadow
and rebuild it on demand, using the same shadow/free/reallocate pattern
as llama_model weights. No-op for a CPU-backed encoder. This lets the
server drop the ~hundreds-of-MiB vision encoder from VRAM when it is not
encoding an image.

Assisted-by: Claude
2026-11-22 20:38:32 +01:00
..
2026-11-12 20:59:09 +02:00
2026-11-04 10:10:22 +02:00
2026-11-12 20:59:09 +02:00
2026-11-12 20:59:09 +02:00