llama: add default load-mode auto, which avoids mmap on iGPUs (#26081)

* llama: add new default load-mode auto which picks mmap unless a non-Metal iGPU is used

* Update ggml/src/ggml-hexagon/ggml-hexagon.cpp

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>

* set mmap_support to false on OpenCL backend

* fix order of load modes

* use -1 for auto

* resolve load mode auto earlier to correctly pick gpu host or cpu memory

* add load mode auto to llama-bench

* bump virtgpu api version, regenerate docs

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Ruben Ortlam
2026-08-11 09:20:46 +03:00
committed by GitHub
co-authored by Max Krasnyansky Piotr Wilkin Georgi Gerganov
parent b3df57286c
commit 153d324bcf
33 changed files with 75 additions and 29 deletions
+1
View File
@@ -17891,6 +17891,7 @@ static void ggml_backend_vk_device_get_props(ggml_backend_dev_t dev, struct ggml
/* .host_buffer = */ true,
/* .buffer_from_host_ptr = */ false,
/* .events = */ true,
/* .mmap_support = */ !ctx->is_integrated_gpu,
};
}