Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

llama: add default load-mode auto, which avoids mmap on iGPUs (#26081)

Browse Source 
* llama: add new default load-mode auto which picks mmap unless a non-Metal iGPU is used

* Update ggml/src/ggml-hexagon/ggml-hexagon.cpp

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>

* set mmap_support to false on OpenCL backend

* fix order of load modes

* use -1 for auto

* resolve load mode auto earlier to correctly pick gpu host or cpu memory

* add load mode auto to llama-bench

* bump virtgpu api version, regenerate docs

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Ruben Ortlam
2026-08-11 09:20:46 +03:00
committed by GitHub
co-authored by Max Krasnyansky Piotr Wilkin Georgi Gerganov
parent b3df57286c
commit 153d324bcf
33 changed files with 75 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-backend-meta.cpp
+2
View File
@@ -132,6 +132,7 @@ static void ggml_backend_meta_device_get_props(ggml_backend_dev_t dev, ggml_back
/* .host_buffer = */ false, // Not implemented.
/* .buffer_from_host_ptr = */ false, // Not implemented.
/* .events = */ false, // Not implemented.
/* .mmap_support = */ true,
};
for (ggml_backend_dev_t simple_dev : meta_dev_ctx->simple_devs) {
ggml_backend_dev_props tmp_props;
@@ -140,6 +141,7 @@ static void ggml_backend_meta_device_get_props(ggml_backend_dev_t dev, ggml_back
props->caps.host_buffer = props->caps.host_buffer && tmp_props.caps.host_buffer;
props->caps.buffer_from_host_ptr = props->caps.buffer_from_host_ptr && tmp_props.caps.buffer_from_host_ptr;
props->caps.events = props->caps.events && tmp_props.caps.events;
props->caps.mmap_support = props->caps.mmap_support && tmp_props.caps.mmap_support;
}
}