llama: add default load-mode auto, which avoids mmap on iGPUs (#26081)
* llama: add new default load-mode auto which picks mmap unless a non-Metal iGPU is used * Update ggml/src/ggml-hexagon/ggml-hexagon.cpp Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com> * set mmap_support to false on OpenCL backend * fix order of load modes * use -1 for auto * resolve load mode auto earlier to correctly pick gpu host or cpu memory * add load mode auto to llama-bench * bump virtgpu api version, regenerate docs --------- Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com> Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
co-authored by
Max Krasnyansky
Piotr Wilkin
Georgi Gerganov
parent
b3df57286c
commit
153d324bcf
@@ -154,6 +154,8 @@ extern "C" {
|
||||
bool buffer_from_host_ptr;
|
||||
// event synchronization
|
||||
bool events;
|
||||
// mmap is supported for loading
|
||||
bool mmap_support;
|
||||
};
|
||||
|
||||
// all the device properties
|
||||
|
||||
Reference in New Issue
Block a user