llama : apply the iGPU lazy-mode fallback only when mmap is off (#31957)
On a device without mmap support, --lazy-mode auto resolved to off even with --load-mode mmap, so llama_prefetch_rows never ran and the rows of the large tensors faulted in one page at a time. Resolve it to off only when mmap is off; the default iGPU load path is unchanged. Fixes #31943
This commit is contained in:
1 parent
d5bbcd1f31
commit
58e38bdf65
1 file changed
+2
-2
+2
-2
@@ -1697,8 +1697,8 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
|
||||
}
|
||||
}
|
||||
|
||||
// resolve AUTO on systems without mmap support (e.g. iGPUs): fall back to OFF; see #28160
|
||||
if (ml.lazy.mode == LLAMA_LAZY_MODE_AUTO) {
|
||||
// resolve AUTO on systems without mmap support (e.g. iGPUs): fall back to OFF unless mmap is in use; see #28160, #31943
|
||||
if (ml.lazy.mode == LLAMA_LAZY_MODE_AUTO && !ml.use_mmap) {
|
||||
for (const auto & dev : devices) {
|
||||
ggml_backend_dev_props props;
|
||||
ggml_backend_dev_get_props(dev.dev, &props);
|
||||
|
||||
Reference in new issue
Block a user