Files
llama.cpp/ggml
Lumpiasty cfdc33ec83 ggml-vulkan: pre-stage host weights when import is unavailable
VK_EXT_external_memory_host import fails on RADV for file-backed mmap pages,
so register_host_buffer no-op'd and every MoE-expert upload paid a slow
single-threaded pageable memcpy into the staging buffer each eval (~3 GB/s
effective on the 35B, ~25% of PCIe bandwidth).

When the import fails, fall back to a one-time copy of the region into a
host-visible Vulkan buffer registered in device->pinned_memory under the
mmap's address range. The existing pinned fast path then DMAs straight from
that buffer each eval at full PCIe bandwidth - no per-eval CPU memcpy and no
blocking sync. Costs one-time host-visible memory equal to the CPU-resident
weight region.

Assisted-by: opencode
2026-07-26 21:58:43 +02:00
..
2024-07-13 18:12:39 +02:00