Commit Graph
2 Commits
Author SHA1 Message Date
Lumpiasty aa01a4c1c2 tmp: probe CONCAT shapes 2026-07-15 21:30:08 +02:00
Lumpiasty f4b0af3f1e ggml-vulkan: pin mmap CPU weights for faster H2D uploads
Export register_host_buffer/unregister via the backend reg so the existing
GGML_CUDA_REGISTER_HOST path in llama-model-loader pins mmap'd expert
weights on Vulkan. Imports host pages through VK_EXT_external_memory_host
into device->pinned_memory, letting the existing pinned fast path in
ggml_vk_buffer_write_2d_async DMA straight from system RAM instead of
bouncing through the staging buffer + blocking host memcpy.

A single Vulkan buffer cannot cover a multi-GB mmap (capped at
device->max_buffer_size), so the region is imported in page-aligned
chunks; a bound check makes tensors straddling a chunk boundary fall back
to staging instead of reading out of bounds. Opt-in via
GGML_CUDA_REGISTER_HOST=1 or GGML_VK_REGISTER_HOST=1, no-op otherwise.

Assisted-by: opencode
2026-07-15 20:43:18 +02:00