Commit Graph
1 Commits
Author SHA1 Message Date
thecodacusandLumpiasty 5a0c899424 llama : pin mmap-backed CPU weights for faster H2D uploads
Wire the existing GGML_CUDA_REGISTER_HOST path back up: after model load,
cudaHostRegister the mmap pages backing weights kept in system memory.
Recovers pageable-copy losses when MoE experts are streamed to the GPU
during prefill (n-cpu-moe): Qwen3.6-35B-A3B pp2048 1144 -> 1385 t/s on
RTX 3060. Opt-in via GGML_CUDA_REGISTER_HOST=1, unchanged otherwise.
2026-07-22 21:04:20 +02:00