Files
llama.cpp/common
Aman Gupta 2c192444c9 fit : account for skipped NextN layers when setting n_gpu_layers (#31569)
* fit : account for skipped NextN layers when setting n_gpu_layers

Without MTP, fit counts n_layer + 1 layers while the model loader counts
n_layer_all + 1 for n_gpu_layers, including the NextN layers which hold no
weights. A fit result that offloads all layers then leaves layer 0 on the CPU
("offloaded 65/66 layers" for Qwen3.8-27B on two GPUs).

Add the skipped NextN layers to n_gpu_layers and to the tensor split of the
device that holds the output layer so the loader assigns the layers that
fit measured.

Fixes #31531

* fit : pass hp_ngl_skip by reference like the other hparams

* fit : add hp_ngl_skip to common_get_device_memory_data
2026-11-05 10:31:48 +02:00
..
2026-09-21 10:32:07 +02:00
2026-10-09 10:06:22 -05:00