Files
llama.cpp/conversion
Daniel Bevenius 9c9074e14e conversion : resolve quantized_layers from source tensor names (#31611)
* conversion : resolve quantized_layers from source tensor names

This commit changes how the per layer NVFP4 quant_algo are resolved from
hf_quant_config.json quantized_layers.

The keys in quantized_layers are module paths in the source checkpoint,
and these can be either be modules like 'lm_head' or parent modules
like 'layers.0.ffn.experts'. Currently these keys are passed through
filter_tensors, but they are not tensor names. So a filter_tensors that
keeps an explicit list of tensor names, like Nemotron's with --mtp
would drop them. That means there was no `_nvfp4_layer_algo entry`, and
`_tag_prec_a4` fell back to `_nvfp4_global_algo`.

This commit records the mapping from source tensor name to filtered
tensor name in index_tensors. Then in prepare_tensors each source
tensor is resolved to its closest quantized_layers parent, and
the algo is stored under the filtered tensor name, which is what
_tag_prec_a4 later looks up.

Converting Nemotron 3.5 Lightning NVFP4 with --mtp now produces:
```console
    general.tensor_extra.name = ['output.weight']
    general.tensor_extra.prec_a4 = [False]
```
whereas on master no general.tensor_extra fields are written. For the
non-MTP conversion, the general.tensor_extra fields are identical to
master.

Also manually verified (using pdb) that this works for
DeepSeek-V4.1-Flash-1017-NVFP4.

Refs: https://github.com/ggml-org/llama.cpp/pull/31602#issuecomment-6603337403

* conversion : shorten comment

* conversion : also remove '.bias' suffixes
2026-11-05 10:05:06 +01:00
..
2026-10-17 00:07:55 +02:00
2026-11-02 18:37:27 +01:00
2026-10-20 18:33:32 +02:00
2026-10-19 23:01:40 +02:00
2026-09-07 21:10:06 +02:00
2026-10-29 15:43:33 +01:00
2026-10-27 11:49:12 +01:00
2026-10-29 18:14:37 +01:00
2026-08-17 10:15:11 +02:00