Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Dflash support for nemotron-3.5 (#26905)

Browse Source 
* conversion: skip untrained DFlash embeddings

* Add Nemotron DFlash support

* Add DFlash NVFP4 support

* Address review comments

* add missing output_s for nvfp4

* Include change for keeping residual for last layer also if requested in future dflash models

* Update conversion/qwen.py

Defensive check, not needed

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Fixing bug introduced by merge conflict

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
This commit is contained in:
lnigam
2026-08-11 18:46:26 +05:30
committed by GitHub
co-authored by Sigbjørn Skjæret
parent 6e62ba5384
commit cc078b45b6
6 changed files with 42 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
conversion/base.py
+1 -1
View File
@@ -829,7 +829,7 @@ class ModelBase:
elif any(str(v.get("quant_algo")).endswith("NVFP4") for v in quant_layers.values() if isinstance(v, dict)):
quant_algo = "NVFP4"
self._is_nvfp4 = quant_algo == "NVFP4"
self._is_nvfp4 = quant_algo in ("NVFP4", "W4A16_NVFP4")
self._is_mxfp4 = quant_method == "mxfp4"
# NVFP4 weights are repacked and written directly to gguf_writer.