Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

model: support speculators-format checkpoints for DSpark (#26275)

Browse Source 
* dspark: support speculators-format checkpoints (SpecForge exports)

Speculators-format DSpark drafts (e.g. SpecForge exports for the
Gemma-4-26B-A4B target) differ from the dense DeepSpec checkpoints in
three ways:

- the config nests the backbone hparams under transformer_layer_config
  and gives the extract layers as aux_hidden_state_layer_ids
- the block is the DFlash 1+N fill-in layout: the anchor slot is a bonus
  token, not a prediction slot. Written as dflash.bonus_anchor; such
  drafts build the block and read the mask positions exactly like
  DFlash (n_max drafts from a 1+n_max block), only the Markov/confidence
  sampling comes from DSpark
- the draft output vocab may be reduced (draft_vocab_size < vocab_size)
  with a d2t remap table. The converter expands lm_head/markov_w2 back
  to the full vocab and synthesizes an lm_head bias of -1e9 on the rows
  the draft cannot produce, so the runtime needs no d2t remapping. Such
  drafts ship their own (now optional) token_embd/output tensors instead
  of sharing the target's

Verified against gemma4-26b-a4b-dspark: greedy outputs are byte-identical
with and without the draft; acceptance 0.46, mean draft len 3.7 (n_max 6).

Co-authored-by: desovo7 <942845546@qq.com>
Assisted-by: Claude Fable 5

* dspark: fold the speculators draft class into DSparkModel

One class now covers every DSpark variant. What used to pick the class is
a single flag, because the arch name turns out to be the only thing that
separates the two families: SpecForge also exports a flat schema that
carries no speculators_* fields yet still uses the 1+N bonus-anchor block,
so keying on those fields would silently mis-read its drafts.

Also rename i0 to i_first_pred in the draft read loop and the Markov head,
and give the head a real bonus_anchor bool instead of testing i0 > 0.

Converting the Qwen3-8B DeepSpec draft and both gemma-4 speculators drafts
produces byte-identical GGUFs. The one behaviour change is that the
markov_head_type check now also covers the DeepSpec checkpoints, which
previously skipped it.

Co-authored-by: desovo7 <942845546@qq.com>
Assisted-by: Claude Opus 5

* dspark: address review comments

- rename bonus_anchor to sample_from_anchor (GGUF key and code), matching
  the checkpoint config field; absent key still means anchor-first
- rework the reduced draft vocab to match EAGLE3: d2t is written as I64
  absolute target ids and the logits are scattered at runtime, instead of
  expanding lm_head/markov_w2 and synthesizing an output bias at conversion
- move the t2d skip to modify_tensors, like EAGLE3
- drop _is_specforge: the arch name only picks the sample_from_anchor
  default, embed/lm_head sharing is decided by the draft vocab size
- deduplicate the tok_embd create_tensor left behind by the rebase

Verified with the RedHat gemma-4-31b speculator draft: greedy output is
byte-identical with and without the draft; acceptance 0.26 (n_max 7).

Co-authored-by: desovo7 <942845546@qq.com>
Assisted-by: Claude Fable 5

* dspark: fold the sample_from_anchor read into the block_size block

* dspark: fix flake8 continuation indent

* clean up

* dspark: key the sample_from_anchor default off the export format

  Co-authored-by: desovo7 <942845546@qq.com>
  Assisted-by: Claude Fable

* dspark: drop t2d in filter_tensors

  Co-authored-by: desovo7 <942845546@qq.com>
  Assisted-by: Claude Fable

* dspark: map model.lm_head instead of bypassing the dflash prefix

  Co-authored-by: desovo7 <942845546@qq.com>
  Assisted-by: Claude Fable

---------

Co-authored-by: desovo7 <942845546@qq.com>
Co-authored-by: ruixiang63 <wangruixiang07@outlook.com>
This commit is contained in:
王金旭
2026-08-17 13:51:06 +02:00
committed by GitHub
co-authored by desovo7 ruixiang63
parent 7077abbe14
commit 9cd719af21
8 changed files with 154 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gguf-py/gguf/constants.py
+3
View File
@@ -162,6 +162,7 @@ class Keys:
TARGET_LAYERS = "{arch}.target_layers"
TARGET_HIDDEN_SIZE = "{arch}.target_hidden_size"
BLOCK_SIZE = "{arch}.block_size"
SAMPLE_FROM_ANCHOR = "{arch}.sample_from_anchor"
NORM_BEFORE_RESIDUAL = "{arch}.norm_before_residual"
NORM_BEFORE_FC = "{arch}.norm_before_fc"
@@ -4819,6 +4820,7 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
],
MODEL_ARCH.DFLASH: [
MODEL_TENSOR.TOKEN_EMBD,
MODEL_TENSOR.OUTPUT,
MODEL_TENSOR.OUTPUT_NORM,
MODEL_TENSOR.ATTN_NORM,
MODEL_TENSOR.ATTN_Q,
@@ -4858,6 +4860,7 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
MODEL_TENSOR.FFN_UP_SHEXP,
MODEL_TENSOR.FC,
MODEL_TENSOR.ENC_OUTPUT_NORM,
MODEL_TENSOR.D2T,
# optional DSpark heads
MODEL_TENSOR.DSPARK_MARKOV_W1,
MODEL_TENSOR.DSPARK_MARKOV_W2,
gguf-py/gguf/gguf_writer.py
+3
View File
@@ -981,6 +981,9 @@ class GGUFWriter:
def add_block_size(self, value: int) -> None:
self.add_uint32(Keys.LLM.BLOCK_SIZE.format(arch=self.arch), value)
def add_sample_from_anchor(self, value: bool) -> None:
self.add_bool(Keys.LLM.SAMPLE_FROM_ANCHOR.format(arch=self.arch), value)
def add_target_layers(self, value: Sequence[int]) -> None:
self.add_array(Keys.LLM.TARGET_LAYERS.format(arch=self.arch), value)
gguf-py/gguf/tensor_mapping.py
+2 -2
View File
@@ -76,14 +76,14 @@ class TensorNameMap:
# Output
MODEL_TENSOR.OUTPUT: (
"embed_out", # gptneox
"lm_head", # gpt2 mpt falcon llama-hf baichuan qwen mamba dbrx jais nemotron exaone olmoe olmo2 phimoe plamo2
"lm_head", # gpt2 mpt falcon llama-hf baichuan qwen mamba dbrx jais nemotron exaone olmoe olmo2 phimoe plamo2 llama4
"output", # llama-pth bloom internlm2
"word_embeddings_for_head", # persimmon
"lm_head.linear", # phi2
"output_layer", # chatglm
"head", # rwkv
"head.out", # wavtokenizer
"lm_head", # llama4
"model.lm_head", # dflash
"model.transformer.ff_out", # llada
"head.decoder", # modern-bert
),