Commit Graph
2 Commits
Author SHA1 Message Date
Pascal cb55ffe6e2 graph: skip the LM head for pooled embeddings (#30445)
* graph: skip the LM head for pooled embeddings

A context that extracts pooled embeddings outputs one vector per
sequence, but every model still expanded its LM head, so the graph
computed n_vocab logits per token and the context kept a host buffer
for them.

The models now only declare t_logits, and set_outputs() expands it
unless the context extracts embeddings with a pooling type other than
NONE. Since the toggle changes the graph shape for a pooled context,
set_embeddings() schedules a re-reserve in that case.

* graph: drop the redundant t_embd expand

Address review from @CISC: with embeddings on, build_pooling() already
expands its output for every pooling type, and a decoder graph without
embeddings does not read t_embd, so set_outputs() only expands
t_logits.

* graph: add build_logits() for the LM head output

Address review from @ggerganov: the LM head is expanded by a dedicated
build_logits(), called from llama_model::build_graph() like
build_pooling(), and set_outputs() only marks the outputs again.

* graph: call build_logits() before build_sampling()

Address review from @ggerganov: build_sampling() reads t_logits, so the
LM head is built first. A pooled context then skips the sampling layers
too, since it has no logits to sample from.

* nit
2026-10-13 16:16:14 +02:00
49c0dc82b8 model : add Tencent Hy 4 (hy_v4) preview architecture support (#28127)
* model: add Tencent Hy 4 (hy_v4) preview architecture support

Adds support for the Tencent Hy 4 model (Hugging Face architecture
HYV4ForCausalLM, GGUF arch hy_v4):

Add HF -> GGUF conversion script (conversion/hy_v4.py) and wire it into the conversion registry
Register hy_v4 GGUF constants, arch enum, and writer support
Implement the hy-v4 model graph, hparams, vocab and context changes
Register the new arch in llama-arch and models registry
Extend arch tests to cover hy_v4

Assisted by Claude Opus 5

* Update convert_hf_to_gguf_update.py

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* Update conversion/base.py

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>

* convert : move hy_v4 entry to the same place as in convert_hf_to_gguf_update.py

* model : apply changes related to n_ff_exp becoming per-layer in Hy4-preview

* n_layer_all

---------

Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-09-04 14:31:36 +02:00