llama: support both embd + raw tokens in batch (#29622)

* llama: support both embd + raw tokens in batch

* add to test-llama-archs

* also check case llm_arch_supports_mixed_batch = false

* constant graph topology

* have dedicated input for mixed case

* rm set_tensor_backend

* is_embd --> type

* consolidate m-rope pos handling into one place

* nits
This commit is contained in:
Xuan-Son Nguyen authored and GitHub committed 2026-10-05 01:35:49 +02:00
1 parent 2ca15f5404
commit 0bb496dbd3
29 files changed
+506 -99

No files matched your search

+1
View File
@@ -221,6 +221,7 @@ llama_model_deepseek2::graph_mtp::graph_mtp(const llama_model & model, const llm
ggml_set_input(inp->embd);
ggml_tensor * tok_embd;
ASSERT_EMBD_OR_TOKEN(ubatch);
if (ubatch.token) {
ggml_tensor * tok_embd_w = layer.nextn.embed_tokens
? layer.nextn.embed_tokens