mtmd: fix silent prompt truncation on embedded NUL (#25548)

* mtmd: fix silent prompt truncation on embedded NUL

mtmd_input_text carried the prompt as a bare const char* with no
length, so a NUL byte in message content cut the prompt at the
tokenizer boundary and dropped every later message plus the assistant
marker, with no log. Add an explicit text_len and thread it through,
matching llama_tokenize and the text only path.

* cleanup

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
Pascal
2026-07-13 00:47:25 +02:00
committed by GitHub
co-authored by Xuan Son Nguyen
parent 0c4fa7a989
commit 4114ba18b2
4 changed files with 8 additions and 4 deletions
+1
View File
@@ -67,6 +67,7 @@ struct mtmd_batch;
struct mtmd_input_text {
const char * text;
size_t text_len;
bool add_special;
bool parse_special;
};