 Xuan-Son NguyenandPascal
|
0713275082
|
mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) (#26254)
* convert text model
* main model load ok
* convert encoder ok
* speaker encoder loading ok
* speaker enc graph
* adapt vocab for backbone (with some tricks)
* add suppress_tokens
* poc new mtmd gen api
* convert code_predictor to gguf
* load gen_code model ok
* add clip_encode
* wire up
* code gen cgraph init version
Co-authored-by: Pascal <admin@serveurperso.com>
* code2wav convert to gguf
* code2wav graph ok
* wire up in/out
* (wip) subgraph
* wire up
* wip, correct code2wav
* demo (to be removed)
* code2wav preserve kv between calls
* demo voice clone
* llama: add llama_model_get_tok_embd
* mtmd_helper_gen_audio API
* fix clamp cold prefix
Co-authored-by: Pascal <admin@serveurperso.com>
* fuse snake op
Co-authored-by: Pascal <admin@serveurperso.com>
* demo: use proper sampling
* update dev docs
* polymorphism helper
* revamp llama-tts binary
* update docs
* fix compile
* fix lint
* nits
* add guide + docs
* more timings info
* clean up code comments
* security fixes
* update docs
* use ggml_build_forward_select, clean up comments
* fix ci
* use ISO 639-1 language code
* rename CODE2WAV --> GEN_WAV, update docs
* clean up
* clean up tts.cpp
* add seq_id
* add step_prompt()
* mtmd_helper_model_can_chat
* clean up comments
---------
Co-authored-by: Pascal <admin@serveurperso.com>
|
2026-08-04 17:26:15 +02:00 |
|
|
Georgi Gerganov
|
e3cab403bf
|
mtmd : add post-decode callback (#24645)
Assisted-by: pi:llama.cpp/Qwen3.6-27B
|
2026-06-15 16:02:05 +03:00 |
|
|
Xuan-Son Nguyen
|
e37abd6b5f
|
mtmd: add batching API (#24384)
* mtmd: add batching API
* wip
* first working version (gemma4v)
* add arg
* nits
* wire up support_batch()
* fix 0.0 output embd
* fix audio
* nits
* refactor a bit
* nits
* fix non-batching case
* fix comment
|
2026-06-13 00:10:29 +02:00 |
|
|
Xuan-Son Nguyen
|
8f83d6c271
|
mtmd : add video input support (#24269)
* wip
* ok: lazy bitmap API
* remember to free lazy text
* wip
* add mtmd_helper_video
* support video input on server (base64 input)
* add MTMD_VIDEO config
* add timestamp
* update CLI
* cli: allow auto-completion for video
* add --video arg
* fix build
* update docs
* rename as suggested
|
2026-06-08 14:40:12 +03:00 |
|
|
Xuan-Son Nguyen
|
f5c6ae1827
|
mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)
* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing
* fast path skip preproc for placeholder
* fix build
* correct the api
* add server endpoint + tests
* add object name
* update docs
* add proxy handling
* fix build
* fix audio input path
* use is_placeholder in process_mtmd_prompt()
* nits
* nits (2)
* docs: clarify chat/completions/input_tokens is not official
* fix merge problem
|
2026-06-06 11:06:51 +02:00 |
|
|
Xuan-Son Nguyen
|
19124078be
|
mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082)
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos
* fix build
|
2026-04-19 11:57:21 +02:00 |
|
|
65a
|
268d61e178
|
mtmd: add missing struct tag (#22023)
|
2026-04-17 10:48:33 +02:00 |
|
|
Xuan-Son Nguyen
|
707c0b7a6e
|
mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851)
* mtmd: add mtmd_image_tokens_get_decoder_pos() API
* consistent naming
* fix build
|
2026-04-14 16:07:41 +02:00 |
|
|
Xuan-Son Nguyen
|
9b17d74ab7
|
mtmd: add mtmd_log_set (#17268)
|
2025-11-14 15:56:19 +01:00 |
|
|
Xuan-Son Nguyen
|
10961339b2
|
mtmd : move helpers to dedicated library (⚠️ breaking change) (#13866)
* mtmd : move helpers to dedicated library
* fix server build
* rm leftover cmakelist code
|
2025-05-28 22:35:22 +02:00 |
|