* mtmd: support YuE2-3B * improve speed * add CFG * fix streaming * add cfg control from api * check ctx size * fix n_predict * add flow_steps config * nits * fix batch api * nits * adapt * mtmd: yue2 samples from its own saved logits The context logits belong to the last decode of any slot, so with several server slots the score and the unguided codes were sampled from another sequence. Keep the logits of each own decode in save_logits(), which also does the guidance mix. * mtmd: let the yue2 snake chain fuse The beta reshape landed between sqr and mul, so the mul, sin, sqr, mul, add chain was not consecutive and the backends never fused it. Expand both reshapes first, like qwen3tts does. --------- Co-authored-by: Pascal <admin@serveurperso.com>
26 lines
1.3 KiB
Markdown
26 lines
1.3 KiB
Markdown
# llama.cpp TTS
|
|
|
|
This is a tool to demonstrate audio generation capability in llama.cpp via `libmtmd`. It was added via PR [#26254](https://github.com/ggml-org/llama.cpp/pull/26254)
|
|
|
|
Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.
|
|
|
|
## Common usage
|
|
|
|
Simple usage:
|
|
|
|
```sh
|
|
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav
|
|
```
|
|
|
|
Note: it's highly recommended to always provide a speaker reference voice (via `--tts-speaker-file`); otherwise, the model's performance may be degraded.
|
|
|
|
Common params:
|
|
- Sampling params such as `--top-k`, `--top-p`, `--temp`, etc. `--temp` also sets the temperature of the audio frames for models that support it, without it they use the model default
|
|
- `-n <number_of_frames>` limits the output length, e.g. `-n 500` (default: the limit of the model, 512 frames for most models). Note that how many milliseconds each frame represents varies by model
|
|
- `--tts-speaker-text` (optional) gives the transcript of `--tts-speaker-file`, some models put it in the prompt (e.g. Fish Audio S2 Pro)
|
|
- Core inference params such as `-ngl`, `-b`, `-ub`, etc.
|
|
|
|
## Supported models
|
|
|
|
See [docs/multimodal/tts.md](../../docs/multimodal/tts.md) for the list of supported models, their params and GGUF conversion notes.
|