# Text-to-speech (TTS) llama.cpp supports audio generation via `libmtmd`. The following tools can be used: - [llama-tts](../../tools/tts/README.md), see its README for common usage and params - [llama-server](../../tools/server/README.md) via the `/tts` endpoint and the OpenAI-compatible `/v1/audio/speech` API Supported models: - [Qwen3-TTS](#qwen3-tts) - [Pocket TTS](#pocket-tts) - [Soprano](#soprano) - [LFM3-Audio](#lfm3-audio) - [Chatterbox](#chatterbox) - [Drama 3](#drama-3) - [ZONOS3](#zonos3) The sections below list the params supported by each model, with `llama-tts` examples. In `llama-server`, the same options are set with `--tts-lang` and `--tts-voice`, or per request (see the [`/tts` endpoint](../../tools/server/README.md#post-tts-generate-speech-audio-from-text)). ## Qwen3-TTS Available params: - `--tts-lang` can be `zh`, `en`, `de`, `it`, `pt`, `es`, `ja`, `ko`, `fr`, `ru` (default: `en`) - `--tts-speaker-file` should point to a speaker reference audio file (wav, mp3) Example usage: ```sh llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \ -p "Hello world" \ --tts-lang english \ --tts-speaker-file speaker.mp3 \ --output out.wav ``` ## Pocket TTS Available params: - `--tts-speaker-file` should point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it - `--tts-voice` selects one of the predefined voices shipped with the weights (e.g. `giovanni`), when the GGUF holds them. The converter reads them from `embeddings/*.safetensors` next to the weights, of both repositories, and logs how closely each one is reproduced (some packs ship voice states computed with other weights, the converter warns above 5% and skips a voice only when it does not belong to the model at all) - Note: `lang` is not used, the language is a property of the weights Example usage: ```sh llama-tts -m pocket-tts.gguf \ -mm mmproj-pocket-tts.gguf \ -p "Hello world" \ --tts-speaker-file speaker.mp3 \ --output out.wav ``` The weights [without voice cloning](https://huggingface.co/kyutai/pocket-tts-without-voice-cloning) ship a set of predefined voices (`embeddings/*.safetensors`) next to each language; the converter stores them in the mmproj, as the voice prompt embeddings, and leaves out the zeroed voice encoder. With such a model `--tts-speaker-file` is rejected and a voice is selected with `--tts-voice` (the first one when it is not given): ```sh llama-tts -m pocket-tts.gguf \ -mm mmproj-pocket-tts.gguf \ -p "Ciao a tutti" \ --tts-voice giovanni \ --output out.wav ``` **Note for GGUF conversion:** The [upstream repository](https://huggingface.co/kyutai/pocket-tts) holds one complete model per language under `languages/`, next to a set of shared files at the root. Convert one of the `languages/` directories, **not** the root directory: ```sh python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf ``` ## Soprano Available params: - `--tts-speaker-file` is not supported; the model uses a fixed voice - Note: `lang` is not used. Supply normalized English text Example usage: ```sh llama-tts -m soprano.gguf \ -mm mmproj-soprano.gguf \ -p "Hello world" \ --temp 0 \ --output out.wav ``` **Note for GGUF conversion:** Both [Soprano-80M](https://huggingface.co/ekwek/Soprano-80M) and [Soprano-1.1-80M](https://huggingface.co/ekwek/Soprano-1.1-80M) are supported. Convert both files from the same model directory, including `decoder.pth`: ```sh python convert_hf_to_gguf.py path/to/Soprano-1.1-80M --outtype f32 --outfile soprano.gguf python convert_hf_to_gguf.py path/to/Soprano-1.1-80M --mmproj --outtype f16 --outfile mmproj-soprano.gguf ``` ## LFM3-Audio Available params: - `--tts-lang` can be `ar`, `da`, `de`, `el`, `en`, `en-gb`, `es`, `fi`, `fr`, `he`, `it`, `ja`, `ko`, `nl`, `pl`, `pt`, `sv`, `tr`, `zh` (default: `en`), other values are used as the language name - `--tts-voice` can be `female` (default) or `male` - Note: `--tts-speaker-file` is not used, the model has no voice cloning - Note: `--top-k` and `--top-p` are not used, audio frames are sampled with the settings of the model card (LFM3-Audio: temperature 1.0, top-k 128, repetition penalty 1.1 over 4096 frames; LFM3-Omni: temperature 0.4, repetition penalty 1.2 over 64 frames), the `audio_*` keys of `generation_config.json` override them at conversion - Each frame is 80 ms, generation stops at the end-of-audio frame Example usage: ```sh llama-tts -m lfm3-audio.gguf \ -mm mmproj-lfm3-audio.gguf \ -p "Hello world" \ --tts-voice male \ --output out.wav ``` The mmproj holds the audio encoder (for ASR), the depthformer and the codec decoder, the latter two are exported when the checkpoint has an `audio_tokenizer` directory: ```sh python convert_hf_to_gguf.py path/to/lfm3-audio --outfile lfm3-audio.gguf python convert_hf_to_gguf.py path/to/lfm3-audio --mmproj --outfile mmproj-lfm3-audio.gguf ``` ## Chatterbox Two variants are supported: [Chatterbox Turbo](https://huggingface.co/ResembleAI/chatterbox-turbo) (English) and [Chatterbox Multilingual V3](https://huggingface.co/ResembleAI/chatterbox) (23 languages). Available params: - `--tts-speaker-file` should point to a speaker reference audio file (wav, mp3). Optional, the predefined voice is used without it - `--tts-voice` can be `default`, the voice shipped with the model (`conds.pt`) - `--tts-lang` sets the language of the multilingual variant, for example `fr` (default: none). It is not used by Turbo Example usage: ```sh llama-tts -m chatterbox-mtl.gguf \ -mm mmproj-chatterbox-mtl.gguf \ -p "Bonjour tout le monde" \ --tts-lang fr \ --tts-speaker-file speaker.mp3 \ --output out.wav ``` Note: the multilingual variant runs classifier-free guidance with a second sequence. `llama-tts` sets this up by itself, but `llama-server` does not support it yet. **Note for GGUF conversion:** The upstream repositories ship raw checkpoints without `config.json`. Download the files below into a directory, then convert it as usual (the variant is detected from the talker file): - Turbo: `t3_turbo_v1.safetensors`, `s3gen_meanflow.safetensors`, `ve.safetensors`, `conds.pt` and the GPT-2 tokenizer files (`vocab.json`, `merges.txt`, `added_tokens.json`, `special_tokens_map.json`, `tokenizer_config.json`) - Multilingual V3: `t3_mtl23ls_v3.safetensors`, `s3gen_v3.safetensors`, `ve.safetensors`, `conds.pt`, `mtl_tokenizer.json` ```sh python convert_hf_to_gguf.py path/to/chatterbox --outfile chatterbox.gguf python convert_hf_to_gguf.py path/to/chatterbox --mmproj --outfile mmproj-chatterbox.gguf ``` ## Drama 3 [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3): a Qwen3.5-MoE backbone (35B, 3B active) samples each frame's main token, a codebook tower samples the 9 other codes, and the Fish DAC decodes them to 44.1 kHz audio. Available params: - Speaker tags `<|speaker:0|>` to `<|speaker:15|>` in the text give each character its own voice, a text without a tag starts with `<|speaker:0|>` - The language is the one of the text, `--tts-lang` is not used - Note: `--tts-speaker-file` is not supported yet (voice cloning) - Note: `--top-k` and `--top-p` are not used, frames are sampled with the settings of the reference (main token: temperature 1.0, top-p 0.9, repetition-aware redraw over the last 16 frames; other codes: temperature 1.0, top-k 30, top-p 0.9) - Each frame is 2048 samples (about 46 ms), generation stops at the end-of-speech token - The codebook tower keeps the whole sequence in its KV cache, the prompt and the generated frames are limited to 4096 positions Example usage: ```sh llama-tts -m drama3.gguf \ -mm mmproj-drama3.gguf \ -p "<|speaker:0|> Did you remember to buy the bread? <|speaker:1|> Of course I did, it is on the kitchen table." \ -n 1000 \ --output out.wav ``` **Note for GGUF conversion:** The routed experts of the checkpoint are FP8, they are dequantized at conversion. The mmproj holds the codebook tower and the decoder half of `codec.pth`, it is best kept in BF16: ```sh python convert_hf_to_gguf.py path/to/drama-3 --outtype q8_0 --outfile drama3.gguf python convert_hf_to_gguf.py path/to/drama-3 --mmproj --outtype bf16 --outfile mmproj-drama3.gguf ``` ## ZONOS3 [Zyphra ZONOS3 Mini](https://huggingface.co/Zyphra/ZONOS3-Mini): a hybrid Gated DeltaNet / attention MoE backbone (800M, 100M active) predicts the 10 codebooks of each frame with an inter-codebook delay, and the ZAC codec decodes them to 44.1 kHz audio. Available params: - `--tts-voice` picks one of the predefined voices of the release (`Savannah_en-American_F_1` by default); `llama-server` lists them in `GET /props`. A voice is the prompt prefix of its reference codes, the default speaker mode of the reference - The language is the one of the text; `--tts-lang` (`lang` in `llama-server`) only selects the text normalization - Numbers are spelled out by the `tts` chat template of the GGUF: cardinals, decimals, money, percents, clock times, common ordinal forms (`1st`, `1er`, `1ยบ`), English and German years. A 4-digit number is read as a year only after a word like `in`, `since`, a month name, or `im Jahr`, `seit` in German (`in 1990` -> "in nineteen ninety", `1234 people` -> "one thousand two hundred thirty-four people"). Languages: `en`, `fr`, `de`, `es`, `it`, `pt`, `ja` and `zh` (or `cmn`), the region is ignored (`fr_fr`, `pt_br`); English when the language is empty, other languages are left as written. Set the language to the one of the text: the separators differ, e.g. `1.234` is a thousand in German - Note: the reference uses NeMo text normalization, which also covers units, dates, abbreviations and Korean; supply normalized text for them - `--tts-speaker-file` clones the voice of a clip (the first 60 s are used) with the speaker encoder of the release, the "embedding" speaker mode of the reference - `llama-server` also takes `speed` (relative to 15 UTF-8 bytes per second) and `loudness` (LUFS), see the `/tts` endpoint; they pick the speaking-rate and loudness condition buckets of the checkpoint, the defaults leave them unknown - Note: `loudness` has a strong effect with a cloned voice; a predefined voice mostly keeps the loudness of its reference clip - Note: `--top-k` and `--top-p` are not used, frames are sampled with the settings of the reference: temperature 0.75 (`--temp` changes it), top-k 80, top-p 0.69, repetition penalty 2.4 over the last 96 frames of codebooks 0 to 5 - Each frame is 2940 samples (about 67 ms), generation stops at the end-of-audio code Example usage: ```sh llama-tts -m zonos3-mini.gguf \ -mm mmproj-zonos3-mini.gguf \ -p "The old lighthouse keeper climbed the spiral stairs every evening." \ --tts-voice Imogen_en-British_F_1 \ --output out.wav ``` **Note for GGUF conversion:** Convert the root of the release repository, not the `lm/` subdirectory. The mmproj holds the codebook heads, the predefined voices, the speaker encoder and the `keep2_plain` codec decoder, it is best kept in F32: ```sh python convert_hf_to_gguf.py path/to/ZONOS3-Mini --outtype q8_0 --outfile zonos3-mini.gguf python convert_hf_to_gguf.py path/to/ZONOS3-Mini --mmproj --outtype f32 --outfile mmproj-zonos3-mini.gguf ```