* add chatterbox TTS support Co-authored-by: Pascal <admin@serveurperso.com> * add built-in voice support * rm redundant code * clean up * clean up 2 * Apply batched suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * improve --------- Co-authored-by: Pascal <admin@serveurperso.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
llama.cpp TTS
This is a tool to demonstrate audio generation capability in llama.cpp via libmtmd. It was added via PR #26254
Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.
Common usage
Simple usage:
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav
Note: it's highly recommended to always provide a speaker reference voice (via --tts-speaker-file); otherwise, the model's performance may be degraded.
Common params:
- Sampling params such as
--top-k,--top-p,--temp, etc. -n <number_of_frames>limits the output length, e.g.-n 500. Note that how many milliseconds each frame represents varies by model- Core inference params such as
-ngl,-b,-ub, etc.
Qwen3-TTS
Available params:
--tts-langcan bezh,en,de,it,pt,es,ja,ko,fr,ru(default:en)--tts-speaker-fileshould point to a speaker reference audio file (wav, mp3)
Example usage:
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
-p "Hello world" \
--tts-lang english \
--tts-speaker-file speaker.mp3 \
--output out.wav
Pocket TTS
Available params:
--tts-speaker-fileshould point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it--tts-voiceselects one of the predefined voices shipped with the weights (e.g.giovanni), when the GGUF holds them. The converter reads them fromembeddings/*.safetensorsnext to the weights, of both repositories, and logs how closely each one is reproduced (some packs ship voice states computed with other weights, the converter warns above 5% and skips a voice only when it does not belong to the model at all)- Note:
langis not used, the language is a property of the weights
Example usage:
llama-tts -m pocket-tts.gguf \
-mm mmproj-pocket-tts.gguf \
-p "Hello world" \
--tts-speaker-file speaker.mp3 \
--output out.wav
The weights without voice cloning ship a set of predefined voices (embeddings/*.safetensors) next to each language; the converter stores them in the mmproj, as the voice prompt embeddings, and leaves out the zeroed voice encoder. With such a model --tts-speaker-file is rejected and a voice is selected with --tts-voice (the first one when it is not given):
llama-tts -m pocket-tts.gguf \
-mm mmproj-pocket-tts.gguf \
-p "Ciao a tutti" \
--tts-voice giovanni \
--output out.wav
Note for GGUF conversion:
The upstream repository holds one complete model per language under languages/, next to a set of shared files at the root. Convert one of the languages/<name> directories, not the root directory:
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf
Soprano
Available params:
--tts-speaker-fileis not supported; the model uses a fixed voice- Note:
langis not used. Supply normalized English text
Example usage:
llama-tts -m soprano.gguf \
-mm mmproj-soprano.gguf \
-p "Hello world" \
--temp 0 \
--output out.wav
Note for GGUF conversion:
Both Soprano-80M and Soprano-1.1-80M are supported. Convert both files from the same model directory, including decoder.pth:
python convert_hf_to_gguf.py path/to/Soprano-1.1-80M --outtype f32 --outfile soprano.gguf
python convert_hf_to_gguf.py path/to/Soprano-1.1-80M --mmproj --outtype f16 --outfile mmproj-soprano.gguf
LFM3-Audio
Available params:
--tts-langcan bear,da,de,el,en,en-gb,es,fi,fr,he,it,ja,ko,nl,pl,pt,sv,tr,zh(default:en), other values are used as the language name--tts-voicecan befemale(default) ormale- Note:
--tts-speaker-fileis not used, the model has no voice cloning - Note:
--top-kand--top-pare not used, audio frames are sampled with the settings of the model card (LFM3-Audio: temperature 1.0, top-k 128, repetition penalty 1.1 over 4096 frames; LFM3-Omni: temperature 0.4, repetition penalty 1.2 over 64 frames), theaudio_*keys ofgeneration_config.jsonoverride them at conversion - Each frame is 80 ms, generation stops at the end-of-audio frame
Example usage:
llama-tts -m lfm3-audio.gguf \
-mm mmproj-lfm3-audio.gguf \
-p "Hello world" \
--tts-voice male \
--output out.wav
The mmproj holds the audio encoder (for ASR), the depthformer and the codec decoder, the latter two are exported when the checkpoint has an audio_tokenizer directory:
python convert_hf_to_gguf.py path/to/lfm3-audio --outfile lfm3-audio.gguf
python convert_hf_to_gguf.py path/to/lfm3-audio --mmproj --outfile mmproj-lfm3-audio.gguf
Chatterbox
Two variants are supported: Chatterbox Turbo (English) and Chatterbox Multilingual V3 (23 languages).
Available params:
--tts-speaker-fileshould point to a speaker reference audio file (wav, mp3). Optional, the predefined voice is used without it--tts-voicecan bedefault, the voice shipped with the model (conds.pt)--tts-langsets the language of the multilingual variant, for examplefr(default: none). It is not used by Turbo
Example usage:
llama-tts -m chatterbox-mtl.gguf \
-mm mmproj-chatterbox-mtl.gguf \
-p "Bonjour tout le monde" \
--tts-lang fr \
--tts-speaker-file speaker.mp3 \
--output out.wav
Note: the multilingual variant runs classifier-free guidance with a second sequence. llama-tts sets this up by itself, but llama-server does not support it yet.
Note for GGUF conversion:
The upstream repositories ship raw checkpoints without config.json. Download the files below into a directory, then convert it as usual (the variant is detected from the talker file):
- Turbo:
t3_turbo_v1.safetensors,s3gen_meanflow.safetensors,ve.safetensors,conds.ptand the GPT-2 tokenizer files (vocab.json,merges.txt,added_tokens.json,special_tokens_map.json,tokenizer_config.json) - Multilingual V3:
t3_mtl23ls_v3.safetensors,s3gen_v3.safetensors,ve.safetensors,conds.pt,mtl_tokenizer.json
python convert_hf_to_gguf.py path/to/chatterbox --outfile chatterbox.gguf
python convert_hf_to_gguf.py path/to/chatterbox --mmproj --outfile mmproj-chatterbox.gguf