mtmd: support kyutai/hibiki realtime full-duplex (#32491)

* mtmd: support hibiki realtime duplex

* fix and clean up

* fix long session quality

* fix ci

* tests : add the moshi fusion baseline for Metal

The Metal fusion check found 5 RMS_NORM+MUL fusions in the new moshi
arch and no baseline for them.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Xuan-Son NguyenandPascal authored and GitHub committed 2026-11-21 14:25:19 +01:00
1 parent c661033349
commit c622f623fc
31 files changed
+1845 -231

No files matched your search

+4
View File
@@ -120,6 +120,10 @@ struct mtmd_cli_context {
init_vision_context(params);
mtmd_cap = mtmd_helper_get_capabilities(model, ctx_vision.get());
if (mtmd_cap.audio_duplex) {
LOG_ERR("Full duplex models only run in llama-server, see the /v1/realtime endpoint\n");
exit(1);
}
// a realtime transcription model has no chat, it streams the audio below
if (!mtmd_cap.chat && !mtmd_cap.audio_stream) {
LOG_ERR("Model does not support chat mode\n");