mtmd: support kyutai/hibiki realtime full-duplex (#32491)

* mtmd: support hibiki realtime duplex

* fix and clean up

* fix long session quality

* fix ci

* tests : add the moshi fusion baseline for Metal

The Metal fusion check found 5 RMS_NORM+MUL fusions in the new moshi
arch and no baseline for them.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Xuan-Son NguyenandPascal authored and GitHub committed 2026-11-21 14:25:19 +01:00
1 parent c661033349
commit c622f623fc
31 files changed
+1845 -231

No files matched your search

+1 -1
View File
@@ -6132,7 +6132,7 @@ void server_routes::init_routes() {
// models that answer after the audio hold a conversation, see server-realtime.h
// live transcription: Voxtral Realtime (no chat), or an "asr" model in a session opened with ?intent=transcription
const bool live_asr = !ctx_server.asr_prompt.empty() && req.get_param("intent") == "transcription";
if (ctx_server.mtmd_cap.audio_stream && ctx_server.mtmd_cap.chat && !live_asr) {
if (ctx_server.mtmd_cap.audio_duplex || (ctx_server.mtmd_cap.audio_stream && ctx_server.mtmd_cap.chat && !live_asr)) {
server_realtime_conversation({queue_tasks, queue_results, *meta, params, ctx_server.mctx, ctx_server.mtmd_cap, ctx_server.vocab}, conn);
return;
}