Feiteng and Xuan Son Nguyen
34ff2e047c
mtmd: drama3 voice cloning ( #32586 )
...
* mtmd: drama3 voice cloning
The Fish S2 codec encoder (fish_s2_spkenc) now also serves drama3:
- conversion: drama3 writes the codec encoder and the audio encoder KV, shared with fish_s2
- fish_s2_spkenc: outputs the codes of the clip (gen-side reference) and, in a drama3_gen mmproj,
builds the drama3 backbone rows (E_lm[semantic] + E_cb[1][code 1]) / sqrt(2)
- drama3_gen tower: teacher forcing when the caller gives the codes of the next frame
- drama3 pipeline: reference-bracket-v1 prompt with one reference (vLLM-Omni fish_omni), the
reference frames go through the backbone as codec rows and through the tower with their codes
- mtmd_gen_state_get_ref_codes(): read-only access to the reference codes of a gen state
Claude-Session: https://claude.ai/code/session_01PpX3DmaxNSmJbxGaxnUBrX
* fix correctness
* clean up
* rm redundant code
* fish_s2: rm unused import
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
2026-11-20 18:34:52 +01:00
..
2026-11-20 09:51:41 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-09-24 08:57:31 +02:00
2026-11-10 13:20:56 +01:00
2026-11-20 09:51:41 +02:00
2026-11-10 21:54:14 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-11-08 15:49:06 +01:00
2026-10-05 14:02:40 +02:00
2026-11-02 18:37:27 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-10-07 20:07:50 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-11-10 21:54:14 +01:00
2026-11-20 09:51:41 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-20 18:34:52 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-11-20 18:34:52 +01:00
2026-10-21 23:26:22 +02:00
2026-10-20 18:33:32 +02:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-10 13:00:03 +01:00
2026-08-17 10:15:11 +02:00
2026-11-16 23:49:57 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-10-20 18:54:27 +02:00
2026-11-15 21:25:34 +01:00
2026-09-16 15:18:45 +02:00
2026-11-10 21:54:14 +01:00
2026-09-07 15:20:58 +02:00
2026-11-16 20:39:32 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-10-06 18:42:24 +02:00
2026-11-06 23:42:46 +01:00
2026-11-10 21:54:14 +01:00
2026-11-10 11:28:47 +01:00
2026-11-17 22:25:44 +01:00
2026-10-17 13:00:17 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-10-27 23:36:32 +01:00
2026-11-08 15:49:06 +01:00
2026-08-17 10:15:11 +02:00
2026-10-29 10:13:41 +02:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-10-19 23:01:40 +02:00
2026-10-20 22:48:45 +02:00
2026-11-07 19:36:44 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-09-14 14:04:05 +03:00
2026-10-16 10:20:03 +02:00
2026-11-03 16:24:45 +01:00
2026-09-25 11:15:33 +08:00
2026-11-10 21:54:14 +01:00
2026-11-17 12:53:19 +01:00
2026-11-15 17:39:59 +01:00
2026-10-23 07:31:30 +02:00
2026-10-29 15:43:33 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-05-15 15:18:12 +02:00
2026-10-07 11:34:16 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-11-14 15:05:30 +01:00
2026-11-17 18:59:11 +01:00
2026-10-27 11:49:12 +01:00
2026-11-17 22:25:01 +08:00
2026-11-12 15:54:58 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-11-05 19:55:15 +01:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-10-21 21:54:16 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-10-24 01:09:50 +02:00
2026-09-06 17:43:58 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-08-17 10:15:11 +02:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-18 00:04:15 +01:00
2026-10-16 01:46:42 +02:00
2026-11-15 21:25:34 +01:00
2026-10-12 15:01:52 +02:00
2026-10-12 13:39:55 +02:00
2026-11-19 02:39:05 +01:00
2026-08-17 10:15:11 +02:00
2026-11-05 17:47:27 +01:00
2026-11-10 21:54:14 +01:00
2026-08-17 10:15:11 +02:00
2026-11-16 20:22:23 +01:00
2026-11-18 15:46:49 +01:00
2026-11-15 19:44:44 +01:00