Commit Graph
12608 Commits
Author SHA1 Message Date
shalinib-ibm 93cb37ea7b ggml-cpu: fix nc aligment in tinyBLAS_PPC_Q0 (#32659)
Fixes https://github.com/ggml-org/llama.cpp/issues/31680

The matmul_tiled() implementation in tinyBLAS_Q0_PPC requires nc to be a multiple of 8 to perform matrix multiplication correctly.

The previous alignment logic contained a special case for n == 4, which allowed n_aligned to be 4. Consequently, the tiled path could be selected with nc = 4, violating the kernel's tile-width requirement.

This caused incorrect matrix multiplication results, observed as garbled inference output when running  Flint Q4_K_M and Gemma 3 27B Instruct.
2026-11-21 12:58:49 +02:00
Aaron Teo 1fdda89a4d ci: switch ppc64le runners to ubuntu 26.04 (#32679)
* ci: switch ppc64le runners to ubuntu 26.04

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: rm problem matchers experiment

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-11-21 10:54:00 +01:00
Georgi Gerganov e412c6f468 llama : remove fused lightning indexer gating (#32650)
The fused lightning indexer is always used now, so drop the cparams flags, the auto probe and the non-fused fallbacks.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-21 10:26:55 +02:00
Xuan-Son Nguyen 40b7a546de ci: temporary disable no realloc case for mtmd (#32708) 2026-11-21 10:02:26 +02:00
Daniel Kuts 46eb1742db llama-grammar: Fix integer overflow (#32551)
* llama-grammar: Fix integer overflow

* Check incorrect values at parsing and bail out nicely

* fix style
2026-11-21 07:00:30 +01:00
Aman Gupta 6792507b7f llama : score only the k-pools of the ubatch sequences in QSA (#32519) 2026-11-21 13:22:36 +08:00
cwriter 418fe97b5f sycl: Improve Grouped XMX GEMM by avoiding SLM staging (#31428) 2026-11-20 22:37:40 -05:00
Ashutosh Singh 7476e47261 docs: openvino backend details, fix SYCL docs (#30967) 2026-11-20 22:08:53 -05:00
Pedro Cuenca 29b52d398b model : Inkling 2 Mini (#32694)
* Conversion

* Separate model_type

* Add gate_rank to allow building from metadata while testing

* Base model

* Loop graph

* Support the new is_varloop attribute

* Adopt naming scheme from https://github.com/ggml-org/llama.cpp/pull/32588

* Rename to varying
2026-11-20 23:01:07 +01:00
Pelayo Garcia BárcenaandJohannes Gäßler e3c3f666a4 hip: add RDNA1 MMVQ parameter table (1 warp per block for ncols=1) (#32389)
* hip: add RDNA1 MMVQ parameter table (1 warp per block for ncols=1)

* hip: write the RDNA1 MMVQ values explicitly instead of recursing into the generic table

* hip: list the RDNA1 MMVQ table before RDNA2 in the enum

* Update ggml/src/ggml-cuda/mmvq.cu

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2026-11-20 22:52:19 +01:00
Ravi Panchumarthy b2e659ec24 ggml-openvino: update openvino CI runs triggers (#32593)
* ci : run the OpenVINO GPU job only for OpenVINO changes

The self-hosted OpenVINO GPU job ran for most PRs and every master push
on a shared runner, so it queued for hours.

- ci-self-hosted-openvino.yml (GPU): runs only for changes to
  ggml/src/ggml-openvino.
- build-openvino.yml: keeps its triggers (OpenVINO changes on PRs, every
  source change pushed to master). The Linux job runs on a GitHub-hosted
  runner and tests the OpenVINO CPU device only.

* address review comments

* ci : limit the OpenVINO PR CMake trigger to ggml

* ci : drop the OpenVINO Ubuntu job

* ci : trigger OpenVINO PR runs on tests/CMakeLists.txt
2026-11-20 22:51:49 +01:00
Xuan-Son NguyenandPascal 9197945004 convert: qwen3-asr: put "language ..." output into thinking block (#32571)
* convert: qwen3-asr: put "language ..." output into thinking block

* convert: qwen3-asr: read the chat template from models/templates

The converter embedded its own copy of Qwen3-ASR.jinja. Read the
bundled file instead, as the Kimi-K3 and Kokoro converters do, so
the GGUF and test-chat share a single template.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-20 20:54:57 +01:00
Tarek Dakhran a83d68b562 metal : allow the norm chain in either operand of the fused MUL/ADD (#32300)
Assisted-by: Claude Opus 5.5
2026-11-20 19:55:44 +01:00
Aleksander Grygier e46069c409 ui: fix the flaky reset check of the MermaidPreview Flowchart story (#32675)
Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 19:43:24 +01:00
Aleksander Grygier 427a836734 ui: stage Llama Voice's sources for CMake and Docker UI builds (#32680)
Llama Chat compiles @ggml-org/llama-voice from ui/lib/voice/src, but the
CMake stage and the Docker UI stages copied only ui/lib/ui, so their npm
build could not resolve the voice imports. Stage ui/lib/voice/src next to
Llama UI and track it as an input of the CMake build.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 19:42:49 +01:00
Daniel Bevenius 374d90a9fc ci : disable musa.Dockerfile from docker.yml (#32674)
* ci : disable musa.Dockerfile from docker.yml

This commit disables musa in docker.yml due dl.mthreads.com is sometime
unavailable.

Refs: https://github.com/ggml-org/llama.cpp/pull/32672#issuecomment-7042886858

* remove the SKIP_MUSA option
2026-11-20 18:58:11 +01:00
Xuan-Son Nguyen 98709fd82b server: add more decision tests (#32578)
* server: add more decision tests

* fix ci

* also test audio
2026-11-20 18:45:31 +01:00
Georgi Gerganov 7716ec8cbc ggml : major version bump reminders + deprecate ggml_ftype (#32647) 2026-11-20 19:36:42 +02:00
FeitengandXuan Son Nguyen 34ff2e047c mtmd: drama3 voice cloning (#32586)
* mtmd: drama3 voice cloning

The Fish S2 codec encoder (fish_s2_spkenc) now also serves drama3:
- conversion: drama3 writes the codec encoder and the audio encoder KV, shared with fish_s2
- fish_s2_spkenc: outputs the codes of the clip (gen-side reference) and, in a drama3_gen mmproj,
  builds the drama3 backbone rows (E_lm[semantic] + E_cb[1][code 1]) / sqrt(2)
- drama3_gen tower: teacher forcing when the caller gives the codes of the next frame
- drama3 pipeline: reference-bracket-v1 prompt with one reference (vLLM-Omni fish_omni), the
  reference frames go through the backbone as codec rows and through the tower with their codes
- mtmd_gen_state_get_ref_codes(): read-only access to the reference codes of a gen state

Claude-Session: https://claude.ai/code/session_01PpX3DmaxNSmJbxGaxnUBrX

* fix correctness

* clean up

* rm redundant code

* fish_s2: rm unused import

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-20 18:34:52 +01:00
Ruben Ortlam 6f5c9ad217 vulkan: add support for COL2IM_1D misalign bytes and split workgroups (#32673) 2026-11-20 19:11:59 +02:00
Meet Amin 184862801a gemma4: Optimized prompt processing for multi-seq queries (#32342) 2026-11-20 23:41:02 +08:00
Aleksander GrygierandNicolas Mowen 79e81b0bd4 ui: voice messages and dictation with live waveforms and transcripts (#32531)
* ui: add a voice engine for capture, peaks and transcription

Voice input needs a live waveform, a clip that can be sent and drawn,
and a transcript, whether the model streams text while the user speaks
or not. This adds the engine for it in src/lib/voice, with no imports
of app stores or components, so it can move into a package later.

- VoiceCapture: one microphone start gives live levels (RMS on a dB
  curve, fixed history, newest last), PCM16 mono chunks at 16 kHz from
  an inline AudioWorklet (ScriptProcessor fallback), and on stop a
  16-bit mono WAV clip with its duration and peaks. Tracks, nodes and
  the context are released on stop, cancel, a failed start and page hide.
- computePeaks / computePeaksFromAudio: normalised peaks of samples, a
  buffer, or a stored base64 or Blob file.
- transcribeClip: POST /v1/audio/transcriptions, with the server's own
  error message on failure.
- RealtimeTranscriber: the /v1/realtime?intent=transcription socket,
  queueing audio until session.created, a final commit, and a typed
  RealtimeUnsupportedError when the model cannot stream or the socket
  is refused (a browser websocket cannot send the API key header).
- startVoiceSession and VoiceRecorder: try realtime, fall back to one
  HTTP transcription of the clip, and expose levels and live text.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add voice waveform and transcript components to Llama UI

Voice input needs three visuals that the app can feed with its own
recording and transcription logic: the live waveform while recording,
a playable waveform of a recorded clip, and a clip with its transcript
that opens on hover, keyboard focus or a tap.

VoiceWaveformLive draws input levels as many bars as fit the width,
newest on the right, and holds still with reduced motion.
VoiceWaveform plays a clip through its own audio element, fills the
played part in the primary colour, seeks by press, drag and arrow keys
(a slider) and pauses any other clip that was playing.
VoiceTranscriptReveal keeps the transcript in the DOM for screen
readers while it is closed, and shows transcribing and error states.

The waveform math (resampling peaks, bar capacity, seek and time
formatting) is in src/lib/voice with unit tests. The components are
exported from the package root and from @ggml-org/llama-ui/voice, which
Llama Chat resolves to the sources like the other entry points.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add stories and tests for the voice components

Storybook gets UI/Blocks/Voice: the live waveform fed by a fake
microphone meter in a chat form row, the clip waveform in its sizes and
states, and the transcript reveal in each state and on touch. The demo
audio is a WAV synthesised in the story, so no binary file is added.

The play functions check seeking by keys and by press, the reveal on
hover, keyboard focus and a tap, and a clip without audio. Pressing play
needs a real click before a browser plays audio, so a client test covers
play and pause and one clip pausing another.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: store voice messages with their transcript and send it to models that cannot hear

A voice message recorded in the chat form is an audio attachment like an
uploaded file, plus what the app needs to draw and read it. The audio
attachment gets four optional fields: voice (recorded in the app),
transcript, peaks and durationMs. Stored rows, export and import carry
them as they are, so no migration is needed and older messages keep
working.

The transcript is for reading. A model with audio input still gets the
audio alone; a model without it (the user switched models later) gets the
transcript as a labelled text part in place of the audio. When the
model's modalities are not known yet, or there is no transcript, the
audio goes as before.

A transcript often comes back after the message was sent. The new
voiceTranscriptsStore follows each pending transcript by the clip's name,
hands it to a send that happens after it arrived, and otherwise writes it
into the stored attachment of the message it went out with, in the open
chat or, if the user left it, in the database.

Copying a message now includes its voice transcript.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add Voice settings and resolve what the microphone does per model

The user should not have to think about which kind of voice input is in
use, so the app decides it from the conversation's model and two
settings. resolveVoiceMode is a small pure function: with Voice input on
Automatic and a chat model that has audio input, a recording is a voice
message, transcribed by that same model so a router swaps nothing.
Otherwise it is dictation, with the chat model when it can hear, else,
in router mode, with the chosen transcription model. When no model can
transcribe, the microphone is unavailable and says why.

Settings get a Voice section of their own rather than more rows in
General, which is already long: these four belong together and one of
them lists the server's models.

- Voice input: Automatic (default) or Always dictate.
- Transcription model: Same as chat model (default) or, in router mode,
  any model with audio input; on a single-model server it only says
  whether that model can transcribe. It notes that dictating with
  another model loads it beside the chat model.
- Transcribe voice messages: on by default.
- Transcription language: empty for automatic.

The keys live in the settings registry like the others, so defaults,
storage, reset and export need nothing new. The model picker is drawn by
SettingsChatVoiceTab, the way Extensions and Tools draw their sections.

Assisted-by: Claude (Opus 5.5 Medium)
Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com>

* ui: record voice messages and dictation from the chat form

The microphone now does what the conversation's model allows, without
the user choosing a mode: its label says "Send a voice message" or
"Dictate", and when no model can take audio it is disabled and says why
(shown in router mode only when some model there has audio input, so the
reason can be acted on; a single-model server without audio keeps the
send button as before). The microphone errors keep their warning state
and toast, now also for the voice engine's own capture errors.

Recording goes through the voice engine's VoiceRecorder instead of the
MediaRecorder flow. While it records, the actions row becomes a
recording row: cancel, the live waveform, the elapsed time and stop.
Escape or cancel drops the recording. The form keeps its two rows while
recording, so the message stays visible on desktop and the row fits the
phone layouts and the expanded sheet.

- Dictation writes the live transcript into the message at the caret
  as it streams. DictationSpan replaces only the dictated stretch, moves
  it when the user types before it, keeps what is typed after it, and
  hands text the user edits over to them. When nothing streams, the row
  shows Transcribing... after stop and the text goes in once it is back.
  Cancel takes the dictated text out again. No audio is attached.
- A voice message streams its live text, if any, under the waveform,
  not into the message. On stop the clip is attached as a voice file
  with its peaks and length, and its transcript is followed by the voice
  transcripts store (unless Transcribe voice messages is off).

Realtime transcription is skipped for servers reached through a tunnel
or with an API key, which a browser websocket cannot carry. The first
dictation with a separate router model shows a one-time note that it
loads beside the chat model.

The form takes an optional voiceRecorder, so tests and stories can run
it without a microphone.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: draw audio attachments as playable waveforms with their transcript

A voice message should read as a clip, not as a file chip. Audio
attachments now render with Llama UI's VoiceWaveform inside
VoiceTranscriptReveal, through a direct branch in the core attachment
item (voice is core, so it does not go through the extension renderers):
the removable sm chip in the chat form, and the md clip in a sent
message, on the user bubble's surface.

- The audio plays from an object URL made from the stored base64 (or the
  uploaded file's data URL), revoked when the clip leaves the screen.
- A clip stored without peaks, an older voice message or an uploaded
  audio file, gets them computed from its audio once per session, in
  memory only; uploaded files keep their name under the waveform.
- The transcript opens under the clip on hover, keyboard focus or a tap,
  and shows Transcribing... while the voice transcripts store waits for
  it, Transcription unavailable when it failed, and nothing when none
  was asked for.
- The full-screen preview keeps its player and shows the transcript.

Tests run the chat form with a fake recorder: the microphone per mode,
the recording row, dictation around typed text, the Transcribing...
state, cancel, and a voice message chip whose transcript arrives later
and goes with the send. Stories show the form recording each kind, a
recorded chip, and sent voice messages with the transcript, pending,
and computed peaks.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: drop the old audio recorder and proxy realtime transcription in dev

The voice engine replaced AudioRecorder, so it goes. The dev server now
proxies WebSockets on /v1, so realtime transcription works under vite dev.
The VoiceCapture test waits for the tone to be recorded instead of a fixed
700 ms, which a loaded test run could miss.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: dictate with a model that is already loaded when the chat model cannot hear

The Transcription model setting's Automatic choice now falls back to a
router model with audio input that is already loaded, so dictating with a
text-only chat model works without picking a model and loads nothing.
A picked model still wins, and the empty value is a named constant.
Idea from upstream PR 27567, by NickM-27.

Assisted-by: Claude (Opus 5.5 Medium)
Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com>

* ui: fill the recording waveform, and open voice message transcripts

- The live waveform fills the row from the start: a recording shorter
  than the row is stretched over all its bars, a longer one scrolls with
  the newest level on the right. No dotted placeholder.
- A voice message's transcript is open under its clip, in the chat form
  while it comes in and in the sent message, with the toggle beside it
  to fold it.
- Settings > Voice > Collapse transcripts (off by default) keeps them
  folded instead, opening on hover, keyboard focus or a tap.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit 2a93f2f948ed976e40f693ce5d3acf1168284b05)

* ui: record voice without ducking other audio, from the first word

- Voice input asks for the raw microphone signal: no echo cancellation,
  noise suppression or auto gain. Echo cancellation made the system treat
  a recording as a call, which lowers or interrupts other playback, and
  auto gain faded the first words in.
- One AudioContext serves every recording, with the capture worklet
  loaded once, suspended in between and, where the browser allows it,
  without an output device, so recording never touches the speakers.
- The chat form prepares it when it gets focus (no microphone, so no
  recording indicator), and a mouse starts recording on press.
- A recording starts counting once audio flows: until then the bar says
  Starting... instead of running the timer. Development builds log how
  long each step of a start took.
- Settings > Voice > Microphone: Automatic (the system default, or the
  built-in microphone when the default is a Bluetooth headset, which
  would switch to its low-quality call mode), System default, or a
  listed microphone.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit 76643e7210d21617d734fdfde677e84af7273153)

* ui: show voice text live in the message box, and follow transcripts as spoken

- A model that hears but has no live ASR (e.g. LFM3-Omni) now gives text
  while you record: the realtime client commits the audio piece by piece,
  at the pauses the levels show, one piece at a time, and the final
  commit takes only what is left.
- A voice message's live text shows as the empty message box's
  placeholder. Clicking into the box makes it text to edit there, and
  what is said next follows it. On stop the text leaves the box and goes
  with the clip; when it was edited, the edit is the clip's transcript.
- Llama UI gains VoiceSpokenText and spokenLength(): spoken words in the
  full foreground colour, which they keep, the rest muted. A clip's
  transcript follows its playback (estimated over the parts with speech,
  as there are no word timings), and live text brightens as it arrives.
- The recording bar has a send button that stops and sends in one go.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit ce773017f902f8a5690ab9ecca368cecb6427422)

* ui: let the capture test start with silence

A capture starts with its first audio block, which can come before a
stream's first sound, so the steady tone is checked from where it begins.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit a41ee5337b83fef685a5cbaff176925f1647ce9d)

* ui: show live voice text sooner, and keep silence from becoming "yeah"

- A model that transcribes per commit (LFM3-Omni) gets shorter pieces:
  cut at a 0.35 s pause after speech, or at the next quiet moment once a
  piece reaches 3 s, so the text follows the speaker within a few
  seconds in dictation, the message box and voice mode.
- When the recording stops, the audio after the last piece is committed
  only if speech was heard in it. LFM3-Omni transcribes a commit of pure
  silence as "Yeah." (or "はい。"), which ended up at the end of the text.
  A recording whose only piece is that rest still commits it.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit 43e52cbab0fba9d8e84b9491890a406e3c6c90f3)

* ui: transcribe a recording cut at pauses once more as a whole

Live text comes piece by piece, cut at the speaker's short pauses, and a
model reads each piece as a sentence of its own: "I went. To the store."
When the recording ends, one pass over the whole clip gives the final
text, punctuated by meaning, in dictation and in voice mode alike. The
pieces' text stays the live text until then, and is kept when that pass
fails or the recording is longer than two minutes.

Assisted-by: Claude (Opus 5.5 Medium)
(cherry picked from commit 359dd9632e6c88000a1d23c2575bd1e6fcdd9ec1)

* ui: give live text over HTTP for models without a realtime session

A model that hears but cannot hold a realtime session, such as
Qwen3-ASR, was transcribed only after the recording, so dictation
showed no text while speaking. The recording is now cut at the
speaker's pauses as with a realtime session, and each piece sent to
/v1/audio/transcriptions on its own, one at a time, its text joined
live. The audio is kept from the start, so what is said while the
socket answers goes with the first piece. The whole clip is still
transcribed once more at the end, and the pieces' text is kept if
that fails. A server a websocket cannot reach (an API key, a remote
server) gets live text the same way.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: say when the browser processes the microphone despite the raw request

The browser keeps one processing setting per microphone: while a call,
another tab or voice mode has it open with echo cancellation, a
dictation that asked for the raw signal is processed too, sounds
distant, and the system may lower other sound. The capture now warns
when it gets processing it did not ask for, and its development log
names the microphone, its rate and what processing is on.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the microphone beside the stop and send buttons

While a reply streams, or once the message has text, the microphone no longer
gives way to the main button: it stays beside it as a quieter, secondary
button, so more can be dictated without clearing the message or waiting for
the reply.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: show the voice components' stories with Llama UI

VoiceWaveform and VoiceTranscriptReveal are Llama UI components, so their
stories move to the library's Storybook, under Llama UI/Voice, with the demo
audio they play. Llama Chat's Storybook shows them there too, and its own
voice stories take the demo audio from the library. VoiceWaveformLive stays
with Llama Chat, as its demo records through the chat form's microphone.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a noise suppression setting for voice input

Recordings stay raw by default. Settings > Voice can now switch the
browser's noise suppression on; echo cancellation and automatic gain stay
off either way, as they interrupt other sound and fade the first words in.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep live voice text coming when the room is never quiet

Pieces were cut only at a quiet moment, so with music or talk in the room
none was ever cut and no text showed until the recording ended. A piece half
as long again as its longest length is now cut where it is.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add echo cancellation and automatic gain settings for voice input

Settings > Voice now has a switch for each kind of microphone processing the
browser offers. All three stay off by default, so recordings are raw; each
one says what switching it on costs.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: put the microphone after the stop button, filled like it

While a reply streams, the microphone is the last button of the row and
looks like the stop and skip buttons beside it, instead of a bare icon
before them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: say when dictated text is being refined, and send it refined

After a recording with live text, the whole clip is transcribed once more
so pauses do not end sentences. The form now says so while that pass is
out, and a send asked for meanwhile waits for it, so the message goes out
with the refined text rather than the pieces.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the send button in place and put the microphone beside it

The send button no longer gives way to the microphone on an empty
message. The microphone stands before it as a quieter, tertiary button,
with the same tooltip, warning and recording states as before.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: let a recording be sent while a reply streams, once it has words

While a reply streams, the recording row now shows its send button as
soon as the first transcribed words are in the message, so the
recording no longer has to be stopped first. The microphone also stays
beside the send button once a voice message is attached.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: raise quiet speech for transcription, and say when it is too quiet

A quiet microphone gave the model near silence to transcribe. A new
Voice setting, Boost quiet speech (on by default), raises speech to a
level speech models hear well, up to eight times louder: the audio
streamed for live text as it goes, and the whole recording in one pass
when it stops, so its first words get the same gain as the rest. Only
speech is measured, never the silence around it, loud speech is left
as it is, and a soft limiter keeps raised peaks from clipping. When
speech stays quiet even at the highest gain, the recording row says so.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the voice engine free of app code

The engine in lib/voice imported the microphone messages from the app's
utils and the WAV type from its enums, the only two things keeping it
from moving into a package of its own. The microphone checks and
messages move into the engine (voice/support.ts), next to the capture
that needs them; the uploaded-file helpers stay in the app's utils.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: start voice sessions from one voice service

How a recording reaches the server (routes, API key, whether the
realtime socket can be used) and how the Voice settings become capture
and transcribe options were built inside the chat form's voice hook,
and copied into every other feature that records. VoiceService builds
them in one place: a feature asks it for its session options and adds
only its own callbacks.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: tell speech from the room in one place

The pause detector, the speech gain and (in voice mode) the listener for
speech over a reply each followed a noise floor with their own numbers.
A SpeechDetector now does it for all of them, in dB, with one set of
settings in VOICE_SPEECH.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: let any text field take dictation

Writing dictated text into a field (at the caret, around the user's
typing, replaced by the whole recording's text when it comes) moves out
of the chat form's hook into a plain createDictation(). The hook keeps
what is the chat form's own: voice messages, toasts and the hint.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: move the recording row into Llama UI

VoiceRecordingBar draws the row shown while recording (cancel, waveform
and time, send, and a caption or warning under it) from data alone, so
voice mode or a notes page can show the same row. The chat form passes
its labels and its microphone button as the stop button.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the voice engine's Svelte recorder behind its own entry

VoiceRecorder, the one part of lib/voice written with runes, moves to
lib/voice/svelte, so the rest of the engine is plain TypeScript any app
can use without Svelte.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: move voice input into its own package, Llama Voice

The voice engine (capture, the speech signal, transcription over the
realtime socket or HTTP) and dictation into a text field move from the
app to ui/lib/voice as @ggml-org/llama-voice, so another app, such as a
notes app, can use them. The Svelte recorder is its ./svelte entry.
Llama Chat compiles the sources, as it does Llama UI's, and keeps its
server and settings wiring. The package has its own type check, tests
and build, and a CI job for them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: show which model does what in voice input

The model picker and the Models manager mark each model that hears:
Dictation on the one that transcribes dictation, Speech to text on
transcription models such as Qwen3-ASR, and Voice on chat models that
take voice messages. A loaded model kept only for dictation sits apart
from the chat models, a model can be made the dictation model from its
row, and Settings > Voice groups the transcription models apart from
the chat models with audio input. A voice store now resolves what the
microphone does, for the chat form and the model lists alike.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: warn of a quiet microphone inside the recording row

The too-quiet line under the waveform made the form grow and jump. The
warning now stays inside the row: the waveform turns the warning colour,
an icon sits by the time, and the advice opens above the row in a
tooltip for a few seconds, then on hover or focus. Screen readers still
hear it when it appears.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: open the microphone while the pointer is on its button

The first seconds of a recording could come out quiet while the
microphone settled, and the too-quiet warning showed for them. With the
pointer or the focus on the microphone button, the microphone now opens
ahead of the press (only when the page may already use it, so nothing
asks), and the recording takes it at once; it is let go shortly after
the pointer leaves without a press. The warning also waits until the
speech has stayed too quiet for two seconds.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: tidy voice input after review

Fix a microphone left open when a recording is cancelled while it starts
or the pointer leaves while it opens, a capture that is still letting go
silencing the next one, and live text stopping after a realtime error.
Share the dictation model choice, the transcript lookup, base64 decoding,
PCM joining and stopping a stream instead of repeating them, and export
only Llama Voice's public API.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: pin Llama Voice's dev dependencies to Llama UI's versions

Assisted-by: Claude (Opus 5.5 Medium)

* ui: let demos speak into a synthetic microphone

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the microphone for voice input, off the audio files entry

The attach menu's Audio Files entry used a microphone, which now sits
beside the form's own microphone button. The microphone stands for
recording and dictation only; a model's audio modality is headphones, and
the audio files entry is a file with headphones.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: set voice input on the slimmed design tokens

Voice input's components use the type roles, the one corner scale and
shadcn's utility names for the neutral colours, as the rest of Llama UI
and Llama Chat do since the design tokens were slimmed down.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name voice states, platform strings, patterns and tunings

Addresses the review on the voice PR:

- Llama Voice: VoiceCaptureState, VoiceRecorderStatus, RealtimeState and
  RealtimeUnsupportedReason are enums (src/enums.ts, exported); DOMException
  names, AudioContext states, transcription form fields, socket protocols,
  device kinds, worklet messages, error messages, regexes and tuning values
  are named constants.
- Llama UI: VoiceTranscriptStatus, VoiceWaveformSize and
  VoiceRecordingBarStatus are enums; KeyboardKey, MouseButton and PointerType
  join its UI enums; the voice components' sizes, labels, timings, selectors
  and patterns move to constants/voice.constants.ts. The voice unit tests
  move from src to tests/unit.
- App: ChatFormVoiceState is an enum, the recording bar's status comes from a
  map, and the voice code uses the enums above, BooleanString, HEADERS and
  new voice constants in place of inline strings, regexes and numbers.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep all of Llama UI's tests under tests

The Model UI formatting spec moves to tests/unit and the button spec to
tests/client, next to the voice specs; the test projects now read only
tests/unit and tests/client.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: show a transcript's words without the model's own format

Qwen3-ASR names the language before its text (language French<asr_text>...) and
VibeVoice-ASR answers with timed segments as JSON, which the server passes on as they
are. The voice engine now takes both off, over HTTP and over the realtime socket, also
while segments still stream, so the message box and the clip show the words alone.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: dictate when a speech-to-text model is picked for the conversation

A speech-to-text model has audio input, so on Automatic the microphone recorded a voice
message for it, which such a model has nothing to answer with. It now dictates, as the
model's badge already says it is for.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: say why a live-only model cannot transcribe behind an API key

A browser websocket cannot send the API key, so the realtime socket is left out on a
server that needs one, and a model that only transcribes over it (Voxtral Realtime)
answers the HTTP route with a 501. The clip and the dictation toast now say so instead of
only that transcription is unavailable.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: pick the dictation model with a microphone on its row

The model that transcribes dictation was marked with a filled Dictation badge. It is now
marked by a red microphone among the row's actions in the model picker, beside the heart,
and every other model that hears shows the same microphone on hover: a click makes it the
transcription model, and a click on the chosen one hands the choice back to Automatic. The
badge keeps saying what kind of hearing model a row is, and the Models manager marks the
dictation model with a small microphone beside its name.

Assisted-by: Claude (Opus 5.5 Medium)

---------

Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com>
2026-11-20 16:34:52 +01:00
Aleksander Grygier 059cac2848 ui: search message content and Discover models from the search dialog (#32670)
* ui: search message content and Discover models from the search dialog

The search dialog matched chat titles only. It now also scans what was
said in each chat (content, reasoning, tool calls and results, attachment
names and text), newest chat first, stopping at the first match per chat
and after 20 chats, and shows a snippet around the match. Ids, timestamps
and base64 payloads are not searched.

With Discover on, it also lists matching catalog models and, once typing
settles, Hugging Face repos; picking one opens Discover on that repo.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the search dialog one width whatever it lists

The shared dialog sizes itself to its content from md up, so the search
was narrow while empty and widened as results came in. It now takes the
full max-w-2xl there, less a margin on narrower windows.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 16:20:00 +01:00
Aleksander Grygier cf4153a9ba ui: publish llama-ui as an alias of @ggml-org/llama-ui (#32667)
The unscoped llama-ui on npm is a 0.0.1 placeholder. After the scoped
publish, the workflow renames the same build and publishes it again at
the same version, with a readme note pointing at the scoped package.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 15:46:18 +01:00
yoch 8e7862411d tts : forward -t to the mtmd context (#32664)
llama-tts built the mtmd context from mtmd_context_params_default(), which hardcodes 4 threads, and never copied n_threads over, so -t never reach the mtmd side.
Copy params.cpuparams.n_threads into mtmd_params.n_threads, as mtmd-cli and the server do; with no -t this now follows the resolved default.

Verified: -t 1 now runs single-threaded, -t 4 is unchanged, and the produced WAV is identical.

Assisted-by: Claude
2026-11-20 15:34:58 +01:00
Xuejun ZhaiandSigbjørn Skjæret e626db7caa server-bench : fix TypeError in histogram generation (#32518)
* fix(server-bench): prevent float bin-count TypeError in histogram generation

Symptom: benchmark crashes while generating gen_rate histogram.

Root cause: np.ceil(...) returns numpy.float64, and that value was used where an integer count/shape argument is required.

Fix: cast ceil result to int before +1, ensuring bin count is an integer.

Result: histogram path completes and gen_rate.png is generated reliably.

* Update scripts/server-bench.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-20 15:16:52 +01:00
Aleksander Grygier c6a25fc622 ui: load highlight.js languages on demand (#32661)
* ui: load highlight.js on first use

highlight.js and every lowlight language (~900 KB raw) moved out of
Llama Chat's first load into a chunk that loads with the first code block.

- @ggml-org/llama-ui/code: highlightCode stays synchronous. Until
  highlight.js has loaded it returns the code escaped as plain text and
  starts the load; a $derived or $effect that called it runs again once
  it has loaded. New loadHighlighter() and isHighlighterLoaded().
- Markdown: rehype-highlight runs through a lazy rehype step, like
  KaTeX's: the first message with a code block waits for it, every
  other one renders in the same pass.

First-load JS of the chat page: 3.47 -> 2.56 MB raw, 1.05 -> 0.75 MB
gzip, 0.84 -> 0.60 MB brotli.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: load each highlight.js language on first use

highlight.js now loads one language at a time instead of all ~190 at
once with the first code block: every language is its own chunk, and
the core loads with the first one.

- @ggml-org/llama-ui/code: highlightCode looks a language or alias up
  in a generated table (scripts/generate-highlight-languages.mjs), loads
  it with the languages it embeds (CSS and JavaScript inside HTML, say)
  and returns plain text until then; a $derived or $effect that called
  it runs again once it has loaded. Auto-detection picks among lowlight's
  37 common languages, loaded the first time something auto-detects.
  New loadHighlightLanguages(), isHighlightLanguageLoaded(),
  resolveHighlightLanguage() and getLoadedHighlightLanguages() replace
  loadHighlighter() and isHighlighterLoaded().
- Markdown: rehypeLazyHighlight highlights with lowlight's core and the
  languages a message uses, waiting only for one it is the first to use.
  rehype-highlight is no longer in the pipeline, since it bundles
  lowlight's common languages.

First-load JS of the chat page stays at 2.58 MB raw, 0.76 MB gzip,
0.61 MB brotli: no highlight.js code, only the 14 KB language table.
The first code block now loads its language (1-12 KB for most) and the
20 KB core instead of 900 KB.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name the code-block class names and shell language as constants

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 15:06:10 +01:00
Xuan-Son Nguyen 5f837145d8 server: add more audio tests (#32579) 2026-11-20 14:21:58 +01:00
Xuan-Son Nguyen 36cd3aff40 server: add --ui-mcp-proxy-allow-private-urls (#32633) 2026-11-20 14:10:06 +01:00
Aleksander Grygier 28de970cc1 ui : focus the chat form when typing with nothing focused (#32570)
A printable key pressed on the chat screen while no field, editor,
dialog or menu has focus now moves focus to the chat form, and the
character lands there.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 14:01:50 +01:00
Georgi Gerganov 4b69c79cca cuda : update GDN supports_op check (#32660) 2026-11-20 14:35:20 +02:00
Daniel HanandSigbjørn Skjæret a1f4222608 model : load ffn_gate_up_exps scales (#32657)
* model: load `ffn_gate_up_exps` scales

The generic scale pass loads per-expert scales for ffn_gate_exps,
ffn_up_exps and ffn_down_exps but not for the merged ffn_gate_up_exps.
Load ~ffn_gate_up_exps_s there~ and expose it as ffn_up_exps_s ~when the layer
has no separate up tensor,~ since build_moe_ffn applies up_exps_s to the
merged gate_up output. Inkling 2 now passes the expert scales to
build_moe_ffn like the other MoE models.

* Update src/llama-model.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Update src/llama-model.h

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-20 13:11:28 +01:00
Aman Gupta 4783b1145c llama : move the indexer key rotation choice into the model classes (#32653)
* llama : move the indexer key rotation choice into the model classes

The KV cache forced a Hadamard rotation for the lightning indexer keys
based on a list of architectures. The virtual llama_model::indexer_k_rot()
now returns true for the same models. DeepSeek4.1 inherits from DeepSeek4
but was not in the list, so it overrides it back to false.

* llama : rename indexer_k_rot() to lid_k_hdmd()
2026-11-20 18:49:16 +08:00
Sigbjørn Skjæret 6abcbe5d35 remove ui-lib publish verification step (#32658) 2026-11-20 11:45:46 +01:00
Aleksander Grygier b81aa847c2 ui: slim the design tokens and set all text through type roles (#32615)
* ui: drop component tokens that only alias a semantic token

A component token now exists only for what a component decides itself:
a value no semantic token has, or one that only this component changes
between themes. Components that take a semantic token as it is read it
directly, so 88 of the 118 component tokens go, with no change to what
is drawn.

Also drops semantic tokens that repeat another one (status-* is fill-*,
elevation-tooltip is elevation-surface), the unused info lamp tokens and
14 primitives nothing reads, and adds fg-soft for the label of an option
that is not the picked one (tabs, toggle group).

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a short summary on top of DESIGN.md

The rules most changes need, in one screen: components, colour, type,
shape, sizes, surfaces, tokens and no arbitrary values. The sections
below keep the detail.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: derive design tokens by rule and drop the alias layers

Semantic tokens are either a pick (a primitive per theme) or a derivation,
mixed from another semantic token by a fixed rule and declared once: hover
fills, status surfaces and lines, dark shadows (--elevation-strength), the
middle of a matte edge, a lamp's rim and glyph.

Primitives keep colours only. shadcn's colour names stay as Tailwind
utilities and no longer exist as variables; neutral semantic utilities that
duplicated them are gone.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: set all text through type roles and move off-scale values onto the scale

Tailwind's text sizes and bracketed sizes become the type roles, with a new
text-caption-sm (10px) for tags and counts. Body and caption roles no longer
set a weight, and line heights are written as Tailwind writes its own, so the
swap from text-sm and text-xs changes no pixel. A unit test keeps old sizes
from coming back.

Arbitrary values with a scale equivalent use it (z-20, max-w-4/5, min-w-32),
and the two easing curves repeated across the app are ease-out-quint and
ease-sheet. DESIGN.md is updated and regenerated.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: bump @ggml-org/llama-ui to 0.5.0

Removed and renamed tokens are breaking for an app that reads them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a display type role for the biggest headline

The greeting on a new chat keeps its 30px from md up, now as text-display
instead of a Tailwind size.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: one corner scale, one set of shadows and three more derived tokens

Six corner roles named values Tailwind's rounded-md to rounded-4xl
already have, and components used the Tailwind names nine times out of
ten. The roles are gone; the whole scale is computed from --radius.

Tailwind's shadow sizes were restated next to the elevation tokens and
did not follow the theme, so they were five times fainter in dark. The
five uses take elevation tokens and a unit test keeps both from coming
back.

A button group's seam is the button's own text colour at 25%, the dark
status text is the dark status fill, and the pressed shadow scales with
--elevation-strength like the inset one.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 11:26:26 +01:00
Aleksander Grygier 9d6f04fa99 ui: show vision as glasses and audio as headphones (#32617)
The model modality icons match the ones the Llama apps use: glasses for
vision, headphones for audio. The attach menu's list of what the model
takes reads the same icons instead of its own microphone and picture.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 10:33:34 +01:00
Aman Gupta 652d559edf llama : move the graph node budget into the model classes (#32648)
graph_max_nodes() picked the nodes-per-tensor factor from lists of
architectures. Each model class now gives it through a new virtual,
llama_model::graph_nodes_per_tensor(), which defaults to 8. The
architectures that need 32 or 64 override it in models.h, and DFlash
keeps its hparams-based rule in dflash.cpp.

LoRA adapter nodes are now added for every architecture, not only for
the ones on the default factor.
2026-11-20 17:19:14 +08:00
Aleksander Grygier 6aee50801d ui: set the reasoning effort from the model picker (#32632)
* ui: move the reasoning effort into the model picker

The chat form's row no longer holds the reasoning button: the model
picker ends with a Reasoning row, in place of Manage models, that shows
the level in use and opens the levels under it. A phone's model drawer
has the same row. A desktop server with a single model, whose selector
opens the manager instead of a picker, keeps the button beside it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: show a reasoning effort that is not the default on the model selector

Next to the model on the closed selector, a set level shows as its word,
and reasoning that is off as a small crossed-out lightbulb. At the
default, which leaves it to the model, nothing more shows.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the model picker's footer in place while it opens in Safari

WebKit pins a bottom-sticky element against the scaled box of a parent
that zooms in, so the Reasoning row rode up over the model list until
the opening animation ended. The desktop picker now scrolls its search
and list in a box of their own, with the footer under it.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 09:58:11 +01:00
Aleksander Grygier 55b8d8d898 ui: full-screen diagram preview that zooms around what is on screen, inert diagrams (#32591)
* ui: keep Pele diagrams inert

Pele turns a node with `click ... href` into a live link, where Mermaid's
strict security level left diagrams without links. Unwrap every link in
Pele's output, keeping its children, so the inline block, the live-preview
editor, the preview dialog and the downloaded svg stay inert. The Diagrams
story's flowchart now carries click directives and its play test checks
that no link or href is left.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: open diagrams full screen, zoom around what is on screen

The diagram preview dialog fills the window on every device. The preview
places the diagram from its top left corner, so a zoom keeps a screen point
fixed: the wheel and a trackpad pinch zoom around the pointer, the buttons
and +/- keys around the middle of the screen, wherever the diagram has been
panned to. Two-finger pinch, double-click and arrow keys work too. The
diagram opens fitted and keeps fitting until panned or zoomed; Reset view
fits it again.

Adds MermaidPreview and DialogMermaidPreview stories with play tests and a
unit test for the zoom math.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a diagrams trace to the demo fixtures

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-20 09:57:44 +01:00
Aman Gupta 4c4a0f0a42 llama : reserve graph nodes for the MoE cache (#32636)
The MoE cache adds a slot lookup to every MoE layer, and with
--moe-cache-cpu-share a second expert path, but graph_max_nodes() did
not account for it. Models on the default 8*n_tensors budget with few
tensors per layer (cohere2moe: 443 tensors, 49 layers) overflow the
graph context when the cache is combined with a quantized KV cache
(K/V rotation), aborting in ggml_new_tensor with GGML_ASSERT(obj_new).

Fixes #32584

Assisted-by: Claude Opus 5.5
2026-11-20 16:13:00 +08:00
nicholas-mainardi-domyn 55d323507d model: add Domyn Edge 4B Model (#32559)
* feat: add domyn-edge model

* Generate tokenizer hash for conversion + fix tests

* Address changes according to code-review skill

* catch ValueError in convert_hf_to_gguf_update.py

* renegerate fusion test baseline + fix model naming style
2026-11-20 09:51:41 +02:00
Aaron Teo bed8ec2f8d llamafile(s390x): add QK data type support (#32600)
* llamafile: add QK support for s390x

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* llamafile: clean up for s390x

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-11-20 09:47:41 +02:00
sys2008 52e8db6550 kleidiai : allow OHOS (HarmonyOS) AArch64 target (#32265)
* kleidiai : allow OHOS (HarmonyOS) AArch64 target

* kleidiai : allow OHOS AArch64 target

Assisted-by: GLM
2026-11-20 09:46:59 +02:00
Nikhil Jain d7d1b632da webgpu : fix 2D indexing for copy kernel (#32576)
* Only run webgpu CI on my fork

* Add webgpu only workflow

* resolve merge conflicts

* Add 2d indexing to copy kernel

* move test
2026-11-20 09:45:44 +02:00
Teng Mei ea84d70903 CUDA: optimize Orin single-column MMVQ (#31268)
* CUDA: optimize Orin single-column MMVQ

* Create a separate Orin branch in `calc_rows_per_block`
2026-11-20 09:44:04 +02:00
Chinmay ShrivastavaandJonSnow1807 9fbe30c2a0 CUDA: keep the MUL_MAT_ID precision in the per-expert fallback (#31345)
* tests: add CUDA-only MUL_MAT/MUL_MAT_ID cases with results above the fp16 range

IQ1_M weights have no MMQ kernel, so once the batch is bigger than the MMVQ
limit the path is dequantize + cuBLAS. These cases produce results around
1.3e5, past the fp16 max of 65504, so an fp16 result buffer turns them into inf
while the CPU reference stays finite. They are only added for the CUDA backend.
Vulkan deliberately accumulates quantized matmuls in fp16 at default precision
and would fail them by design.

* CUDA: use an fp32 result for quantized src0 in the cuBLAS fallback

On the F16 compute path, quantized weights went through cuBLAS with fp16
accumulation and an fp16 result buffer that was converted to fp32 afterwards.
Anything above 65504 became inf and the next layer turned into NaN (see #30951:
gpt-oss-120b with IQ1_M experts at -ub 512). Set prefer_f32_output for
quantized src0 so the GEMM still uses fp16 operands but accumulates and stores
in fp32, the same mode already used on Volta, RDNA4 and CDNA. Every other
quantized path already accumulates in fp32.

Diagnosis and fix by denizyuret in #30951.

* CUDA: keep the MUL_MAT_ID precision in the per-expert fallback

The per-expert dst_slice kept op = GGML_OP_NONE, so ggml_prec_get_acc returned UNDEFINED and a requested precision was ignored. Set the op to GGML_OP_MUL_MAT. Also revert the fp32 result change and remove the tests.

---------

Co-authored-by: JonSnow1807 <JonSnow1807@users.noreply.github.com>
2026-11-20 09:42:16 +02:00
Amadeus dyw c0b97be0b2 vulkan: optimize RMS_NORM_MUL_ROPE loop control (#31149) 2026-11-20 09:38:43 +02:00
Ruben OrtlamandJeff Bolz 16f80bd0c0 vulkan: chunked GATED_DELTA_NET implementation (#30207)
* vulkan: add cm2 chunked GATED_DELTA_NET shader

* fix shmem bank conflict bottleneck

* optimize further

* add coopmat maintenance1 variant

* optimize for RDNA4

* add gmem version RDNA4

* optimize for Ampere

* fix nvidia shmem overuse

* fix cm2 validation errors

* improve cm2 perf

* improve cm1 perf

* clean up

* enable by default

* show cm maintenance1 marker in device info

* properly check for cm_m1 features

* add tests

* add v32 mode, improve chunked condition

* allow other subgroup sizes

* vulkan: specialize the GDN CM2 workgroup size

Assisted-by: Codex

* vulkan: coopmat2 GDN implementation

* work around Nvidia shader compiler crash

* remove shared shader cm2 code

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
2026-11-20 09:38:14 +02:00
Leon 2d8d003937 webgpu : add COL2IM_1D support (#31708) 2026-11-20 09:36:58 +02:00
7e960f40fb opencl: fold GQA head for f16 FA for better efficiency (#32631)
Assisted-by: Codex

Co-authored-by: Li He <lih@qti.qualcomm.com>
Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>
2026-11-19 22:26:28 -08:00