2ca30e5f350def921fc5deb2bb55ba1237c48280
813
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5d28d64bf2 |
ggml-webgpu : Update to a recent version of Dawn (#32779)
Dawn v20260908.214631 doesn't compile with emsdk latest, that is 7.0.1. |
||
|
|
1fdda89a4d |
ci: switch ppc64le runners to ubuntu 26.04 (#32679)
* ci: switch ppc64le runners to ubuntu 26.04 Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: rm problem matchers experiment Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> |
||
|
|
40b7a546de | ci: temporary disable no realloc case for mtmd (#32708) | ||
|
|
b2e659ec24 |
ggml-openvino: update openvino CI runs triggers (#32593)
* ci : run the OpenVINO GPU job only for OpenVINO changes The self-hosted OpenVINO GPU job ran for most PRs and every master push on a shared runner, so it queued for hours. - ci-self-hosted-openvino.yml (GPU): runs only for changes to ggml/src/ggml-openvino. - build-openvino.yml: keeps its triggers (OpenVINO changes on PRs, every source change pushed to master). The Linux job runs on a GitHub-hosted runner and tests the OpenVINO CPU device only. * address review comments * ci : limit the OpenVINO PR CMake trigger to ggml * ci : drop the OpenVINO Ubuntu job * ci : trigger OpenVINO PR runs on tests/CMakeLists.txt |
||
|
|
374d90a9fc |
ci : disable musa.Dockerfile from docker.yml (#32674)
* ci : disable musa.Dockerfile from docker.yml This commit disables musa in docker.yml due dl.mthreads.com is sometime unavailable. Refs: https://github.com/ggml-org/llama.cpp/pull/32672#issuecomment-7042886858 * remove the SKIP_MUSA option |
||
|
|
79e81b0bd4 |
ui: voice messages and dictation with live waveforms and transcripts (#32531)
* ui: add a voice engine for capture, peaks and transcription Voice input needs a live waveform, a clip that can be sent and drawn, and a transcript, whether the model streams text while the user speaks or not. This adds the engine for it in src/lib/voice, with no imports of app stores or components, so it can move into a package later. - VoiceCapture: one microphone start gives live levels (RMS on a dB curve, fixed history, newest last), PCM16 mono chunks at 16 kHz from an inline AudioWorklet (ScriptProcessor fallback), and on stop a 16-bit mono WAV clip with its duration and peaks. Tracks, nodes and the context are released on stop, cancel, a failed start and page hide. - computePeaks / computePeaksFromAudio: normalised peaks of samples, a buffer, or a stored base64 or Blob file. - transcribeClip: POST /v1/audio/transcriptions, with the server's own error message on failure. - RealtimeTranscriber: the /v1/realtime?intent=transcription socket, queueing audio until session.created, a final commit, and a typed RealtimeUnsupportedError when the model cannot stream or the socket is refused (a browser websocket cannot send the API key header). - startVoiceSession and VoiceRecorder: try realtime, fall back to one HTTP transcription of the clip, and expose levels and live text. Assisted-by: Claude (Opus 5.5 Medium) * ui: add voice waveform and transcript components to Llama UI Voice input needs three visuals that the app can feed with its own recording and transcription logic: the live waveform while recording, a playable waveform of a recorded clip, and a clip with its transcript that opens on hover, keyboard focus or a tap. VoiceWaveformLive draws input levels as many bars as fit the width, newest on the right, and holds still with reduced motion. VoiceWaveform plays a clip through its own audio element, fills the played part in the primary colour, seeks by press, drag and arrow keys (a slider) and pauses any other clip that was playing. VoiceTranscriptReveal keeps the transcript in the DOM for screen readers while it is closed, and shows transcribing and error states. The waveform math (resampling peaks, bar capacity, seek and time formatting) is in src/lib/voice with unit tests. The components are exported from the package root and from @ggml-org/llama-ui/voice, which Llama Chat resolves to the sources like the other entry points. Assisted-by: Claude (Opus 5.5 Medium) * ui: add stories and tests for the voice components Storybook gets UI/Blocks/Voice: the live waveform fed by a fake microphone meter in a chat form row, the clip waveform in its sizes and states, and the transcript reveal in each state and on touch. The demo audio is a WAV synthesised in the story, so no binary file is added. The play functions check seeking by keys and by press, the reveal on hover, keyboard focus and a tap, and a clip without audio. Pressing play needs a real click before a browser plays audio, so a client test covers play and pause and one clip pausing another. Assisted-by: Claude (Opus 5.5 Medium) * ui: store voice messages with their transcript and send it to models that cannot hear A voice message recorded in the chat form is an audio attachment like an uploaded file, plus what the app needs to draw and read it. The audio attachment gets four optional fields: voice (recorded in the app), transcript, peaks and durationMs. Stored rows, export and import carry them as they are, so no migration is needed and older messages keep working. The transcript is for reading. A model with audio input still gets the audio alone; a model without it (the user switched models later) gets the transcript as a labelled text part in place of the audio. When the model's modalities are not known yet, or there is no transcript, the audio goes as before. A transcript often comes back after the message was sent. The new voiceTranscriptsStore follows each pending transcript by the clip's name, hands it to a send that happens after it arrived, and otherwise writes it into the stored attachment of the message it went out with, in the open chat or, if the user left it, in the database. Copying a message now includes its voice transcript. Assisted-by: Claude (Opus 5.5 Medium) * ui: add Voice settings and resolve what the microphone does per model The user should not have to think about which kind of voice input is in use, so the app decides it from the conversation's model and two settings. resolveVoiceMode is a small pure function: with Voice input on Automatic and a chat model that has audio input, a recording is a voice message, transcribed by that same model so a router swaps nothing. Otherwise it is dictation, with the chat model when it can hear, else, in router mode, with the chosen transcription model. When no model can transcribe, the microphone is unavailable and says why. Settings get a Voice section of their own rather than more rows in General, which is already long: these four belong together and one of them lists the server's models. - Voice input: Automatic (default) or Always dictate. - Transcription model: Same as chat model (default) or, in router mode, any model with audio input; on a single-model server it only says whether that model can transcribe. It notes that dictating with another model loads it beside the chat model. - Transcribe voice messages: on by default. - Transcription language: empty for automatic. The keys live in the settings registry like the others, so defaults, storage, reset and export need nothing new. The model picker is drawn by SettingsChatVoiceTab, the way Extensions and Tools draw their sections. Assisted-by: Claude (Opus 5.5 Medium) Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com> * ui: record voice messages and dictation from the chat form The microphone now does what the conversation's model allows, without the user choosing a mode: its label says "Send a voice message" or "Dictate", and when no model can take audio it is disabled and says why (shown in router mode only when some model there has audio input, so the reason can be acted on; a single-model server without audio keeps the send button as before). The microphone errors keep their warning state and toast, now also for the voice engine's own capture errors. Recording goes through the voice engine's VoiceRecorder instead of the MediaRecorder flow. While it records, the actions row becomes a recording row: cancel, the live waveform, the elapsed time and stop. Escape or cancel drops the recording. The form keeps its two rows while recording, so the message stays visible on desktop and the row fits the phone layouts and the expanded sheet. - Dictation writes the live transcript into the message at the caret as it streams. DictationSpan replaces only the dictated stretch, moves it when the user types before it, keeps what is typed after it, and hands text the user edits over to them. When nothing streams, the row shows Transcribing... after stop and the text goes in once it is back. Cancel takes the dictated text out again. No audio is attached. - A voice message streams its live text, if any, under the waveform, not into the message. On stop the clip is attached as a voice file with its peaks and length, and its transcript is followed by the voice transcripts store (unless Transcribe voice messages is off). Realtime transcription is skipped for servers reached through a tunnel or with an API key, which a browser websocket cannot carry. The first dictation with a separate router model shows a one-time note that it loads beside the chat model. The form takes an optional voiceRecorder, so tests and stories can run it without a microphone. Assisted-by: Claude (Opus 5.5 Medium) * ui: draw audio attachments as playable waveforms with their transcript A voice message should read as a clip, not as a file chip. Audio attachments now render with Llama UI's VoiceWaveform inside VoiceTranscriptReveal, through a direct branch in the core attachment item (voice is core, so it does not go through the extension renderers): the removable sm chip in the chat form, and the md clip in a sent message, on the user bubble's surface. - The audio plays from an object URL made from the stored base64 (or the uploaded file's data URL), revoked when the clip leaves the screen. - A clip stored without peaks, an older voice message or an uploaded audio file, gets them computed from its audio once per session, in memory only; uploaded files keep their name under the waveform. - The transcript opens under the clip on hover, keyboard focus or a tap, and shows Transcribing... while the voice transcripts store waits for it, Transcription unavailable when it failed, and nothing when none was asked for. - The full-screen preview keeps its player and shows the transcript. Tests run the chat form with a fake recorder: the microphone per mode, the recording row, dictation around typed text, the Transcribing... state, cancel, and a voice message chip whose transcript arrives later and goes with the send. Stories show the form recording each kind, a recorded chip, and sent voice messages with the transcript, pending, and computed peaks. Assisted-by: Claude (Opus 5.5 Medium) * ui: drop the old audio recorder and proxy realtime transcription in dev The voice engine replaced AudioRecorder, so it goes. The dev server now proxies WebSockets on /v1, so realtime transcription works under vite dev. The VoiceCapture test waits for the tone to be recorded instead of a fixed 700 ms, which a loaded test run could miss. Assisted-by: Claude (Opus 5.5 Medium) * ui: dictate with a model that is already loaded when the chat model cannot hear The Transcription model setting's Automatic choice now falls back to a router model with audio input that is already loaded, so dictating with a text-only chat model works without picking a model and loads nothing. A picked model still wins, and the empty value is a named constant. Idea from upstream PR 27567, by NickM-27. Assisted-by: Claude (Opus 5.5 Medium) Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com> * ui: fill the recording waveform, and open voice message transcripts - The live waveform fills the row from the start: a recording shorter than the row is stretched over all its bars, a longer one scrolls with the newest level on the right. No dotted placeholder. - A voice message's transcript is open under its clip, in the chat form while it comes in and in the sent message, with the toggle beside it to fold it. - Settings > Voice > Collapse transcripts (off by default) keeps them folded instead, opening on hover, keyboard focus or a tap. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit 2a93f2f948ed976e40f693ce5d3acf1168284b05) * ui: record voice without ducking other audio, from the first word - Voice input asks for the raw microphone signal: no echo cancellation, noise suppression or auto gain. Echo cancellation made the system treat a recording as a call, which lowers or interrupts other playback, and auto gain faded the first words in. - One AudioContext serves every recording, with the capture worklet loaded once, suspended in between and, where the browser allows it, without an output device, so recording never touches the speakers. - The chat form prepares it when it gets focus (no microphone, so no recording indicator), and a mouse starts recording on press. - A recording starts counting once audio flows: until then the bar says Starting... instead of running the timer. Development builds log how long each step of a start took. - Settings > Voice > Microphone: Automatic (the system default, or the built-in microphone when the default is a Bluetooth headset, which would switch to its low-quality call mode), System default, or a listed microphone. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit 76643e7210d21617d734fdfde677e84af7273153) * ui: show voice text live in the message box, and follow transcripts as spoken - A model that hears but has no live ASR (e.g. LFM3-Omni) now gives text while you record: the realtime client commits the audio piece by piece, at the pauses the levels show, one piece at a time, and the final commit takes only what is left. - A voice message's live text shows as the empty message box's placeholder. Clicking into the box makes it text to edit there, and what is said next follows it. On stop the text leaves the box and goes with the clip; when it was edited, the edit is the clip's transcript. - Llama UI gains VoiceSpokenText and spokenLength(): spoken words in the full foreground colour, which they keep, the rest muted. A clip's transcript follows its playback (estimated over the parts with speech, as there are no word timings), and live text brightens as it arrives. - The recording bar has a send button that stops and sends in one go. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit ce773017f902f8a5690ab9ecca368cecb6427422) * ui: let the capture test start with silence A capture starts with its first audio block, which can come before a stream's first sound, so the steady tone is checked from where it begins. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit a41ee5337b83fef685a5cbaff176925f1647ce9d) * ui: show live voice text sooner, and keep silence from becoming "yeah" - A model that transcribes per commit (LFM3-Omni) gets shorter pieces: cut at a 0.35 s pause after speech, or at the next quiet moment once a piece reaches 3 s, so the text follows the speaker within a few seconds in dictation, the message box and voice mode. - When the recording stops, the audio after the last piece is committed only if speech was heard in it. LFM3-Omni transcribes a commit of pure silence as "Yeah." (or "はい。"), which ended up at the end of the text. A recording whose only piece is that rest still commits it. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit 43e52cbab0fba9d8e84b9491890a406e3c6c90f3) * ui: transcribe a recording cut at pauses once more as a whole Live text comes piece by piece, cut at the speaker's short pauses, and a model reads each piece as a sentence of its own: "I went. To the store." When the recording ends, one pass over the whole clip gives the final text, punctuated by meaning, in dictation and in voice mode alike. The pieces' text stays the live text until then, and is kept when that pass fails or the recording is longer than two minutes. Assisted-by: Claude (Opus 5.5 Medium) (cherry picked from commit 359dd9632e6c88000a1d23c2575bd1e6fcdd9ec1) * ui: give live text over HTTP for models without a realtime session A model that hears but cannot hold a realtime session, such as Qwen3-ASR, was transcribed only after the recording, so dictation showed no text while speaking. The recording is now cut at the speaker's pauses as with a realtime session, and each piece sent to /v1/audio/transcriptions on its own, one at a time, its text joined live. The audio is kept from the start, so what is said while the socket answers goes with the first piece. The whole clip is still transcribed once more at the end, and the pieces' text is kept if that fails. A server a websocket cannot reach (an API key, a remote server) gets live text the same way. Assisted-by: Claude (Opus 5.5 Medium) * ui: say when the browser processes the microphone despite the raw request The browser keeps one processing setting per microphone: while a call, another tab or voice mode has it open with echo cancellation, a dictation that asked for the raw signal is processed too, sounds distant, and the system may lower other sound. The capture now warns when it gets processing it did not ask for, and its development log names the microphone, its rate and what processing is on. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the microphone beside the stop and send buttons While a reply streams, or once the message has text, the microphone no longer gives way to the main button: it stays beside it as a quieter, secondary button, so more can be dictated without clearing the message or waiting for the reply. Assisted-by: Claude (Opus 5.5 Medium) * ui: show the voice components' stories with Llama UI VoiceWaveform and VoiceTranscriptReveal are Llama UI components, so their stories move to the library's Storybook, under Llama UI/Voice, with the demo audio they play. Llama Chat's Storybook shows them there too, and its own voice stories take the demo audio from the library. VoiceWaveformLive stays with Llama Chat, as its demo records through the chat form's microphone. Assisted-by: Claude (Opus 5.5 Medium) * ui: add a noise suppression setting for voice input Recordings stay raw by default. Settings > Voice can now switch the browser's noise suppression on; echo cancellation and automatic gain stay off either way, as they interrupt other sound and fade the first words in. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep live voice text coming when the room is never quiet Pieces were cut only at a quiet moment, so with music or talk in the room none was ever cut and no text showed until the recording ended. A piece half as long again as its longest length is now cut where it is. Assisted-by: Claude (Opus 5.5 Medium) * ui: add echo cancellation and automatic gain settings for voice input Settings > Voice now has a switch for each kind of microphone processing the browser offers. All three stay off by default, so recordings are raw; each one says what switching it on costs. Assisted-by: Claude (Opus 5.5 Medium) * ui: put the microphone after the stop button, filled like it While a reply streams, the microphone is the last button of the row and looks like the stop and skip buttons beside it, instead of a bare icon before them. Assisted-by: Claude (Opus 5.5 Medium) * ui: say when dictated text is being refined, and send it refined After a recording with live text, the whole clip is transcribed once more so pauses do not end sentences. The form now says so while that pass is out, and a send asked for meanwhile waits for it, so the message goes out with the refined text rather than the pieces. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the send button in place and put the microphone beside it The send button no longer gives way to the microphone on an empty message. The microphone stands before it as a quieter, tertiary button, with the same tooltip, warning and recording states as before. Assisted-by: Claude (Opus 5.5 Medium) * ui: let a recording be sent while a reply streams, once it has words While a reply streams, the recording row now shows its send button as soon as the first transcribed words are in the message, so the recording no longer has to be stopped first. The microphone also stays beside the send button once a voice message is attached. Assisted-by: Claude (Opus 5.5 Medium) * ui: raise quiet speech for transcription, and say when it is too quiet A quiet microphone gave the model near silence to transcribe. A new Voice setting, Boost quiet speech (on by default), raises speech to a level speech models hear well, up to eight times louder: the audio streamed for live text as it goes, and the whole recording in one pass when it stops, so its first words get the same gain as the rest. Only speech is measured, never the silence around it, loud speech is left as it is, and a soft limiter keeps raised peaks from clipping. When speech stays quiet even at the highest gain, the recording row says so. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the voice engine free of app code The engine in lib/voice imported the microphone messages from the app's utils and the WAV type from its enums, the only two things keeping it from moving into a package of its own. The microphone checks and messages move into the engine (voice/support.ts), next to the capture that needs them; the uploaded-file helpers stay in the app's utils. Assisted-by: Claude (Opus 5.5 Medium) * ui: start voice sessions from one voice service How a recording reaches the server (routes, API key, whether the realtime socket can be used) and how the Voice settings become capture and transcribe options were built inside the chat form's voice hook, and copied into every other feature that records. VoiceService builds them in one place: a feature asks it for its session options and adds only its own callbacks. Assisted-by: Claude (Opus 5.5 Medium) * ui: tell speech from the room in one place The pause detector, the speech gain and (in voice mode) the listener for speech over a reply each followed a noise floor with their own numbers. A SpeechDetector now does it for all of them, in dB, with one set of settings in VOICE_SPEECH. Assisted-by: Claude (Opus 5.5 Medium) * ui: let any text field take dictation Writing dictated text into a field (at the caret, around the user's typing, replaced by the whole recording's text when it comes) moves out of the chat form's hook into a plain createDictation(). The hook keeps what is the chat form's own: voice messages, toasts and the hint. Assisted-by: Claude (Opus 5.5 Medium) * ui: move the recording row into Llama UI VoiceRecordingBar draws the row shown while recording (cancel, waveform and time, send, and a caption or warning under it) from data alone, so voice mode or a notes page can show the same row. The chat form passes its labels and its microphone button as the stop button. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the voice engine's Svelte recorder behind its own entry VoiceRecorder, the one part of lib/voice written with runes, moves to lib/voice/svelte, so the rest of the engine is plain TypeScript any app can use without Svelte. Assisted-by: Claude (Opus 5.5 Medium) * ui: move voice input into its own package, Llama Voice The voice engine (capture, the speech signal, transcription over the realtime socket or HTTP) and dictation into a text field move from the app to ui/lib/voice as @ggml-org/llama-voice, so another app, such as a notes app, can use them. The Svelte recorder is its ./svelte entry. Llama Chat compiles the sources, as it does Llama UI's, and keeps its server and settings wiring. The package has its own type check, tests and build, and a CI job for them. Assisted-by: Claude (Opus 5.5 Medium) * ui: show which model does what in voice input The model picker and the Models manager mark each model that hears: Dictation on the one that transcribes dictation, Speech to text on transcription models such as Qwen3-ASR, and Voice on chat models that take voice messages. A loaded model kept only for dictation sits apart from the chat models, a model can be made the dictation model from its row, and Settings > Voice groups the transcription models apart from the chat models with audio input. A voice store now resolves what the microphone does, for the chat form and the model lists alike. Assisted-by: Claude (Opus 5.5 Medium) * ui: warn of a quiet microphone inside the recording row The too-quiet line under the waveform made the form grow and jump. The warning now stays inside the row: the waveform turns the warning colour, an icon sits by the time, and the advice opens above the row in a tooltip for a few seconds, then on hover or focus. Screen readers still hear it when it appears. Assisted-by: Claude (Opus 5.5 Medium) * ui: open the microphone while the pointer is on its button The first seconds of a recording could come out quiet while the microphone settled, and the too-quiet warning showed for them. With the pointer or the focus on the microphone button, the microphone now opens ahead of the press (only when the page may already use it, so nothing asks), and the recording takes it at once; it is let go shortly after the pointer leaves without a press. The warning also waits until the speech has stayed too quiet for two seconds. Assisted-by: Claude (Opus 5.5 Medium) * ui: tidy voice input after review Fix a microphone left open when a recording is cancelled while it starts or the pointer leaves while it opens, a capture that is still letting go silencing the next one, and live text stopping after a realtime error. Share the dictation model choice, the transcript lookup, base64 decoding, PCM joining and stopping a stream instead of repeating them, and export only Llama Voice's public API. Assisted-by: Claude (Opus 5.5 Medium) * ui: pin Llama Voice's dev dependencies to Llama UI's versions Assisted-by: Claude (Opus 5.5 Medium) * ui: let demos speak into a synthetic microphone Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the microphone for voice input, off the audio files entry The attach menu's Audio Files entry used a microphone, which now sits beside the form's own microphone button. The microphone stands for recording and dictation only; a model's audio modality is headphones, and the audio files entry is a file with headphones. Assisted-by: Claude (Opus 5.5 Medium) * ui: set voice input on the slimmed design tokens Voice input's components use the type roles, the one corner scale and shadcn's utility names for the neutral colours, as the rest of Llama UI and Llama Chat do since the design tokens were slimmed down. Assisted-by: Claude (Opus 5.5 Medium) * ui: name voice states, platform strings, patterns and tunings Addresses the review on the voice PR: - Llama Voice: VoiceCaptureState, VoiceRecorderStatus, RealtimeState and RealtimeUnsupportedReason are enums (src/enums.ts, exported); DOMException names, AudioContext states, transcription form fields, socket protocols, device kinds, worklet messages, error messages, regexes and tuning values are named constants. - Llama UI: VoiceTranscriptStatus, VoiceWaveformSize and VoiceRecordingBarStatus are enums; KeyboardKey, MouseButton and PointerType join its UI enums; the voice components' sizes, labels, timings, selectors and patterns move to constants/voice.constants.ts. The voice unit tests move from src to tests/unit. - App: ChatFormVoiceState is an enum, the recording bar's status comes from a map, and the voice code uses the enums above, BooleanString, HEADERS and new voice constants in place of inline strings, regexes and numbers. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep all of Llama UI's tests under tests The Model UI formatting spec moves to tests/unit and the button spec to tests/client, next to the voice specs; the test projects now read only tests/unit and tests/client. Assisted-by: Claude (Opus 5.5 Medium) * ui: show a transcript's words without the model's own format Qwen3-ASR names the language before its text (language French<asr_text>...) and VibeVoice-ASR answers with timed segments as JSON, which the server passes on as they are. The voice engine now takes both off, over HTTP and over the realtime socket, also while segments still stream, so the message box and the clip show the words alone. Assisted-by: Claude (Opus 5.5 Medium) * ui: dictate when a speech-to-text model is picked for the conversation A speech-to-text model has audio input, so on Automatic the microphone recorded a voice message for it, which such a model has nothing to answer with. It now dictates, as the model's badge already says it is for. Assisted-by: Claude (Opus 5.5 Medium) * ui: say why a live-only model cannot transcribe behind an API key A browser websocket cannot send the API key, so the realtime socket is left out on a server that needs one, and a model that only transcribes over it (Voxtral Realtime) answers the HTTP route with a 501. The clip and the dictation toast now say so instead of only that transcription is unavailable. Assisted-by: Claude (Opus 5.5 Medium) * ui: pick the dictation model with a microphone on its row The model that transcribes dictation was marked with a filled Dictation badge. It is now marked by a red microphone among the row's actions in the model picker, beside the heart, and every other model that hears shows the same microphone on hover: a click makes it the transcription model, and a click on the chosen one hands the choice back to Automatic. The badge keeps saying what kind of hearing model a row is, and the Models manager marks the dictation model with a small microphone beside its name. Assisted-by: Claude (Opus 5.5 Medium) --------- Co-authored-by: Nicolas Mowen <14866235+NickM-27@users.noreply.github.com> |
||
|
|
cf4153a9ba |
ui: publish llama-ui as an alias of @ggml-org/llama-ui (#32667)
The unscoped llama-ui on npm is a 0.0.1 placeholder. After the scoped publish, the workflow renames the same build and publishes it again at the same version, with a readme note pointing at the scoped package. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
6abcbe5d35 | remove ui-lib publish verification step (#32658) | ||
|
|
5812201a81 |
ci : fix build-opencl.yml (#32594)
Signed-off-by: Adrien Gallouët <angt@huggingface.co> |
||
|
|
b5dbca1e44 |
ci : use LLVM 22 (#32565)
* ci : fix build-vulkan.yml Signed-off-by: Adrien Gallouët <angt@huggingface.co> * ci : fix behavior-check.yml Signed-off-by: Adrien Gallouët <angt@huggingface.co> --------- Signed-off-by: Adrien Gallouët <angt@huggingface.co> |
||
|
|
edec462cc0 |
ggml, ci : use llvm 22.1.8 (#32548)
* ggml, ci : use llvm 22.1.8 Signed-off-by: Adrien Gallouët <angt@huggingface.co> * Update build-cpu.yml too Signed-off-by: Adrien Gallouët <angt@huggingface.co> * Fix clang 22 Signed-off-by: Adrien Gallouët <angt@huggingface.co> * Fix release.yml (arm64 with vulkan) Signed-off-by: Adrien Gallouët <angt@huggingface.co> * Fix server.yml Signed-off-by: Adrien Gallouët <angt@huggingface.co> --------- Signed-off-by: Adrien Gallouët <angt@huggingface.co> |
||
|
|
2cbfc3cf34 |
[CUDA] : Add SKU-aware FP16/BF16 accumulation precision gating for NVIDIA GPUs (#30672)
* cuda: select cuBLAS precision by GPU SKU Assisted-by: Codex * cuda: fix remaining GB10 macro reference * cuda: review comment addressed, store accumulation preference in device info * Apply batched suggestions from code review Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * addressed Oliver's comment regarding initialize NVML once and install headers in Windows CI * move precision selection helper into init --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de> |
||
|
|
a693a5f66f |
ui: llama-chat publish drops the registry check, bin paths without ./ (#32543)
* ui: llama-chat publish waits for the registry, bin paths without ./ The first publish of @ggml-org/llama-chat took about four minutes to show up on the registry, so the 60 seconds the check after publishing waited failed a release that had gone through. It now waits up to ten minutes. npm normalizes "./bin/cli.js" to "bin/cli.js" on publish and warns that the bin was "invalid and removed", which reads as if the command was dropped. Both package.json files now give the paths as npm writes them. Assisted-by: Claude (Opus 5.5 Medium) * ui: drop the registry check after llama-chat publishes npm publish already fails when the registry refuses a package, and the check after it only failed a release that had gone through, while npm was still processing it. Nothing waits on the registry any more. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
97069ff6c4 |
ui: run Llama Chat with npx @ggml-org/llama-chat (#32524)
* ui: run Llama Chat with npx @ggml-org/llama-chat Makes @ggml-org/llama-chat publishable: drops private, adds bin, files, exports, repository and a prepack build. Its CLI serves the prebuilt dist on localhost (3000 by default) and forwards every other request to a llama-server (--server, default http://127.0.0.1:8080), with the headers llama-server sends for the UI. A thin unscoped llama-chat package in alias/ forwards to it, so npx llama-chat works. Assisted-by: Claude (Opus 5.5 Medium) * ui: llama-chat revalidates service-worker.js and asks for a build before packing SvelteKit 3 adds service-worker.js next to sw.js, and llama-server revalidates it, so the CLI does too. The folder's .npmrc sets ignore-scripts, so prepack never ran: it goes, and the publishing steps build dist explicitly and check the packed file list. Assisted-by: Claude (Opus 5.5 Medium) * ui: llama-chat keeps llama-server's DNS rebinding and cross-origin guards The CLI called llama-server with its own Host and dropped every Origin, so a rebound host name or another page got past the checks llama-server makes when --cors-origins is not *. It now drops only its own page's Origin, passes other Origins and Sec-Fetch-Site through, and, when it listens on loopback, answers only to localhost and loopback addresses. `start` on Windows gets its empty title verbatim. Assisted-by: Claude (Opus 5.5 Medium) * ui: llama-chat build writes the app to a folder for any static server `npx @ggml-org/llama-chat build [--out <dir>] [--force]` copies the prebuilt app, without source maps, to ./llama-chat or the given folder, so a plain file server can host it next to the CLI serving it. It refuses a folder that is not empty unless forced, and the serving flags and build flags refuse each other's command. The README shows serving the folder with http-server forwarding the API to llama-server. Assisted-by: Claude (Opus 5.5 Medium) * ui: publish Llama Chat to npm from CI when its version is bumped ui-chat-publish.yml publishes @ggml-org/llama-chat and its llama-chat alias the way ui-lib-publish.yml publishes @ggml-org/llama-ui: the version is set by hand, and a commit that reaches master with a version npm does not have builds dist, checks the packed files and publishes both with trusted publishing. PRs that change the packages get a dry run. npm-version-check.mjs takes the package folder, so both workflows share it. ui/README.md says how releases work, and Chat's AGENTS.md says not to bump versions unless asked to release. Assisted-by: Claude (Opus 5.5 Medium) * ui: call the app Llama Chat in its README The README's title and a few lines still called the app llama-ui or Llama UI, the name of the library it is built on. They now say Llama Chat, and the intro points to Llama UI in ui/lib/ui. Assisted-by: Claude (Opus 5.5 Medium) * ui: llama-chat connects to llama-server's default port, 9931 The CLI's --server default and its examples used 8080, which is not where llama-server listens. They now use 9931, its default port, so npx llama-chat reaches a llama-server started without --port. Assisted-by: Claude (Opus 5.5 Medium) * ui: a shorter Llama Chat README that matches the code The README grew to nearly 900 lines, and its tables of components, hooks, stores and services had drifted from the code. It now keeps what a reader needs: running it with npx and self-hosting, a short feature list, development commands, the architecture (the line between Llama UI and the app, a table of the layers and where they live, a flowchart and one sequence diagram for sending a message), a few patterns worth knowing, cross-origin isolation, how the build is embedded in llama-server, the project tree and publishing. The folders now carry the detail the tables used to. ASCII replaces the unicode arrows. Assisted-by: Claude (Opus 5.5 Medium) * ui: llama-chat serves only static files, without the API proxy The CLI forwarded every request it had no file for to llama-server, from loopback. With --host 0.0.0.0 that let any LAN client reach a llama-server started on 127.0.0.1 as a local request, tools included, which llama-server on its own refuses without an API key. The browser now talks to each llama-server directly, so llama-server's own rules apply. --server is gone, and anything that is not a file in dist is a 404, or a 405 for other methods. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
6da36139e8 |
ui: run the Storybook deploy on HF Jobs, with a token that can write the Space (#32525)
The deploy failed on master with a 403: HF_TOKEN_UI_STATIC_OUTPUT is scoped to the UI bucket and cannot write ggml-org/llama-ui-storybook, so hf upload tried to create the Space. The upload now uses HF_TOKEN_UI_STORYBOOK when that secret exists, and the job runs on hf-jobs-cpu-upgrade instead of a GitHub-hosted runner. That image gets python3-venv from apt for the hf CLI, as the other HF Jobs workflows do. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
65758d01f5 |
ui: Storybook follow-ups: generated DESIGN.md tokens, shared helpers and one Llama Mono (#32523)
* ui: generate DESIGN.md's token sections from the CSS Add scripts/design-md.js in ui/apps/chat (npm run design:tokens, and npm run check:design in CI), which writes the colors, typography and rounded sections of DESIGN.md's front matter from llama-brand's colors.css and Llama UI's primitives, semantic tokens and font stacks. The front matter had drifted: it gave Matte's surfaces as the light base and system-ui for every text role. Twins are now written wherever a token resolves differently in dark mode (-dark) or in Matte (-matte). Describe the floating panel's rim as the CSS draws it, on the border, and say which parts of DESIGN.md are generated and which are kept by hand. Assisted-by: Claude (Opus 5.5 Medium) * ui: say the Storybooks show accessibility findings without enforcing them Both Storybooks keep a11y test 'todo': under 'error' 98 of the stories fail, most on colour contrast of the muted foreground and on nested interactive controls, which need design changes. Reword the Uses tile and record the reason next to the setting. Assisted-by: Claude (Opus 5.5 Medium) * ui: list every Llama Chat group in the Introduction's Sections figure The figure said it was in sidebar order but left out Forms, Actions, Badges, Misc, Servers and PWA. Its list moves to sections.ts, and a unit test checks it against storySort in .storybook/preview.ts, which has to stay inline there because Storybook parses it as written. Assisted-by: Claude (Opus 5.5 Medium) * ui: load the package's Llama Mono in both Storybooks The library's Storybook declared Llama Mono a second time from tests/stories/brand/fonts, a 72276-byte file that differs from the 76896-byte one the package ships, so its pages showed a Llama Mono that does not ship. Drop both Storybooks' copies: theme.css already loads the package's font. Assisted-by: Claude (Opus 5.5 Medium) * ui: take Llama Chat's shared Storybook helpers from Llama UI Llama Chat kept copies of the Editorial, Chapters, StateGallery, StateGalleryItem, svx.d.ts and docs.css that live with the library's stories. Its stories now import them as $llama-ui-stories/..., an alias its Storybook config and tsconfig.json map onto ui/lib/ui/tests/stories, the way it already compiles the library from source; docs.css and svx.d.ts are referenced by path. The library's Editorial takes Llama Chat's newer list numbering. An empty svelte.config.js next to the stories keeps Llama Chat's svelte-check from loading the library's vite.config.ts, as src/svelte.config.js does for the sources. Assisted-by: Claude (Opus 5.5 Medium) * ui: pin the library's Storybook to Llama Chat's versions ui/lib/ui used ^10.6.2 with @storybook/addon-mcp at 10.6.2, while ui/apps/chat pins 10.6.6, so one tree had two Storybooks. Pin the library's Storybook packages exactly to Llama Chat's: 10.6.6, addon-svelte-csf 5.1.6 and @chromatic-com/storybook 5.5.1. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
9e93cd8001 |
ui: a Storybook for Llama UI and Llama Chat (#32485)
* ui: Llama UI's own Storybook in ui/lib/ui The library gets its Storybook (`npm run storybook`, `npm run build-storybook`) with a story for everything it holds, under Llama UI: the foundations, the 29 base components (as actions, forms, display, overlays and navigation), the app components (ActionIcon, SearchInput, SyntaxHighlightedCode), the Model UI and the native app mockups. The stories import the package by its published names, aliased onto src/lib, run as tests under `npm test` (the storybook project in vite.config.ts), and are type-checked with the rest of the package. `npm run dev` starts Storybook next to the SvelteKit check page; the toolbar switches the color mode and the surface theme, and pages that are mostly prose are .svx, through mdsvex. Assisted-by: Claude (Opus 5.5 Medium) * ui: a Storybook for Llama Chat Llama Chat's Storybook gets an introduction, Agents › Design guidelines (DESIGN.md), Llama UI's stories as the library has them, and Llama Chat: its own foundations (safe areas, the splash screens), then one section per src/lib/components/app/<area> (Chat with the chat screen, form, messages, attachments and tabs; Content, Models, MCP, Navigation, Settings, Dialogs, Forms, Actions, Badges, Misc, Servers and PWA), each accessibility story next to its component's. It replaces the flat pages it had (chat message, chat form, markdown content, the PWA refresh alert and their fixtures, and the ChatForm single-row exploration). The stories run on the new tokens and surfaces with the color mode and surface theme in the toolbar (by mode-watcher's key, the ColorMode and SurfaceTheme enums and applySurfaceTheme), and pages that are mostly prose are .svx in an editorial frame (AGENTS.md says how to write them and where a story goes). The story tests run Llama Chat's stories and leave the library's to the library. The brand and communications pages are in ggml-org/llama-brand; the logo files come from @ggml-org/llama-brand. Assisted-by: Claude (Opus 5.5 Medium) * ui: let the ChatScreen story store its reply before the next one starts The ContinueConversation story ended while its reply was still being stored, and the next story reset the stores under it. It now waits for the reply to finish. Assisted-by: Claude (Fable 5.2) * ui: publish the Storybook to its Hugging Face Space on every UI change A new UI Storybook workflow builds ui/apps/chat's Storybook whenever ui/apps/chat or ui/lib/ui changes on master (or on demand) and uploads it to the static Space ggml-org/Llama-UI-storybook, with a Space card naming the commit it was built from. Files the new build no longer has are removed; the Space's .gitattributes stays. It runs only in ggml-org/llama.cpp, takes the UI's Hugging Face token (HF_TOKEN_UI_STATIC_OUTPUT), and the Space can be changed with the HF_SPACE_UI_STORYBOOK variable. Assisted-by: Claude (Opus 5.5 Medium) * ui: run the library's tests, its stories among them, in CI The Library checks job type-checked and packed ui/lib/ui but never ran its tests, so the library's stories, which now hold every library component's, went untested in CI. It installs Chromium and runs `npm test`. Assisted-by: Claude (Opus 5.5 Medium) * ui: draw a floating panel's rim on its border, so its edge is one line Menus, popovers, dialogs and the other matte-float panels carry a 1px border that the surface made transparent, and drew their inner highlight as an inset shadow. An inset shadow sits inside the border, so the 1px of fill between it and the outer ring showed as a second edge. The border now draws the highlight, right inside the ring. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
744c68cd14 |
ui: publish @ggml-org/llama-ui from CI, and release 0.4.0 (#32480)
* ui: publish @ggml-org/llama-ui from CI when its version moves ui/lib/ui reaches npm only when someone remembers to publish it by hand, which let 0.3.0 sit a day behind master. The version in its package.json becomes the release: a commit landing on master with a version npm does not have gets built, linted and published with a provenance attestation, through OIDC rather than a stored token, as release-publish.yml already does for GitHub releases. A library change carrying no bump publishes nothing and says so, on the PR and on the run. Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS) * ui: bump @ggml-org/llama-ui to 0.4.0 The library on npm, published before the design system PR merged, holds no entry points for the Model UI, the app components or the shared constants and enums, and no dependency on @ggml-org/llama-brand, whose colours the primitives now build on. Consumers import the new subpaths and get ERR_PACKAGE_PATH_NOT_EXPORTED. Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS) * ui: name no environment for the Llama UI publish job Creating a repository environment needs admin rights the UI maintainers do not hold, so a job that names one cannot be set up by them. The job works without it: npm's trusted publisher matches on the repository and the workflow file, and its environment field stays blank to match. Nothing else in the job depends on an environment. Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS) * ui: address review on the Llama UI publish job - the advisory version check leaves ui.yml for the publish workflow, on a pull_request trigger whose path filter keeps it off PRs that touch only Llama Chat: a chat-only PR was told to bump a library it did not change - publish and verify run on master only. The trusted publisher matches on the repository and the workflow file, not the branch, so a dispatch from a PR branch - anything with write access can start one - could publish code no one reviewed. With no environment, this was the only branch protection in place. - the registry read after a publish retries six times rather than sleeping once, so a release the registry is slow to serve is not reported as a failure - a build metadata suffix (1.2.3+build) is not a prerelease, and versions order by semver's prerelease rules, so 0.4.0-beta.1 behind a released 0.4.0 is refused instead of shipped to next - the summary step passes its values through env like the others, actions: read is gone, and the concurrency comment claims only what the setting does - npm older than 11.5.1 is refused up front, as it cannot publish with an OIDC token Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS) * ui: note the two trusted-publishing traps in the Llama UI README A publisher names a workflow file that must already be in .github/workflows, so it is added after this workflow lands rather than before, and one created now publishes only through staged publishing until npm publish is permitted for it. Provenance comes free with either, and only on a GitHub-hosted runner. Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS) * Update .github/workflows/ui-lib-publish.yml Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> |
||
|
|
7f106b5459 |
ui: Llama UI design system: tokens, surfaces and fonts (#32468)
* ui: bring @ggml-org/llama-ui into ui/lib/ui The design system package (tokens, fonts, surfaces, components and Model UI), so far in the temporary ggml-org/llama-ui repository at edaf518, moves to its home in ui/lib/ui, built with @sveltejs/package and published as @ggml-org/llama-ui from here. ui/lib holds the packages, one folder each. Its references to tools/ui now name ui/apps/chat, its package.json points at this repository, and the UI workflow type-checks and packs it. ui/README.md describes the layout. Assisted-by: Claude (Opus 5.5 Medium) * ui: Llama UI design system: tokens, surfaces and brand The design system from allozaur/docs/storybook-overhaul: three tiers of tokens, matte glass surfaces, Llama Sans and Llama Mono, and the base components on the new tokens (Tabs and GlassKey among them). It also restyles the stack's features with it, replacing the plain stand-ins they used until now. DESIGN.md is generated from the tokens. The Storybook that documents it is the next PR. Assisted-by: Claude (Opus 5.5 Medium) * ui: have Llama Chat use @ggml-org/llama-ui for its base components, tokens and fonts Llama Chat kept its own copy of the design system: 29 base components, the three tiers of tokens, the surfaces, the theme and the fonts, next to the 12 components ui/lib/ui held. The copy goes, and the library gets what it lacked. ui/lib/ui - the 17 base components the app had and the library did not (alert, alert-dialog, button-group, checkbox, collapsible, context-menu, dialog, drawer, dropdown-menu, empty, hover-card, radio-group, scroll-area, skeleton, table, tabs, textarea), with vaul-svelte for the drawer - the inset-group and selected-row tokens and surfaces - sources import each other by relative path instead of $lib, which in an app points at the app's own src/lib - theme.css no longer imports tw-animate-css: it is a peer dependency that the consumer imports before the theme, as it does Tailwind - 0.3.0 ui/apps/chat - imports the base components as @ggml-org/llama-ui/ui/<name> and cn/tv as @ggml-org/llama-ui/utils; app.css imports the library's theme.css - compiles the library from source, with no install or build of it: scripts/vite-plugin-llama-ui.ts resolves its entry points and what it imports, and svelte.config.js gives TypeScript the same paths - src/lib/components/ui, styles/tokens, styles/surfaces.css and assets/fonts are removed; components.json moves to the library Builds - CMake stages ui/lib/ui next to the app, keeping the layout of ui/, and rebuilds when a library source changes - the Dockerfiles copy the library's sources; the Nix derivation takes ui/ as its source The built CSS has the same rules in the same order, apart from one zero-specificity rule and the @font-face rules moving, plus ten utilities and two tokens that only the library's own components use. Assisted-by: Claude (Opus 5.5 Medium) * ui: AGENTS.md places Llama UI in ui/lib/ui Llama Chat's AGENTS.md says where each part lives: Llama UI is ui/lib/ui, and Chat imports its tokens, fonts and base components from there. Assisted-by: Claude (Opus 5.5 Medium) * ui: Llama Sans and Llama Mono from llama-brand's latest, and shell commands marked in bash code The fonts in ui/lib/ui come from llama-brand's latest, with optical sizes from 10 to 36. Bash code blocks mark shell commands and flags. Assisted-by: Claude (Opus 5.5 Medium) * ui: open submenus outside the menu they come from A submenu was rendered inside its parent menu, so the menu's scroll box clipped it: the glass menus' backdrop filter, and the filter and transform of the open animation, make the menu the containing block even for the fixed-position submenu. Dropdown and context submenus now render in a portal, like the menus themselves. Assisted-by: Claude (Opus 5.5 Medium) * ui: draw submenus like the menu they open from Submenus had smaller corners than their menus, and context menus had none of the dropdown's glass. Context-menu parts now match the dropdown's, and both submenus take the menu's corners and height cap, with their first row level with the row that opens them, 4px past the menu's edge. Sub-triggers take the items' radius and pointer. The submenus' height cap and zoom origin named the dropdown's CSS variables, which bits-ui does not set on submenus; they now use its --bits-menu-* ones. The chat group submenu takes the actions menu's z-[999999] layer, as it no longer renders inside that menu. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep a late current-node write off a closed or switched conversation updateCurrentNode checked the active conversation, awaited the database write, then wrote currNode to whatever was active after it. When the conversation closed during the write it threw, and the reply ended in a streaming error dialog. When another conversation opened, that one got the wrong node. Assisted-by: Claude (Fable 5.2) * ui: have Llama Chat use the library's Model UI and app components Llama Chat still kept its own copies of what the library holds above the base components. It now takes them from @ggml-org/llama-ui too, and its copies go. ui/lib/ui - ModelId gains what Chat's had: the model's own draft kind, draft sidecars with their quant (ModelDraftSidecars, new), aliases, hideName and a tooltip, in the same badge order - entry points for them, like ui/<name>: @ggml-org/llama-ui/app/<group> (actions, content, forms, misc) and @ggml-org/llama-ui/models ui/apps/chat - ActionIcon, SearchInput, SyntaxHighlightedCode, Logo, ModelCapabilityIcons, ModelDownloadProgressBar and ModelDraftSidecars come from the library; Chat's copies and the two constants only they used are removed - ModelId and ModelOrgAvatar stay as thin wrappers: ModelId parses the raw id and applies the user's display settings, ModelOrgAvatar hides the avatar while Hub metadata is off - the svelte/* TypeScript path falls back to Svelte's types/index.d.ts: svelte/reactivity has no file of its own, so a library source importing it got its types from ui/lib/ui's own install when there was one Assisted-by: Claude (Opus 5.5 Medium) * ui: Llama UI takes the brand's colours and logo from llama-brand @ggml-org/llama-brand is now a dependency of the library, by the same GitHub tarball Llama Chat already installs, which now names it @ggml-org/llama-brand too. - primitives.css drops the 25 tokens it held copies of (the neutral greys, sand, the skies and cinder) and builds on llama-brand's css/colors.css instead; the skies' gradients point at llama-brand's. Apps import that file before theme.css, as they do Tailwind and tw-animate-css: the library's stylesheets import no package, so an app that compiles them from source resolves it from its own node_modules - Logo draws llama-brand's icon, in currentColor, and assets/logo.svg goes - Llama Chat's TypeScript paths resolve llama-brand from its own node_modules, like the library's other dependencies The fonts stay as the library's WOFF2 copies until llama-brand ships WOFF2 builds of them. Assisted-by: Claude (Opus 5.5 Medium) * ui: have Llama Chat take the library's constants, enums and helpers Llama Chat redefined most of what the library exports next to its components. Where the two were the same, Chat now re-exports the library's from the files that held them, so its own imports stay as they are. - the library gets entry points for them: @ggml-org/llama-ui/constants, @ggml-org/llama-ui/enums and @ggml-org/llama-ui/code, and @ggml-org/llama-ui/models carries the model data with the Model UI - from the library now: the shared CSS classes and code-block constants, the byte units, the memory-fit constants and minMemoryTierGb, the bit depth labels, the capability and modality icons, labels and order, ColorMode, TooltipSide, BooleanString, the model enums, ModelModalities, formatParameters, highlightCode and trimCodePadding - the library's modality order follows Chat's: vision, video, audio - Chat keeps its own where they differ: UI_DATA_ATTRS and ModelDownloadProgress are supersets of the library's (the latter extends it now), and formatFileSize counts in 1024s Assisted-by: Claude (Opus 5.5 Medium) * ui: give the chat form's add key and model selector a tighter shadow In light, the two keys inside the chat form, white on its white glass, took the control elevation, whose soft spread blended into the form. They take a new elevation-key instead: a 0.5px hairline all round and a 1px drop. In dark they keep the control elevation. Chosen from the options in Scrapbook › UI › Composer keys. - semantic tokens: --elevation-key (light: the hairline and the drop; dark: the control elevation) - component token: --chat-form-key-shadow, for the add key and every model selector trigger - DESIGN.md names it next to the other elevations Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
3a7660371d |
ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)
* ui: move tools/ui to ui/apps/chat The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library (ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS, .editorconfig, the git hooks, the dev and demo scripts and the docs follow it. Assisted-by: Claude (Opus 5.5 Medium) * ui: name Llama Chat's package @ggml-org/llama-chat It was still called llama-ui, the name of the library that now sits beside it in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name and the llama-ui.zip build artifact are unchanged. Assisted-by: Claude (Opus 5.5 Medium) * ui: call the built-in web UI Llama Chat in docs, help text and comments The app in tools/ui is Llama Chat; Llama UI is the component library it is built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now names it: the READMEs, SECURITY.md, the --ui help text, the CMake option description, one server log line, code comments and the Storybook intro. Names that others depend on are unchanged: the deprecated --webui flag aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings field, the storage migration keys and the Nix attributes. Assisted-by: Claude (Opus 5.5 Medium) * ui: name tools/ui as a former path of Llama Chat in the code-review skill Changes under the old tools/ui path, as well as tools/server/webui, still get the Llama Chat checklist. Assisted-by: Claude (Opus 5.5 Medium) * ui: name Llama Chat's path in the security policy's exclusions Llama Chat no longer lives under tools/server/, so the exclusion listed under it now names ui/apps/chat. What is covered stays the same. Assisted-by: Claude (Opus 5.5 Medium) |
||
|
|
95db855bce |
ci : use vX.Y.Z tags as docker tags (#32388)
* ci : use vX.Y.Z tags as docker tags When the Docker workflow is manually dispatched from a vX.Y.Z tag, use that tag verbatim for the image tags instead of appending the build number and commit hash. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB) * ci : require a tag for manual docker runs Manual dispatches of the Docker workflow must now use a tag ref, so the image tags are derived from the selected tag. The check can be skipped with the new skip_ref_check input. Scheduled nightly builds are unchanged. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB) |
||
|
|
6b01a4767f |
CI: trigger spec dec testing for relevant PRs (#32281)
* ci: trigger spec dec testing after refactoring * add options to run spec dec ci nightly manually |
||
|
|
278a89cb78 |
CUDA/HIP: Refactor the use of _sync warp primitives (#31617)
Uses the rocm-native primitives, increases the rocm version requirement to 6.3 and makes top-k selectable warp. |
||
|
|
82503b42fa | ci : set windows-cuda ccache size to 1G (#32185) | ||
|
|
827466b9ef | ci : build ui on non-slim runner (#32138) | ||
|
|
664f031f49 |
ci: move test models to ggml-org-dev (#32096)
* ci: move test models to ggml-org-dev * fix test spec |
||
|
|
956591f457 | ci : remove dead snakes (#32084) | ||
|
|
b326dd6752 |
devops : add missing MKL runtime DLLs to the Windows SYCL release package (#32005)
ggml-sycl.dll built with oneAPI 2026.2 imports mkl_intel_thread.3.dll and libimalloc.dll, which are not part of the release package. LoadLibrary fails and the SYCL backend is silently skipped, llama.cpp falls back to CPU. Adding both DLLs fixes the load: measured on Arc B570 with LFM2.5-8B-A1B Q4_K_M, pp4096 goes from 404 t/s (CPU fallback) to 2232 t/s (SYCL) with the extracted b12244 package. Assisted-by: GLM (z-ai/glm-5.3-flash) Co-authored-by: Asahi-Prv <Asahi-Prv@users.noreply.github.com> |
||
|
|
582eab7186 |
ci : bump ty to 0.0.100 (#32030)
* bump ty to 0.0.100 * rename rules * type fixes * type fix * proper type fix |
||
|
|
202c1a229f |
llama : Add FP8 KV-cache support (#28392)
* Add FP8 KV-cache support * Add tests for scaled, FP8 KV-cache to test-backend-ops * Fix get_*_scale mapping and dequnatize FP8 also before rope shift * Normalizes cache-scale names independently of projection layout. While fused KQV is currently impractival for non-shared scales, this is an improvement for future use regardless * Enable Gemma 4 Assistant with FP8 by storing and fetching scale ptrs * Update comment wording * Move E2E FP8 KV cache tests to test-llama-archs They belong there as they are not just a test for ggml's backend ops * Increase support surface, remove seed * make scaled kv generic * Fix CI failings for test-kv-cache https://github.com/ggml-org/llama.cpp/actions/runs/38718763447/job/116367842578?pr=28392 * CI: execute kv-cache-scale tests * exclude fp8-kv tests from OpenVINO * Fix linting errors of test_kv_cache_scales * Disable kv-ffload=false + multi-GPU tests for FP8 See https://github.com/ggml-org/llama.cpp/actions/runs/38952722084/job/117148353699?pr=28413 * Adjust test label for test-kv-cache.cpp * Fix conflict-resolution mistakes in test-kv-cache.cpp https://github.com/ggml-org/llama.cpp/actions/runs/40554674463/job/122367426461?pr=28392 * Move KV tests to test-fp8.cpp https://github.com/ggml-org/llama.cpp/pull/27512#discussion_r4410518158 * Remove tmp file for test-kv-cache-scales.py on Windows as well * Fix fp8 kv cache test by population kv_source_impl * Exclude models that consume KV cache directly * Remove obsolete comment * Allow BF16 scales as well for KV cache * Rename *_cache_scale to *_cache_s https://github.com/ggml-org/llama.cpp/pull/28392#discussion_r4517194999 * Move `is_type_scaled()` to `llama-kv-cache.cpp` |
||
|
|
d5bbcd1f31 |
ci: update oneAPI 2026.2, oneDNN 2026.1 and Level Zero SDK 1.34.0 (#31540)
Co-authored-by: Neo Zhang <zhang.jianyu@outlook.com> |
||
|
|
359dc5858e |
server : functionality test for new spec dec (#31027)
* server : functionality test to make sure every time when we add new spec dec method the existing ones are not broken. * server: use compact models for speculative decoding CI * ci: avoid GHA cache for speculative models * ci: run speculative tests on DGX Spark * server: address speculative test review * server: use consolidated speculative models * server: refine speculative test calibration * docs: record speculative Spark validation * ci : run the server-speculative steps with bash * server: recalibrate speculative test floors on GB10 --------- Co-authored-by: Pascal <admin@serveurperso.com> |
||
|
|
fc0be86182 |
ci : Run gfx1151 ROCm job on Strix Halo Devlab runners (#31567)
* fix(rocm,ci): Update ci workflow to use devlab ci runners for gfx1151
* ci : move gpu-rocm job to Strix Halo devlab runners
Assisted-by: Claude
* ci : ROCm 10 wheels with per-GPU device packages for gpu-rocm; add gfx1201
The runners' system ROCm 7.2.1 cannot compile the current HIP code for
gfx1151 ("Invalid dpp_ctrl value" in mmf-instance-ncols_*.cu). Go back
to the TheRock ROCm 10 wheels with the device-<gfx> extra, so rocBLAS
has the target's kernels, and keep ci/run.sh's results and models under
$RUNNER_TEMP instead of the runners' RAM-backed home. Add a gfx1201
(Radeon AI PRO R9700) row with HIP graphs disabled.
Assisted-by: Claude
---------
Co-authored-by: David Hekimian <dhekimian@users.noreply.github.com>
|
||
|
|
2277e75297 |
ci: move runners to Ubuntu 26.04 (#31529)
* ci: update to ubuntu26.04 Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: bring back toolchain workaround Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: remove workaround completely Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: more switching to ubuntu 26.04 Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: pin android cmake version Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: restore ubuntu 24.04 for s390x Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix missing s390x ubuntu 24.04 release link Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: switch release-publish to ubuntu-latest Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * devops: revert --break-system-packages Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: bring back ubuntu 24.04 for arm release Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: revert ubuntu 26.04 from release.yml Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix broken links Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * ci: fix openvino job naming Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * devops: revert vulkan dockerfile to use system python3 Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> |
||
|
|
9a1d03c7ca |
tests : override hardware_concurrency in CI (#31797)
The self-hosted CPU runners are containers with 8 CPUs, but std::thread::hardware_concurrency reports the cores of the host. The table mode of test-llama-context sizes its pool from that count, so it runs 4 models x 8 threads on an 8-CPU job and hangs. Add TEST_ARG_HARDWARE_CONCURRENCY and TEST_ARG_PARALLEL_WORKERS as the defaults for the auto-selected counts of test-llama-context and of test-backend-ops, and set them to 8 and 4 for cpu-x64-high-perf. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB) |
||
|
|
3ed7284f2d |
ci : bump CUDA 13 release builds to 13.5 (#31788)
* ci : bump CUDA 13 release builds to 13.5 * docker : bump cuda13 images to 13.5.0 |
||
|
|
186eab1cb8 | ci: fix build-cmake-pkg ccache (#31705) | ||
|
|
6023692b1e |
llama: fix kv shifting, add tests, add per-model build_rope (#31683)
* llama: fix kv shifting and add tests * llama: add graph build_rope (#31693) * llama: add graph build_rope * improve * migrate new models * update review skill * fix ci * add --kv-shift to CI |
||
|
|
3de52ac388 | ci: run self hosted cpu jobs on hf or github (#31439) | ||
|
|
0943ca5472 |
ui: add Playwright harness for recording demo videos (#31318)
* ui: add Playwright harness for recording demo videos
Scripted walkthroughs under tools/ui/demo are recorded with Playwright and
exported as H.264 mp4 for pull requests (demo/pr) and docs (demo/docs).
Fixture MCP servers and seeded settings keep recordings deterministic, a
drawn cursor and captions make the flow readable, and DEMO_UI_DIR points
the recording at another checkout such as a PR worktree.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record demo scripts from PR descriptions and a configurable storage
Demo scripts no longer live in the repo. `npm run demo:pr -- <number>`
records the `ts demo` block from a PR description against the PR head,
uploads the mp4, a GIF preview and the script to DEMO_STORAGE, and links
the preview in a `## Demo` section of the PR description.
`npm run demo:docs` records the docs scripts kept in DEMO_STORAGE.
DEMO_STORAGE (hf:// buckets or file://) and the public URL are read
from tools/ui/.env.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: import generated conversations into demo recordings
Demos can now start with existing chats: `test.use({ traces })` imports
llama.app JSONL traces through Settings > Import/Export before the video
starts. Built-in traces are generated deterministically, other names are
downloaded from DEMO_TRACES_DATASET, and `demo:traces` pushes the built-in
ones there. Adds `demo.scroll` for wheel scrolling.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep demo captions clear of the chat form and isolate concurrent runs
Captions move to the top of the frame, since at the bottom they covered
the chat form a demo is often about. Each recording run gets its own
Playwright output folder, so a run starting elsewhere no longer clears
another run's video before it is saved.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record demos at 2x and 60 fps with human-like timing
Recording goes through Chromium's screencast instead of Playwright's
video, which encoded VP8 at about 1 Mbit/s at 1x. Frames are captured at
the device pixel ratio (DEMO_SCALE, default 2) and encoded into a 60 fps
H.264 mp4, starting once the page has settled.
The cursor now glides faster along a slight arc with Fitts-like
durations, settles briefly before each click, and typing and pauses vary
a little. The randomness is seeded per demo, so re-recording gives the
same motion.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: document the demo harness prerequisites
The demo README now lists the tools the harness needs and what each is
for, and walks through creating the Hugging Face bucket and the optional
traces dataset, with the matching .env settings.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: version linked demo URLs so re-recorded previews refresh
GitHub's image proxy caches a preview by URL, so a demo re-recorded at
the same PR head kept showing the old GIF. The Demo section now links
the files with a content hash in the query string.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: label PRs that feature a video demo
Adds a `video demo` label and a workflow that applies it when a PR
description has a demo written by the UI demo pipeline (the managed
demo:start/demo:end markers, or a preview linking to a demo video),
and removes it when that section is gone.
Assisted-by: Claude (Opus 5.5 Medium)
* ci: label video demos with gh instead of github-script
The label workflow now runs the gh CLI in a shell step, like
pr-draft-label.yml, so it needs no action outside the repository's
allowed list. The video link check only counts videos uploaded for the
PR itself, so a description linking other PRs' demos isn't labeled.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: zoom demos in on details with a two-finger pinch
demo.zoom pinches in on a control or a point, holds, and pinches back out
to 1x, so a detail too small to read in the frame can be shown up close.
It drives Chromium's own pinch gesture, so the page's fixed bars stay put,
and works at desktop and phone sizes alike.
The pinch is evened out where Chromium misses: one anchored near the top
does not zoom at all, so it starts lower, and one near the bottom
overshoots its ratio, so a small pinch from the middle corrects it. The
way out reads the real scale until the page is back at 1x, which the next
gesture needs.
The overlay also draws a fingertip per touch, so a pinch on a touch
device shows as two fingers.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: wait out a cold dev server before a demo starts
A fresh checkout's dev server optimizes its dependencies on the first
load and reloads the page, which can take well past an action's timeout,
so a demo failed before it started. The demo now waits up to 90 seconds
for the app; the recording only starts once it is ready.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record a video per test in a PR demo, shown in a table
A large PR can split its demo into several shorter videos: each test()
in the ts demo block is recorded as its own video (<sha>-1.mp4, ...),
and the Demo section lays their previews out in a table, each column
headed by its test's title.
GIF previews now keep their longer side at DEMO_PREVIEW_WIDTH and shrink
until they are under 5 MB, since GitHub's image proxy refuses anything
larger and shows a broken image instead.
Importing traces works on a phone viewport too, where Settings sits in
the sidebar under the page until the menu key slides it aside.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: fix demo trace imports on desktop and lay landscape demos out two to a row
Importing traces failed on a desktop whose collapsed sidebar sits under
the page: it now expands the sidebar from the page's menu key on a phone
or from the sidebar's logo on a desktop, whichever makes Settings
reachable. Afterwards it forgets the expanded sidebar, so a demo starts
with it collapsed, as on a first visit.
A PR demo with several videos shows up to four portrait (phone) videos
to a row, or two landscape ones, so each preview stays readable.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: only record other people's PR demos at a reviewed head commit
demo:pr runs the PR's code and demo script on this machine, with the
gh and hf logins, so a PR by someone else now needs --trust <sha>
naming the head commit that was read, and is refused again after new
pushes. Recordings also launch Chromium with its sandbox on, which
Playwright turns off by default.
Assisted-by: Claude (Opus 5.5 Medium)
|
||
|
|
27b8464e82 | ci : add missing ccache-clear to windows behavior check (#31413) | ||
|
|
e216f4ec87 |
ci : extract most apple jobs into daily workflow (#31408)
This commit extracts all apple jobs except for macos-latest-arm64 into a new daily workflow. The motivation for this is to reducd the load on the macos runners. |
||
|
|
68d20b884c |
Quant: OCP FP8 E4M3 support (#27512)
* WIP OCP FP8 E4M3 support * Fix python type-checker * Materialize input_scales also for FP8 when converting * Fix conversion of https://huggingface.co/RedHatAI/Qwen3.8-27B-NVFP4 1. Per row FP8 wrongly fell to NVFP4 conversion, as NVFP4 checked on dims alone. Also check on dtype for FP8 2. Need to reshape FP8 QKV projections scales in the same way that weights are reshaped * Allow remaining FP8 tensors to be dequantized during conversion * Allow tensor-strategy for FP8-only compressed-tensor ckpts * Fix and test gguf-py behavior for FP8_E4M3 * Disallow QKV-fusion for FP8 temporarily * Fix conversion of fused qkvz & FP8 for Qwen3-Next * Avoid fusing FP8 for --fuse-gate-up-exps as we may have different ws * Emit warnings when not fusing FP8 during conversion * More FP8 conversion fixes * Add supported for strided copies to/from FP8 in CPU * remove unneeded scratch buffer from FP8 cpu cpy/dup path * add explanatory comments and TODOs * Add dedicated encoder and decoder tests for FP8 * Whitelist ggml_op_get_rows in metal instead of blacklisting required to fix failing CI, and white-listing should be more future-proof than adding FP8 to the black-list https://github.com/ggml-org/llama.cpp/actions/runs/38433916498/job/115422880633?pr=27512 * Fix more conversion bugs * Defer weight materialization during conversion * Blacklist fp8 in metal backend instead of white-list * Adjust indents * Fix lint + execute tests * Add fast CPU kernel for F8_E4M3 * Conversion fixes * Fix remaining gaps * Enable tied weights with shared scales * Make mandatory-F32 rules apply to FP8 during conversion * Fix python type checks https://github.com/ggml-org/llama.cpp/actions/runs/39765670052/job/119780379595?pr=27512 * Fix kaleidai unit-test failures https://github.com/ggml-org/llama.cpp/actions/runs/39765670125/job/119780383150 * Relax numeric precision constraints for pre-check * Leave fast-FP8 CPU on RISC-V for follow-up We already exclude arm from the fast-path * Allow output scale tensors to be located in any gguf shard * Move test_conversion_fp8.py test execution to behavior-check.yaml * Apply batched suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Move mixed compressed-tensor logic into dequant_fp8 + add unit-tests for it * Install all requirements also on Windows for behavior-check Fixes https://github.com/ggml-org/llama.cpp/actions/runs/40535852590/job/122300758223?pr=27512 * Unmap lazy tensors before clean-up in Windows Should fix https://github.com/ggml-org/llama.cpp/actions/runs/40539797824/job/122314782459?pr=27512 * LF line endings https://github.com/ggml-org/llama.cpp/actions/runs/40539798168/job/122314791118?pr=27512 * Extract FP8 tests from llama-arch to its own test-fixture See https://github.com/ggml-org/llama.cpp/pull/27512#discussion_r4400628221 * Update gguf-py/gguf/scripts/gguf_convert_endian.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> |
||
|
|
ff0d396e3d | ci(s390x): switch to Ubuntu 26.04 runners (#31317) | ||
|
|
fae9ac561f | ci : use ccache depend mode for ROCm release builds (#31231) | ||
|
|
a8276b70cc | ci : force python 3.12 on ubuntu 26.04 (#31222) | ||
|
|
4244ad7acd | ci : add windows behavior check (#31191) | ||
|
|
48c36110ab |
ci : drop the macos-latest-x64 job (#31175)
|
||
|
|
344be0a600 | ci : update CPU job assignments (#31136) | ||
|
|
71255fefc0 |
ci : pass per-runner results dir to self-hosted jobs on T4 runners (#31100)
The two T4 runners in the ggml-ci group (ggml-6-x86-vulkan-t4, ggml-8-x86-nvidia-a10) share the same labels and can both pick up the same jobs, so extend the per-runner results dir from #31046 to them. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) |