* ui: use a live-preview markdown editor for the chat form input
Replace the contenteditable rich input with a CodeMirror editor that shows
the markdown source of the block being edited and renders every other block
through the MarkdownContent pipeline and stylesheet, so they match chat
messages. Plain text types exactly as before, the value stays the exact
markdown source, and mention links show as atomic badges.
The editor lives in a shared MarkdownEditor component, and the highlight.js
theme loader is shared between MarkdownContent and SyntaxHighlightedCode.
The old rich input, its tokenizer, source history and their tests go away.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: give the production build more memory, note what mdsvex is for
The single-bundle build now includes CodeMirror and runs past Node's default
heap limit on CI runners. Raise it in the build script, so local and CI builds
behave the same on every platform.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep the live preview's blocks while only the cursor moves
Moving the cursor re-walked the syntax tree and rebuilt every block's
decoration. Keep the blocks with the decorations and rebuild only when the
text changes or the cursor enters another block.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: point the chat form story at the editor and drop the engraved placeholder until the design system lands
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name the markdown editor for screen readers
CodeMirror's content is a textbox with no name of its own, so the chat form
input was announced as an unnamed text box, and could not be found by its name
as the old input could. It takes an aria-label, its placeholder by default,
which makes the chat form's "Type a message..." again.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: format the chat form input's classes
Assisted-by: Claude (Opus 5.5 Medium)
* ui: share the markdown styles and diagram drawing between messages and the editor
The editor and MarkdownContent each @imported markdown-content.css into their own
<style>, so the bundle carried two scoped copies of it. Both now render into a
MarkdownSurface, the one element that carries the stylesheet; the few rules for
MarkdownContent's own template elements are global under .markdown-content.
Drawing mermaid and svg blocks and the actions of their header buttons move to
markdown-diagrams.ts, so the editor's rendered blocks can draw diagrams the same
way messages do.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: harden the live preview editor after review
- onChange skips only the editor's own syncs from `value`, marked by an
annotation, instead of reporting only edits that carry a user event
- Enter and the mention badges ask the syntax tree whether they are in code, so
~~~ fences and indented code count, and a closing fence ends a block
- a picked mention is one undoable edit, through replaceText
- rendered blocks draw mermaid and svg diagrams with their header buttons, a $$
math block with blank lines renders as one block, and a link reference
definition stays raw instead of rendering to nothing
- rendered blocks render again when the mention path settings change
- a drag in a rendered block selects its text and copy takes it, across blocks;
a click still opens the block
- the block margins are the ones a chat message renders (each heading is a first
child there), checked by a test; the blank lines after a raw block are its gap
- the tree comes from syntaxTree, without forcing a full parse on every edit
- placeholder, label, extensions, mention widgets and render key follow prop
changes after mount, through compartments
- the editor uses the markdown language alone, which drops the html, css and
javascript parsers markdown() pulls in for html tag completion (-70 KB gzip)
Assisted-by: Claude (Opus 5.5 Medium)
* ui: pin the CodeMirror packages and set the build heap in .npmrc
The new dependencies are pinned like the rest of package.json. The build heap
moves from a node call on vite's bin path to npm's node-options, so the build
script is plain `vite build` again and the setting reaches it on every platform.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: update the live preview block by block and estimate block heights
An edit used to build every block's decorations and widgets again, which the
editor then compared one by one. The previous decorations, and the mention badges
of raw blocks, now move through the edit, and only the blocks whose range, text,
raw or rendered state, or gap changed get new ones; the texts of the blocks an
edit left alone are reused instead of sliced again. A test drives random edits,
cursor moves and focus changes and checks each result against a full build.
Rendered blocks report the height they were last measured at, or a guess from
their text, so CodeMirror's height map for blocks off screen in a long document
stays close and scrolling stays steady.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: skip the editor's redundant syncs, reconfigures and theme reloads
- syncing `value` compares it with the text the editor last reported or synced, so
a keystroke coming back through onChange no longer serializes the document again
- the prop effects reconfigure the editor only when a prop changed, not once more
right after it is created (which rebuilt the whole preview)
- loadHighlightTheme leaves the theme's style element alone while the mode stays,
so a message or an editor mounting no longer restyles the whole page
Tests check that typing in one block leaves the other rendered blocks without a
single DOM mutation, and that mounting reconfigures nothing.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: simplify the live preview to a plain rebuild in three small modules
Rebuilding the decorations of every block on an edit measures the same as
patching only the changed ones (0.09 ms against 0.11 ms per keystroke at
200 blocks, 0.9 ms against 1.1 ms at 2000), and equal widgets keep their
DOM either way, so the incremental update and its test go.
- markdown-blocks.ts collects the document's blocks, markdown-block-widget.ts
renders one, markdown-live-preview.ts decides which ones show raw text
- one bounded cache holds a block's HTML and its measured height
- the render key is a plain option: a new one reconfigures the preview,
which keeps the editor's focus
- MarkdownEditor follows its props with one helper instead of five effects
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep setext headings raw in the live preview editor
`test` over `---` is a setext heading, so the preview rendered it as an h2
and the underline vanished once the cursor left the block. The editor now
shows a setext heading's text as typed at all times, like a link
reference definition.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: add a setting to turn the live markdown preview off
"Live markdown preview in the message input" (General) is on by default.
Off, the input shows the plain markdown text, with file mention badges
still in place; switching it takes effect without a reload.
Assisted-by: Claude (Opus 5.5 Medium)
* ggml-rpc: fail gracefully on unreachable endpoints and failed handshakes
rpc_dispatcher::start() aborted the whole process (GGML_ABORT -> SIGABRT) when
the endpoint could not be parsed, the transport failed to initialise, the socket
could not connect, or the HELLO handshake failed. A typo in --rpc or a
temporarily down node killed the host application, and the reason went to a log
stream the host may never see.
Make the connect-time failure reportable instead of fatal: start() returns bool
(LOG_ERROR + false; the dispatcher thread only starts after a successful
handshake), negotiate_hello() returns false on a half-open socket,
get_dispatcher() returns nullptr and does not cache failed attempts, and every
consumer treats nullptr as "endpoint did not answer" with the conservative
behaviour of its domain (allocation -> nullptr, alloc-size -> lower bound,
device memory -> 0/0, events -> no-op, device count -> 0, and add_server()
already maps 0 devices to nullptr).
Mid-stream failures keep the existing RPC_STATUS_ASSERT by design (fail fast
against silent corruption) and are intentionally out of scope.
Field motivation: on Android (llama.rn + phone RPC cluster) an unreachable
worker crashed the app; the crash buffer showed
abort <- ggml_abort <- rpc_dispatcher::start <- ggml_backend_rpc_add_server.
* tests : add a regression test for unreachable rpc endpoints
Calls ggml_backend_rpc_add_server with a socket that nothing listens on and with a
malformed endpoint, and expects nullptr from both. Without the fix in this branch
the first call aborts the process, so the test cannot even fail cleanly.
* ggml-rpc remove low-value comments
* tests : drop the internal device-count check after the rpc registry refactor
---------
Co-authored-by: figkcoci-debug <figkcoci-debug@users.noreply.github.com>
* server: add oai-compat /v1/audio/speech endpoint
* add e2e pytest
* server: share the s16 rounding, type speech form fields
The WAV writer of the audio helper truncates while pcm_f32_to_s16
rounds, so wav and pcm differ by one LSB. Both now round.
pcm_f32_to_s16 takes a pointer and a count and converts the TTS
bytes without an intermediate copy.
/tts and /v1/audio/speech parse their body with parse_tts_body, so
multipart fields such as speed get their type before the speech
conversion checks them.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* mtmd: support gemma 4 input audio stream
* use 480ms step
* server: enable realtime embeddings only for audio generation
The realtime slot forced embeddings on, which marks every row of a
batch as an output. Only a model that generates audio reserves
n_batch outputs, so a chat model with an audio stream (Gemma 4)
aborted on its first batch of audio rows. The hidden state is only
read by the audio generation pipeline.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
A cold dev server reloads the page once it has optimized its
dependencies, and that reload can land right after the harness opened
the phone sidebar, closing it again before Settings is clicked. Repeat
the pass over the menu keys up to three times while Settings is not
reachable.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: reveal the mobile sidebar by sliding the page aside
On a phone the sidebar now stays put under the page, and the page slides
right to uncover it, as a rounded card with the card border. The menu
button rides on the page; a swipe from the left edge opens the sidebar,
and a swipe back or a tap on the page closes it.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: polish the mobile sidebar reveal and add a hold menu to rows
Holding a conversation row on a phone opens its action menu anchored to
the row, in place of the hidden three-dot trigger. The page no longer
widens the document when it slides aside, so iOS stops panning it on the
first open.
The page and sidebar backgrounds start in the page color and blend into
the sidebar tint as the sidebar opens, following a drag frame by frame,
and a theme-color tag follows them for the browser bars. The sidebar's
contents fade in on a log curve and scale up slightly, and sit at the
same inset as the menu toggle, which now uses a two-line icon in a
filled circle.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: pin the context gauge on phones and lock the slid-aside page
On a phone the context gauge leaves the chat form for the top right of
the chat screen, in a round button across from the menu toggle. Both
take the scroll-down button's frosted look.
While the sidebar is out, and until the page is fully back in place,
the page neither scrolls nor takes taps; a tap or drag on it still
closes the sidebar.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: open the mobile sidebar with a swipe from anywhere
A rightward swipe from anywhere on the page now opens the phone sidebar
too, beside the edge swipe that competes with the browser's own back and
forward gesture. It starts after a short sideways move, skips text
fields and content that scrolls sideways, and keeps the page from
scrolling under it. Letting go opens or closes the sidebar past 30% of
the way or on a flick.
The top buttons and the chat form now hold their place on screen while
the page is slid aside, instead of jumping to the far ends of a
scrolled conversation.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: hide the model org name on mobile unless a setting keeps it
A phone's model selector is narrow, so it now drops the organization
from model names by default. A new Display setting, "Show organization
name on mobile", brings it back; it applies only while the organization
name is shown at all.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: close the mobile sidebar before following a tap in it
On a phone, picking a conversation, New chat, Search, MCP or Settings
in the open sidebar now slides it shut first and acts 180ms later, so
the page or dialog appears once the sidebar is out of the way. With the
sidebar already closed, or on a desktop, the action runs at once.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: make the app feel native on phones and run it under the status bar
Applies the mobile-native checklist: no tap highlight or landscape text
inflation, instant taps on buttons and links, no label selection or
callout on a held control, hover styles only where a pointer can hover,
and 16px textareas on phones so focusing them does not zoom the page.
The page now runs under the notch (viewport-fit=cover), so the installed
iOS app tints its status bar from the page and the sidebar. The menu
toggle, the context gauge, the sidebar and the top of the chat and
models pages pad themselves clear of it, and the locked, slid page
reaches down by the status bar's height, which iOS leaves out of it.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: add a viewport height readout for debugging on phones
With VITE_DEBUG set, a dev build shows the window, viewport unit and
safe-area heights, and the elements that reach past the window's bottom
and make the page scroll. It is loaded on demand, so production builds
leave it out.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: fill the screen from launch in the installed iOS app
iOS 26 gives an installed app with viewport-fit=cover and a translucent
status bar a viewport short of that bar (WebKit bug 301108), clipping
the bottom of the page and lifting the chat form, but only while the
page is shorter than the screen. In the installed app the page now
reaches the screen plus the status bar, capped at the screen, so it
gets the full screen from the start without ever scrolling.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: raise the phone's corner controls in the installed app
The installed app's status bar already sets the menu toggle, the context
gauge and the open sidebar's logo apart from the screen's edge, so they
sit just below it rather than a full 1rem lower. They share one value,
which stays at 1rem below the inset in a browser tab.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: add safe-area utilities for what is pinned to a screen edge
The page runs under the status bar and over the home indicator, which an
installed iPhone app always shows. New pt-safe, pb-safe, top-safe and
bottom-safe utilities, each with a spacing step (pt-safe-12 is 3rem plus
the top inset), keep pinned content clear of them instead of hand-written
calc()s. A --safe-area-bottom joins --safe-area-top, and the chat screen and
Models page use pt-safe-12.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: open search in a full-height drawer on phones
A phone used to leave for the /search page. It now opens the same search
as the desktop dialog, in a bottom drawer that fills the screen below the
status bar. Keyboard shortcuts, the Enter hint and the keyboard highlight
stay on desktop, and so does the models manager, as in the phone sidebar.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: show every dialog full screen on phones, clear of the status bar
A phone now shows every dialog full screen, whatever size it asks for on a
desktop, instead of each full-screen dialog repeating its own classes. Its
top and bottom borders take the status bar's and the home indicator's
height, transparent, so an installed app's dialogs keep their content clear
of both with no change to their padding. The attachment preview, which has
its own full-screen content, pads itself and lowers its close button the
same way.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: fade the page out under the status bar of the installed app on phones
Assisted-by: Claude (Opus 5.5 Medium)
* ui: show search in the full-screen dialog on phones instead of a drawer
Assisted-by: Claude (Opus 5.5 Medium)
* ui: start a phone page's title a row below the corner controls
Assisted-by: Claude (Opus 5.5 Medium)
* ui: pin the slid page's bottom bars against the screen in the installed iOS app
Assisted-by: Claude (Opus 5.5 Medium)
* ui: cast a soft shadow from the slid page onto the phone sidebar
Assisted-by: Claude (Opus 5.5 Medium)
* ui: hide the scroll-down key at the real bottom on iOS and show it only well above
Assisted-by: Claude (Opus 5.5 Medium)
* ui: fade the status-bar fade out as the page slides aside for the sidebar
Assisted-by: Claude (Opus 5.5 Medium)
* ui: draw the slid page's shadow above its rounded frame, so it follows the corners
Assisted-by: Claude (Opus 5.5 Medium)
* ui: show a phone chat's title and working directory at the top, and pick it in a drawer
Assisted-by: Claude (Opus 5.5 Medium)
* ui: give iOS Safari a solid top and fade the bottom of the screen on phones
Assisted-by: Claude (Opus 5.5 Medium)
* ui: let iOS Safari find the top strip, which it skips when it takes no taps
Assisted-by: Claude (Opus 5.5 Medium)
* ui: lengthen the top fade and lift the chat form a little in phone browsers
Assisted-by: Claude (Opus 5.5 Medium)
* ui: give iOS Safari's bottom toolbar the page's colour, not the chat form's
Assisted-by: Claude (Opus 5.5 Medium)
* ui: draw the phone's context gauge key like the menu key across from it
Assisted-by: Claude (Opus 5.5 Medium)
* ui: confirm opening the phone sidebar, its hold menu and its rows with a haptic tap
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep the screen-edge fades on the page as it slides aside for the sidebar
Fading them out with the slide made the page flicker; the rounded frame over the
slid page already clips them at its corners.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep the working directory chip optional for hosts with their own trigger
The phone layout opens the picker from the chat's header, so ChatForm hides the
chip there. Master's version of the chip has no such switch; bring it back.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: style layer 5's additions with master's tokens until the design system lands
Assisted-by: Claude (Opus 5.5 Medium)
* ui: drop the design system's stylesheets and fonts from app.css until it lands
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep master's font stacks until the design system brings its fonts
Assisted-by: Claude (Opus 5.5 Medium)
* ui: restore master's colour palette in app.css until the design system lands
Resolving the safe-area commit took the design system's app.css, which drops
the palette for its token files and aliases the shadcn names to semantic
tokens. With the tokens gone those aliases point at themselves, so every
colour went invalid: transparent surfaces and currentColor borders.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: center alert dialogs on phones instead of sliding them up as drawers
An alert is a short question, not a drawer: on a phone it is now a centered
card, 90% of the screen wide and as tall as its text, scrolling only past the
screen's height. It zooms in as on desktop, and its footer drops the margin
the sheet needed.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: space the working directory drawer for a phone
The drawer reused the popover's compact picker: Browse sat against the search
field, the divider ran past the rows and each row had its own inset. In the
drawer the picker now takes the header's inset, gives Browse room, lines its
rows up with the search results and sizes its text for a phone.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: tap the haptic engine when a message sends, and once per hold
A phone now confirms a sent message by feel, at the moment the send goes
through (not on every press of the button), and before any await, so it still
runs inside the tap or keypress that iOS needs to play its tick. The hold that
opens a row's menu now taps the engine from the long-press hook itself, once:
the hook vibrated and the row tapped as well, which buzzed twice on Android.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name the gauge dial sizes, the chat form's bottom offset and the sidebar fade
Review follow-ups: the context gauge dial reads its grid, radius, stroke and
track from one table per size instead of ternaries; the chat form's distance
from the screen's bottom per platform and the sidebar content's fade duration
are named constants. The viewport debug readout moves out of src, next to the
tests, as it is a debug tool the app only loads in a dev build.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: say why the microphone can't record instead of failing silently
A llama-server opened over plain HTTP on the local network or over Tailscale
is not a secure context, where browsers hide the microphone, so a tap on the
mic did nothing but log to the console. The button now shows a crossed-out mic
in the warning tone with the reason in its tooltip, and a tap says it in a
toast, as a phone shows no tooltip. A start that fails names the browser's
reason (access denied, no microphone, in use by another app) the same way,
and the next tap tries again. The recorder passes the browser's error on
instead of replacing it with a generic one.
Assisted-by: Claude (Opus 5.5 Medium)
* model : allow loading external PLE files
Assisted-by: Codex
* convert : add separate PLE exports
Assisted-by: Codex
* tests : fix PLE export test lint
Assisted-by: Codex
* model : make --ple an alias of --shard, allow multiple shard locations
Reuse the multi-shard loading path for external PLE files instead of a
separate option.
* --ple is an alias of --shard: both take a comma separated list and may
be given more than once, and the shards may live anywhere
* metadata comes from the main file; numbered shards are still
auto-discovered and strictly validated
* explicit files outside the discovered set are appended as extra shards,
exempt from split.no and split.count checks
* -isnm/--ignore-shard-num-mismatch loads only the given files, in order,
with no auto-discovery and no split metadata validation
* ple_path, path_ple and llama_model_loader::add_ple_file are removed;
common_model_load() uses llama_model_load_from_splits for two or more
paths
* common/fit.{h,cpp} take the shard path vector, so --fit and the router
memory measurement load every shard
Assisted-by: GPT-6-Astra
* rm -isnm
* make sure to dedup paths
* rm test_convert_ple
* address comments
* allow ple shard to override tensor
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* server : functionality test to make sure every time when we add new spec dec method the existing ones are not broken.
* server: use compact models for speculative decoding CI
* ci: avoid GHA cache for speculative models
* ci: run speculative tests on DGX Spark
* server: address speculative test review
* server: use consolidated speculative models
* server: refine speculative test calibration
* docs: record speculative Spark validation
* ci : run the server-speculative steps with bash
* server: recalibrate speculative test floors on GB10
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* server : accept top-level schema in response_format with type json_schema
The OpenAI-compatible chat completions endpoint only applied the schema
when it was nested inside response_format.json_schema.schema. When the
schema was provided directly as response_format.schema (as documented in
the server README, and used by some clients), it was silently ignored
and generation ran unconstrained.
Also mention json_schema in the error message for invalid response_format
types, document the OpenAI wrapper form in the README, and add a test
case for the top-level schema form.
Fixes#10732
* server : let the json_schema wrapper take precedence by presence
Pick the schema source based on whether response_format has a
json_schema key, instead of falling back when the parsed wrapper schema
is empty. An empty schema {} is valid (any JSON value), so the old check
let a top-level schema override an explicit {} in the wrapper. Use
json_value() for the top-level schema too, like the rest of the
function, and add tests for the precedence.
---------
Co-authored-by: linsen <251731047+linsen458-spec@users.noreply.github.com>
* ui : fold the sidebar's conversation sections
Pinned and Recent conversations fold under their heading. The whole heading
row is the toggle; its chevron follows the title, shows on hover and stays
while the section is folded. Folded sections are remembered across reloads.
Assisted-by: Claude (Opus 5.5 Medium)
* ui : show a sidebar section's first six chats
Pinned and Recent list their first six chats and end with a Show more
row that opens the whole section; Show less, sticking to the bottom of
the sidebar while the section scrolls under it, closes it back. The
expanded sections last for the session.
Assisted-by: Claude (Opus 5.5 Medium)
* ui : rearrange the sidebar's sections and chats by dragging
Pinned and Recent can be dragged by their title. The real section follows
the pointer, tinted like the sidebar's resize handle and edged with a
faint border, the other sliding aside as it passes; let go, it glides from
there into its new slot. The order is remembered across reloads, and a
section never moved keeps its default place. All sections now scroll
together.
A chat dragged onto Pinned is pinned, onto Recent unpinned. A lifted copy
of the row follows the pointer while the row slides out of its section.
The section under the pointer takes the same tint and border; a section
ignores its own chats. Let go, the row slides in where its section sorts
it, the rows below making room, and the copy glides into it; anywhere
else, it slides back into its own section.
While dragging, the rows and headings passed over take no hover. Dragging
needs a mouse or a pen, so a finger still scrolls the sidebar; the sidebar
scrolls near its edges, and Escape cancels.
Assisted-by: Claude (Opus 5.5 Medium)
* ui : set the sidebar's sections apart from their rows
A section heading now reads above its chats: larger, semibold and in the
full foreground colour. Chat titles drop to a regular weight in a slightly
softened colour, full again when the chat is open or hovered, and "Show
more" drops to a regular weight in the muted colour. Chats inside a
section sit closer together, sections further apart, and the sidebar's
actions get more room above and below.
Assisted-by: Claude (Opus 5.5 Medium)
* mtmd, server : add temperature to /tts, default Pocket TTS to 0.3
Assisted-by: Claude Opus 5.5
* feat: add temp parameter in llama-tts
* temp: fix default hardcoded values
* fix: remove too narrow test
* fix: addressing PR comments
* llama, mtmd: follow the reference precision of ggml_prec_set_acc calls
Requests that the reference implementation does not keep in F32 move
to GGML_PREC_BF16: the GLM4 family down and output projections, the
MiMoVL vision matmuls and the Qwen3-TTS speaker encoder conv1d. The
DeepSeek V4/V4.1 routers and the MiniMax-M3 indexer stay in F32 with
a link to the reference line that computes them in F32.
* ggml: handle GGML_PREC_BF16 the same way as GGML_PREC_F32 in backends
Backends that reject GGML_PREC_F32 on the src1 of MUL_MAT_ID now also
reject GGML_PREC_BF16 (CUDA, Hexagon, OpenCL, WebGPU, Vulkan), and
backends that accept GGML_PREC_F32 for flash attention now also accept
GGML_PREC_BF16 with the same F32 accumulation (CPU, SpaceMIT, ET).
Vulkan keeps the F32 transfer between devices for both. The CUDA
cuBLAS path already handles GGML_PREC_BF16 and is unchanged.
* tests: cover GGML_PREC_BF16 in the flash attention tests
The flash attention cases run with GGML_PREC_BF16 next to F32 and
DEFAULT, and var_to_str prints bf16 so these cases keep their own name.
* llama: request GGML_PREC_BF16 for the KQ multiplication
* cont : add TODO in clip.cpp
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* mtmd: support MageVL-Cowork model
* mtmd: keep magevl rope_freqs at model level and check its shape
The rope factors are shared by every layer, so they live in clip_model
instead of a pointer copied into each clip_layer. The loader now checks
that v.rope_freqs is F32 with d_head elements and that d_head splits
into the 4:6:6 pairs the graph expects.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* moe-cache : compute part of the misses on the CPU
--moe-cache-cpu-share F computes a share F of the MoE cache misses on the CPU
from the host experts instead of uploading them. The graph builds the expert
FFN twice: on the device from the cache slots, where the misses left to the CPU
read zero slots, and on the CPU from the host experts, where the experts
computed on the device have id -1. The two results are added before the down
bias.
- ggml-cpu: MUL_MAT_ID skips negative ids and writes zero rows
- sched: the copy callback is called for all the inputs of a split, and before
the host sync when there are no events
- the slot lookup callback reads the inputs of the CPU part together with the
selected experts, so the CPU part does not wait for the device, and its
result is copied back asynchronously
- each token uses distinct zero slots, mul_mat_id does not support an expert
selected twice by a token
- LLAMA_MOE_CACHE_TRACE writes the selected experts of each lookup, for a
simulator of the cache
* scripts : add moe-cache-tune.py to pick the MoE cache arguments
The script wraps a llama-server command: it starts the server at two cache
sizes and three CPU shares, fits the upload and CPU speeds of the machine, and
replays the routing of the runs through a copy of the cache logic to predict
other sizes and shares. It prints the recommended --moe-cache-mib and
--moe-cache-cpu-share.
docs/moe-cache.md describes the MoE cache arguments and the script.
* add am17an as codeowner for MoE cache
* moe-cache : upload all the misses until the cache is full
With a high --moe-cache-cpu-share, few misses are uploaded, so an empty cache
fills slowly and more misses than needed go to the CPU. The CPU share now
starts once all the slots of a cache group hold an expert. moe-cache-tune.py
replays the same rule.
* add backend sync for async transfer queue
* moe-cache : let the scheduler copy the results of the CPU part
* Fix MSVC compilation
* remove sync change
* llama-graph : extract the CPU share condition into can_cpu_share
* llama-graph : make llm_graph_params::moe_cache const
The graph construction no longer updates the MoE cache directly: the CPU
parts of the layers are accumulated in the graph result and applied to
the cache in llama_context::process_ubatch, after the graph inputs are
set and before the graph is computed.
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (Mac Studio, M5 Ultra, 256GB)
* scripts : replay the ClassLRU policy in moe-cache-tune.py
* moe-cache : pick CPU misses by CRF instead of random (#31853)
* moe-cache : remove the backend logic of the CPU part from the graph
The CPU part reads the host experts, so the scheduler places it on the
CPU without set_tensor_backend. The cache gives no CPU map for batches
that a device would offload. Graphs with a CPU part look up their own
slot map, so the graph does not register the part with the cache.
ggml-backend : the inputs of the next split that are already computed
are copied to host memory before the current split runs, so a CPU split
does not wait for the device split before it. This replaces the stash
of the cache, and the copy callback is again called for host weights
only.
* llama-graph : return the MoE cache slots and CPU ids in a struct
The scheduler prefetch no longer checks that the next split runs on
another backend: the inputs of a split are never on its own backend,
so the input loop already skips that case.
---------
Co-authored-by: Gaurav Garg <gaugarg@nvidia.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Alexey Kopytko <alexey@kopytko.com>
* server: visit parent slots before child slots
A parent slot wakes its waiting child slots once it finishes the shared
prompt prefix, while slots are visited in id order. When a child task
lands on a lower slot id than its parent, the child is visited before
the parent wakes it and waits an extra batch to process its suffix.
Non-child slots are now visited before active child slots, so every
child woken by its parent joins the same batch.
* server: check systemone test events through test tags
The systemone tests matched free-form debug messages, including the
formatted f_keep value of the prompt cache. The server now emits
__TEST_TAG_WAIT_PARENT_SLOT__ when a child slot waits for its parent,
__TEST_TAG_PROMPT_CACHE_RESTORE__ on every prompt cache restore and
__TEST_TAG_PROMPT_CACHE_FULL_RESTORE__ when the whole cached prompt is
a prefix of the new one, compared on token counts.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* checkpoints: reduce overhead
Assisted-by: Codex
* rebase
Adapt the slot save/restore checkpoint appendix from #26004 to the
shared_ptr checkpoint list and the common_state_data buffers.
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: cwriter <cwriter@localhost>
Prevent malformed realtime session updates from throwing an uncaught
exception and terminating the server
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* server : realtime: send an error event instead of crashing on exceptions
* server : route websocket handlers through ex_wrapper, remove the realtime test
* server : realtime: keep the previous tools when the chat template rejects new ones
* server : realtime: reject tool schemas the grammar cannot convert, drop ws_ex_wrapper
* server : realtime: clarify the tool schema check comment
* ui : share directory resolution and the cwd row
Move the lookup of a browser-picked folder to ToolsService.resolveDirectoryPath, let the working directory picker run without its chip, and split the 'Set working directory to' row into its own component, so other views can pick and show a folder the same way.
Assisted by: Claude (Opus 5.5 Medium)
* ui : open an item's actions on right-click
Sidebar conversations and models manager rows open the same actions as
their "more actions" menu on right-click. Both menus render one shared
list of actions, so they cannot drift apart. Phones keep the browser's
long-press; the dropdown's drawer already holds the actions there.
An action can render a flyout submenu in place of its row, through the
kit of the menu it is in, and close the menu after a pick.
Adds the shadcn-svelte context-menu component, mirroring the
dropdown-menu one.
Assisted by: Claude (Opus 5.5 Medium)
* ui : close conversation tabs from a right-click menu
A tab's right-click menu closes it, the other tabs or the tabs to its
right, and stops a response the tab is generating. Closing tabs that
include the active one keeps the tab the menu was opened on.
Assisted by: Claude (Opus 5.5 Medium)
* ui : open a message's actions on right-click
User and assistant messages open the same actions as their action icons
on right-click. Selected text, links, media, code and fields keep the
browser's own menu, so copying them works as before. The fork dialog
opens from the actions context, like the delete one, so both the icon
and the menu can open it.
Assisted by: Claude (Opus 5.5 Medium)
* ui : scope right-click menu shortcuts and selection to their item
Show the conversation shortcuts only on the active row, as they act on
the active conversation, and keep the browser menu only for a selection
inside the message. Name the close-tab shortcut, and drop the unused
chip toggle, menu submenus and shared cwd row.
Assisted-by: Claude (Opus 5.5 Medium)
* fix(ui): show extracted text in PDF attachment preview
Assisted-by: Claude (Opus 5.5 Medium)
* feat(ui): show page thumbnails for PDF attachments sent as images
Assisted-by: Claude (Opus 5.5 Medium)
* ui: stop retrying PDF preview conversion after it fails
A failed PDF-to-image conversion or text extraction reset its loading
flag, which re-ran the effect and retried forever. Remember the failed
item for each view and only retry when the item changes, so the error
state stays on screen.
Add a PdfConversionError story that feeds a broken PDF.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep the text of PDFs sent as images and tidy the PDF preview
Store the extracted text of a PDF sent as images in its content, so the
Text view reads it instead of parsing the PDF again; only messages stored
with the old placeholder extract it. The model still receives the images
only. Guard retries on the error state, as the item remounts per id, skip
an empty base64 payload, and share the status markup between views.
Assisted-by: Claude (Opus 5.5 Medium)
Playwright's bundled Chromium lags behind Chrome security releases
(1.56.1 ships 141, and even 1.64.1 ships 156.0.8078.4, before the
156.0.8078.73 fixes), and Playwright launches it with --no-sandbox by
default. Outside CI, use the installed, auto-updated Chrome with the
sandbox enabled. CI keeps the bundled Chromium, so the self-hosted
runner's preinstalled 1.56.1 browsers still work.
Assisted-by: Claude (Opus 5.5 Medium)
* mtmd : add Nemotron 3.5 Transcribe support
Assisted-by: Claude
* convert : fix vocab of draft models for Nemotron-H targets
Assisted-by: Claude
* convert : reject non-BF16 Transcribe checkpoints
Assisted-by: Claude
* mtmd : make speaker FIFO predictions local
Assisted-by: Claude
* mtmd : normalize the mel in encode_windows
Assisted-by: Claude
* convert : map the generated speaker kernel
Assisted-by: Claude
* mtmd : validate Nemotron Transcribe mmproj shapes
Assisted-by: Claude
* ui: add Playwright harness for recording demo videos
Scripted walkthroughs under tools/ui/demo are recorded with Playwright and
exported as H.264 mp4 for pull requests (demo/pr) and docs (demo/docs).
Fixture MCP servers and seeded settings keep recordings deterministic, a
drawn cursor and captions make the flow readable, and DEMO_UI_DIR points
the recording at another checkout such as a PR worktree.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record demo scripts from PR descriptions and a configurable storage
Demo scripts no longer live in the repo. `npm run demo:pr -- <number>`
records the `ts demo` block from a PR description against the PR head,
uploads the mp4, a GIF preview and the script to DEMO_STORAGE, and links
the preview in a `## Demo` section of the PR description.
`npm run demo:docs` records the docs scripts kept in DEMO_STORAGE.
DEMO_STORAGE (hf:// buckets or file://) and the public URL are read
from tools/ui/.env.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: import generated conversations into demo recordings
Demos can now start with existing chats: `test.use({ traces })` imports
llama.app JSONL traces through Settings > Import/Export before the video
starts. Built-in traces are generated deterministically, other names are
downloaded from DEMO_TRACES_DATASET, and `demo:traces` pushes the built-in
ones there. Adds `demo.scroll` for wheel scrolling.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep demo captions clear of the chat form and isolate concurrent runs
Captions move to the top of the frame, since at the bottom they covered
the chat form a demo is often about. Each recording run gets its own
Playwright output folder, so a run starting elsewhere no longer clears
another run's video before it is saved.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record demos at 2x and 60 fps with human-like timing
Recording goes through Chromium's screencast instead of Playwright's
video, which encoded VP8 at about 1 Mbit/s at 1x. Frames are captured at
the device pixel ratio (DEMO_SCALE, default 2) and encoded into a 60 fps
H.264 mp4, starting once the page has settled.
The cursor now glides faster along a slight arc with Fitts-like
durations, settles briefly before each click, and typing and pauses vary
a little. The randomness is seeded per demo, so re-recording gives the
same motion.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: document the demo harness prerequisites
The demo README now lists the tools the harness needs and what each is
for, and walks through creating the Hugging Face bucket and the optional
traces dataset, with the matching .env settings.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: version linked demo URLs so re-recorded previews refresh
GitHub's image proxy caches a preview by URL, so a demo re-recorded at
the same PR head kept showing the old GIF. The Demo section now links
the files with a content hash in the query string.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: label PRs that feature a video demo
Adds a `video demo` label and a workflow that applies it when a PR
description has a demo written by the UI demo pipeline (the managed
demo:start/demo:end markers, or a preview linking to a demo video),
and removes it when that section is gone.
Assisted-by: Claude (Opus 5.5 Medium)
* ci: label video demos with gh instead of github-script
The label workflow now runs the gh CLI in a shell step, like
pr-draft-label.yml, so it needs no action outside the repository's
allowed list. The video link check only counts videos uploaded for the
PR itself, so a description linking other PRs' demos isn't labeled.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: zoom demos in on details with a two-finger pinch
demo.zoom pinches in on a control or a point, holds, and pinches back out
to 1x, so a detail too small to read in the frame can be shown up close.
It drives Chromium's own pinch gesture, so the page's fixed bars stay put,
and works at desktop and phone sizes alike.
The pinch is evened out where Chromium misses: one anchored near the top
does not zoom at all, so it starts lower, and one near the bottom
overshoots its ratio, so a small pinch from the middle corrects it. The
way out reads the real scale until the page is back at 1x, which the next
gesture needs.
The overlay also draws a fingertip per touch, so a pinch on a touch
device shows as two fingers.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: wait out a cold dev server before a demo starts
A fresh checkout's dev server optimizes its dependencies on the first
load and reloads the page, which can take well past an action's timeout,
so a demo failed before it started. The demo now waits up to 90 seconds
for the app; the recording only starts once it is ready.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: record a video per test in a PR demo, shown in a table
A large PR can split its demo into several shorter videos: each test()
in the ts demo block is recorded as its own video (<sha>-1.mp4, ...),
and the Demo section lays their previews out in a table, each column
headed by its test's title.
GIF previews now keep their longer side at DEMO_PREVIEW_WIDTH and shrink
until they are under 5 MB, since GitHub's image proxy refuses anything
larger and shows a broken image instead.
Importing traces works on a phone viewport too, where Settings sits in
the sidebar under the page until the menu key slides it aside.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: fix demo trace imports on desktop and lay landscape demos out two to a row
Importing traces failed on a desktop whose collapsed sidebar sits under
the page: it now expands the sidebar from the page's menu key on a phone
or from the sidebar's logo on a desktop, whichever makes Settings
reachable. Afterwards it forgets the expanded sidebar, so a demo starts
with it collapsed, as on a first visit.
A PR demo with several videos shows up to four portrait (phone) videos
to a row, or two landscape ones, so each preview stays readable.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: only record other people's PR demos at a reviewed head commit
demo:pr runs the PR's code and demo script on this machine, with the
gh and hf logins, so a PR by someone else now needs --trust <sha>
naming the head commit that was read, and is refused again after new
pushes. Recordings also launch Chromium with its sandbox on, which
Playwright turns off by default.
Assisted-by: Claude (Opus 5.5 Medium)
* feat: support PocketTTS voices with no voice cloning
The kyutai/pocket-tts-without-voice-cloning weights ship predefined voices as the
backbone KV state after the voice prompt. The converter turns each state back into
the prompt embeddings and stores them in the mmproj (voice.<name>), the zeroed voice
encoder is left out, and the "voice" field of /tts and --tts-voice select one of them.
A speaker reference is rejected when the model has no voice encoder.
Assisted-by: Claude Fable 5.1
* convert : pocket-tts voices never fail the conversion, log the drift of the state
The voice states shipped by Kyutai are not always the ones the pack's
weights give for the prompt: italian, german and spanish match within the
bf16 rounding, english_2026-04 is 2% off and french 20% (its layer 0 state
comes from other weights), while the reference loads them as is and they
still sound like the same voice. The converter now re-prompts the backbone
with the recovered embeddings, compares the whole state and logs the
distance per voice, warning above 5%. A state that does not belong to the
weights at all is skipped, so the packs of kyutai/pocket-tts convert too.
Assisted-by: Claude Fable 5.1
* mtmd : list the predefined voices, expose them in GET /props
mtmd_gen_audio_info gains n_voices and mtmd_gen_audio_voice_name() the
names, with the voices worded as a feature of any audio generation model.
The server lists them under "voices" in /props when the model ships some.
Assisted-by: Claude Fable 5.1
* server : an unknown voice is a bad request
A model with predefined voices answers 400 naming the voice instead of 500
"failed to process TTS prompt". A model without them ignores "voice" with a
warning, as the other audio generation models do.
Assisted-by: Claude Fable 5.1
* refactor api
* fix conversion
* small renaming
* fix lint
* tts : a model with predefined voices speaks with the first one by default
Without a voice prompt such a model only produces near silence. When neither
a voice nor a speaker reference is given, llama-tts and the server now take
the first predefined voice and say so in the log, as the LFM3-Audio template
does with its default voice.
Assisted-by: Claude Fable 5.1
* mtmd : one comment per voice accessor, converter states the rule only
The clip.h comment described clip_gen_get_voice but sat above all three
functions; the converter comment quoted the drift measured per pack, which
goes stale with the next upload of the weights.
Assisted-by: Claude Fable 5.1
* clean up redundant code and comments
* mtmd: match the LFM3-Audio voice name case-insensitively
The tts chat template lowers the voice name, so Male speaks with
the male voice; the voice lookup lowers it the same way and names the
bitmap after the canonical voice.
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pascal <admin@serveurperso.com>
The router only removes models it took from the Hugging Face cache, but
the models manager offered "Delete from disk" for every model, so a model
from --models-preset or --models-dir failed with a "not removable" toast.
Read the source and can_remove fields of /models and show the delete
action disabled for those models, with a tooltip (a hint line in the
mobile drawer) saying how to remove them instead.
Assisted-by: Claude (Opus 5.5 Medium)
Use the popover background, softer border, rounded-xl corners and
shadow-md like dropdowns, hover cards and the settings dialog, and
match the dialog title line height.
Assisted-by: Claude (Opus 5.5 Medium)
A message queued while a conversation was busy was re-sent through
sendMessage, which targets the active conversation, so switching chats
before the turn ended delivered it into the wrong one. Re-send only when
the owning conversation is active; otherwise keep it queued there and
flush it when that conversation is opened again.
Assisted-by: Claude (Opus 5.5 Medium)