Commit Graph
1575 Commits
Author SHA1 Message Date
Xuan-Son Nguyen 9d925e7c02 mtmd: fix video pixel aspect ratio and video rotation (#32267) 2026-11-14 14:57:21 +01:00
Jhen-Jie HongandXuan Son Nguyen e3483e40b0 mtmd : allow custom video decoder (#32188)
* mtmd : allow custom video decoder

* also adapt to wllama requirements

* clean up

* add test

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-14 14:12:07 +01:00
Hera 0cce7580b8 mtmd : avoid padding-only Whisper audio chunks (#30403)
Assisted-by: Claude Opus 5
2026-11-14 13:33:59 +01:00
Ruixiang Wang 32bdc3a89e spec + model: refactor DFlash based speculative decoding methods (#32016)
* dflash : move the DSpark markov head and backbone graphs to dspark.cpp

Move-only: build_dspark_markov_head, graph_dsv4, graph_dsv41 and
graph_mistral_large4 move verbatim. build_inp_target_features and
build_dspark_markov_head become static members of llama_model_dflash so
both files can use them.

* dflash : detect the head and the backbone once, split the loaders per file

load_arch_hparams sets llama_model_dflash::variant (DFlash, DFlash2,
DSpark, LiliCorr) and ::backbone (base, DSV4, DSV4.1, Mistral Large 4).
The head, backbone and base layer loaders become member helpers in
dflash.cpp, dflash2.cpp, lilicorr.cpp and dspark.cpp. build_arch_graph
dispatches on the backbone.

The DFlash2 and LiliCorr loaders create the DFlash2 conv tensors through
a helper in dflash2.cpp, right after their head tensors instead of in
the base layer loop; the other tensors keep their creation order. The
DFlash2 loader drops its copy of the conv group size check, which master
never reached.

* dflash : share the injection input, token embeddings and lm head lookups

build_inject_inp (target features + fc + encoder norm), get_tok_embd and
get_lm_head replace the copies in the base, DSV4, DSV4.1 and Mistral
Large 4 graphs. build_inp_target_features is file-static again.

* dflash : split the DFlash decoder into a graph per head

graph_base creates the shared inputs and provides build_inject,
build_inp_tokens, build_layers and build_logits. graph_dflash
(dflash.cpp), graph_dflash2 (dflash2.cpp), graph_dspark (dspark.cpp) and
graph_lilicorr (lilicorr.cpp) assemble them and add their head; the
selector, scorer and markov head builders move to the file of their
graph, and the DFlash2 local conv run by build_layers to dflash2.cpp.
build_arch_graph dispatches on the backbone, then on the head.

The op order and the tensor names are unchanged.

* speculative : split the DFlash driver into a class per head

common_speculative_impl_draft_dflash stays the DFlash implementation and
becomes the base of draft_dspark, draft_dflash2 and draft_lilicorr. Its
protected constructor only reads the draft metadata; init() then sets up
the contexts with the options of the draft (draft length, backend
sampling, device features, masked h_nextn). Each derived class overrides
draft() with the path of its head from the former draft(). DFlash2 also
overrides begin() to reseed its generator per request, and LiliCorr
overrides process() to store the context features of each injected
chunk, which its noise block carries as its state.

The factory keeps the current routing: draft-dspark picks DSpark,
draft-dflash picks DFlash2 or LiliCorr from the draft metadata and
DFlash otherwise. draft-dspark with a DFlash2 or LiliCorr draft now runs
the DSpark reader instead of a mix of both. sample_from_anchor is logged
by DSpark only.

* speculative : add the draft-dflash2 and draft-lilicorr types

draft-dflash2 and draft-lilicorr name the DFlash2 and LiliCorr drafts,
which draft-dflash keeps running too: it picks them from the draft
metadata. draft-dflash2 and draft-lilicorr must match the draft model,
draft-dspark rejects a DFlash2 or LiliCorr draft, a model that is not a
DFlash draft is rejected, and a run uses at most one DFlash family type;
these errors leave speculative decoding uninitialized.

The type detected from the draft GGUF is now the concrete one, and
draft-dflash2 and draft-lilicorr download the dflash sidecar.

* docs : describe the DFlash2 and LiliCorr drafts

Add the DFlash2 and LiliCorr sections and the new types, and explain
that draft-dflash also runs DFlash2 and LiliCorr drafts.

* speculative : drop the unused DFlash is_mrope

Nothing reads it since the migration to common_batch (#29385), which
picks the number of M-RoPE position rows from the rope type of the draft.

* doc: update spec dec

* address review comments
2026-11-14 09:49:27 +01:00
AMIR 5193ef1d4f ui: configure allow-remote=root in .npmrc for npm 12+ (#32158) (#32196) 2026-11-13 20:54:49 +01:00
Pascal 4f9dfa0e36 requirements, Fix CI: support Python 3.15 (#32199)
* requirements : support Python 3.15

torch 2.11.0, numpy 2.4, matplotlib 3.10, pandas 2.3, pillow 11.3 and
cffi 1.16 have no Python 3.15 wheels, so the CI runner on 3.15 cannot
install the requirements.

Python 3.15 gets the first releases that ship 3.15 wheels through
version markers, and every version below 3.15 stays as is.
pyproject.toml now allows 3.15 and picks torch 2.15.0 there.

* requirements : apply review suggestions

numpy 2.5.4 from Python 3.14, matplotlib 3.11.2 and pandas 3.0.6 from
3.11, cffi 2.1.1 and pillow 12.3.0 for every version. cffi 1.16.0 has
no 3.14 wheel and fails to build there, so this also fixes the
tokenizer test requirements on 3.14.
2026-11-13 17:50:03 +01:00
Georgi Gerganov 28d37fe2e2 examples : clean-up old/obsolete stuff (#32184)
* examples : clean-up old/obsolete stuff

* examples : update references to removed files

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

[no ci]
2026-11-13 10:33:17 +02:00
Pascal 0a66ac7189 server, fix CI: read a small temp file in the read_file tool test [no release] (#32168)
* server: read a small temp file in the read_file tool test

test_tools_builtin_read_file read its own file, which grows with every
added test. On Windows the CRLF checkout puts it at 16664 bytes, over
the 16384 byte read limit, and the tool refuses it.

The test now writes a small file of its own and checks the exact
content read back.

* server: use a temp cwd in the x-tool-cwd tool test

test_tools_builtin_cwd_header read test_tools_builtin.py through the
x-tool-cwd header, which hits the same 16384 byte read limit on the
Windows CRLF checkout, and wrote its shell marker into the source tree.

The test now uses a temp directory as cwd, reads a small file from it
and checks the shell marker lands there.
2026-11-12 23:47:14 +01:00
Pascal 892b40b910 server: allow private media URLs in the vision tests [no ci] (#32167)
The vision tests download their images from huggingface.co, which
resolves to a private address on the runners inside the HF network.
The media URL check refuses it with "host is not a public address"
and the server-cuda job fails.

The vision fixture enables media_allow_private_urls. The private
address check keeps its own coverage in
test_vision_image_url_private_address.
2026-11-12 23:15:28 +01:00
Pascal e970101470 server: skip LAN address tests when the host cannot reach it [no ci] (#32166)
The same-origin tests over the LAN address block forever on the Metal
runners: the request to the host's own LAN address never returns and
has no timeout, so the job runs until the 6h limit.

get_lan_ip() now probes /health on that address with a short timeout
and returns None when the host cannot reach it, which skips the test.
The test requests themselves use the default request timeout so a
hang fails the test instead of stalling the job.
2026-11-12 23:13:37 +01:00
Georgi Gerganov 8ca5a8008b llama : prune (#30565)
* llama : remove deprecated `llama_set_warmup`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : remove remaining deprecated API

Drop the APIs marked DEPRECATED in llama.h, the deprecated
llama_context_params::defrag_thold field and the --defrag-thold option.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* fit-params : remove stale `llama-ext.h` include

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : document `llama_model_dflash_selector_top_k`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : promote the NextN / speculative API from `llama-ext.h`

Move the NextN, layer input, embd row and state row APIs to `llama.h`
and rename `llama_set_embd_state_src` to `llama_set_ctx_state_src`, with
the internal `llama_context::set_state_src` following the same name.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : polish the NextN API promotion

Move the NextN draft model queries next to the Adapters section, rename
the `src` parameters of `llama_set_ctx_state_src` and
`llama_context::set_ctx_state_src` to `ctx_src`, and drop the now-redundant
`llama-ext.h` includes from `common/speculative.cpp` and `tools/imatrix`.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-12 20:59:09 +02:00
Hamish M. Blair d8c8cc4d9b server : cancel MCP tool calls on client disconnect or timeout (#32025)
* server : cancel MCP tool calls on client disconnect or timeout

* server : shorten MCP cancellation tests and fix a type-check error
2026-11-12 17:19:26 +01:00
Xuan-Son Nguyen 8954f19483 server: fix SSRF for CORS proxy and media URLs (#32115) 2026-11-12 12:33:48 +01:00
Xuan-Son Nguyen a1a68eb797 server: explicit opt-in tools via non-loopback addr, check cors host port (#32113)
* server: explicit opt-in tools via non-loopback addr, check host port

* fix ci
2026-11-12 12:16:50 +01:00
Xuan-Son Nguyen e182fdad4a server: disallow direct access child instance from browser (#32120) 2026-11-12 12:16:29 +01:00
PascalandVenkata Vineel 09452cdedd server: keep the server awake while a request uses its state (#32133)
With --sleep-idle-seconds, a handler passes wait_until_no_sleep()
while the server is awake, then keeps using the server state, for
example to parse the body and tokenize. If the idle timer expires in
that window, the main loop enters sleep and the callbacks destroy the
model: the handler reads freed vocab memory, and a task posted after
that stays in the queue with nobody awake to process it.

Count the requests that passed wait_until_no_sleep() and are not done
yet, and do not enter sleep while the count is positive. The response
generator releases its count in its destructor, so every return path
of a handler gives it back.

Co-authored-by: Venkata Vineel <vyalamar@users.noreply.github.com>
2026-11-12 12:16:16 +01:00
Pascal cfb9d4b754 mtmd: require an F32 position embedding for HunyuanVL (#32131)
HunyuanVL resamples its position embedding on the host into a float
buffer sized for F32, then reads ggml_nbytes() of the tensor into it.
The graph never uses the tensor, so any type loads, and an F64 tensor
overflows the buffer at the first image request.

Require the position embedding to be present and F32 when loading the
HunyuanVL tensors.
2026-11-12 12:15:28 +01:00
Pascal 72f4603c7c mtmd: read the MiniCPM-V projection width from ne[1] (#32132)
The MiniCPM-V resampler ends with build_mm(mm_model_proj), whose output
width is ne[1], while n_mmproj_embd() returns ne[0]. Both are equal in
real models, the projection is square. A non square projection passes
the text model n_embd check and only aborts at the encode, on the
output buffer size check.

Return ne[1], so the width matches the graph output.
2026-11-12 12:14:50 +01:00
Pascal 8c388364e1 mtmd: require the MLP projector and patch embedding tensors (#32129)
* mtmd: require the MLP projector and patch embedding tensors

The loader marks the patch embedding and the LLaVA projector tensors
as optional, while the graphs and n_mmproj_embd() dereference them
unconditionally. A GGUF without them crashes on a null tensor during
the warmup.

Load the patch embedding as required for the vision encoders that
start with it, which is all of them except GEMMA3NV, DEEPSEEKOCR and
DEEPSEEKOCR2. Load mm.0 and mm.2 as required for MLP, and mm.0, mm.1,
mm.3 and mm.4 for the Yi-type MLP_NORM variant, selected by mm.3.

The graph no longer tests mm.2 and the yasa2 assert on the patch
embedding is covered by the loader.

* mtmd: keep the Yi-type llava TODO

The Yi-type variant is still detected by the presence of mm.3, the
TODO on that detection stays.
2026-11-12 12:14:36 +01:00
Pascal 08f49598c1 mtmd: require a 2x2 spatial merge for the Qwen-VL projectors (#32130)
The Qwen2-VL, Qwen2.5-VL, Qwen3-VL and Ling3-VL graphs build their
input with a temporal merge that assumes a 2x2 patch grid, and the
Qwen2-VL merger reshapes by 4. Any other spatial_merge_size from the
GGUF reaches a GGML_ASSERT at the first image request and aborts the
server.

Reject a spatial_merge_size other than 2 when loading the hparams.
2026-11-12 12:14:10 +01:00
Xuan-Son NguyenandAleksander Grygier b19f1d9e63 ui: add search_chats tool (#32051)
* ui: add search_chats tool

* add from_timestamp / to_timestamp

* ui: search_chats review fixes

- keep search_chats disabled on import, copy tool policy on fork
- stream messages without attachment payloads, one transaction for time ranges
- keep the output within budget, cut whole messages from the older end
- reject impossible date-times, quote titles and attachment names
- typed readOnly tables, move localIsoString to formatters as formatLocalIso

Assisted-by: Claude (Opus 5.5 Medium)

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
2026-11-12 09:44:35 +01:00
Aleksander Grygier d90cfd4a9b ui: install llama-brand from a GitHub tarball instead of git (#32112)
npm 11.20 refuses git dependencies by default (EALLOWGIT), so npm ci
failed on fresh installs. Pin llama-brand to the same commit through
an HTTPS tarball, which also adds an integrity hash to the lockfile.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-12 09:36:28 +01:00
Xuan-Son Nguyen 60d46a5abe server: when --tools-runtime is specified, ignore x-tool-runtime (#32119) 2026-11-12 09:34:16 +01:00
Aleksander Grygier dd6b945be2 server: use the moved test model in the router CORS test (#32117)
test_router_restricted_cors_same_origin_reaches_child still named
ggml-org/tinygemma3-GGUF, which the router no longer finds since the test
models moved to ggml-org-dev, so the Server test job fails on master.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-12 02:01:19 +01:00
Xuan-Son Nguyen 7bcbc04fbf ui: move js sandbox setting out of "developer" (#32054)
* ui: move js sandbox setting out of "developer"

* make js sandbox enabled by default
2026-11-11 23:42:34 +01:00
Aleksander Grygier 1d81992784 ui: keep downloads in Discover during search and fix their progress with a draft (#31467)
* ui : keep downloads in Discover sidebar during search, sort base-org repos first

- Show the sticky Downloading section regardless of an active search query
- Order search results so repos published by their base model's org lead

Assisted-by: Claude (Opus 5.5 Medium)

* server : throttle download progress per file

Files of a download run in parallel threads, but progress updates were
throttled with one shared timer. A small draft sidecar reports far more
often than the main weights, so it took nearly every slot and the main
file's updates, including the first one carrying its total size, were
dropped until the draft finished. The UI then showed the draft's progress
first and restarted from 0% for the model.

Throttle each file on its own and always pass a file's first update.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: list a downloading repo once and match base orgs case-insensitively

A search hit that is downloading already shows under Downloading, so the
results leave it out. Base model orgs compare without case, as the hub
treats them.

Assisted-by: Claude (Opus 5.5 Medium)

* server : tell clients a download stopped on request

When a client paused or cancelled a download, every other client following
/models/sse got a bare download_failed and could not tell it from an error,
so the web UI showed "Download failed" for a pause made in the Llama app.
A stop asked for through unload or delete now fails with {"stopped": true}.

Deleting a paused download, which has no instance left, now sends
model_remove as deleting a running one does, so other clients drop its row.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : follow downloads other clients start, pause or cancel

A download another client started (e.g. the Llama app) already showed up
from the feed; it now also picks up the sidecars the list reports for it.
A stop another client asked for reads as a pause instead of a failure, and
a removal clears the download row even when the list does not carry it yet.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-11 23:26:39 +01:00
Aleksander Grygier bd717e08f3 ui: take the logo and favicons from the llama-brand package (#32039)
* ui : take the logo and favicons from llama-brand

The web UI's logo mark and favicons now come from llama-brand, added as a
dev dependency from GitHub and pinned to a commit, so the UI draws the
same icon as the README cover instead of its own copy of an older one.
Logo.svelte inlines the brand's icon and fills it with currentColor, and
the PWA asset generator starts from the brand's web favicons, which carry
their own margin and optical centring, so the UI's colouring and padding
script, its test and its logo.svg go.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : follow llama-brand's master instead of a pinned commit

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-11 23:25:35 +01:00
Xuan-Son NguyenandPascal 2cbe0ec429 server: fix --cors-origins handling + protect against DNS rebinding (#32098)
* server: fix --cors-origins localhost handling

* server: validate Host on loopback connections against DNS rebinding

A page whose own name resolves to 127.0.0.1 reaches the server with
that name in Host and a matching Origin, so the same-origin shortcut
let it through. On a loopback connection, Host must now name loopback
or the host of an allowed --cors-origins entry. The LAN tests reach
the server through a real interface instead of a forged Host.

* fix python type check

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-11 21:17:46 +01:00
Xuan-Son Nguyen 664f031f49 ci: move test models to ggml-org-dev (#32096)
* ci: move test models to ggml-org-dev

* fix test spec
2026-11-11 21:17:16 +01:00
JamieBordoni 71a6a114bc quantize: fix imatrix GGUF filepath metadata leak (#32000) 2026-11-11 19:13:28 +08:00
Pascal adde94c11b ui: open the virtual keyboard in the chat editor on Android (#32060)
CodeMirror takes text input through EditContext on Chrome for Android
only, and there the chat editor never brings up the virtual keyboard.
The editor now always takes its input through contenteditable, as on
desktop and iOS.

A press in the input box outside the text also focuses the editor, in
the collapsed form too, so the whole box is a tap target again as the
textarea was.
2026-11-11 10:42:02 +01:00
Aleksander Grygier b5cb8f6f47 ui: keep a hidden sidebar from intercepting clicks again (#32035)
The hidden desktop strip relied on inherited `visibility: hidden` to keep
its content from catching clicks. Since the sidebar's sections fold
(#31936), an open section's CollapsibleRegion sets `visibility: visible`
again, and its rows' context-menu triggers carry an inline
`pointer-events: auto`, so they caught clicks meant for the page.

Make the hidden strip's content inert, which no descendant can undo, and
add a client test that hit-tests the hidden sidebar's area.

Fixes #31971

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-10 22:44:23 +01:00
Aleksander Grygier 2b66bfbf71 ui: show audio in Discover for models whose mmproj has an audio encoder (#31837)
* ui: show audio in Discover for models whose mmproj has an audio encoder

Discover took audio only from the pipeline tag, so omni models tagged
any-to-any, such as Gemma 4 and LFM 3 Omni, showed vision but never audio,
though their mmproj carries an audio encoder.

Read the encoders from the mmproj's GGUF metadata instead: a ranged read of
its first 64 KB, once per repo, for clip.has_vision_encoder and
clip.has_audio_encoder. The Discover list and details now show audio when
either the pipeline tag or the mmproj says so.

Assisted by: Claude (Opus 5.5 Medium)

* ui: take Discover modalities from the mmproj's encoders in one place

Review fixes for the mmproj header read:

- drop vision for audio-only mmprojs: the converter writes
  clip.has_vision_encoder only when true, so a header read in full without
  it has no vision encoder; a header cut short leaves vision as it was
- accept only a 206 to the ranged read, so a server that ignores Range
  never streams the whole mmproj
- encode the mmproj path in the resolve URL
- derive the list and details modalities with one HuggingFaceService
  helper, so both views agree
- point the key comment at tools/mtmd/mtmd-impl.h, where the keys live now

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep a gated repo's mmproj from being read again on every mount

A gated, private or missing mmproj answers 401, 403 or 404 to every read,
so caching nothing made each mount of its Discover row request it again and
log an error. Cache it as unknown instead: no audio, and vision from the
mmproj being there, as before the header was read.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-10 22:34:46 +01:00
Xuan-Son Nguyen 7c2653acc7 mtmd: fix string format, rename to mtmd_format (#32040) 2026-11-10 20:23:25 +01:00
Aleksander Grygier 7592b61d3b ui: move Models, MCP and Settings to a sidebar footer (#32022)
* ui : move Models, MCP and Settings to a sidebar footer

The sidebar's actions split in two: New chat, Search and Projects stay at
its top, and Models, MCP and Settings move to a footer at its bottom, under
a thin rule, in the expanded sidebar and its collapsed strip alike. On a
phone the footer keeps clear of the home indicator, and Models stays
desktop-only as before. The search dialog still lists every action.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : group the sidebar footer in an inset surface

The design system gains inset-group-surface, for rows grouped inside a glass
panel: white on the light glass and a faint tint of the text colour on the
dark one, with the sidebar's own ambient edge, no inner highlight, a faint
wide shade under it and the card radius. It is matte-glass pointed at its
own component tokens, like the sidebar and chat form surfaces, and reads as
part of the panel: no tight contact shadow or solid border like a button,
no inset shade like a field.

The sidebar footer uses it in place of the rule above Models, MCP and
Settings. DESIGN.md and the Surfaces page document it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : give the sidebar's logo button the add button's key look

Whenever the sidebar shows, open or peeking, its logo button takes the same
key as the chat form's add button (matte-key, white in light mode, a light
tint in dark), in place of its own muted fill. While the sidebar is hidden
the logo has no fill and no shadow, and hover and focus never change the
fill. It keeps its round shape and its own size.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : put search in a round key at the top of the phone sidebar

On a phone the sidebar's Search row moves to a round key at the right end
of its top row, across from the logo, with the same key look as the logo
and the chat form's add button. It opens search as the row did. Desktop
keeps the Search row.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : size the phone sidebar for a thumb and list its footer with the actions

On a phone the sidebar's actions and rows are 48px tall at 17px, flush one under
the other with 16px corners, its section headings 15px and grey, and its groups
28px apart. MCP and Settings join the top actions there: the footer is a desktop
pattern.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : mark the selected sidebar row with a tinted inset surface

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the phone sidebar's round keys white on the grey page

Assisted-by: Claude (Opus 5.5 Medium)

* ui : open Settings and MCP as pages on a phone, dialogs elsewhere

On a phone the sidebar opens them at #/settings and #/mcp-servers, with the menu key and a
title like any phone page. Pinned under the status bar, Settings' section tabs fill the
screen above them in the page's colour.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : open a phone sidebar's pages behind it, settled before they slide back

A tap in the phone sidebar now opens its page while the current one is still aside and slides
back once it has rendered, instead of swapping pages mid-slide. The slid page's fixed bars are
pinned again as the page under them changes, the lock ends with the slide itself, and a
phone's new chat shows its form and greeting at once, without the move and entry they have
on a desktop.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: style the sidebar footer group and the selected row with master's classes until the design system lands

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add GlassKey with master's styles for the phone sidebar's round keys

Assisted-by: Claude (Opus 5.5 Medium)

* ui: center the top edge of a new chat's form on the screen

The form rested 42px below the middle up to 2xl and 6px above it past
that; one drop, from the resting card's height, now puts its top edge on
the middle at every width from md up.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: round the sidebar's action rows concentric with the footer group

The sidebar stacks four surfaces: the page, the sidebar, the footer group
inset in it and a row in the group. app.css now names the group's radius
and padding, and derives the row radius from them (16px less 4px), so the
footer's rows run parallel to its corners; the actions at the top of the
sidebar share that radius. The footer moves to its own component, the
README documents the pattern, and a client test holds the rule on the
rendered footer.

A selected action row was bg-accent, the same grey as a hovered row in
light and darker than the footer's fill in dark, so it read as hovered or
not at all; it is now raised and edged, while a hover only tints.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: import demo chats through the Settings page on a phone

A phone opens Settings as a page now, not a dialog, so the demo harness
waited for a dialog that never came and every phone demo with traces
failed; it now imports through whichever of the two opens.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: count the sidebar's rows from their items in the surface test

The test expected three footer rows and two top actions, which the
stacked PRs change as they add Projects, Library and Servers; it now
expects as many rows as the sidebar has items.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a line on Meta+Enter off macOS too

The chat form takes Meta+Enter as Ctrl+Enter and leaves the newline to
the editor, but the editor bound only Mod-Enter, which is Ctrl off
macOS, so on Linux Meta+Enter neither sent nor added a line. This failed
the client test on CI's Linux runners, on master too.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-10 18:58:57 +01:00
Oliver Simons 202c1a229f llama : Add FP8 KV-cache support (#28392)
* Add FP8 KV-cache support

* Add tests for scaled, FP8 KV-cache to test-backend-ops

* Fix get_*_scale mapping and dequnatize FP8 also before rope shift

* Normalizes cache-scale names independently of projection layout.

While fused KQV is currently impractival for non-shared scales, this is
an improvement for future use regardless

* Enable Gemma 4 Assistant with FP8 by storing and fetching scale ptrs

* Update comment wording

* Move E2E FP8 KV cache tests to test-llama-archs

They belong there as they are not just a test for ggml's backend ops

* Increase support surface, remove seed

* make scaled kv generic

* Fix CI failings for test-kv-cache

https://github.com/ggml-org/llama.cpp/actions/runs/38718763447/job/116367842578?pr=28392

* CI: execute kv-cache-scale tests

* exclude fp8-kv tests from OpenVINO

* Fix linting errors of test_kv_cache_scales

* Disable kv-ffload=false + multi-GPU tests for FP8

See https://github.com/ggml-org/llama.cpp/actions/runs/38952722084/job/117148353699?pr=28413

* Adjust test label for test-kv-cache.cpp

* Fix conflict-resolution mistakes in test-kv-cache.cpp

https://github.com/ggml-org/llama.cpp/actions/runs/40554674463/job/122367426461?pr=28392

* Move KV tests to test-fp8.cpp

https://github.com/ggml-org/llama.cpp/pull/27512#discussion_r4410518158

* Remove tmp file for test-kv-cache-scales.py on Windows as well

* Fix fp8 kv cache test by population kv_source_impl

* Exclude models that consume KV cache directly

* Remove obsolete comment

* Allow BF16 scales as well for KV cache

* Rename *_cache_scale to *_cache_s

https://github.com/ggml-org/llama.cpp/pull/28392#discussion_r4517194999

* Move `is_type_scaled()` to `llama-kv-cache.cpp`
2026-11-10 18:08:32 +02:00
Georgi Gerganov 494d0f47bd ggml : add MUL_MAT src1 precision tests and gating (#31896) 2026-11-10 17:03:27 +02:00
Aleksander GrygierandPascal 281e7dad7c ui: gather chats into named groups in the sidebar (#32012)
* ui: let a menu action open a submenu

Group and project pickers open as flyouts from a chat's menus. Master's menu
list has no submenus, so bring them back next to its tooltips, with a close
callback for picks made inside them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : store groups and let conversations join them

Groups live in their own IndexedDB table (schema version 2) with a name.
groupsStore creates, renames and deletes them; deleting a group keeps its
chats, out of any group. A conversation records the group it is sorted
into, and a group picked before the first message is kept pending and
applied when the conversation is created.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : show each group as a sidebar section

Every group is a sidebar section next to Pinned and Recent, folding,
showing its first six chats and dragged into place like them. A chat
dropped on a group joins it and leaves Pinned; dropped on Recent, it
leaves its group. Recent is called Ungrouped once a group exists, and its
menus create a group in a small naming dialog; so does a right-click
anywhere in the list that no row or heading takes, down to the empty space
under the last section. A group's heading starts a new chat in it, and its
menus rename or delete it; deleting keeps its chats, back in Ungrouped.

Assisted-by: Claude (Opus 5.5 Medium)

* ui : move chats into groups from their menus

A chat's menus, in the sidebar and on right-click, move it into a group
or take it out of its group. Desktop menus show a searchable group flyout
whose New group entry names a group in the group dialog and moves the chat
into it; a phone gets a picker dialog instead, which creates a group from
what is typed.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: open the group picker from a phone's hold menu

The hold menu is anchored to the row, so it showed the desktop flyout,
which ran off the left edge of the screen and whose New group closed
without naming one. On a phone, Move to group now opens the picker
dialog, as intended.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: open a new chat in a tab of its own

With conversation tabs on, New chat from the sidebar, a group's heading
or the shortcut replaced the open chat's tab. It now adds a tab, as the
tab bar's "+" does, and the open chat keeps its own.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: list a phone's groups under the picker's search

On a phone the picker fills the screen, and its grid spread the rows
over the height, so the list sat in the middle. It is now a column: the
list starts under the search and scrolls in the rest of the screen.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: fix gaps in chat groups found in review

- A phone shows a group heading's New chat and menu, and the Ungrouped
  heading's menu, which were reachable only by hover or right-click.
- A pinned chat moved into a group from its menu leaves Pinned, as a
  drag onto the group already did, so the move shows.
- Only the sidebar's New chat, a group's New chat and the new-chat
  shortcut open a new tab; stepping past the last chat, search and the
  models drawer show the new-chat screen in the current tab again.
- An export carries each chat's group, and an import sorts the chat
  into the group of that id or name, or creates it; a group the file
  does not carry is dropped.
- The group picked with a group's New chat survives leaving the
  new-chat tab and coming back.
- A phone's menus skip any submenu in the shared menu list, not only
  the group one.
- Deleting a group forgets its section's place and fold.
- Groups load newest first by creation, so a rename no longer moves one
  after a reload.
- groupsStore looks groups up in a map and answers which live group a
  chat is in, for the three places that checked it by hand.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep a forked chat in its source chat's group

A fork copied the working directory but not the group, so it landed in
Ungrouped.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: move a chat with its whole fork family between groups

Moving a chat into a group, or out of one, takes its parents and its
children along, so a fork tree never splits across sidebar sections. A
pinned member that joins a group leaves Pinned with the rest. Group
sections render the tree with its depth, as Ungrouped does.

* ui: show the group name prompt as a small centered card on phones

The new and rename group prompt opened full screen on a phone, like a
large dialog. It is now built on the alert dialog, as the chat rename
prompt is, so a phone shows it as a card in the middle of the screen.
Its Cancel is a plain button, so Enter in the name field saves.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: space a phone's group sections like Ungrouped

On a phone a group or Pinned section put its first chat right under the
heading, where Ungrouped leaves a row gap. They now leave the same gap,
and the sections sit a little further apart than the rows inside them.

Assisted-by: Claude (Opus 5.5 Medium)

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-10 15:44:46 +01:00
Hamish M. Blair 90e3841d9f ui: show settings save button even while focused (#32010) 2026-11-10 14:14:11 +01:00
Aleksander Grygier 54cde7106a ui: expand the chat form into a large editor for longer messages (#31973)
* ui: expand the chat form into a large editor for longer messages

The chat form expands into a large editor once a message has markdown
beyond plain paragraphs or reaches four lines, and a Maximize button in its
top-right corner toggles it by hand. The expanded form starts around half the
screen height and grows with the text up to the space below the tabs bar,
animating its height by mirroring the editor's content height.

On a new chat the expanded form keeps the collapsed form's top edge and the
greeting only fades out, waiting for a collapse to settle before it fades
back in. In a conversation the messages follow the form as it grows, so the
latest one stays in view above it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: make the expanded chat form a full-screen sheet on phones

In the expanded editor Enter adds a line and Cmd/Ctrl+Enter sends.

On a phone the expanded form is a full-screen sheet with no corners or
border. The page around it and Safari's bars take its colour, and the
sidebar's logo button and the working-directory chip step aside while it is
open. It moves by relative offsets and a clip on the main thread, so its
edges, text, actions and chip stay in step in every engine, and its content
keeps the collapsed card's places and widths, so nothing reflows when it
settles. Expanding does not focus the editor on a phone.

On a new chat on desktop the form is lifted by its sticky bottom offset
instead of a translate, and the expanded size is built on the collapsed
card's real height, so its top edge holds still through any sequence of
quick toggles.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: hide the phone menu toggle and the sheet's focus edge while expanded

On a phone the menu toggle rides on the page, above the chat form: it now
fades out while the expanded form covers the screen, and comes back when
it collapses. The expanded sheet also drops the glass rim and inner ring,
so focusing the editor no longer outlines the whole screen.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the expanded phone sheet clear of the status bar

The expanded chat form fills the screen on a phone, under the status bar
an installed app draws over the page. Its text now starts below the bar
(pt-safe) and its collapse button sits below it too (top-safe-2.5).

Assisted-by: Claude (Opus 5.5 Medium)

* ui: hide the phone's context gauge while the chat form is expanded

The gauge pinned to the phone's top right sat over the expanded sheet's
collapse key. Like the menu toggle across from it, it now fades out while
the expanded form covers the screen and comes back when it collapses.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: lift the expanded phone sheet's actions above the iOS keyboard

Assisted-by: Claude (Opus 5.5 Medium)

* ui: move the chat form with a short, quick drift, without sliding after it lands

Its position transition came back on before the browser had laid out the new position,
so it slid the form there after the fade; the drift is now 10px each way and quicker.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: style the expanded chat form with master's surfaces until the design system lands

Assisted-by: Claude (Opus 5.5 Medium)

* ui: tighten the expanding chat form after review

- Expand on markdown only in finished lines: a lone `-`, `#` or `1.` on the
  line being typed no longer opens the large editor.
- Show "Cmd/Ctrl+Enter to send" by the send button while expanded.
- Move the expansion state, the phone sheet and the held collapsed height
  into useChatFormExpansion, and the sheet's geometry into pure functions.
- Drop the working-directory chip's phone animation, which never ran since
  the phone chip moved to the + menu.
- Clearing or sending mid-move cancels the sheet's animation.
- A click only continues at the end of the text when it lands below it.
- Reuse ActionIcon for the toggle and INPUT_CLASSES for the card, derive the
  centered form's offset from its drop, name the shared easing and fade.
- The greeting owns its fade behind the expanded form.
- MarkdownEditor's view is reactive, so following its content starts once it
  exists.

Assisted-by: Claude (Fable 5.2)

* ui: keep the expanded chat form to the message being written

- The form expands on its own only for what is written or pasted, and
  another conversation starts collapsed: a restored draft no longer opens
  the large editor, or the phone sheet, on arriving at a chat.
- In the expanded editor the pickers open by the line being written, above
  it or, short of room, below, instead of above the form's top edge.
- The sidebar's swipe is off while the phone sheet covers the screen.
- The send hint shows only beside the send button, where the row has room.
- The inner area keeps master's focus shadow, except on the phone sheet.
- The card and the form's position honour reduced motion.
- The phone greeting's place accounts for the wrapper it is counted from.
- The sheet's animation tolerates the form going away mid-move.

Assisted-by: Claude (Fable 5.2)

* ui: turn the typed pickers off on phones, and scroll long messages

- On a phone, typing `/` or `@` opens no picker for now: each mobile
  browser moves the page its own way as the keyboard opens, so a list by
  the form can not stay clear of the text being written. A note marks it
  for later.
- The editor scrolls inside the form again: MarkdownEditor's own box sat
  between the input and the editor without taking its height, so a message
  longer than the form ran under the actions instead of scrolling.

Assisted-by: Claude (Fable 5.2)

* ui: settle the expanding chat form's review findings

- A phone expands the form only by hand: the sheet switching in mid-typing
  moves the text under the keyboard.
- The typed pickers are off wherever a finger is the main pointer, not
  below a viewport width, so a narrow desktop window keeps them and an
  iPad loses them. The prompt picker opened from the + menu floats by the
  sheet's actions on a phone.
- A message put back after a failed send expands the form again.
- Only the left button moves the caret from a click below the text.
- The sheet's fill is one token, used by the sheet, the page and Safari's bars.
- The collapsed-height hold is its own module and no longer finds the
  editor's elements by class name.
- Shorter comments.

Assisted-by: Claude (Sonnet 5.5)

* ui: keep a one-line message on the chat form's actions row

On a desktop the main chat form is a single pill while its message is one
line: the add button, the text, then the model selector and send. A second
line, or a line that wraps beside the actions, brings back the two rows,
and they stay until the text is shorter than where it wrapped, so the
wider row can not flip it straight back. The editor stays mounted: the
actions join the text's row by CSS alone.

The single row's corners are `--chat-form-single-row-radius` in app.css, a
pill by default; the "Scrapbook / ChatForm single row" story shows the real
form with other radii.

Assisted-by: Claude (Opus 5.5)

* ui: give the one-line chat form 1.25rem corners

Assisted-by: Claude (Opus 5.5)

* ui: pick the working directory from the + menu on desktop too

The chip under the chat form is gone: the + menu offers the working
directory everywhere but a phone's chat, whose header shows it. The menu
item names the directory once one is set, and the picker gains a row to
clear it, which the chip's X used to do.

Assisted-by: Claude (Opus 5.5)

* ui: put the single-row chat form behind a Display setting

"Single-row chat form for short messages", on by default.

Assisted-by: Claude (Opus 5.5)

* ui: send the chat form on Enter, and add a line with Shift or Ctrl/Cmd

Enter sends wherever the caret is, the expanded editor and code blocks
included; Shift+Enter and Ctrl/Cmd+Enter add a line, continuing lists and
quotes. With "Send message on Enter" off the two swap as before. The
expanded editor's send hint and the code-block rule are gone with it.

Assisted-by: Claude (Opus 5.5)

* ui: start the chat form on the row layout it will settle on

Assisted-by: Claude (Opus 5.5)

* ui: show a set working directory under the chat form again

On desktop the chip under the form is back once a directory is set, and
the + menu offers the directory only until then.

Assisted-by: Claude (Opus 5.5)

* ui: add a line on Enter with an on-screen keyboard

A phone's Return has no Shift, so on a touch screen Enter adds a line and
the send button sends, as "Send message on Enter" turned off does; on a
desktop Enter still sends.

Assisted-by: Claude (Opus 5.5)
2026-11-10 13:33:50 +01:00
Xuan-Son Nguyen a591cbe17c mtmd: share compute buffer among subcontext (#31945)
* shared compute buffer

* add load-time alloc for audio gen
2026-11-10 11:45:09 +01:00
Xuan-Son Nguyen 203b5f404c server: fix output lang for realtime api (#31982) 2026-11-10 11:35:30 +01:00
Xuan-Son Nguyen 351025ec53 mtmd: add kimi K3.1 vision (#31980)
* mtmd: support kimi k3.1 vision

* fix preproc (both k2.5 and k3.1) and projector mlp

* small renaming
2026-11-10 11:28:47 +01:00
Aleksander Grygier ac007054e7 ui: use a live-preview markdown editor for the chat form input (#31955)
* ui: use a live-preview markdown editor for the chat form input

Replace the contenteditable rich input with a CodeMirror editor that shows
the markdown source of the block being edited and renders every other block
through the MarkdownContent pipeline and stylesheet, so they match chat
messages. Plain text types exactly as before, the value stays the exact
markdown source, and mention links show as atomic badges.

The editor lives in a shared MarkdownEditor component, and the highlight.js
theme loader is shared between MarkdownContent and SyntaxHighlightedCode.
The old rich input, its tokenizer, source history and their tests go away.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: give the production build more memory, note what mdsvex is for

The single-bundle build now includes CodeMirror and runs past Node's default
heap limit on CI runners. Raise it in the build script, so local and CI builds
behave the same on every platform.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep the live preview's blocks while only the cursor moves

Moving the cursor re-walked the syntax tree and rebuilt every block's
decoration. Keep the blocks with the decorations and rebuild only when the
text changes or the cursor enters another block.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: point the chat form story at the editor and drop the engraved placeholder until the design system lands

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name the markdown editor for screen readers

CodeMirror's content is a textbox with no name of its own, so the chat form
input was announced as an unnamed text box, and could not be found by its name
as the old input could. It takes an aria-label, its placeholder by default,
which makes the chat form's "Type a message..." again.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: format the chat form input's classes

Assisted-by: Claude (Opus 5.5 Medium)

* ui: share the markdown styles and diagram drawing between messages and the editor

The editor and MarkdownContent each @imported markdown-content.css into their own
<style>, so the bundle carried two scoped copies of it. Both now render into a
MarkdownSurface, the one element that carries the stylesheet; the few rules for
MarkdownContent's own template elements are global under .markdown-content.

Drawing mermaid and svg blocks and the actions of their header buttons move to
markdown-diagrams.ts, so the editor's rendered blocks can draw diagrams the same
way messages do.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: harden the live preview editor after review

- onChange skips only the editor's own syncs from `value`, marked by an
  annotation, instead of reporting only edits that carry a user event
- Enter and the mention badges ask the syntax tree whether they are in code, so
  ~~~ fences and indented code count, and a closing fence ends a block
- a picked mention is one undoable edit, through replaceText
- rendered blocks draw mermaid and svg diagrams with their header buttons, a $$
  math block with blank lines renders as one block, and a link reference
  definition stays raw instead of rendering to nothing
- rendered blocks render again when the mention path settings change
- a drag in a rendered block selects its text and copy takes it, across blocks;
  a click still opens the block
- the block margins are the ones a chat message renders (each heading is a first
  child there), checked by a test; the blank lines after a raw block are its gap
- the tree comes from syntaxTree, without forcing a full parse on every edit
- placeholder, label, extensions, mention widgets and render key follow prop
  changes after mount, through compartments
- the editor uses the markdown language alone, which drops the html, css and
  javascript parsers markdown() pulls in for html tag completion (-70 KB gzip)

Assisted-by: Claude (Opus 5.5 Medium)

* ui: pin the CodeMirror packages and set the build heap in .npmrc

The new dependencies are pinned like the rest of package.json. The build heap
moves from a node call on vite's bin path to npm's node-options, so the build
script is plain `vite build` again and the setting reaches it on every platform.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: update the live preview block by block and estimate block heights

An edit used to build every block's decorations and widgets again, which the
editor then compared one by one. The previous decorations, and the mention badges
of raw blocks, now move through the edit, and only the blocks whose range, text,
raw or rendered state, or gap changed get new ones; the texts of the blocks an
edit left alone are reused instead of sliced again. A test drives random edits,
cursor moves and focus changes and checks each result against a full build.

Rendered blocks report the height they were last measured at, or a guess from
their text, so CodeMirror's height map for blocks off screen in a long document
stays close and scrolling stays steady.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: skip the editor's redundant syncs, reconfigures and theme reloads

- syncing `value` compares it with the text the editor last reported or synced, so
  a keystroke coming back through onChange no longer serializes the document again
- the prop effects reconfigure the editor only when a prop changed, not once more
  right after it is created (which rebuilt the whole preview)
- loadHighlightTheme leaves the theme's style element alone while the mode stays,
  so a message or an editor mounting no longer restyles the whole page

Tests check that typing in one block leaves the other rendered blocks without a
single DOM mutation, and that mounting reconfigures nothing.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: simplify the live preview to a plain rebuild in three small modules

Rebuilding the decorations of every block on an edit measures the same as
patching only the changed ones (0.09 ms against 0.11 ms per keystroke at
200 blocks, 0.9 ms against 1.1 ms at 2000), and equal widgets keep their
DOM either way, so the incremental update and its test go.

- markdown-blocks.ts collects the document's blocks, markdown-block-widget.ts
  renders one, markdown-live-preview.ts decides which ones show raw text
- one bounded cache holds a block's HTML and its measured height
- the render key is a plain option: a new one reconfigures the preview,
  which keeps the editor's focus
- MarkdownEditor follows its props with one helper instead of five effects

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep setext headings raw in the live preview editor

`test` over `---` is a setext heading, so the preview rendered it as an h2
and the underline vanished once the cursor left the block. The editor now
shows a setext heading's text as typed at all times, like a link
reference definition.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: add a setting to turn the live markdown preview off

"Live markdown preview in the message input" (General) is on by default.
Off, the input shows the plain markdown text, with file mention badges
still in place; switching it takes effect without a reload.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-10 09:51:01 +01:00
figkcoci-debugandfigkcoci-debug 6122b566b0 ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes (#30415)
* ggml-rpc: fail gracefully on unreachable endpoints and failed handshakes

rpc_dispatcher::start() aborted the whole process (GGML_ABORT -> SIGABRT) when
the endpoint could not be parsed, the transport failed to initialise, the socket
could not connect, or the HELLO handshake failed. A typo in --rpc or a
temporarily down node killed the host application, and the reason went to a log
stream the host may never see.

Make the connect-time failure reportable instead of fatal: start() returns bool
(LOG_ERROR + false; the dispatcher thread only starts after a successful
handshake), negotiate_hello() returns false on a half-open socket,
get_dispatcher() returns nullptr and does not cache failed attempts, and every
consumer treats nullptr as "endpoint did not answer" with the conservative
behaviour of its domain (allocation -> nullptr, alloc-size -> lower bound,
device memory -> 0/0, events -> no-op, device count -> 0, and add_server()
already maps 0 devices to nullptr).

Mid-stream failures keep the existing RPC_STATUS_ASSERT by design (fail fast
against silent corruption) and are intentionally out of scope.

Field motivation: on Android (llama.rn + phone RPC cluster) an unreachable
worker crashed the app; the crash buffer showed
abort <- ggml_abort <- rpc_dispatcher::start <- ggml_backend_rpc_add_server.

* tests : add a regression test for unreachable rpc endpoints

Calls ggml_backend_rpc_add_server with a socket that nothing listens on and with a
malformed endpoint, and expects nullptr from both. Without the fix in this branch
the first call aborts the process, so the test cannot even fail cleanly.

* ggml-rpc remove low-value comments

* tests : drop the internal device-count check after the rpc registry refactor

---------

Co-authored-by: figkcoci-debug <figkcoci-debug@users.noreply.github.com>
2026-11-10 09:40:35 +02:00
Matt Thompson 711f9f4c11 server: responses api: support conversion of text.format.json_schema to completions (#26145) 2026-11-10 00:43:38 +01:00
Aldehir Rojas 26d52b0f95 server : use common_chat_input in realtime api (#31978) 2026-11-09 23:59:16 +01:00
Xuan-Son NguyenandPascal 08f945ffb5 server: add oai-compat /v1/audio/speech endpoint (#31763)
* server: add oai-compat /v1/audio/speech endpoint

* add e2e pytest

* server: share the s16 rounding, type speech form fields

The WAV writer of the audio helper truncates while pcm_f32_to_s16
rounds, so wav and pcm differ by one LSB. Both now round.

pcm_f32_to_s16 takes a pointer and a count and converts the TTS
bytes without an intermediate copy.

/tts and /v1/audio/speech parse their body with parse_tts_body, so
multipart fields such as speed get their type before the speech
conversion checks them.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-09 23:58:52 +01:00
Xuan-Son NguyenandPascal 20c2e96655 mtmd: support gemma 4 input audio stream (#31965)
* mtmd: support gemma 4 input audio stream

* use 480ms step

* server: enable realtime embeddings only for audio generation

The realtime slot forced embeddings on, which marks every row of a
batch as an output. Only a model that generates audio reserves
n_batch outputs, so a chat model with an audio stream (Gemma 4)
aborted on its first batch of audio rows. The hidden state is only
read by the audio generation pipeline.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-09 23:36:31 +01:00
Xuan-Son Nguyen 3d94ffe452 mtmd: add mtmd_helper_resampler (#31964)
* mtmd: add mtmd_helper_resampler

* update docs

* add test

* MA_MAX_FILTER_ORDER
2026-11-09 23:04:39 +01:00