Commit Graph
1612 Commits
Author SHA1 Message Date
FeitengandXuan Son Nguyen 34ff2e047c mtmd: drama3 voice cloning (#32586)
* mtmd: drama3 voice cloning

The Fish S2 codec encoder (fish_s2_spkenc) now also serves drama3:
- conversion: drama3 writes the codec encoder and the audio encoder KV, shared with fish_s2
- fish_s2_spkenc: outputs the codes of the clip (gen-side reference) and, in a drama3_gen mmproj,
  builds the drama3 backbone rows (E_lm[semantic] + E_cb[1][code 1]) / sqrt(2)
- drama3_gen tower: teacher forcing when the caller gives the codes of the next frame
- drama3 pipeline: reference-bracket-v1 prompt with one reference (vLLM-Omni fish_omni), the
  reference frames go through the backbone as codec rows and through the tower with their codes
- mtmd_gen_state_get_ref_codes(): read-only access to the reference codes of a gen state

Claude-Session: https://claude.ai/code/session_01PpX3DmaxNSmJbxGaxnUBrX

* fix correctness

* clean up

* rm redundant code

* fish_s2: rm unused import

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-20 18:34:52 +01:00
yoch 8e7862411d tts : forward -t to the mtmd context (#32664)
llama-tts built the mtmd context from mtmd_context_params_default(), which hardcodes 4 threads, and never copied n_threads over, so -t never reach the mtmd side.
Copy params.cpuparams.n_threads into mtmd_params.n_threads, as mtmd-cli and the server do; with no -t this now follows the resolved default.

Verified: -t 1 now runs single-threaded, -t 4 is unchanged, and the produced WAV is identical.

Assisted-by: Claude
2026-11-20 15:34:58 +01:00
Xuan-Son Nguyen 5f837145d8 server: add more audio tests (#32579) 2026-11-20 14:21:58 +01:00
Xuan-Son Nguyen 36cd3aff40 server: add --ui-mcp-proxy-allow-private-urls (#32633) 2026-11-20 14:10:06 +01:00
JamePeng 53ecea05a0 mtmd : fix YuE2 projector loading (#32606)
- Allow cls.output.weight to be absent from the mmproj. It is an optional
Qwen3 reranking head that the YuE2 NAR decoder never uses.

- Without this fix, mtmd_init_from_file() fails with
"tensor cls.output.weight is missing from the mmproj", preventing
audio generation from starting.
2026-11-19 19:11:11 +01:00
Bhargav Krish c7468107ef cuda: Scatter kernel for set_n (#32489)
* cuda: add scatter for batched tensor writes

* update test case to mirror get_n test case

* save restore timing logs

* refactor code into batch-copy.cu file
2026-11-19 20:14:47 +05:30
Xuan-Son Nguyen e4b620af19 mtmd: support voxcpm2 (#32443)
* mtmd: support voxcpm

* speed up a bit

* various fixes
2026-11-19 02:39:05 +01:00
Xuan-Son Nguyen b044890408 server: fix sleep state for v1/realtime (#32567) 2026-11-18 23:48:02 +01:00
Tarek Dakhran bafa0420c1 server : enable packing for d1-2B and d1-Omni-2B models (#32553)
* llama-bench : size the batch to the prompt for models without memory

Assisted-by: Claude Opus 5.5

* server : read up to 16 lfm3-d1 questions in one pass

Assisted-by: Claude Opus 5.5
2026-11-18 20:15:43 +01:00
Xuan-Son Nguyen 48a32194ee server: fix transcription prompt for v1/realtime (#32507) 2026-11-18 12:09:22 +01:00
Xuan-Son NguyenandPascal 52975342ff server: slot_sched: fix lora and recurrent (#32434)
* server: fix lora and recurrent

* tests: run the hybrid SWA checkpoint test with one token per ubatch

With and without the cache, the prompt was not split into the same
ubatches, and the generated tokens are decoded one at a time while the
uncached run processes them in a batch. On CUDA this moves the top
logprobs of the tiny model by up to 0.03 and swaps near ties, so the
test failed while the cache behaved as expected.

With one token per ubatch both runs go through the same kernels and
the output is compared exactly.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-18 00:12:20 +01:00
Xuan-Son Nguyen 3574df84b7 mtmd: support tiny-voca (#32427)
* mtmd: support tiny-voca

* improve G2P
2026-11-18 00:04:15 +01:00
Xuan-Son Nguyen 307e3f1551 mtmd: add kokoro tts support (#32413)
* jinja: impl scope-chain key lookup

* add test for call_statement lambda

* chat: add tts_dict for kokoro G2P template

* more testing

* value_float_t range check

* no json-in-meta, add llama_model_meta_arr_str/_n

* mtmd: add kokoro tts support

* detailed guide about G2P and text norm port

* refine G2P

* clean up

* migrate batch api

* nits
2026-11-17 22:25:44 +01:00
James PrevettandXuan Son Nguyen 8c16babba5 mtmd: Add support for Qwen3-TTS' ICL/reference mode (#32438)
* Added support for Qwen3-TTS ICL/reference mode

Assisted-By: Claude Opus 5.5

* fix correctness

* attach transcription to bitmap

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-17 18:59:11 +01:00
eddiechen 955c4e8ee5 server : return 400 when grammar and ignore_eos are on (#32377)
* sampling : fix crash when both ignore_eos and grammar are on

* server : rework, throw 400 when grammar and ignore_eos are on

* sampling : move the check into the sampler (review suggestion)

* tests : remove redundant tests
2026-11-17 17:03:21 +01:00
Xuan-Son Nguyen d92ea9bb12 mtmd: support Minimax-Music3 (#32435)
* mtmd: support Minimax-Music3

* fix and clean up
2026-11-17 12:53:19 +01:00
Xuan-Son Nguyen 8d1599fca1 chat: add tts_dict for kokoro G2P template (#32397)
* chat: add tts_dict for kokoro G2P template

* more testing

* value_float_t range check

* no json-in-meta, add llama_model_meta_arr_str/_n
2026-11-17 12:52:50 +01:00
Aleksander Grygier 67cf8967eb ui: move Llama Chat to SvelteKit 3 and its built-in service worker (#32463)
* ui: replace deprecated packages and fix npm audit findings

Swap the mdast stub for @types/mdast, move the katex override to 0.19.0,
and override glob, source-map and html-encoding-sniffer under
workbox-build and http-server so installs no longer pull deprecated
packages. Apply the non-breaking npm audit fixes to the lockfile.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: migrate Llama Chat to SvelteKit 3 and its built-in service worker

Move to SvelteKit 3, Vite 8, TypeScript 6 and Storybook 10.6. The
SvelteKit options move from svelte.config.js into sveltekit() in
vite.config.ts, $lib imports become #lib subpath imports from
package.json, and $app/environment becomes $app/env. Modules named
server.* or living under a server/ folder are server-only in SvelteKit 3,
so the llama-server components, store and enums move to llama-server
names.

@vite-pwa/sveltekit and Workbox give way to SvelteKit's own service
worker in src/service-worker, registered by usePwa(), which precaches the
app files and leaves API requests to the network. A small plugin emits
manifest.webmanifest. static/sw.js removes the worker of older builds, and
llama-server serves a self-removing service-worker.js as well as sw.js
when the built-in UI is off or replaced.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: use SvelteKit 3's update detection and Svelte 5's current patterns

A page the service worker doesn't control now learns of a new build
from SvelteKit itself: `updated` from $app/state checks
_app/version.json on navigation, on focus and every hour. It replaces
the version Llama Chat kept in localStorage and compared by hand.
Reload on the update prompt pulls in the new worker and activates it
before reloading, so one click lands on the new build, whether or not
a worker controlled the page.

Following Svelte's current guidance:
- usePwa keeps its state in $state rather than a store, and no effect
  checks `browser`
- nearViewport, autoResizeTextarea and the custom CSS injection are
  attachments ({@attach}) rather than actions (use:)
- class lists use arrays rather than class: directives
- the chat contexts come from createContext, typed, in place of
  setContext/getContext with string keys
- ActionIconCopyToClipboard takes $props() rather than `export let`

Assisted-by: Claude (Opus 5.5 Medium)

* ui: move to ESLint 10

ESLint 9 is out of support. Every plugin the app uses supports 10:
typescript-eslint, eslint-plugin-svelte, perfectionist,
simple-import-sort, eslint-plugin-storybook and eslint-config-prettier.
@eslint/js and @eslint/compat follow.

ESLint 10's recommended rules ask for two things the code now does:
errors rethrown from a catch keep the caught error as their `cause`,
and no variable starts with a value that every path overwrites.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: override minimatch to clear npm audit's last finding

ESLint 10.14, @eslint/config-array 0.23.5 and typescript-eslint 8.72
depend on minimatch 10, which GHSA-g3hv-75wg-m249 (denial of service
via globparts amplification) affects below 11.0.1. Their fixed releases
move to minimatch 11 but are all younger than the repo's 7-day
min-release-age, so an override pins minimatch 11.0.5, the newest the
rule allows. The three only call new Minimatch(), match() and
minimatch(), which minimatch 11 keeps. npm audit now finds nothing.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-17 11:43:36 +01:00
Sidhartha Reddy Potu 9581154a22 mtmd : fix Muse Glimmer vision embedding order (#32433)
Assisted-by: Claude
2026-11-17 00:29:02 +01:00
Xuan-Son NguyenandPascal 92192d31e4 mtmd: support YuE2-3B (#32369)
* mtmd: support YuE2-3B

* improve speed

* add CFG

* fix streaming

* add cfg control from api

* check ctx size

* fix n_predict

* add flow_steps config

* nits

* fix batch api

* nits

* adapt

* mtmd: yue2 samples from its own saved logits

The context logits belong to the last decode of any slot, so with
several server slots the score and the unguided codes were sampled
from another sequence. Keep the logits of each own decode in
save_logits(), which also does the guidance mix.

* mtmd: let the yue2 snake chain fuse

The beta reshape landed between sqr and mul, so the mul, sin, sqr,
mul, add chain was not consecutive and the backends never fused it.
Expand both reshapes first, like qwen3tts does.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-16 20:22:23 +01:00
Xuan-Son Nguyen a68c6136bb mtmd: support fish audio s2 (#32330)
* add fisk-audio-s2 support

* add voice cloning

* cont merge master

* nits

* nits

* clean up

* nits

* use new batch api
2026-11-16 17:32:27 +01:00
Aleksander Grygier 3a7660371d ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)
* ui: move tools/ui to ui/apps/chat

The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's package @ggml-org/llama-chat

It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: call the built-in web UI Llama Chat in docs, help text and comments

The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.

Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name tools/ui as a former path of Llama Chat in the code-review skill

Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's path in the security policy's exclusions

Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-16 17:23:36 +01:00
Shubham RoyandDeveloper e8698ac163 fix:- Misc. bug: Settings/Tools is lying about tool picks inside a ch… (#32223)
* fix:- Misc. bug: Settings/Tools is lying about tool picks inside a chat #32214

* checks run lint error

---------

Co-authored-by: Developer <dev@example.com>
2026-11-16 17:15:16 +01:00
Niklas Wenzel 62a6d3a302 llama : migrate to llama_batch_ext (#32356)
* llama : migrate to llama_batch_ext

* mtmd : migrate zonos3-mini code

Added in https://github.com/ggml-org/llama.cpp/pull/32020
2026-11-16 15:16:12 +01:00
Ruixiang Wang 6b01a4767f CI: trigger spec dec testing for relevant PRs (#32281)
* ci: trigger spec dec testing after refactoring

* add options to run spec dec ci nightly manually
2026-11-16 11:14:07 +01:00
Pascal a946eb541a server: answer reasoning_effort levels the template rejects with 400 (#32332)
The reasoning effort probe already renders every candidate level, but
it dropped the ones the template raises on, so a request with such a
level (e.g. max on Qwen3.8) reached the template and came back as a
500 on /v1/chat/completions, /v1/responses and /v1/messages.

Keep the raising levels in reasoning_effort_rejected and reject them
in oaicompat_chat_params_parse with an invalid_argument, which the
server answers with a 400 listing the supported levels. Aliases and
levels a template ignores are not rejected, so high on Qwen3.8
still renders as xhigh.

Also trim the output_config.effort test to a single level.
2026-11-16 06:55:56 +01:00
Evan Huus 1719dbb0ee server : skip MCP discovery for router children (#32359) 2026-11-15 23:55:33 +01:00
Xuan-Son Nguyen fc504f91d8 mtmd: support higgs v3 STT and vibevoice-ASR (#32329)
* mtmd: support higgs v3 and vibevoice-ASR

* use proper mapping

* address review comment

* nits

* rm hacky get_audio_length, add mtmd_helper_get_media_meta
2026-11-15 21:25:34 +01:00
Xuan-Son Nguyen 7ae3c6b231 mtmd: add zonos3-mini TTS model (#32020)
* zonos3 init

* add audio gen pipeline

* fix string_format

* add default voice as gguf metadata

* better ref

* support voice clone

* add delivery params

* optimize speed

* nits

* nits 2

* add text normalization

* nits
2026-11-15 19:44:44 +01:00
Xuan-Son Nguyen 75fd7f9ec9 docs: move per-model TTS to /docs, refer it from other pages (#32304) 2026-11-15 18:35:27 +01:00
Roshan Kumar 56fd8a375f server : ignore empty stop strings (#32093) 2026-11-15 17:23:05 +01:00
eddiechen 4e5116f476 server : fix output_config.effort being dropped in /v1/messages (#32321) 2026-11-15 11:09:46 +01:00
Xuan-Son Nguyen dd099a6bf3 chat: fix inkling 2 begin/end of image tokens placement (#32294)
* chat: fix inkling 2 begin/end of image tokens placement

* add test
2026-11-15 09:41:44 +01:00
Xuan-Son Nguyen 429460dc0e server: refactor, add server_slot_sched, prompt running hash (#32154)
* server: refactor, add server_slot_sched, prompt running hash

* use sha256 hash
2026-11-15 09:37:17 +01:00
Aayan Karmali 58a4a6b49c server : clear slot cache when the global lora config changes (#30367) 2026-11-15 09:05:10 +01:00
Piotr Wilkin (ilintar)andXuan Son Nguyen 5c008e0008 feat: add Inkling 2 support (text + vision) (#32217)
* convert : add Inkling2 text GGUF conversion

Assisted-by: Codex

* llama : implement Inkling2 text inference

Assisted-by: Codex

* tests : cover Inkling2 text support

Assisted-by: Codex

* conversion : dequantize packed Inkling2 experts in row chunks

Decode per-layer NVFP4 and MXFP8 schemes, validate companions, and use stored scale shapes with a warning for the declared MXFP8 [32,32] grid. Reorder packed w13 rows and their scale rows together. Keep normal GGUF outtype selection and 3D expert tensors.

Stream 128 rows per expert through LazyChunkedTensor. Real w13 (384x4608x4608) needs a 2.25 MiB F32 chunk instead of a 30.375 GiB expanded layer. Real-shaped sparse probes peak below 5.8 MiB of Python/NumPy chunk allocations, plus 5.2 MiB of callback metadata: budget about 16 MiB per projection including mapped pages, excluding runtime, other tensors and OS cache. Full-model chunk metadata is roughly 370 MiB.

Packed synthetic GGUF weights match the independent pinned vLLM decode bit-for-bit. CPU prefill/decode logits remain within 3.1e-6 absolute error, with 100% argmax agreement. Float fixtures, original-name conversion and CPU architecture checks pass. Real checkpoint conversion remains untested.

Assisted-by: Codex

* convert : add Inkling2 vision mmproj support

Assisted-by: Codex

* mtmd : implement Inkling2 still-image encoder

Assisted-by: Codex

* inkling2 : scope image loading

Assisted-by: Codex

* inkling2 : enable flash attention with folded bias and sinks

Assisted-by: Codex

* cuda : route per-head flash attention masks through vec

Assisted-by: Codex

* mtmd : support Inkling2 temporal frame means

Assisted-by: Codex

* style : restore shared image comment

Assisted-by: Codex

* tests : enlarge Inkling2 context fixture

Assisted-by: Codex

* inkling2 : load per-layer expert feed-forward widths

Assisted-by: Codex

* tests : add Inkling2 Metal fusion baselines

Assisted-by: Codex

* inkling2 : group and align registry entries

Assisted-by: Fable

* convert : regenerate the Inkling2 tokenizer hash with the update script

Assisted-by: Fable

* conversion : revert the print_registered_models change

Assisted-by: Fable

* conversion : sort the Inkling2 model map entries

Assisted-by: Fable

* convert : map Inkling2 vision checkpoint names in the tensor table

Assisted-by: Fable

* convert : map Inkling2 text checkpoint names in the tensor table

Assisted-by: Fable

* tests : document the Inkling2 packed expert conversion test

Assisted-by: Fable

* tests : cover Inkling2 checkpoint and transformers tensor names

Assisted-by: Fable

* llama : place Inkling2 fields and list entries by file convention

Assisted-by: Fable

* gguf-py : align Inkling2 tensor mapping comments

Assisted-by: Fable

* mtmd : place Inkling2 entries by file convention

Assisted-by: Fable

* tests : place Inkling2 cases after the existing ones

Assisted-by: Fable

* cuda : index per-head flash attention masks in the tile and mma kernels

Assisted-by: Claude
Claude-Session: https://claude.ai/code/session_01GbYSza1yxgmTz8wdr6uV8Q

* inkling2 : project the relative bias before the gather, drop arch branches

Assisted-by: Claude
Claude-Session: https://claude.ai/code/session_01GbYSza1yxgmTz8wdr6uV8Q

* convert : require Inkling2 hparams, add inkling2 pre-tokenizer

Assisted-by: Claude
Claude-Session: https://claude.ai/code/session_01GbYSza1yxgmTz8wdr6uV8Q

* mtmd : cap Inkling2 image size, drop video frame averaging

Assisted-by: Claude
Claude-Session: https://claude.ai/code/session_01GbYSza1yxgmTz8wdr6uV8Q

* inkling2 : fix CI build, Meta device assert and Windows conversion test

Assisted-by: Claude

* tests : use the default sliding window for the Inkling2 test model

Assisted-by: Claude

* simplify mtmd part

* give _v arch for vision

* remove alpha channel handling, it should be a follow up

* explicit rule about audio + vision models

* add audio support

* rm audio

* rm gguf mtmd rope base

* address review comment

* nits

* do not normalize embd from mtmd

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-15 00:51:31 +01:00
Srijan Keshri cf9a9957da server : Fix connection reuse after streamed responses (#30492)
Assisted-by: Codex
2026-11-15 00:29:28 +01:00
Xuan-Son Nguyen 9d925e7c02 mtmd: fix video pixel aspect ratio and video rotation (#32267) 2026-11-14 14:57:21 +01:00
Jhen-Jie HongandXuan Son Nguyen e3483e40b0 mtmd : allow custom video decoder (#32188)
* mtmd : allow custom video decoder

* also adapt to wllama requirements

* clean up

* add test

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-11-14 14:12:07 +01:00
Hera 0cce7580b8 mtmd : avoid padding-only Whisper audio chunks (#30403)
Assisted-by: Claude Opus 5
2026-11-14 13:33:59 +01:00
Ruixiang Wang 32bdc3a89e spec + model: refactor DFlash based speculative decoding methods (#32016)
* dflash : move the DSpark markov head and backbone graphs to dspark.cpp

Move-only: build_dspark_markov_head, graph_dsv4, graph_dsv41 and
graph_mistral_large4 move verbatim. build_inp_target_features and
build_dspark_markov_head become static members of llama_model_dflash so
both files can use them.

* dflash : detect the head and the backbone once, split the loaders per file

load_arch_hparams sets llama_model_dflash::variant (DFlash, DFlash2,
DSpark, LiliCorr) and ::backbone (base, DSV4, DSV4.1, Mistral Large 4).
The head, backbone and base layer loaders become member helpers in
dflash.cpp, dflash2.cpp, lilicorr.cpp and dspark.cpp. build_arch_graph
dispatches on the backbone.

The DFlash2 and LiliCorr loaders create the DFlash2 conv tensors through
a helper in dflash2.cpp, right after their head tensors instead of in
the base layer loop; the other tensors keep their creation order. The
DFlash2 loader drops its copy of the conv group size check, which master
never reached.

* dflash : share the injection input, token embeddings and lm head lookups

build_inject_inp (target features + fc + encoder norm), get_tok_embd and
get_lm_head replace the copies in the base, DSV4, DSV4.1 and Mistral
Large 4 graphs. build_inp_target_features is file-static again.

* dflash : split the DFlash decoder into a graph per head

graph_base creates the shared inputs and provides build_inject,
build_inp_tokens, build_layers and build_logits. graph_dflash
(dflash.cpp), graph_dflash2 (dflash2.cpp), graph_dspark (dspark.cpp) and
graph_lilicorr (lilicorr.cpp) assemble them and add their head; the
selector, scorer and markov head builders move to the file of their
graph, and the DFlash2 local conv run by build_layers to dflash2.cpp.
build_arch_graph dispatches on the backbone, then on the head.

The op order and the tensor names are unchanged.

* speculative : split the DFlash driver into a class per head

common_speculative_impl_draft_dflash stays the DFlash implementation and
becomes the base of draft_dspark, draft_dflash2 and draft_lilicorr. Its
protected constructor only reads the draft metadata; init() then sets up
the contexts with the options of the draft (draft length, backend
sampling, device features, masked h_nextn). Each derived class overrides
draft() with the path of its head from the former draft(). DFlash2 also
overrides begin() to reseed its generator per request, and LiliCorr
overrides process() to store the context features of each injected
chunk, which its noise block carries as its state.

The factory keeps the current routing: draft-dspark picks DSpark,
draft-dflash picks DFlash2 or LiliCorr from the draft metadata and
DFlash otherwise. draft-dspark with a DFlash2 or LiliCorr draft now runs
the DSpark reader instead of a mix of both. sample_from_anchor is logged
by DSpark only.

* speculative : add the draft-dflash2 and draft-lilicorr types

draft-dflash2 and draft-lilicorr name the DFlash2 and LiliCorr drafts,
which draft-dflash keeps running too: it picks them from the draft
metadata. draft-dflash2 and draft-lilicorr must match the draft model,
draft-dspark rejects a DFlash2 or LiliCorr draft, a model that is not a
DFlash draft is rejected, and a run uses at most one DFlash family type;
these errors leave speculative decoding uninitialized.

The type detected from the draft GGUF is now the concrete one, and
draft-dflash2 and draft-lilicorr download the dflash sidecar.

* docs : describe the DFlash2 and LiliCorr drafts

Add the DFlash2 and LiliCorr sections and the new types, and explain
that draft-dflash also runs DFlash2 and LiliCorr drafts.

* speculative : drop the unused DFlash is_mrope

Nothing reads it since the migration to common_batch (#29385), which
picks the number of M-RoPE position rows from the rope type of the draft.

* doc: update spec dec

* address review comments
2026-11-14 09:49:27 +01:00
AMIR 5193ef1d4f ui: configure allow-remote=root in .npmrc for npm 12+ (#32158) (#32196) 2026-11-13 20:54:49 +01:00
Pascal 4f9dfa0e36 requirements, Fix CI: support Python 3.15 (#32199)
* requirements : support Python 3.15

torch 2.11.0, numpy 2.4, matplotlib 3.10, pandas 2.3, pillow 11.3 and
cffi 1.16 have no Python 3.15 wheels, so the CI runner on 3.15 cannot
install the requirements.

Python 3.15 gets the first releases that ship 3.15 wheels through
version markers, and every version below 3.15 stays as is.
pyproject.toml now allows 3.15 and picks torch 2.15.0 there.

* requirements : apply review suggestions

numpy 2.5.4 from Python 3.14, matplotlib 3.11.2 and pandas 3.0.6 from
3.11, cffi 2.1.1 and pillow 12.3.0 for every version. cffi 1.16.0 has
no 3.14 wheel and fails to build there, so this also fixes the
tokenizer test requirements on 3.14.
2026-11-13 17:50:03 +01:00
Georgi Gerganov 28d37fe2e2 examples : clean-up old/obsolete stuff (#32184)
* examples : clean-up old/obsolete stuff

* examples : update references to removed files

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

[no ci]
2026-11-13 10:33:17 +02:00
Pascal 0a66ac7189 server, fix CI: read a small temp file in the read_file tool test [no release] (#32168)
* server: read a small temp file in the read_file tool test

test_tools_builtin_read_file read its own file, which grows with every
added test. On Windows the CRLF checkout puts it at 16664 bytes, over
the 16384 byte read limit, and the tool refuses it.

The test now writes a small file of its own and checks the exact
content read back.

* server: use a temp cwd in the x-tool-cwd tool test

test_tools_builtin_cwd_header read test_tools_builtin.py through the
x-tool-cwd header, which hits the same 16384 byte read limit on the
Windows CRLF checkout, and wrote its shell marker into the source tree.

The test now uses a temp directory as cwd, reads a small file from it
and checks the shell marker lands there.
2026-11-12 23:47:14 +01:00
Pascal 892b40b910 server: allow private media URLs in the vision tests [no ci] (#32167)
The vision tests download their images from huggingface.co, which
resolves to a private address on the runners inside the HF network.
The media URL check refuses it with "host is not a public address"
and the server-cuda job fails.

The vision fixture enables media_allow_private_urls. The private
address check keeps its own coverage in
test_vision_image_url_private_address.
2026-11-12 23:15:28 +01:00
Pascal e970101470 server: skip LAN address tests when the host cannot reach it [no ci] (#32166)
The same-origin tests over the LAN address block forever on the Metal
runners: the request to the host's own LAN address never returns and
has no timeout, so the job runs until the 6h limit.

get_lan_ip() now probes /health on that address with a short timeout
and returns None when the host cannot reach it, which skips the test.
The test requests themselves use the default request timeout so a
hang fails the test instead of stalling the job.
2026-11-12 23:13:37 +01:00
Georgi Gerganov 8ca5a8008b llama : prune (#30565)
* llama : remove deprecated `llama_set_warmup`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : remove remaining deprecated API

Drop the APIs marked DEPRECATED in llama.h, the deprecated
llama_context_params::defrag_thold field and the --defrag-thold option.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* fit-params : remove stale `llama-ext.h` include

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : document `llama_model_dflash_selector_top_k`

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : promote the NextN / speculative API from `llama-ext.h`

Move the NextN, layer input, embd row and state row APIs to `llama.h`
and rename `llama_set_embd_state_src` to `llama_set_ctx_state_src`, with
the internal `llama_context::set_state_src` following the same name.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* llama : polish the NextN API promotion

Move the NextN draft model queries next to the Adapters section, rename
the `src` parameters of `llama_set_ctx_state_src` and
`llama_context::set_ctx_state_src` to `ctx_src`, and drop the now-redundant
`llama-ext.h` includes from `common/speculative.cpp` and `tools/imatrix`.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-12 20:59:09 +02:00
Hamish M. Blair d8c8cc4d9b server : cancel MCP tool calls on client disconnect or timeout (#32025)
* server : cancel MCP tool calls on client disconnect or timeout

* server : shorten MCP cancellation tests and fix a type-check error
2026-11-12 17:19:26 +01:00
Xuan-Son Nguyen 8954f19483 server: fix SSRF for CORS proxy and media URLs (#32115) 2026-11-12 12:33:48 +01:00