Commit Graph
802 Commits
Author SHA1 Message Date
mmalpe-nvandJohannes Gäßler 2cbfc3cf34 [CUDA] : Add SKU-aware FP16/BF16 accumulation precision gating for NVIDIA GPUs (#30672)
* cuda: select cuBLAS precision by GPU SKU

Assisted-by: Codex

* cuda: fix remaining GB10 macro reference

* cuda: review comment addressed, store accumulation preference in device info

* Apply batched suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* addressed Oliver's comment regarding initialize NVML once and install headers in Windows CI

* move precision selection helper into init

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2026-11-18 18:29:56 +01:00
Aleksander Grygier a693a5f66f ui: llama-chat publish drops the registry check, bin paths without ./ (#32543)
* ui: llama-chat publish waits for the registry, bin paths without ./

The first publish of @ggml-org/llama-chat took about four minutes to show
up on the registry, so the 60 seconds the check after publishing waited
failed a release that had gone through. It now waits up to ten minutes.

npm normalizes "./bin/cli.js" to "bin/cli.js" on publish and warns that
the bin was "invalid and removed", which reads as if the command was
dropped. Both package.json files now give the paths as npm writes them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: drop the registry check after llama-chat publishes

npm publish already fails when the registry refuses a package, and the
check after it only failed a release that had gone through, while npm
was still processing it. Nothing waits on the registry any more.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-18 15:29:54 +01:00
Aleksander Grygier 97069ff6c4 ui: run Llama Chat with npx @ggml-org/llama-chat (#32524)
* ui: run Llama Chat with npx @ggml-org/llama-chat

Makes @ggml-org/llama-chat publishable: drops private, adds bin,
files, exports, repository and a prepack build. Its CLI serves the
prebuilt dist on localhost (3000 by default) and forwards every other
request to a llama-server (--server, default http://127.0.0.1:8080),
with the headers llama-server sends for the UI. A thin unscoped
llama-chat package in alias/ forwards to it, so npx llama-chat works.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: llama-chat revalidates service-worker.js and asks for a build before packing

SvelteKit 3 adds service-worker.js next to sw.js, and llama-server
revalidates it, so the CLI does too. The folder's .npmrc sets
ignore-scripts, so prepack never ran: it goes, and the publishing steps
build dist explicitly and check the packed file list.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: llama-chat keeps llama-server's DNS rebinding and cross-origin guards

The CLI called llama-server with its own Host and dropped every Origin,
so a rebound host name or another page got past the checks llama-server
makes when --cors-origins is not *. It now drops only its own page's
Origin, passes other Origins and Sec-Fetch-Site through, and, when it
listens on loopback, answers only to localhost and loopback addresses.
`start` on Windows gets its empty title verbatim.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: llama-chat build writes the app to a folder for any static server

`npx @ggml-org/llama-chat build [--out <dir>] [--force]` copies the
prebuilt app, without source maps, to ./llama-chat or the given folder,
so a plain file server can host it next to the CLI serving it. It
refuses a folder that is not empty unless forced, and the serving flags
and build flags refuse each other's command. The README shows serving
the folder with http-server forwarding the API to llama-server.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: publish Llama Chat to npm from CI when its version is bumped

ui-chat-publish.yml publishes @ggml-org/llama-chat and its llama-chat
alias the way ui-lib-publish.yml publishes @ggml-org/llama-ui: the
version is set by hand, and a commit that reaches master with a version
npm does not have builds dist, checks the packed files and publishes both
with trusted publishing. PRs that change the packages get a dry run.
npm-version-check.mjs takes the package folder, so both workflows share
it. ui/README.md says how releases work, and Chat's AGENTS.md says not
to bump versions unless asked to release.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: call the app Llama Chat in its README

The README's title and a few lines still called the app llama-ui or
Llama UI, the name of the library it is built on. They now say Llama
Chat, and the intro points to Llama UI in ui/lib/ui.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: llama-chat connects to llama-server's default port, 9931

The CLI's --server default and its examples used 8080, which is not
where llama-server listens. They now use 9931, its default port, so
npx llama-chat reaches a llama-server started without --port.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: a shorter Llama Chat README that matches the code

The README grew to nearly 900 lines, and its tables of components,
hooks, stores and services had drifted from the code. It now keeps what
a reader needs: running it with npx and self-hosting, a short feature
list, development commands, the architecture (the line between Llama UI
and the app, a table of the layers and where they live, a flowchart and
one sequence diagram for sending a message), a few patterns worth
knowing, cross-origin isolation, how the build is embedded in
llama-server, the project tree and publishing. The folders now carry the
detail the tables used to. ASCII replaces the unicode arrows.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: llama-chat serves only static files, without the API proxy

The CLI forwarded every request it had no file for to llama-server, from
loopback. With --host 0.0.0.0 that let any LAN client reach a llama-server
started on 127.0.0.1 as a local request, tools included, which llama-server
on its own refuses without an API key. The browser now talks to each
llama-server directly, so llama-server's own rules apply. --server is gone,
and anything that is not a file in dist is a 404, or a 405 for other methods.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-18 12:58:16 +01:00
Aleksander Grygier 6da36139e8 ui: run the Storybook deploy on HF Jobs, with a token that can write the Space (#32525)
The deploy failed on master with a 403: HF_TOKEN_UI_STATIC_OUTPUT is
scoped to the UI bucket and cannot write ggml-org/llama-ui-storybook, so
hf upload tried to create the Space. The upload now uses
HF_TOKEN_UI_STORYBOOK when that secret exists, and the job runs on
hf-jobs-cpu-upgrade instead of a GitHub-hosted runner. That image gets
python3-venv from apt for the hf CLI, as the other HF Jobs workflows do.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-18 12:07:23 +01:00
Aleksander Grygier 65758d01f5 ui: Storybook follow-ups: generated DESIGN.md tokens, shared helpers and one Llama Mono (#32523)
* ui: generate DESIGN.md's token sections from the CSS

Add scripts/design-md.js in ui/apps/chat (npm run design:tokens, and npm
run check:design in CI), which writes the colors, typography and rounded
sections of DESIGN.md's front matter from llama-brand's colors.css and
Llama UI's primitives, semantic tokens and font stacks. The front matter
had drifted: it gave Matte's surfaces as the light base and system-ui for
every text role. Twins are now written wherever a token resolves
differently in dark mode (-dark) or in Matte (-matte).

Describe the floating panel's rim as the CSS draws it, on the border,
and say which parts of DESIGN.md are generated and which are kept by hand.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: say the Storybooks show accessibility findings without enforcing them

Both Storybooks keep a11y test 'todo': under 'error' 98 of the stories
fail, most on colour contrast of the muted foreground and on nested
interactive controls, which need design changes. Reword the Uses tile
and record the reason next to the setting.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: list every Llama Chat group in the Introduction's Sections figure

The figure said it was in sidebar order but left out Forms, Actions,
Badges, Misc, Servers and PWA. Its list moves to sections.ts, and a unit
test checks it against storySort in .storybook/preview.ts, which has to
stay inline there because Storybook parses it as written.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: load the package's Llama Mono in both Storybooks

The library's Storybook declared Llama Mono a second time from
tests/stories/brand/fonts, a 72276-byte file that differs from the
76896-byte one the package ships, so its pages showed a Llama Mono that
does not ship. Drop both Storybooks' copies: theme.css already loads the
package's font.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: take Llama Chat's shared Storybook helpers from Llama UI

Llama Chat kept copies of the Editorial, Chapters, StateGallery,
StateGalleryItem, svx.d.ts and docs.css that live with the library's
stories. Its stories now import them as $llama-ui-stories/..., an alias
its Storybook config and tsconfig.json map onto ui/lib/ui/tests/stories,
the way it already compiles the library from source; docs.css and
svx.d.ts are referenced by path. The library's Editorial takes Llama
Chat's newer list numbering. An empty svelte.config.js next to the
stories keeps Llama Chat's svelte-check from loading the library's
vite.config.ts, as src/svelte.config.js does for the sources.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: pin the library's Storybook to Llama Chat's versions

ui/lib/ui used ^10.6.2 with @storybook/addon-mcp at 10.6.2, while
ui/apps/chat pins 10.6.6, so one tree had two Storybooks. Pin the
library's Storybook packages exactly to Llama Chat's: 10.6.6,
addon-svelte-csf 5.1.6 and @chromatic-com/storybook 5.5.1.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-18 09:45:36 +01:00
Aleksander Grygier 9e93cd8001 ui: a Storybook for Llama UI and Llama Chat (#32485)
* ui: Llama UI's own Storybook in ui/lib/ui

The library gets its Storybook (`npm run storybook`, `npm run
build-storybook`) with a story for everything it holds, under Llama UI:
the foundations, the 29 base components (as actions, forms, display,
overlays and navigation), the app components (ActionIcon, SearchInput,
SyntaxHighlightedCode), the Model UI and the native app mockups. The
stories import the package by its published names, aliased onto src/lib,
run as tests under `npm test` (the storybook project in vite.config.ts),
and are type-checked with the rest of the package. `npm run dev` starts
Storybook next to the SvelteKit check page; the toolbar switches the
color mode and the surface theme, and pages that are mostly prose are
.svx, through mdsvex.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: a Storybook for Llama Chat

Llama Chat's Storybook gets an introduction, Agents › Design guidelines
(DESIGN.md), Llama UI's stories as the library has them, and Llama Chat:
its own foundations (safe areas, the splash screens), then one section per
src/lib/components/app/<area> (Chat with the chat screen, form, messages,
attachments and tabs; Content, Models, MCP, Navigation, Settings, Dialogs,
Forms, Actions, Badges, Misc, Servers and PWA), each accessibility story
next to its component's. It replaces the flat pages it had (chat message,
chat form, markdown content, the PWA refresh alert and their fixtures,
and the ChatForm single-row exploration).

The stories run on the new tokens and surfaces with the color mode and
surface theme in the toolbar (by mode-watcher's key, the ColorMode and
SurfaceTheme enums and applySurfaceTheme), and pages that are mostly
prose are .svx in an editorial frame (AGENTS.md says how to write them
and where a story goes). The story tests run Llama Chat's stories and
leave the library's to the library. The brand and communications pages
are in ggml-org/llama-brand; the logo files come from @ggml-org/llama-brand.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: let the ChatScreen story store its reply before the next one starts

The ContinueConversation story ended while its reply was still being stored,
and the next story reset the stores under it. It now waits for the reply to
finish.

Assisted-by: Claude (Fable 5.2)

* ui: publish the Storybook to its Hugging Face Space on every UI change

A new UI Storybook workflow builds ui/apps/chat's Storybook whenever
ui/apps/chat or ui/lib/ui changes on master (or on demand) and uploads it
to the static Space ggml-org/Llama-UI-storybook, with a Space card naming
the commit it was built from. Files the new build no longer has are
removed; the Space's .gitattributes stays. It runs only in
ggml-org/llama.cpp, takes the UI's Hugging Face token
(HF_TOKEN_UI_STATIC_OUTPUT), and the Space can be changed with the
HF_SPACE_UI_STORYBOOK variable.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: run the library's tests, its stories among them, in CI

The Library checks job type-checked and packed ui/lib/ui but never ran its
tests, so the library's stories, which now hold every library component's,
went untested in CI. It installs Chromium and runs `npm test`.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: draw a floating panel's rim on its border, so its edge is one line

Menus, popovers, dialogs and the other matte-float panels carry a 1px border that the
surface made transparent, and drew their inner highlight as an inset shadow. An inset
shadow sits inside the border, so the 1px of fill between it and the outer ring showed
as a second edge. The border now draws the highlight, right inside the ring.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-18 08:27:44 +01:00
Aleksander GrygierandSigbjørn Skjæret 744c68cd14 ui: publish @ggml-org/llama-ui from CI, and release 0.4.0 (#32480)
* ui: publish @ggml-org/llama-ui from CI when its version moves

ui/lib/ui reaches npm only when someone remembers to publish it by hand, which let 0.3.0
sit a day behind master. The version in its package.json becomes the release: a commit
landing on master with a version npm does not have gets built, linted and published with a
provenance attestation, through OIDC rather than a stored token, as release-publish.yml
already does for GitHub releases. A library change carrying no bump publishes nothing and
says so, on the PR and on the run.

Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS)

* ui: bump @ggml-org/llama-ui to 0.4.0

The library on npm, published before the design system PR merged, holds no entry points for
the Model UI, the app components or the shared constants and enums, and no dependency on
@ggml-org/llama-brand, whose colours the primitives now build on. Consumers import the new
subpaths and get ERR_PACKAGE_PATH_NOT_EXPORTED.

Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS)

* ui: name no environment for the Llama UI publish job

Creating a repository environment needs admin rights the UI maintainers do not hold, so a job
that names one cannot be set up by them. The job works without it: npm's trusted publisher
matches on the repository and the workflow file, and its environment field stays blank to
match. Nothing else in the job depends on an environment.

Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS)

* ui: address review on the Llama UI publish job

- the advisory version check leaves ui.yml for the publish workflow, on a pull_request trigger
  whose path filter keeps it off PRs that touch only Llama Chat: a chat-only PR was told to
  bump a library it did not change
- publish and verify run on master only. The trusted publisher matches on the repository and
  the workflow file, not the branch, so a dispatch from a PR branch - anything with write
  access can start one - could publish code no one reviewed. With no environment, this was the
  only branch protection in place.
- the registry read after a publish retries six times rather than sleeping once, so a release
  the registry is slow to serve is not reported as a failure
- a build metadata suffix (1.2.3+build) is not a prerelease, and versions order by semver's
  prerelease rules, so 0.4.0-beta.1 behind a released 0.4.0 is refused instead of shipped to next
- the summary step passes its values through env like the others, actions: read is gone, and the
  concurrency comment claims only what the setting does
- npm older than 11.5.1 is refused up front, as it cannot publish with an OIDC token

Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS)

* ui: note the two trusted-publishing traps in the Llama UI README

A publisher names a workflow file that must already be in .github/workflows, so it is added
after this workflow lands rather than before, and one created now publishes only through
staged publishing until npm publish is permitted for it. Provenance comes free with either,
and only on a GitHub-hosted runner.

Assisted-by: Llama Chat (unsloth/Qwen-3.8-Flash-Next:IQ4_XS)

* Update .github/workflows/ui-lib-publish.yml

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-11-17 23:19:05 +01:00
Aleksander Grygier 7f106b5459 ui: Llama UI design system: tokens, surfaces and fonts (#32468)
* ui: bring @ggml-org/llama-ui into ui/lib/ui

The design system package (tokens, fonts, surfaces, components and Model UI), so far in the temporary ggml-org/llama-ui
repository at edaf518, moves to its home in ui/lib/ui, built with @sveltejs/package and published as @ggml-org/llama-ui
from here. ui/lib holds the packages, one folder each. Its references to tools/ui now name ui/apps/chat, its
package.json points at this repository, and the UI workflow type-checks and packs it. ui/README.md describes the
layout.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: Llama UI design system: tokens, surfaces and brand

The design system from allozaur/docs/storybook-overhaul: three tiers of
tokens, matte glass surfaces, Llama Sans and Llama Mono, and the base
components on the new tokens (Tabs and GlassKey among them). It also
restyles the stack's features with it, replacing the plain stand-ins they
used until now. DESIGN.md is generated from the tokens. The Storybook that
documents it is the next PR.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: have Llama Chat use @ggml-org/llama-ui for its base components, tokens and fonts

Llama Chat kept its own copy of the design system: 29 base components, the
three tiers of tokens, the surfaces, the theme and the fonts, next to the 12
components ui/lib/ui held. The copy goes, and the library gets what it lacked.

ui/lib/ui
- the 17 base components the app had and the library did not (alert,
  alert-dialog, button-group, checkbox, collapsible, context-menu, dialog,
  drawer, dropdown-menu, empty, hover-card, radio-group, scroll-area,
  skeleton, table, tabs, textarea), with vaul-svelte for the drawer
- the inset-group and selected-row tokens and surfaces
- sources import each other by relative path instead of $lib, which in an
  app points at the app's own src/lib
- theme.css no longer imports tw-animate-css: it is a peer dependency that
  the consumer imports before the theme, as it does Tailwind
- 0.3.0

ui/apps/chat
- imports the base components as @ggml-org/llama-ui/ui/<name> and cn/tv as
  @ggml-org/llama-ui/utils; app.css imports the library's theme.css
- compiles the library from source, with no install or build of it:
  scripts/vite-plugin-llama-ui.ts resolves its entry points and what it
  imports, and svelte.config.js gives TypeScript the same paths
- src/lib/components/ui, styles/tokens, styles/surfaces.css and assets/fonts
  are removed; components.json moves to the library

Builds
- CMake stages ui/lib/ui next to the app, keeping the layout of ui/, and
  rebuilds when a library source changes
- the Dockerfiles copy the library's sources; the Nix derivation takes ui/
  as its source

The built CSS has the same rules in the same order, apart from one
zero-specificity rule and the @font-face rules moving, plus ten utilities
and two tokens that only the library's own components use.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: AGENTS.md places Llama UI in ui/lib/ui

Llama Chat's AGENTS.md says where each part lives: Llama UI is
ui/lib/ui, and Chat imports its tokens, fonts and base components
from there.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: Llama Sans and Llama Mono from llama-brand's latest, and shell commands marked in bash code

The fonts in ui/lib/ui come from llama-brand's latest, with optical sizes
from 10 to 36. Bash code blocks mark shell commands and flags.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: open submenus outside the menu they come from

A submenu was rendered inside its parent menu, so the menu's scroll box
clipped it: the glass menus' backdrop filter, and the filter and transform
of the open animation, make the menu the containing block even for the
fixed-position submenu. Dropdown and context submenus now render in a
portal, like the menus themselves.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: draw submenus like the menu they open from

Submenus had smaller corners than their menus, and context menus had none
of the dropdown's glass. Context-menu parts now match the dropdown's, and
both submenus take the menu's corners and height cap, with their first row
level with the row that opens them, 4px past the menu's edge. Sub-triggers
take the items' radius and pointer.

The submenus' height cap and zoom origin named the dropdown's CSS variables,
which bits-ui does not set on submenus; they now use its --bits-menu-* ones.
The chat group submenu takes the actions menu's z-[999999] layer, as it no
longer renders inside that menu.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep a late current-node write off a closed or switched conversation

updateCurrentNode checked the active conversation, awaited the database write, then wrote currNode to whatever was active after it. When the conversation closed during the write it threw, and the reply ended in a streaming error dialog. When another conversation opened, that one got the wrong node.

Assisted-by: Claude (Fable 5.2)

* ui: have Llama Chat use the library's Model UI and app components

Llama Chat still kept its own copies of what the library holds above the
base components. It now takes them from @ggml-org/llama-ui too, and its
copies go.

ui/lib/ui
- ModelId gains what Chat's had: the model's own draft kind, draft
  sidecars with their quant (ModelDraftSidecars, new), aliases, hideName
  and a tooltip, in the same badge order
- entry points for them, like ui/<name>: @ggml-org/llama-ui/app/<group>
  (actions, content, forms, misc) and @ggml-org/llama-ui/models

ui/apps/chat
- ActionIcon, SearchInput, SyntaxHighlightedCode, Logo,
  ModelCapabilityIcons, ModelDownloadProgressBar and ModelDraftSidecars
  come from the library; Chat's copies and the two constants only they
  used are removed
- ModelId and ModelOrgAvatar stay as thin wrappers: ModelId parses the raw
  id and applies the user's display settings, ModelOrgAvatar hides the
  avatar while Hub metadata is off
- the svelte/* TypeScript path falls back to Svelte's types/index.d.ts:
  svelte/reactivity has no file of its own, so a library source importing
  it got its types from ui/lib/ui's own install when there was one

Assisted-by: Claude (Opus 5.5 Medium)

* ui: Llama UI takes the brand's colours and logo from llama-brand

@ggml-org/llama-brand is now a dependency of the library, by the same GitHub
tarball Llama Chat already installs, which now names it @ggml-org/llama-brand
too.

- primitives.css drops the 25 tokens it held copies of (the neutral
  greys, sand, the skies and cinder) and builds on llama-brand's
  css/colors.css instead; the skies' gradients point at llama-brand's.
  Apps import that file before theme.css, as they do Tailwind and
  tw-animate-css: the library's stylesheets import no package, so an app
  that compiles them from source resolves it from its own node_modules
- Logo draws llama-brand's icon, in currentColor, and assets/logo.svg goes
- Llama Chat's TypeScript paths resolve llama-brand from its own
  node_modules, like the library's other dependencies

The fonts stay as the library's WOFF2 copies until llama-brand ships
WOFF2 builds of them.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: have Llama Chat take the library's constants, enums and helpers

Llama Chat redefined most of what the library exports next to its
components. Where the two were the same, Chat now re-exports the library's
from the files that held them, so its own imports stay as they are.

- the library gets entry points for them: @ggml-org/llama-ui/constants,
  @ggml-org/llama-ui/enums and @ggml-org/llama-ui/code, and
  @ggml-org/llama-ui/models carries the model data with the Model UI
- from the library now: the shared CSS classes and code-block constants,
  the byte units, the memory-fit constants and minMemoryTierGb, the bit
  depth labels, the capability and modality icons, labels and order,
  ColorMode, TooltipSide, BooleanString, the model enums, ModelModalities,
  formatParameters, highlightCode and trimCodePadding
- the library's modality order follows Chat's: vision, video, audio
- Chat keeps its own where they differ: UI_DATA_ATTRS and
  ModelDownloadProgress are supersets of the library's (the latter extends
  it now), and formatFileSize counts in 1024s

Assisted-by: Claude (Opus 5.5 Medium)

* ui: give the chat form's add key and model selector a tighter shadow

In light, the two keys inside the chat form, white on its white glass, took
the control elevation, whose soft spread blended into the form. They take a
new elevation-key instead: a 0.5px hairline all round and a 1px drop. In
dark they keep the control elevation. Chosen from the options in
Scrapbook › UI › Composer keys.

- semantic tokens: --elevation-key (light: the hairline and the drop; dark:
  the control elevation)
- component token: --chat-form-key-shadow, for the add key and every model
  selector trigger
- DESIGN.md names it next to the other elevations

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-17 14:39:03 +01:00
Aleksander Grygier 3a7660371d ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)
* ui: move tools/ui to ui/apps/chat

The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's package @ggml-org/llama-chat

It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: call the built-in web UI Llama Chat in docs, help text and comments

The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.

Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name tools/ui as a former path of Llama Chat in the code-review skill

Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's path in the security policy's exclusions

Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-16 17:23:36 +01:00
Georgi Gerganov 95db855bce ci : use vX.Y.Z tags as docker tags (#32388)
* ci : use vX.Y.Z tags as docker tags

When the Docker workflow is manually dispatched from a vX.Y.Z tag, use that tag verbatim for the image tags instead of appending the build number and commit hash.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)

* ci : require a tag for manual docker runs

Manual dispatches of the Docker workflow must now use a tag ref, so the image tags are derived from the selected tag. The check can be skipped with the new skip_ref_check input. Scheduled nightly builds are unchanged.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4.1-Flash-1017-GGUF (Mac Studio, M5 Ultra, 256GB)
2026-11-16 13:27:10 +02:00
Ruixiang Wang 6b01a4767f CI: trigger spec dec testing for relevant PRs (#32281)
* ci: trigger spec dec testing after refactoring

* add options to run spec dec ci nightly manually
2026-11-16 11:14:07 +01:00
uvos 278a89cb78 CUDA/HIP: Refactor the use of _sync warp primitives (#31617)
Uses the rocm-native primitives, increases the rocm version requirement to 6.3 and makes top-k selectable warp.
2026-11-15 10:58:36 +01:00
Sigbjørn Skjæret 82503b42fa ci : set windows-cuda ccache size to 1G (#32185) 2026-11-13 08:53:37 +01:00
Sigbjørn Skjæret 827466b9ef ci : build ui on non-slim runner (#32138) 2026-11-12 12:35:31 +01:00
Xuan-Son Nguyen 664f031f49 ci: move test models to ggml-org-dev (#32096)
* ci: move test models to ggml-org-dev

* fix test spec
2026-11-11 21:17:16 +01:00
Sigbjørn Skjæret 956591f457 ci : remove dead snakes (#32084) 2026-11-11 15:13:14 +01:00
Asahi-PrvandAsahi-Prv b326dd6752 devops : add missing MKL runtime DLLs to the Windows SYCL release package (#32005)
ggml-sycl.dll built with oneAPI 2026.2 imports mkl_intel_thread.3.dll and libimalloc.dll, which are not part of the release package. LoadLibrary fails and the SYCL backend is silently skipped, llama.cpp falls back to CPU.

Adding both DLLs fixes the load: measured on Arc B570 with LFM2.5-8B-A1B Q4_K_M, pp4096 goes from 404 t/s (CPU fallback) to 2232 t/s (SYCL) with the extracted b12244 package.

Assisted-by: GLM (z-ai/glm-5.3-flash)

Co-authored-by: Asahi-Prv <Asahi-Prv@users.noreply.github.com>
2026-11-11 14:14:43 +01:00
Sigbjørn Skjæret 582eab7186 ci : bump ty to 0.0.100 (#32030)
* bump ty to 0.0.100

* rename rules

* type fixes

* type fix

* proper type fix
2026-11-10 21:54:14 +01:00
Oliver Simons 202c1a229f llama : Add FP8 KV-cache support (#28392)
* Add FP8 KV-cache support

* Add tests for scaled, FP8 KV-cache to test-backend-ops

* Fix get_*_scale mapping and dequnatize FP8 also before rope shift

* Normalizes cache-scale names independently of projection layout.

While fused KQV is currently impractival for non-shared scales, this is
an improvement for future use regardless

* Enable Gemma 4 Assistant with FP8 by storing and fetching scale ptrs

* Update comment wording

* Move E2E FP8 KV cache tests to test-llama-archs

They belong there as they are not just a test for ggml's backend ops

* Increase support surface, remove seed

* make scaled kv generic

* Fix CI failings for test-kv-cache

https://github.com/ggml-org/llama.cpp/actions/runs/38718763447/job/116367842578?pr=28392

* CI: execute kv-cache-scale tests

* exclude fp8-kv tests from OpenVINO

* Fix linting errors of test_kv_cache_scales

* Disable kv-ffload=false + multi-GPU tests for FP8

See https://github.com/ggml-org/llama.cpp/actions/runs/38952722084/job/117148353699?pr=28413

* Adjust test label for test-kv-cache.cpp

* Fix conflict-resolution mistakes in test-kv-cache.cpp

https://github.com/ggml-org/llama.cpp/actions/runs/40554674463/job/122367426461?pr=28392

* Move KV tests to test-fp8.cpp

https://github.com/ggml-org/llama.cpp/pull/27512#discussion_r4410518158

* Remove tmp file for test-kv-cache-scales.py on Windows as well

* Fix fp8 kv cache test by population kv_source_impl

* Exclude models that consume KV cache directly

* Remove obsolete comment

* Allow BF16 scales as well for KV cache

* Rename *_cache_scale to *_cache_s

https://github.com/ggml-org/llama.cpp/pull/28392#discussion_r4517194999

* Move `is_type_scaled()` to `llama-kv-cache.cpp`
2026-11-10 18:08:32 +02:00
Asahi-PrvandNeo Zhang d5bbcd1f31 ci: update oneAPI 2026.2, oneDNN 2026.1 and Level Zero SDK 1.34.0 (#31540)
Co-authored-by: Neo Zhang <zhang.jianyu@outlook.com>
2026-11-10 01:18:44 -05:00
nv-vankitandPascal 359dc5858e server : functionality test for new spec dec (#31027)
* server : functionality test to make sure every time when we add new spec dec method the existing ones are not broken.

* server: use compact models for speculative decoding CI

* ci: avoid GHA cache for speculative models

* ci: run speculative tests on DGX Spark

* server: address speculative test review

* server: use consolidated speculative models

* server: refine speculative test calibration

* docs: record speculative Spark validation

* ci : run the server-speculative steps with bash

* server: recalibrate speculative test floors on GB10

---------

Co-authored-by: Pascal <admin@serveurperso.com>
2026-11-09 19:08:20 +01:00
Iswarya AlexandDavid Hekimian fc0be86182 ci : Run gfx1151 ROCm job on Strix Halo Devlab runners (#31567)
* fix(rocm,ci): Update ci workflow to use devlab ci runners for gfx1151

* ci : move gpu-rocm job to Strix Halo devlab runners

Assisted-by: Claude

* ci : ROCm 10 wheels with per-GPU device packages for gpu-rocm; add gfx1201

The runners' system ROCm 7.2.1 cannot compile the current HIP code for
gfx1151 ("Invalid dpp_ctrl value" in mmf-instance-ncols_*.cu). Go back
to the TheRock ROCm 10 wheels with the device-<gfx> extra, so rocBLAS
has the target's kernels, and keep ci/run.sh's results and models under
$RUNNER_TEMP instead of the runners' RAM-backed home. Add a gfx1201
(Radeon AI PRO R9700) row with HIP graphs disabled.

Assisted-by: Claude

---------

Co-authored-by: David Hekimian <dhekimian@users.noreply.github.com>
2026-11-09 13:01:52 +01:00
Aaron Teo 2277e75297 ci: move runners to Ubuntu 26.04 (#31529)
* ci: update to ubuntu26.04

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: bring back toolchain workaround

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: remove workaround completely

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: more switching to ubuntu 26.04

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: pin android cmake version

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: restore ubuntu 24.04 for s390x

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: fix missing s390x ubuntu 24.04 release link

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: switch release-publish to ubuntu-latest

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* devops: revert --break-system-packages

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: bring back ubuntu 24.04 for arm release

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: revert ubuntu 26.04 from release.yml

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: fix broken links

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: fix openvino job naming

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* devops: revert vulkan dockerfile to use system python3

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-11-08 11:47:29 +02:00
Georgi Gerganov 9a1d03c7ca tests : override hardware_concurrency in CI (#31797)
The self-hosted CPU runners are containers with 8 CPUs, but
std::thread::hardware_concurrency reports the cores of the host. The
table mode of test-llama-context sizes its pool from that count, so it
runs 4 models x 8 threads on an 8-CPU job and hangs.

Add TEST_ARG_HARDWARE_CONCURRENCY and TEST_ARG_PARALLEL_WORKERS as the
defaults for the auto-selected counts of test-llama-context and of
test-backend-ops, and set them to 8 and 4 for cpu-x64-high-perf.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)
2026-11-06 14:52:26 +02:00
Samriddha Sinha 3ed7284f2d ci : bump CUDA 13 release builds to 13.5 (#31788)
* ci : bump CUDA 13 release builds to 13.5

* docker : bump cuda13 images to 13.5.0
2026-11-06 09:52:47 +01:00
Eve 186eab1cb8 ci: fix build-cmake-pkg ccache (#31705) 2026-11-05 21:16:56 +00:00
Xuan-Son Nguyen 6023692b1e llama: fix kv shifting, add tests, add per-model build_rope (#31683)
* llama: fix kv shifting and add tests

* llama: add graph build_rope (#31693)

* llama: add graph build_rope

* improve

* migrate new models

* update review skill

* fix ci

* add --kv-shift to CI
2026-11-05 21:10:36 +01:00
Eve 3de52ac388 ci: run self hosted cpu jobs on hf or github (#31439) 2026-11-04 22:34:01 +00:00
Aleksander Grygier 0943ca5472 ui: add Playwright harness for recording demo videos (#31318)
* ui: add Playwright harness for recording demo videos

Scripted walkthroughs under tools/ui/demo are recorded with Playwright and
exported as H.264 mp4 for pull requests (demo/pr) and docs (demo/docs).
Fixture MCP servers and seeded settings keep recordings deterministic, a
drawn cursor and captions make the flow readable, and DEMO_UI_DIR points
the recording at another checkout such as a PR worktree.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record demo scripts from PR descriptions and a configurable storage

Demo scripts no longer live in the repo. `npm run demo:pr -- <number>`
records the `ts demo` block from a PR description against the PR head,
uploads the mp4, a GIF preview and the script to DEMO_STORAGE, and links
the preview in a `## Demo` section of the PR description.
`npm run demo:docs` records the docs scripts kept in DEMO_STORAGE.
DEMO_STORAGE (hf:// buckets or file://) and the public URL are read
from tools/ui/.env.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: import generated conversations into demo recordings

Demos can now start with existing chats: `test.use({ traces })` imports
llama.app JSONL traces through Settings > Import/Export before the video
starts. Built-in traces are generated deterministically, other names are
downloaded from DEMO_TRACES_DATASET, and `demo:traces` pushes the built-in
ones there. Adds `demo.scroll` for wheel scrolling.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep demo captions clear of the chat form and isolate concurrent runs

Captions move to the top of the frame, since at the bottom they covered
the chat form a demo is often about. Each recording run gets its own
Playwright output folder, so a run starting elsewhere no longer clears
another run's video before it is saved.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record demos at 2x and 60 fps with human-like timing

Recording goes through Chromium's screencast instead of Playwright's
video, which encoded VP8 at about 1 Mbit/s at 1x. Frames are captured at
the device pixel ratio (DEMO_SCALE, default 2) and encoded into a 60 fps
H.264 mp4, starting once the page has settled.

The cursor now glides faster along a slight arc with Fitts-like
durations, settles briefly before each click, and typing and pauses vary
a little. The randomness is seeded per demo, so re-recording gives the
same motion.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: document the demo harness prerequisites

The demo README now lists the tools the harness needs and what each is
for, and walks through creating the Hugging Face bucket and the optional
traces dataset, with the matching .env settings.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: version linked demo URLs so re-recorded previews refresh

GitHub's image proxy caches a preview by URL, so a demo re-recorded at
the same PR head kept showing the old GIF. The Demo section now links
the files with a content hash in the query string.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: label PRs that feature a video demo

Adds a `video demo` label and a workflow that applies it when a PR
description has a demo written by the UI demo pipeline (the managed
demo:start/demo:end markers, or a preview linking to a demo video),
and removes it when that section is gone.

Assisted-by: Claude (Opus 5.5 Medium)

* ci: label video demos with gh instead of github-script

The label workflow now runs the gh CLI in a shell step, like
pr-draft-label.yml, so it needs no action outside the repository's
allowed list. The video link check only counts videos uploaded for the
PR itself, so a description linking other PRs' demos isn't labeled.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: zoom demos in on details with a two-finger pinch

demo.zoom pinches in on a control or a point, holds, and pinches back out
to 1x, so a detail too small to read in the frame can be shown up close.
It drives Chromium's own pinch gesture, so the page's fixed bars stay put,
and works at desktop and phone sizes alike.

The pinch is evened out where Chromium misses: one anchored near the top
does not zoom at all, so it starts lower, and one near the bottom
overshoots its ratio, so a small pinch from the middle corrects it. The
way out reads the real scale until the page is back at 1x, which the next
gesture needs.

The overlay also draws a fingertip per touch, so a pinch on a touch
device shows as two fingers.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: wait out a cold dev server before a demo starts

A fresh checkout's dev server optimizes its dependencies on the first
load and reloads the page, which can take well past an action's timeout,
so a demo failed before it started. The demo now waits up to 90 seconds
for the app; the recording only starts once it is ready.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record a video per test in a PR demo, shown in a table

A large PR can split its demo into several shorter videos: each test()
in the ts demo block is recorded as its own video (<sha>-1.mp4, ...),
and the Demo section lays their previews out in a table, each column
headed by its test's title.

GIF previews now keep their longer side at DEMO_PREVIEW_WIDTH and shrink
until they are under 5 MB, since GitHub's image proxy refuses anything
larger and shows a broken image instead.

Importing traces works on a phone viewport too, where Settings sits in
the sidebar under the page until the menu key slides it aside.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: fix demo trace imports on desktop and lay landscape demos out two to a row

Importing traces failed on a desktop whose collapsed sidebar sits under
the page: it now expands the sidebar from the page's menu key on a phone
or from the sidebar's logo on a desktop, whichever makes Settings
reachable. Afterwards it forgets the expanded sidebar, so a demo starts
with it collapsed, as on a first visit.

A PR demo with several videos shows up to four portrait (phone) videos
to a row, or two landscape ones, so each preview stays readable.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: only record other people's PR demos at a reviewed head commit

demo:pr runs the PR's code and demo script on this machine, with the
gh and hf logins, so a PR by someone else now needs --trust <sha>
naming the head commit that was read, and is refused again after new
pushes. Recordings also launch Chromium with its sandbox on, which
Playwright turns off by default.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-03 12:30:49 +01:00
Sigbjørn Skjæret 27b8464e82 ci : add missing ccache-clear to windows behavior check (#31413) 2026-10-30 11:34:25 +01:00
Daniel Bevenius e216f4ec87 ci : extract most apple jobs into daily workflow (#31408)
This commit extracts all apple jobs except for macos-latest-arm64 into a
new daily workflow.

The motivation for this is to reducd the load on the macos runners.
2026-10-30 10:01:28 +01:00
Oliver SimonsandSigbjørn Skjæret 68d20b884c Quant: OCP FP8 E4M3 support (#27512)
* WIP OCP FP8 E4M3 support

* Fix python type-checker

* Materialize input_scales also for FP8 when converting

* Fix conversion of https://huggingface.co/RedHatAI/Qwen3.8-27B-NVFP4

1. Per row FP8 wrongly fell to NVFP4 conversion, as NVFP4 checked on
   dims alone. Also check on dtype for FP8
2. Need to reshape FP8 QKV projections scales in the same way that
   weights are reshaped

* Allow remaining FP8 tensors to be dequantized during conversion

* Allow tensor-strategy for FP8-only compressed-tensor ckpts

* Fix and test gguf-py behavior for FP8_E4M3

* Disallow QKV-fusion for FP8 temporarily

* Fix conversion of fused qkvz & FP8 for Qwen3-Next

* Avoid fusing FP8 for --fuse-gate-up-exps as we may have different ws

* Emit warnings when not fusing FP8 during conversion

* More FP8 conversion fixes

* Add supported for strided copies to/from FP8 in CPU

* remove unneeded scratch buffer from FP8 cpu cpy/dup path

* add explanatory comments and TODOs

* Add dedicated encoder and decoder tests for FP8

* Whitelist ggml_op_get_rows in metal instead of blacklisting

required to fix failing CI, and white-listing should be more
future-proof than adding FP8 to the black-list

https://github.com/ggml-org/llama.cpp/actions/runs/38433916498/job/115422880633?pr=27512

* Fix more conversion bugs

* Defer weight materialization during conversion

* Blacklist fp8 in metal backend instead of white-list

* Adjust indents

* Fix lint + execute tests

* Add fast CPU kernel for F8_E4M3

* Conversion fixes

* Fix remaining gaps

* Enable tied weights with shared scales

* Make mandatory-F32 rules apply to FP8 during conversion

* Fix python type checks

https://github.com/ggml-org/llama.cpp/actions/runs/39765670052/job/119780379595?pr=27512

* Fix kaleidai unit-test failures

https://github.com/ggml-org/llama.cpp/actions/runs/39765670125/job/119780383150

* Relax numeric precision constraints for pre-check

* Leave fast-FP8 CPU on RISC-V for follow-up

We already exclude arm from the fast-path

* Allow output scale tensors to be located in any gguf shard

* Move test_conversion_fp8.py test execution to behavior-check.yaml

* Apply batched suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Move mixed compressed-tensor logic into dequant_fp8 + add unit-tests for it

* Install all requirements also on Windows for behavior-check

Fixes https://github.com/ggml-org/llama.cpp/actions/runs/40535852590/job/122300758223?pr=27512

* Unmap lazy tensors before clean-up in Windows

Should fix https://github.com/ggml-org/llama.cpp/actions/runs/40539797824/job/122314782459?pr=27512

* LF line endings

https://github.com/ggml-org/llama.cpp/actions/runs/40539798168/job/122314791118?pr=27512

* Extract FP8 tests from llama-arch to its own test-fixture

See https://github.com/ggml-org/llama.cpp/pull/27512#discussion_r4400628221

* Update gguf-py/gguf/scripts/gguf_convert_endian.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-29 18:14:37 +01:00
Aaron Teo ff0d396e3d ci(s390x): switch to Ubuntu 26.04 runners (#31317) 2026-10-28 23:32:38 +08:00
Sreeram fae9ac561f ci : use ccache depend mode for ROCm release builds (#31231) 2026-10-27 09:24:00 +01:00
Sigbjørn Skjæret a8276b70cc ci : force python 3.12 on ubuntu 26.04 (#31222) 2026-10-27 09:09:44 +01:00
Sigbjørn Skjæret 4244ad7acd ci : add windows behavior check (#31191) 2026-10-26 23:01:04 +01:00
Georgi Gerganov 48c36110ab ci : drop the macos-latest-x64 job (#31175) 2026-10-26 09:56:19 +02:00
Georgi Gerganov 344be0a600 ci : update CPU job assignments (#31136) 2026-10-25 10:35:11 +02:00
Georgi Gerganov 71255fefc0 ci : pass per-runner results dir to self-hosted jobs on T4 runners (#31100)
The two T4 runners in the ggml-ci group (ggml-6-x86-vulkan-t4,
ggml-8-x86-nvidia-a10) share the same labels and can both pick up the
same jobs, so extend the per-runner results dir from #31046 to them.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
2026-10-24 17:17:04 +03:00
Georgi Gerganov 4d32b29aaa ci : exclude test-speculative-rejection-sampling from macos-latest-arm64 (#31088)
Add test-speculative-rejection-sampling to the ctest -E exclusion list of the macos-latest-arm64 CI job.

Assisted-by: dsh:llama.cpp/ggml-org/Flint-GGUF (RTX 5090 32GB, DDR5 128GB)
2026-10-24 14:14:31 +03:00
Georgi Gerganov 6b9ba4f2e8 ci : fix data race between runners sharing the model cache (#31046)
* ci : fix data race between runners sharing the model cache

Two runners per mac mini share HOME, so concurrent jobs corrupt each
other downloads, converted models and venv under the shared MNT cache.

- gg_wget: download in a temp dir, publish with mv, so readers only
  ever see complete files (-N cannot be combined with -O)
- convert, quantize and wikitext unzip now produce under a per-runner
  scratch dir in OUT, published to MNT via hardlink when missing
- gguf-split and quantize tests get a per-runner TMP_DIR
- python venv moved from MNT to OUT
- metal, vulkan and webgpu mac jobs pass a per-runner OUT path

Assisted-by: dsh:llama.cpp/agent-qwen4 (RTX 5090 machine)

* ci : clean up scratch models after tests

Assisted-by: dsh:llama.cpp/agent-qwen4 (RTX 5090 machine)

* ci : fix q4_0 publish when a cached model exists

cp -n skips an existing file with exit code 1, so the ln fallback
aborted the qwen3 job under set -e. Publish through a unique temp
file with an atomic mv: readers never see a partial file and a
stale cached q4_0 gets replaced.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 machine)

* ci : cache-first gg_wget to avoid re-downloads

HF sends no Last-Modified, so wget -N re-downloads the full files on
every run. Skip the download when a non-empty cache file exists: the
HF CI artifacts are static. Downloads still go through a temp dir and
an atomic mv, so concurrent runners never read partial files.

Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 machine)
2026-10-24 09:21:24 +03:00
Sigbjørn Skjæret f00c9083b2 ci : add behavior check workflow (#31036)
* remove python from tests

* add behavior and backend ctest labels

* add behavior check workflow

[no ci]
2026-10-23 14:53:44 +02:00
Sigbjørn Skjæret b38960f520 ci : remove superfluous webgpu jobs (#30868) 2026-10-20 14:27:55 +02:00
Max Krasnyansky 8466395353 hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (#30854)
* hex-build: bump toolchain, enable v85

* hex-build: bump toolchain to v0.8 in the CI and build scripts

* hex-build: add a little python script for swiv needed on some iot devices

* hex-build: fix flake8 warnings

* hex-arch: update version cap to v85

* hex-inspect: flexible TOOLS_ROOT

* hex-build: update to snake_case naming

* hex-build: update leftover old toolchain refs

* hex-target: update build and run scripts to support ubuntu with adb transport
2026-10-19 23:49:59 -07:00
Mendy Berger 921257f500 ci, docs: add WASI CI and docs (#30754)
* ci, docs: add WASI CI and docs

* accept review feedback
2026-10-20 01:09:55 +02:00
7adf160c41 server: add --connect, allow p2p access to server outside LAN (#28277)
* init

* wip

* wire it up

* make it clear that it's experimental

* add props.connect_code

* nits

* fix windows build

* server: do not advertise a connect code without the tunnel

--connect-code on its own had no effect other than being served in
/props, so the Web UI showed a remote access code that no tunnel was
serving. Clear it when --connect is off, and say so once at startup.

The teardown comment claimed that terminate() closes the child stdout.
It does not: the log thread leaves fgets() only once every writer of
that pipe is gone, so a descendant of llama-connect would block the
join forever. Describe the real contract instead.

Also restore the --reasoning-preserve default in the generated docs.

* Update tools/connect/CMakeLists.txt

Co-authored-by: Adrien Gallouët <angt@huggingface.co>

* server: dial llama-connect on the bound port and a reachable host

Pass ctx_http.port so --port 0 and the GCP override work, pick the
dial host from --host (wildcards map to loopback, IPv6 is bracketed)
and refuse --connect when only unix sockets are bound.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
2026-10-20 00:23:14 +02:00
Aaron Teo fda00f18cd tests: add big-endian conversion tests (#30705)
* ci: add quantize to big-endian tests

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ci: remove incoherent data types from testing

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: add verified hashes

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: rename

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: add result table

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: swap hash and result column, add colours

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: switch to perplexity test

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* tests: update output table

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-10-19 10:14:58 +02:00
Sandro Hanea 83f9389419 ci : added ABI reports as github artifact (#30619) 2026-10-17 11:41:53 +02:00
Zack Li 6420cf5dd5 ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620)
* ci: update device matrix for Snapdragon jobs by removing SM8750

* ci: increase timeout for QDC device tests and update job timeout parameters
2026-10-16 16:25:05 -07:00
Neo ZhangandSigbjørn Skjæret 43d57fb8ca [SYCL] Enable ssl in build script and CI (#30456)
* enable OPENSSL=ON as default in build script, update it in CI and dockerfile

* change to LLAMA_BUILD_BORINGSSL on windows

* refactor

* rm Download UI build

* Apply suggestion from @CISC

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Apply suggestion from @CISC

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Apply suggestion from @CISC

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-15 21:58:07 -04:00