Llama Chat compiles @ggml-org/llama-voice from ui/lib/voice/src, but the
CMake stage and the Docker UI stages copied only ui/lib/ui, so their npm
build could not resolve the voice imports. Stage ui/lib/voice/src next to
Llama UI and track it as an input of the CMake build.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: bring @ggml-org/llama-ui into ui/lib/ui
The design system package (tokens, fonts, surfaces, components and Model UI), so far in the temporary ggml-org/llama-ui
repository at edaf518, moves to its home in ui/lib/ui, built with @sveltejs/package and published as @ggml-org/llama-ui
from here. ui/lib holds the packages, one folder each. Its references to tools/ui now name ui/apps/chat, its
package.json points at this repository, and the UI workflow type-checks and packs it. ui/README.md describes the
layout.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: Llama UI design system: tokens, surfaces and brand
The design system from allozaur/docs/storybook-overhaul: three tiers of
tokens, matte glass surfaces, Llama Sans and Llama Mono, and the base
components on the new tokens (Tabs and GlassKey among them). It also
restyles the stack's features with it, replacing the plain stand-ins they
used until now. DESIGN.md is generated from the tokens. The Storybook that
documents it is the next PR.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: have Llama Chat use @ggml-org/llama-ui for its base components, tokens and fonts
Llama Chat kept its own copy of the design system: 29 base components, the
three tiers of tokens, the surfaces, the theme and the fonts, next to the 12
components ui/lib/ui held. The copy goes, and the library gets what it lacked.
ui/lib/ui
- the 17 base components the app had and the library did not (alert,
alert-dialog, button-group, checkbox, collapsible, context-menu, dialog,
drawer, dropdown-menu, empty, hover-card, radio-group, scroll-area,
skeleton, table, tabs, textarea), with vaul-svelte for the drawer
- the inset-group and selected-row tokens and surfaces
- sources import each other by relative path instead of $lib, which in an
app points at the app's own src/lib
- theme.css no longer imports tw-animate-css: it is a peer dependency that
the consumer imports before the theme, as it does Tailwind
- 0.3.0
ui/apps/chat
- imports the base components as @ggml-org/llama-ui/ui/<name> and cn/tv as
@ggml-org/llama-ui/utils; app.css imports the library's theme.css
- compiles the library from source, with no install or build of it:
scripts/vite-plugin-llama-ui.ts resolves its entry points and what it
imports, and svelte.config.js gives TypeScript the same paths
- src/lib/components/ui, styles/tokens, styles/surfaces.css and assets/fonts
are removed; components.json moves to the library
Builds
- CMake stages ui/lib/ui next to the app, keeping the layout of ui/, and
rebuilds when a library source changes
- the Dockerfiles copy the library's sources; the Nix derivation takes ui/
as its source
The built CSS has the same rules in the same order, apart from one
zero-specificity rule and the @font-face rules moving, plus ten utilities
and two tokens that only the library's own components use.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: AGENTS.md places Llama UI in ui/lib/ui
Llama Chat's AGENTS.md says where each part lives: Llama UI is
ui/lib/ui, and Chat imports its tokens, fonts and base components
from there.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: Llama Sans and Llama Mono from llama-brand's latest, and shell commands marked in bash code
The fonts in ui/lib/ui come from llama-brand's latest, with optical sizes
from 10 to 36. Bash code blocks mark shell commands and flags.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: open submenus outside the menu they come from
A submenu was rendered inside its parent menu, so the menu's scroll box
clipped it: the glass menus' backdrop filter, and the filter and transform
of the open animation, make the menu the containing block even for the
fixed-position submenu. Dropdown and context submenus now render in a
portal, like the menus themselves.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: draw submenus like the menu they open from
Submenus had smaller corners than their menus, and context menus had none
of the dropdown's glass. Context-menu parts now match the dropdown's, and
both submenus take the menu's corners and height cap, with their first row
level with the row that opens them, 4px past the menu's edge. Sub-triggers
take the items' radius and pointer.
The submenus' height cap and zoom origin named the dropdown's CSS variables,
which bits-ui does not set on submenus; they now use its --bits-menu-* ones.
The chat group submenu takes the actions menu's z-[999999] layer, as it no
longer renders inside that menu.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: keep a late current-node write off a closed or switched conversation
updateCurrentNode checked the active conversation, awaited the database write, then wrote currNode to whatever was active after it. When the conversation closed during the write it threw, and the reply ended in a streaming error dialog. When another conversation opened, that one got the wrong node.
Assisted-by: Claude (Fable 5.2)
* ui: have Llama Chat use the library's Model UI and app components
Llama Chat still kept its own copies of what the library holds above the
base components. It now takes them from @ggml-org/llama-ui too, and its
copies go.
ui/lib/ui
- ModelId gains what Chat's had: the model's own draft kind, draft
sidecars with their quant (ModelDraftSidecars, new), aliases, hideName
and a tooltip, in the same badge order
- entry points for them, like ui/<name>: @ggml-org/llama-ui/app/<group>
(actions, content, forms, misc) and @ggml-org/llama-ui/models
ui/apps/chat
- ActionIcon, SearchInput, SyntaxHighlightedCode, Logo,
ModelCapabilityIcons, ModelDownloadProgressBar and ModelDraftSidecars
come from the library; Chat's copies and the two constants only they
used are removed
- ModelId and ModelOrgAvatar stay as thin wrappers: ModelId parses the raw
id and applies the user's display settings, ModelOrgAvatar hides the
avatar while Hub metadata is off
- the svelte/* TypeScript path falls back to Svelte's types/index.d.ts:
svelte/reactivity has no file of its own, so a library source importing
it got its types from ui/lib/ui's own install when there was one
Assisted-by: Claude (Opus 5.5 Medium)
* ui: Llama UI takes the brand's colours and logo from llama-brand
@ggml-org/llama-brand is now a dependency of the library, by the same GitHub
tarball Llama Chat already installs, which now names it @ggml-org/llama-brand
too.
- primitives.css drops the 25 tokens it held copies of (the neutral
greys, sand, the skies and cinder) and builds on llama-brand's
css/colors.css instead; the skies' gradients point at llama-brand's.
Apps import that file before theme.css, as they do Tailwind and
tw-animate-css: the library's stylesheets import no package, so an app
that compiles them from source resolves it from its own node_modules
- Logo draws llama-brand's icon, in currentColor, and assets/logo.svg goes
- Llama Chat's TypeScript paths resolve llama-brand from its own
node_modules, like the library's other dependencies
The fonts stay as the library's WOFF2 copies until llama-brand ships
WOFF2 builds of them.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: have Llama Chat take the library's constants, enums and helpers
Llama Chat redefined most of what the library exports next to its
components. Where the two were the same, Chat now re-exports the library's
from the files that held them, so its own imports stay as they are.
- the library gets entry points for them: @ggml-org/llama-ui/constants,
@ggml-org/llama-ui/enums and @ggml-org/llama-ui/code, and
@ggml-org/llama-ui/models carries the model data with the Model UI
- from the library now: the shared CSS classes and code-block constants,
the byte units, the memory-fit constants and minMemoryTierGb, the bit
depth labels, the capability and modality icons, labels and order,
ColorMode, TooltipSide, BooleanString, the model enums, ModelModalities,
formatParameters, highlightCode and trimCodePadding
- the library's modality order follows Chat's: vision, video, audio
- Chat keeps its own where they differ: UI_DATA_ATTRS and
ModelDownloadProgress are supersets of the library's (the latter extends
it now), and formatFileSize counts in 1024s
Assisted-by: Claude (Opus 5.5 Medium)
* ui: give the chat form's add key and model selector a tighter shadow
In light, the two keys inside the chat form, white on its white glass, took
the control elevation, whose soft spread blended into the form. They take a
new elevation-key instead: a 0.5px hairline all round and a 1px drop. In
dark they keep the control elevation. Chosen from the options in
Scrapbook › UI › Composer keys.
- semantic tokens: --elevation-key (light: the hairline and the drop; dark:
the control elevation)
- component token: --chat-form-key-shadow, for the add key and every model
selector trigger
- DESIGN.md names it next to the other elevations
Assisted-by: Claude (Opus 5.5 Medium)
* ui: move tools/ui to ui/apps/chat
The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name Llama Chat's package @ggml-org/llama-chat
It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: call the built-in web UI Llama Chat in docs, help text and comments
The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.
Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name tools/ui as a former path of Llama Chat in the code-review skill
Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.
Assisted-by: Claude (Opus 5.5 Medium)
* ui: name Llama Chat's path in the security policy's exclusions
Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.
Assisted-by: Claude (Opus 5.5 Medium)
* enable OPENSSL=ON as default in build script, update it in CI and dockerfile
* change to LLAMA_BUILD_BORINGSSL on windows
* refactor
* rm Download UI build
* Apply suggestion from @CISC
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Apply suggestion from @CISC
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* Apply suggestion from @CISC
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
* replace oneAPI package by DLE+oneDNN package
* support to install oneDNN in windows, in release and docker
* for internal test
* remove test code
* Update intel.Dockerfile
* ci : update the oneAPI toolkit to 2026.1
oneDNN is removed from Intel Deep Learning Essentials in 2026.0, so
staying on the deep-learning-essentials path would silently lose oneDNN
support when the toolkit version is updated. Switch both the Ubuntu and
Windows CI jobs to the new unified Intel oneAPI Toolkit installer,
which still includes oneDNN (until 2027.0) and keeps the component IDs
unchanged for the Windows install script.
Measured with the same code (b10899) built with oneAPI 2026.1 vs the
2025.3-based release build on Arc B570: prompt processing 1331 vs 434
t/s (3.1x), token generation 50.1 vs 45.3-48.0 t/s.
Assisted-by: GLM (z-ai/glm-5.3-flash)
* docs : update the SYCL backend build requirements for oneAPI 2026.1
With the 2026.0 release the Base toolkit and the HPC toolkit are
combined into the oneAPI Toolkit, and oneDNN is removed from the Deep
Learning Essentials package. Update the install instructions, the
verified release table and the news section accordingly.
Assisted-by: GLM (z-ai/glm-5.3-flash)
* ci : update the release workflow for oneAPI 2026.1 and Level Zero SDK 1.33.1
Align the release package build with the CI build update:
- oneAPI toolkit 2025.3.3 -> 2026.1 (the unified oneAPI Toolkit)
- Level Zero SDK 1.28.2 -> 1.33.1, and the Debian package names
(level-zero/level-zero-devel -> libze1/libze-dev)
- The Windows DLL copy list for the 2026.1 runtime: sycl9.dll and the
.6/.3 MKL library versions
Assisted-by: GLM (z-ai/glm-5.3-flash)
* ci : remove the removed .spv fallback files from the Windows DLL copy list
oneAPI 2026.1 no longer ships libsycl-fallback-bfloat16.spv and
libsycl-native-bfloat16.spv (the OpenCL fallback mechanism changed), so
the copy step failed with exit 1.
Assisted-by: GLM (z-ai/glm-5.3-flash)
* devops : update the oneAPI toolkit image in the Intel Dockerfile
Assisted-by: GLM (z-ai/glm-5.3-flash)
---------
Co-authored-by: Asahi-Prv <Asahi-Prv@users.noreply.github.com>
* docker: add OCI image labels to all published images
* docker: propagate OCI labels as manifest and index annotations
* docker: drop hardcoded org URL and revert accidental intel version bump
The OCI image url and source are now driven by build args with a sensible default. The workflow passes the actual repository url so fork builds get labels pointing at the fork instead of upstream. Also restores the IGC, compute runtime, and IGDGMM versions in the intel Dockerfile labeled stage which I accidentally bumped in the first commit.
* docker: add skip_s390x workflow_dispatch input for fast test runs
Lets maintainers and PR authors trigger the docker workflow without the s390x build target, which depends on the IBM Z runner and is by far the slowest job in the matrix. The flag filters the s390x row out of the build matrix before merge_matrix is derived, so the merge job sees a consistent shape too.
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
---------
Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
* SYCL: fix multi-GPU system RAM exhaustion by using Level Zero allocations
Replace sycl::malloc_device with zeMemAllocDevice for GPU memory allocation
in the SYCL backend. sycl::malloc_device triggers the xe kernel driver's
DMA-buf/TTM path which mirrors every VRAM allocation 1:1 in system RAM.
zeMemAllocDevice uses the SVM/P2P path with no host staging.
On a dual Intel Arc Pro B70 system (64GB VRAM, 64GB RAM), a 15.6 GiB model
consumed 60 GiB of system RAM via sycl::malloc_device, causing OOM crashes.
With zeMemAllocDevice, the same workload uses ~6.7 GiB of system RAM with
no performance regression.
All Level Zero calls include automatic fallback to the original SYCL
allocation path if Level Zero interop is unavailable.
* SYCL: address review feedback - remove try/catch, check device types, deduplicate
- Remove try/catch from malloc/free/memcpy helpers, check backend and
device type upfront instead (ggml_sycl_is_level_zero, ggml_sycl_is_dgpu)
- Move shared helpers (is_level_zero, is_dgpu, free_device) to common.cpp
and declare in common.hpp to eliminate code duplication
- Use SYCL_CHECK(CHECK_TRY_ERROR()) for fallback sycl::free calls
- Guard dev2dev_memcpy L0 path to dGPU-to-dGPU only, preserving the
host-staged path for iGPU-to-dGPU transfers
- Add Windows Level Zero SDK path detection (LEVEL_ZERO_V1_SDK_PATH)
in CMakeLists.txt (co-authored with @arthw)
* SYCL: add build/runtime flags for Level Zero, address review feedback
Implements the architecture suggested by @arthw: compile-time and runtime
flags to cleanly separate Level Zero and SYCL memory API paths.
- Add GGML_SYCL_SUPPORT_LEVEL_ZERO cmake option (default ON). All Level
Zero code is wrapped in #ifdef so the build works on systems without
the Level Zero SDK installed (e.g. CPU-only CI servers). Both the
loader library and headers are checked before enabling.
- Add GGML_SYCL_ENABLE_LEVEL_ZERO runtime env var (default 1). Controls
whether Level Zero or SYCL memory APIs are used. Only one API style is
used per session, no mixing. If Level Zero is enabled but the devices
don't support the Level Zero backend, it auto-disables with a warning.
- Remove Level Zero code from dpct_malloc. It was unused (dpct::device_memory
is not called anywhere in the backend) and used try/catch for flow control.
- Update SYCL.md with documentation for both new parameters.
Tested on Intel Arc Pro B70 (32GB), single-GPU and dual-GPU, with both
GGML_SYCL_SUPPORT_LEVEL_ZERO=ON and OFF builds. AI-assisted development
(Claude). Code reviewed and tested on my hardware.
* SYCL: unify Level Zero malloc/free call sites, address review feedback
Move ggml_sycl_malloc_device to common.cpp alongside ggml_sycl_free_device.
Both functions are now unconditionally available — Level Zero code is
#ifdef'd inside the functions, not at call sites. All call sites use
uniform SYCL_CHECK(CHECK_TRY_ERROR()) wrapping with no #ifdef blocks.
Addresses arthw's review: wrap all malloc/free in SYCL_CHECK for stack
traces on failure, eliminate duplicated #ifdef/else patterns at 6 call
sites (-29 lines net).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* SYCL: add Level Zero SDK to CI, fix device check and missed alloc paths
Add Level Zero SDK installation to Ubuntu and Windows SYCL CI jobs
so the Level Zero code path is compiled and tested in CI.
Fix two bugs found during extended dual-GPU testing (no
ONEAPI_DEVICE_SELECTOR set):
- The Level Zero backend check was iterating all SYCL devices
including CPU. The OpenCL CPU device caused Level Zero to be
disabled for the GPUs, defeating the fix on multi-GPU systems.
Added is_gpu() filter so only GPU devices are checked.
- sycl_ext_malloc_device/sycl_ext_free (tensor reorder temp buffers)
were still calling sycl::malloc/sycl::free directly, bypassing the
Level Zero path. Routed through ggml_sycl_malloc_device/free_device
for consistency with the other device memory call sites.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* SYCL: address arthw review feedback on Level Zero memory API structure
- Move ggml_sycl_malloc_device to static function in ggml-sycl.cpp;
only ggml_sycl_free_device (used by common.cpp) stays in common.cpp
- Switch both helpers to use g_ggml_sycl_enable_level_zero global
instead of per-call queue backend checks
- Remove #ifdef wrapper from global definition; always declare at 0,
add #else branch in init block so it stays 0 when L0 not compiled in
- Update init loop comment to explain GPU-only device check
- CMakeLists: message(STATUS) before the if block; align option wording
AI-assisted implementation. Reviewed and tested on dual Intel Arc Pro
B70 (32 GB each): test-backend-ops OK on both GPUs, single/dual-GPU
Q4_K_M and Q8_0 bench correct, zeMemAllocDevice GTT delta confirmed
<5 MiB per 4 GiB allocation (vs ~4 GiB shadow with sycl::malloc_device).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* SYCL: remove unused cstdio/cstdlib includes from common.cpp
Leftover from the deleted ggml_sycl_queue_supports_level_zero helper.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Apply suggestions from code review
Co-authored-by: Neo Zhang <zhang.jianyu@outlook.com>
* SYCL: preserve Level Zero allocation path during early malloc
* ci: fix Level Zero package conflict in Intel Docker build
* ci: find Level Zero loader in oneAPI package step
* ci: allow Windows SYCL package without Level Zero DLL
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Neo Zhang <zhang.jianyu@outlook.com>
* upgrade oneAPI to 2025.3.3
* update
* seperate SYCL CI and support release binary package for ubuntu 24
* add dependence
* remove wrong copy lines
* add missed line
* remove other task to test the release for SYCL
* rm more for test release
* fix file name
* correct the error in running
* support build for fp32/fp16
* rm ubuntu-24-sycl-fp16 for duplicated
* refactor build setting
* update guide for ubuntu 24 release package, restore the release.yml for other backend
* user docker replace to install oneAPI
* use download installation package to replace docker
* use wget to download and install oneapi, replace the apt cmd
* enable ccache for oneAPI installation
* fix format error
* enable cache for oneAPI installation
* update guide
* Update .github/workflows/release.yml
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update .github/workflows/release.yml
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update .github/workflows/build-sycl.yml
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update .github/workflows/release.yml
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* update oneapi to 2025.2, use deep-learning-essentials to replace base-tool
* update to 2025.2 use deeplearn essi to replace base toolkit
* add missed dll
* add deep learning essentials
* add sycl-ls
---------
Co-authored-by: Zhang Jianyu <zhang.jianyu@outlook.com>