Files
llama.cpp/ui/apps/chat
Aleksander Grygier 80325823d4 ui: open the context gauge panel on click as a popover (#32685)
* ui: open the context gauge panel on click as a popover

Assisted-by: Claude (Opus 5.5 Medium)

* ui: give the context gauge trigger the ghost button hover

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-21 13:21:47 +01:00
..

Llama Chat

Llama Chat is the chat app for llama.cpp: a web interface for llama-server, built with SvelteKit on Llama UI (@ggml-org/llama-ui), the design system in ui/lib/ui. llama-server embeds it, and npx llama-chat runs it anywhere.

It works with a llama-server in either mode: MODEL (one model) or ROUTER (many models, loaded and unloaded on demand).

Contents

Run it with npx

Llama Chat is published to npm as @ggml-org/llama-chat, with llama-chat as its short name. It serves the prebuilt app as static files on localhost, or writes them to a folder for any static file server (see Self-hosting); it brings no server and no models of its own. It needs Node.js 20 or later.

npx @ggml-org/llama-chat                 # serve it on http://127.0.0.1:3000 and open the browser
npx llama-chat --host 0.0.0.0 --no-open  # the same, by its short name, reachable from other devices
npx llama-chat build                     # write the static app to ./llama-chat instead
Flag Default What it does
--port <n> 3000 Fixed, so the origin, and the chats the browser stores under it, stay the same between runs
--host <addr> 127.0.0.1 0.0.0.0 to open it to the LAN
--open / --no-open open Opens the default browser once it listens
-v, --version, -h, --help As usual

The bare command is the documented form; start is accepted as an alias for it. build takes --out <dir> (default ./llama-chat) and --force, which lets it write into a folder that is not empty.

How it works: the CLI only serves the files of dist, with llama-server's headers for the UI (index.html cross-origin isolated, the PWA files revalidated, hashed assets cached for good), and forwards nothing. The browser talks to each llama-server you add in the app by URL directly, so llama-server's own rules apply: start it with --cors-origins http://127.0.0.1:3000 (the address you open) when it restricts origins, as it does with tools or MCP enabled, and with --api-key when it is reachable from the LAN. An http:// page like this one can reach a plain http:// llama-server on the LAN, which an https:// page cannot.

Self-hosting

The app is static and uses relative paths and a hash router, so any static host can serve it at any path. npx @ggml-org/llama-chat build --out <dir> writes the files (as does npm run build here, to dist); serve them with any static file server, for example npx http-server llama-chat -p 3000. Send the headers listed above with index.html and the PWA files, so the app behaves as it does under llama-server (see Cross-origin isolation).

The app reaches the llama-server on its own origin, if there is one (llama-server serves it with --path, or your reverse proxy puts the API next to the files), and any other llama-server you add by URL, which needs --cors-origins for the app's address.

Features

  • Chat - streaming replies, reasoning blocks, branching by edit or regenerate, tabs, search, import and export
  • Attachments - images, PDFs (as text or as pages for vision models), audio and text files, by drag and drop or paste
  • Rendering - Markdown (GFM), code highlighting, KaTeX math and Mermaid diagrams
  • Models - in ROUTER mode, a selector and a models manager: load and unload, downloads, Discover from the Hugging Face Hub, favorites and hidden models
  • Tools - built-in browser tools, llama-server's tools and MCP servers, each behind a permission prompt
  • Everywhere - light and dark themes, phone layouts, installable as a PWA, keyboard shortcuts (Ctrl/Cmd+K search, Shift+Ctrl/Cmd+O new chat, Ctrl/Cmd+B sidebar)

Development

You need Node.js 20 or later and a llama-server to talk to.

cd ui/apps/chat
npm ci
npm run dev     # the app on http://localhost:5173, Storybook on http://localhost:6006

The dev server forwards the API (/v1, /props, /models, /tools, /slots, /cors-proxy) to llama-server on http://localhost:9931, or to VITE_PUBLIC_SERVER_ORIGIN when it is set.

Command What it does
npm run build The production build, in dist/
npm run check Type checking with svelte-check
npm run lint Prettier and ESLint; npm run format fixes what they can
npm run test:unit Unit tests (Vitest, tests/unit)
npm run test:client Component tests in the browser (Vitest, tests/client)
npm run test:ui Storybook stories run as tests (tests/stories)
npm run test:e2e End-to-end tests (Playwright, tests/e2e), with a llama-server
npm run storybook Storybook on its own

Locally the browser tests run in your installed Google Chrome; CI uses Playwright's bundled Chromium.

Architecture

Llama Chat is the first app on Llama UI, and the reference for the line between library and app: Llama UI draws and handles interaction, the app decides what happens.

  • In Llama UI: components, design tokens, fonts and surfaces, and the logic inside them (keyboard navigation, validation as you type, formatting a file size). Components take data in and report what the user did out.
  • In Llama Chat: talking to llama-server, storing conversations, the agentic loop and its tool permissions, MCP servers, managing models, settings.

The app imports the library as @ggml-org/llama-ui/ui/<name> and compiles its sources in its own build, with no install of the library (see scripts/vite-plugin-llama-ui.ts). Data flows one way, through these layers:

Layer Lives in What it holds
Routes src/routes /, /chat/[id], /models, /mcp-servers, /settings, /search
App components src/lib/components/app Chat, navigation, models, MCP, settings and dialogs, grouped by area
UI components ui/lib/ui shadcn-svelte on bits-ui, from Llama UI
Voice ui/lib/voice Microphone capture, transcription and dictation, from Llama Voice (@ggml-org/llama-voice)
Hooks src/lib/hooks View logic between components and stores: scroll, drag and drop, pickers, keyboard
Stores src/lib/stores Application state as Svelte 5 runes: chat, conversations, the agentic loop, tools, models, settings
Services src/lib/services Stateless I/O: the llama-server API, IndexedDB, MCP, the Hugging Face Hub, the tool sandbox
Storage + API the browser, llama-server IndexedDB and localStorage; llama-server's /v1, /props, /models, /tools, /slots
flowchart LR
    Routes --> Components["App components"]
    Components --> Hooks
    Components --> Stores
    Hooks --> Stores
    Stores --> Services
    Services --> DB[("IndexedDB, localStorage")]
    Services --> API["llama-server"]
    Components -. "imports" .-> UI["Llama UI"]
    Stores -. "reactive state" .-> Components

Sending a message

sequenceDiagram
    participant User
    participant Stores
    participant DB as IndexedDB
    participant API as llama-server

    Note over Stores,API: on start
    Stores->>API: GET /props, then the model list
    User->>Stores: send a message
    opt ROUTER mode, model not loaded
        Stores->>API: POST /models/load, then poll GET /models
    end
    Stores->>DB: save the user message
    Stores->>API: POST /v1/chat/completions (stream)
    loop streaming
        API-->>Stores: SSE chunks, shown as they arrive
    end
    opt the model called tools
        Stores->>Stores: run them (after permission), send the results, stream again
    end
    Stores->>DB: save the assistant message

Patterns worth knowing

  • Stores hold state as Svelte 5 runes ($state, $derived); larger ones live in folders and compose focused sub-stores. Services are stateless and do all I/O.
  • Per-conversation state: streaming, loading and abort controllers are kept per conversation, so several chats can stream at once.
  • Messages form a tree: each message knows its parent and children, and a conversation points at the branch tip it shows, so editing or regenerating branches without losing history.
  • Server role: one codebase serves MODEL and ROUTER mode; serverStore reads the role from /props, and ROUTER-only UI shows only there.
  • Modalities: attachments a model cannot read (images for a text-only model, say) are refused before sending, from the model's /props.
  • Storage: conversations and messages in IndexedDB (Dexie), settings and model preferences in localStorage, server state and Hub details in memory only.

Cross-origin isolation

llama-server serves the UI with Cross-Origin-Embedder-Policy: require-corp and Cross-Origin-Opener-Policy: same-origin, so every cross-origin asset needs the other origin's consent. Hugging Face Hub avatars and metadata are requested in CORS mode, which the Hub allows. MCP server icons and search-result favicons go through llama-server's /cors-proxy when it runs with --ui-mcp-proxy, and fall back to a bundled icon otherwise.

Build and embedding

npm run build generates the PWA icons, bundles the app with Vite and writes it to dist/ (or to LLAMA_UI_OUT_DIR) as static files: index.html, one JS and one CSS bundle under _app/, the service worker, the web manifest and build.json with the version (LLAMA_BUILD_NUMBER, b0000 locally). Paths are relative and routing uses the URL hash, so dist works from any path.

llama-server embeds it: when llama-server is built, scripts/ui-assets.cmake takes ui/apps/chat/dist if it exists, runs npm run build if LLAMA_BUILD_UI is on, or downloads the release's prebuilt dist.tar.gz from Hugging Face, and compiles every file into the binary. The same dist is what npx @ggml-org/llama-chat serves.

ui/apps/chat/
├── src/
│   ├── lib/              # components, hooks, stores, services, types, utils, constants, enums
│   ├── routes/           # SvelteKit routes
│   └── service-worker/   # the PWA's service worker
├── bin/, cli/            # the npx llama-chat command and its server
├── alias/                # the llama-chat npm package, the short name
├── scripts/              # Vite plugins and build scripts
├── static/               # static assets
└── tests/                # unit, client and end-to-end tests, and the Storybook stories

Publishing to npm

Two packages go out with the same version: @ggml-org/llama-chat, which carries the app and the CLI, and llama-chat (in alias/), which depends on it and only forwards to its CLI so that npx llama-chat works. A unit test (tests/unit/cli.test.ts) fails when their versions drift apart.

The version is set by hand and CI publishes it, as for @ggml-org/llama-ui (see Releases). To release, bump the version in the PR that should ship it:

cd ui/apps/chat
npm version <version> --no-git-tag-version
# then set the same version, and the same @ggml-org/llama-chat dependency, in alias/package.json

Once that reaches master, ui-chat-publish.yml builds dist, checks that the package holds dist/index.html and no source maps, and publishes both packages with npm trusted publishing. A version npm already has publishes nothing, and a PR that changes the packages gets a dry run.

To try a package before it is published:

npm run build
npm pack
npx ./ggml-org-llama-chat-<version>.tgz

Build right before packing: the package carries whatever is in dist, and this folder's .npmrc sets ignore-scripts, so npm runs no lifecycle script such as prepack that could build it for you.