Files
llama.cpp/tools/ui/README.md
T
PascalandAleksander Grygier 1a97e387d3 ui: add conversation compaction (#27401)
* ui: add conversation compaction

Add a compaction action that asks the model to summarize the
conversation so far and stores the summary in a new compaction node
appended to the branch. Later requests slice the branch at the last
compaction node, keeping the system prefix and continuing from the
summary instead of the full history, which frees up context while the
complete history stays visible and navigable in the UI.

The slice is applied once at the request funnel, covering the plain
send, the agentic flow and the KV cache pre-encode, plus the continue
path that bypasses it. The summary request is a one-shot outside the
SSE replay machinery, streamed live into the node with a full rollback
on abort, error or empty result.

The action is exposed as a /compact slash command and a Compact button
in the context gauge popup, with an overridable summarization prompt
in the settings.

* ui: pre-fill KV cache and refresh context gauge after compaction

Once the summary is stored, pre-encode the compacted branch and stamp
the returned prompt timings on the compaction node: re-encoding the
reduced prefix is part of the compaction itself, so the server KV
cache and the context gauge reflect the summary right away.

The summary stream is registered under a prefixed identity
(compact::convId), so the visibility kick and byte-offset resume
protect it through tab switches while the reload probe never
reattaches it as a chat turn. The stamped node becomes the shared
context truth for the gauge and the processing state restore, and the
stale live state that kept the gauge full is cleared.

* chore: format

* ui: allow a dedicated model for conversation compaction

An opt-in setting reveals a model selector whose options come from the
live router model list. The chosen model writes the summary and is
loaded on demand when it is not resident, while the default path keeps
resolving to the conversation's own model.

* ui: align the compaction model field with project conventions

* ui: fix the model option helpers import path

The compaction model selector pulls filterModelOptions and
groupModelOptions from the models component folder, where they no
longer live, so the build fails to resolve the module. Point the
import at the navigation folder, which now owns them.

Missed during the rebase: the helpers moved in a commit that does
not touch any compaction file, so nothing conflicted.

* ui: import the model option helpers from models/utils

* ui: follow the moved models selector in the compaction model field

The models manager moved ModelsSelectorList under ModelsSelector and
the model option helpers to /utils, where groupModelOptions takes
the options alone. Drop the stale copy the rebase kept and read the
helpers from their new home.

* ui: drop the dedicated compaction model

A summary written by another model loses the prefix the conversation
model cached, so the whole history is prefilled again, and the model
then resumes from text it did not write. The conversation model
always writes its own summary.

* ui: keep the last steps of a compacted conversation

A compaction now folds the steps the model reads but the last ones,
a step being a user message or an assistant message with its tool
results, and stores how many it kept on the compaction node. The
model then reads the system prefix, the summary, the kept steps and
what follows.

The summary request is the request the chat sends, with its options,
tools and message shaping, cut before the preserved steps: the server
reuses the prefix it cached and only the instruction is new. The
number of preserved steps is a setting, 10 by default.

* ui: run every compaction summary through one path

Three callers needed the same thing - the automatic pre-encode pass, the
action in the context gauge, and re-running a summary from the block - and
each of them had grown its own copy of the request, the streaming and the
storing.

There is one path now: a summarization owns a node, streams into it, and
reports itself through a ledger the rest of the app reads (`isCompacting`,
which node belongs to which conversation). Re-running a summary is the same
call on an existing node, so the two entry points cannot drift apart again.

* ui: give the compaction row its own summary UI

The summary was a line of text in a tool-call-shaped collapsible, which is
how a compaction reads as a side note rather than as the thing the rest of
the conversation now stands on.

The row is its own block: a divider that names the boundary, and the summary
inside it with the model that wrote it, its statistics, and the same actions
an assistant turn has - copy, edit, fork, regenerate and delete all apply to
the summary, and it collapses to the divider once a summary is in, because
the shrunk context is the headline and the prose is the detail.

The divider is a variant of the shared collapsible block, which also needed
the toggle and the title split into snippets so a caller can lay them out
without copying the markup. The action icons no longer take a role: they
render what a message offers, and a compaction offers the same as any other
stored message.

* ui: report compaction progress on the row

The row owns the run, so the row says how it is going: the prompt progress
in the header while the context is being read, the phase once the summary
starts streaming, and the token count once it is in. Before the first
summary token there is nothing to collapse, so the header stands on its own
instead of offering an empty block.

* ui: keep the summarized turns quiet while a compaction runs

The turns being summarized used to keep their own "Processing..." indicator
and live reading stats while the summary streamed, so the conversation said
the same thing twice in two places. The row reports the run, so the turns it
replaces stay silent until it is done.

* ui: hand the compact action to the context gauge

Compacting is a decision about the context, so the action belongs with the
numbers that show the context filling up rather than inside the form: it sits
at the bottom of both the hover card and the drawer, under the breakdown it
acts on, quiet enough to stay a footnote and disabled - with the row's own
spinner - while a summary is already running.

* ui: follow the bottom, otherwise leave the view alone

The screen scrolled itself with a controller of observers, intervals,
direction heuristics and landing timers, and still managed to move the view
while the reader was reading: any mutation that grew the branch was treated
as a reason to keep following.

The page is the scroller, so it follows when the reader sits at the bottom,
and stays where it is otherwise. A resize observer plus native scroll
anchoring does the rest, and `pointerdown` buys the reader a moment of grace
to be caught mid-drag.

* ui: expand collapsible bodies with a height transition

Expanding a block snapped it open: the body was mounted the moment `open`
flipped, so a tall block moved the conversation around it in one frame.

The body now goes through the same region the models table uses. The height
interpolates and the visibility follows it, so a body is unmounted once the
collapse has finished rather than the moment it starts - a closed block
still costs nothing while tokens stream past it.

* ui: open a compaction summary at the top of the screen

Toggling a summary left the reader wherever they were, so a conversation that
had just compacted showed the divider at the bottom of the screen and read
down from there.

A toggle now scrolls the page once and lands the divider a breath under the
tabs, both ways: the summary is read from its beginning, and closing it returns
to the same boundary instead of wherever the page happens to come to rest. The
scroll waits for the body to reach its height, because before that the page is
too short to scroll the divider up there and the call would clamp.

That wait is the shared collapse duration rather than a transition event: an
engine without `interpolate-size` animates the region's grid row instead of its
height, so a listener on `height` would never fire there and the page would not
move at all.

* ui: spin the compaction icon while the summary runs

The divider reported the phase in its title while the icon still showed a
finished block, so a run in progress and a summary that had landed looked the
same in the one place the eye reads first. The icon spins while the summary is
running and folds back once it is in.

* ui: let the compaction summary speak without the reasoning trimmings

The summary is the thing the conversation now stands on, not a note beside
it, and the conversation model always writes it, so the body drops the
reasoning block's rule and indent (`flushBody` keeps them for the blocks
that borrow them) and the model selector under the summary is gone - there
is nothing left to swap.

* ui: drop the model override from the compaction path

The summary row no longer offers a model selector, so nothing hands
regenerateCompaction a model anymore. Remove the parameter from the
whole path, with the on-demand load of another model it carried: the
summary always comes from the model the request funnel resolves.

* ui: let the compaction summary think as the chat does

The summary request forced thinking off, overriding the reasoning
effort the user chose. The summary is the text the conversation
resumes from, the task where reasoning helps the most, so the request
now carries the very options of the chat.

* ui: pre-encode the compacted branch as the next turn reads it

The post-compaction pre-encode sent the branch without tools, while
the tools render ahead of the messages, so it warmed a prefix the next
turn never reads and that turn prefilled the whole compacted context
again. The pre-encode now sends the tools, message shaping and
reasoning handling of the chat, so the next turn only prefills its new
message.

* ui: pre-encode the branch the next turn actually reads

The per-turn pre-encode sent the messages of the request plus one
assistant message holding the streamed text, without tools. With
tools on, it replaced the slot the next turn extends with a prefix
nobody reads again, and after an agentic turn its history did not
match the stored calls and results. Without agentic, it never ran.

Both pre-encodes now read the stored branch up to its leaf, cut at the
last compaction, and send it with the tools, message shaping and
reasoning handling of the chat. The plain completion path triggers it
too.

* ui: refuse a compaction summary the output limit cut

The stream handler read every chunk but dropped the finish reason, so
a summary cut by max_tokens folded the conversation into half a text.
The handlers now hand the finish reason to onComplete, and a summary
that ends on length rolls the compaction back with an error.

The summary was framed through a replacement string, so $& or 32560 in
it changed the text the model read; it is framed as written now.

* ui: wait for the collapse transition in the tool call tests

The collapsible body now unmounts once its collapse animation ends rather
than as soon as the block closes, so the "folds back" assertions ran before
the body was gone. Wait out COLLAPSIBLE_TRANSITION_MS first.

Assisted-by: Claude (Opus 5.5 Medium)

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
2026-10-27 07:49:27 +02:00

756 lines
30 KiB
Markdown

# llama-ui
A modern, feature-rich web interface for llama-server built with SvelteKit. This UI provides an intuitive chat interface with advanced file handling, conversation management, and comprehensive model interaction capabilities.
Llama UI supports two server operation modes:
- **MODEL mode** - Single model operation (standard llama-server)
- **ROUTER mode** - Multi-model operation with dynamic model loading/unloading
---
## Table of Contents
- [Features](#features)
- [Getting Started](#getting-started)
- [Tech Stack](#tech-stack)
- [Build Pipeline](#build-pipeline)
- [Architecture](#architecture)
- [Data Flows](#data-flows)
- [Architectural Patterns](#architectural-patterns)
- [Testing](#testing)
---
## Features
### Chat Interface
- **Streaming responses** with real-time updates
- **Reasoning content** - Support for models with thinking/reasoning blocks
- **Dark/light theme** with system preference detection
- **Responsive design** for desktop and mobile
### File Attachments
- **Images** - JPEG, PNG, GIF, WebP, SVG (with PNG conversion)
- **Documents** - PDF (text extraction or image conversion for vision models)
- **Audio** - MP3, WAV for audio-capable models
- **Text files** - Source code, markdown, and other text formats
- **Drag-and-drop** and paste support with rich previews
### Conversation Management
- **Branching** - Branch messages conversations at any point by editing messages or regenerating responses, navigate between branches
- **Regeneration** - Regenerate responses with optional model switching (ROUTER mode)
- **Import/Export** - JSON format for backup and sharing
- **Search** - Find conversations by title or content
### Advanced Rendering
- **Syntax highlighting** - Code blocks with language detection
- **Math formulas** - KaTeX rendering for LaTeX expressions
- **Markdown** - Full GFM support with tables, lists, and more
### Multi-Model Support (ROUTER mode)
- **Models manager** - one table for every model the server can serve, split into loaded, downloading, favorites, local and hidden sections, with search, context/capability/modality filters, sorting, load/unload, delete from disk and hide. It opens from the sidebar, from a model row or from the selector.
- **Model selector** with Loaded/Available groups
- **Automatic loading** - Models load on selection
- **Modality validation** - Prevents sending images to non-vision models
- **LRU unloading** - Server auto-manages model cache
- **Downloads** - In-flight downloads stay visible above the selector and can be paused, resumed or cancelled from the manager
- **Hub metadata** - Avatars, context length and chat template come from the Hugging Face Hub only while the `Use Hugging Face Hub API for models metadata` setting is on. It is off by default, and the UI then shows what the server reports for `/v1/models`
### Keyboard Shortcuts
| Shortcut | Action |
| ------------------ | -------------------- |
| `Shift+Ctrl/Cmd+O` | New chat |
| `Shift+Ctrl/Cmd+E` | Edit conversation |
| `Shift+Ctrl/Cmd+D` | Delete conversation |
| `Ctrl/Cmd+K` | Search conversations |
| `Ctrl/Cmd+B` | Toggle sidebar |
### Developer Experience
- **Request tracking** - Monitor token generation with `/slots` endpoint
- **Storybook** - Component library with visual testing
- **Hot reload** - Instant updates during development
---
## Getting Started
### Prerequisites
- **Node.js** 18+ (20+ recommended)
- **npm** 9+
- **llama-server** running locally (for API access)
### 1. Install Dependencies
```bash
cd tools/ui
npm ci
```
### 2. Start llama-server
In a separate terminal, start the backend server:
```bash
# Single model (MODEL mode)
./llama-server -m model.gguf
# Multi-model (ROUTER mode)
./llama-server --models-dir /path/to/models
```
### 3. Start Development Servers
```bash
npm run dev
```
This starts:
- **Vite dev server** at `http://localhost:5173` - The main UI frontend app
- **Storybook** at `http://localhost:6006` - Component documentation
The Vite dev server proxies API requests to `SERVER_ORIGIN` (with fallback to default llama-server `9931` port):
```typescript
// vite.config.ts proxy configuration
proxy: {
'/v1': SERVER_ORIGIN,
'/props': SERVER_ORIGIN,
'/models': SERVER_ORIGIN,
'/tools': SERVER_ORIGIN,
'/slots': SERVER_ORIGIN,
'/cors-proxy': SERVER_ORIGIN
},
```
### Development Workflow
1. Open `http://localhost:5173` in your browser
2. Make changes to `.svelte`, `.ts`, or `.css` files
3. Changes hot-reload instantly
4. Use Storybook at `http://localhost:6006` for isolated component development
---
## Tech Stack
| Layer | Technology | Purpose |
| ----------------- | ------------------------------- | -------------------------------------------------------- |
| **Framework** | SvelteKit + Svelte 5 | Reactive UI with runes (`$state`, `$derived`, `$effect`) |
| **UI Components** | shadcn-svelte + bits-ui | Accessible, customizable component library |
| **Drawers** | vaul-svelte | Draggable drawer overlays (mobile pickers, model pane) |
| **Styling** | TailwindCSS 4 | Utility-first CSS with design tokens |
| **Database** | IndexedDB (Dexie) | Client-side storage for conversations and messages |
| **Build** | Vite | Fast bundling with static adapter |
| **Testing** | Playwright + Vitest + Storybook | E2E, unit, and visual testing |
| **Markdown** | remark + rehype | Markdown processing with KaTeX and syntax highlighting |
### Key Dependencies
```json
{
"svelte": "^5.0.0",
"bits-ui": "^2.8.11",
"dexie": "^4.0.11",
"pdfjs-dist": "^5.4.54",
"highlight.js": "^11.11.1",
"rehype-katex": "^7.0.1"
}
```
---
## Build Pipeline
### Development Build
```bash
npm run dev
```
Runs Vite in development mode with:
- Hot Module Replacement (HMR)
- Source maps
- Proxy to llama-server
### Production Build
```bash
npm run build
```
The build process:
1. **Vite Build** - Bundles all TypeScript, Svelte, and CSS
2. **Static Adapter** - Outputs to `../../build/tools/ui/dist` (llama-server's static file directory)
3. **Post-Build Script** - Cleans up intermediate files
4. **Custom Plugin** - Creates `index.html` with:
- Inlined favicon as base64
- GZIP compression (level 9)
- Deterministic output (zeroed timestamps)
```text
tools/ui/ → build → build/tools/ui/dist/
├── src/ ├── index.html (served by llama-server)
├── static/ └── (favicon inlined)
└── ...
```
### SvelteKit Configuration
```javascript
// svelte.config.js
adapter: adapter({
pages: '../../build/tools/ui/dist', // Output directory
assets: '../../build/tools/ui/dist', // Static assets
fallback: 'index.html', // SPA fallback
strict: true
}),
output: {
bundleStrategy: 'inline' // Single-file bundle
}
```
### Integration with llama-server
llama-ui is embedded directly into the llama-server binary:
1. `npm run build` outputs `index.html` to `build/tools/ui/dist/`
2. llama-server compiles this into the binary at build time
3. When accessing `/`, llama-server serves the bundled HTML
This results in a **single portable binary** with the full Llama UI included.
---
## Architecture
Llama UI follows a layered architecture with unidirectional data flow:
```text
Routes → Components → Hooks → Stores → Services → Storage/API
```
### High-Level Architecture
```mermaid
flowchart TB
subgraph Routes["📍 Routes"]
R1["/ (Welcome)"]
R2["/chat/[id]"]
R3["/mcp-servers"]
R4["/search"]
R5["/settings"]
RL["+layout.svelte"]
end
subgraph Components["🧩 Components"]
C_Screen["ChatScreen"]
C_Form["ChatForm"]
C_Messages["ChatMessages"]
C_Sidebar["ChatSidebar"]
C_Models["ModelsSelector"]
C_Settings["ChatSettings"]
C_Mcp["McpServers"]
end
subgraph Hooks["🔌 Hooks"]
H1["use-chat-screen-active-model"]
H2["use-processing-state"]
H3["use-context-gauge"]
H4["use-models-selector"]
H5["use-tools-panel"]
end
subgraph Stores["🗄️ Stores"]
S1["chatStore"]
S2["conversationsStore"]
S3["modelsStore"]
S4["mcpStore"]
S5["agenticStore"]
S6["serverStore"]
S7["settingsStore"]
S8["toolsStore"]
end
subgraph Services["⚙️ Services"]
SV1["ChatService"]
SV2["ModelsService"]
SV3["PropsService"]
SV4["DatabaseService"]
SV5["MCPService"]
SV6["ToolsService"]
SV7["SandboxService"]
end
subgraph Storage["💾 Storage"]
ST1["IndexedDB"]
ST2["LocalStorage"]
end
subgraph APIs["🌐 llama-server"]
API1["/v1/chat/completions"]
API2["/props"]
API3["/models/*"]
API4["/tools"]
end
R1 & R2 --> C_Screen
RL --> C_Sidebar
C_Screen --> C_Form & C_Messages & C_Settings
C_Screen --> H1 & H2 & H3
C_Models --> H4
C_Mcp --> S4
C_Screen --> S1 & S2 & S3
C_Models --> S3
H1 --> S3
S1 --> SV1 & SV4
S2 --> SV4
S3 --> SV2 & SV3
S4 --> SV5
S5 --> SV1 & SV5 & SV6 & SV7
SV4 --> ST1
SV1 --> API1
SV2 --> API3
SV3 --> API2
SV6 --> API4
```
### Layer Breakdown
#### Routes (`src/routes/`)
- **`/`** - Welcome screen, creates new conversation
- **`/chat/[id]`** - Active chat interface
- **`/mcp-servers`** - MCP server management
- **`/search`** - Conversation search
- **`/settings`** - Settings (optional `[[section]]`)
- **`+layout.svelte`** - Sidebar, navigation, global initialization
#### Components (`src/lib/components/`)
Components are organized in `app/` (application-specific) and `ui/` (shadcn-svelte primitives).
**Chat Components** (`app/chat/`):
| Component | Responsibility |
| ------------------ | --------------------------------------------------------------------------- |
| `ChatScreen/` | Main chat container, coordinates message list, input form, and attachments |
| `ChatForm/` | Message input textarea with file upload, paste handling, keyboard shortcuts |
| `ChatMessages/` | Message list with branch navigation, regenerate/continue/edit actions |
| `ChatAttachments/` | File attachment previews, drag-and-drop, PDF/image/audio handling |
| `ChatSettings/` | Parameter sliders (temperature, top-p, etc.) with server default sync |
| `ChatSidebar/` | Conversation list, search, import/export, navigation |
**Dialog Components** (`app/dialogs/`):
| Component | Responsibility |
| ------------------------------ | -------------------------------------------------------- |
| `DialogChatSettings` | Full-screen settings configuration |
| `DialogManageModels` | Models manager: browse, load, download and delete models |
| `DialogModelInformation` | Model details (context size, modalities, parallel slots) |
| `DialogChatAttachmentsPreview` | Full preview for images, PDFs (text or page view), code |
| `DialogConfirmation` | Generic confirmation for destructive actions |
| `DialogConversationRename` | Edit conversation title |
**Server/Model Components** (`app/server/`, `app/models/`):
| Component | Responsibility |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `ServerErrorSplash` | Error display when server is unreachable |
| `ModelsSelector/` | Model dropdown and mobile sheet: favorites, loaded and local sections, rows windowed and folded into families |
| `ModelsManager/` | Models manager table and model pane: sections, filters, sorting, quants folded per repo, downloads |
| `ModelAvatar` | Org avatar of a model, with the quantizer badge; follows the family grouping of the list |
| `ModelId`, `ModelBadge` | Model name, aliases, tags, quantization and draft sidecar badges |
| `ModelContext`, `ModelCapabilities` | Context window and capability icons, filled from the Hub when it is enabled |
**Shared UI Components** (`app/misc/`):
| Component | Responsibility |
| ----------------------------------------- | ----------------------------------------------------------------------- |
| `MarkdownContent` | Markdown rendering with KaTeX, syntax highlighting, copy buttons |
| `SyntaxHighlightedCode` | Code blocks with language detection and highlighting |
| `ActionIcon`, `DropdownMenuActions` | Reusable action button and action menu |
| `BadgesModality`, `BadgeInfo` | Status and capability badges |
| `CollapsibleSection`, `CollapsibleRegion` | Section header with a trigger that expands in place |
| `GroupedList` | Grouped rows with foldable groups, per-group windows and show-more rows |
| `ScrollCarousel`, `TruncatedText` | Horizontal scroll strip and single-line truncation with tooltip |
#### Hooks (`src/lib/hooks/`)
Hooks are the thin view-layer between components and stores: they own UI concerns (scroll, drag-and-drop, keyboard shortcuts, pickers, selection) and translate store state into view state.
| Hook | Responsibility |
| ------------------------------- | ----------------------------------------------------------------------------------------------- |
| `use-chat-screen-active-model` | Active model resolution + modality capability detection |
| `use-processing-state` | View over `chatStore.processing` for streaming progress/tokens |
| `use-context-gauge` | View over `contextStatsStore` for the context usage gauge |
| `use-models-selector` | Model selector state: hidden models, family grouping, windowed rows, selection and load on pick |
| `use-tools-panel` | Tools panel state |
| `use-reasoning-menu` | Reasoning-effort menu state |
| `use-attachment-menu` | Attachment menu + modality flags |
| `use-draft-messages` | Per-chat draft message/files persistence |
| `use-chat-form-pickers` | Chat form pickers (commands, mentions) |
| `use-debounced-search` | Shared debounced async search for pickers |
| `use-picker-navigation` | Picker keyboard navigation |
| `use-chat-message-edit-context` | Message edit context (content + extras) |
| `use-chat-screen-drag-and-drop` | Drag-and-drop state machine |
| `use-chat-screen-file-upload` | File upload queue + capability validation |
| `use-stick-to-bottom` | Follows new content while the reader sits at the bottom of a scroller |
| `use-marquee-selection` | Shift+click / marquee range selection |
| `use-keyboard-shortcuts` | Global keyboard shortcuts |
| `use-settings-navigation` | Settings section navigation |
| `use-pwa` | PWA install/update + version mismatch detection |
#### Stores (`src/lib/stores/`)
Stores own reactive application state as Svelte 5 runes. Larger stores are split into directories and compose focused sub-stores behind a narrow host interface (see Architectural Patterns).
| Store | Responsibility |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `chatStore` | Chat lifecycle, streaming, abort control, error handling; composes `processing`, `activity`, `streams`, `flows` |
| `conversationsStore` | Conversation CRUD, message branching, navigation, import/export; composes `preferences` |
| `modelsStore` | Model list, selection, loading/unloading (ROUTER), recent picks, favorites, hidden models and list open state; composes `props`, `status` |
| `mcpStore` | MCP host role: multi-server lifecycle, tool routing, server icons; composes `health`, `resources` |
| `agenticStore` | Multi-turn agentic loop orchestration, tool execution; composes `gates` |
| `serverStore` | Server connection state, `/props`, role detection, modalities |
| `settingsStore` | User preferences, theme, parameter sync with server defaults |
| `toolsStore` | Tool registry: server + MCP tools, enabled set for the LLM |
| `permissionsStore` | Persisted tool permission grants |
| `contextStatsStore` | Context window usage for the active conversation |
| `draftMessagesStore` | Per-chat draft message/files |
| `deviceStore` | Browser environment signals (mobile, OS, theme) |
| `versionStore` | Build version information |
#### Services (`src/lib/services/`)
Services are a stateless protocol layer: static methods, pure I/O, no reactive state. Stores consume them for all API and storage access.
| Service | Responsibility |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `ChatService` | `/v1/chat/completions` streaming + SSE parsing, message format conversion |
| `ModelsService` | `/models`, `/models/load`, `/models/unload` |
| `PropsService` | `/props`, `/props?model=` |
| `HuggingFaceService` | Hugging Face Hub metadata: model details, avatars, README and file tree, cached per repo and off unless the Hub setting is on |
| `DatabaseService` | IndexedDB operations via Dexie |
| `MCPService` | MCP protocol: transports, connect, list/execute tools, prompts, resources |
| `ToolsService` | Server tool list/execute/stream (`/tools`) |
| `SandboxService` | Browser JS execution in a sandboxed worker |
| `ParameterSyncService` | Syncs settings with server defaults |
| `ConversationTransferService` | Conversation import/export JSONL + ZIP format |
| `MigrationService` | Non-destructive localStorage/IndexedDB migrations |
| `RouterService` | Dynamic route URL construction |
---
## Data Flows
### MODEL Mode (Single Model)
```mermaid
sequenceDiagram
participant User
participant UI
participant Stores
participant DB as IndexedDB
participant API as llama-server
Note over User,API: Initialization
UI->>Stores: initStores() (awaited by route loads)
Stores->>Stores: run migrations
Stores->>DB: load conversations (background)
Stores->>API: GET /props
API-->>Stores: server config
Stores->>API: GET /v1/models
API-->>Stores: single model (auto-selected)
Note over User,API: Chat Flow
User->>UI: send message
Stores->>DB: save user message
Stores->>API: POST /v1/chat/completions (stream)
loop streaming
API-->>Stores: SSE chunks
Stores-->>UI: reactive update
end
Stores->>DB: save assistant message
```
### ROUTER Mode (Multi-Model)
```mermaid
sequenceDiagram
participant User
participant UI
participant Stores
participant API as llama-server
Note over User,API: Initialization
Stores->>API: GET /props
API-->>Stores: {role: "router"}
Stores->>API: GET /models
API-->>Stores: models[] with status
Note over User,API: Model Selection
User->>UI: select model
alt model not loaded
Stores->>API: POST /models/load
loop poll status
Stores->>API: GET /models
end
Stores->>API: GET /props?model=X
end
Stores->>Stores: validate modalities
Note over User,API: Chat Flow
Stores->>API: POST /v1/chat/completions {model: X}
loop streaming
API-->>Stores: SSE chunks + model info
end
```
---
## Architectural Patterns
### 1. Reactive State with Svelte 5 Runes
All stores use Svelte 5's fine-grained reactivity:
```typescript
// Store with reactive state
class ChatStore {
#isLoading = $state(false);
#currentResponse = $state('');
// Derived values auto-update
get isStreaming() {
return $derived(this.#isLoading && this.#currentResponse.length > 0);
}
}
// Exported reactive accessors
export const isLoading = () => chatStore.isLoading;
export const currentResponse = () => chatStore.currentResponse;
```
### 2. Unidirectional Data Flow
Data flows in one direction, making state predictable:
```mermaid
flowchart LR
subgraph UI["UI Layer"]
A[User Action] --> B[Component]
end
subgraph State["State Layer"]
B --> C[Store Method]
C --> D[State Update]
end
subgraph IO["I/O Layer"]
C --> E[Service]
E --> F[API / IndexedDB]
F -.->|Response| D
end
D -->|Reactive| B
```
Components dispatch actions to stores, stores coordinate with services for I/O, and state updates reactively propagate back to the UI.
### 3. Per-Conversation State
Enables concurrent streaming across multiple conversations. Loading is tracked
per conversation by the activity ledger (`chatStore.activity`), while streaming
state and abort controllers live in per-conversation maps:
```typescript
class ChatStore {
chatStreamingStates = new SvelteMap<string, { response: string; messageId: string }>();
abortControllers = new SvelteMap<string, AbortController>();
}
```
### 4. Message Branching with Tree Structure
Conversations are stored as a tree, not a linear list:
```typescript
interface DatabaseMessage {
id: string;
parent: string | null; // Points to parent message
children: string[]; // List of child message IDs
// ...
}
interface DatabaseConversation {
currentNode: string; // Currently viewed branch tip
// ...
}
```
Navigation between branches updates `currentNode` without losing history.
### 5. Layered Service Architecture
Stores handle state; services handle I/O:
```text
┌─────────────────┐
│ Stores │ Business logic, state management
├─────────────────┤
│ Services │ API calls, database operations
├─────────────────┤
│ Storage/API │ IndexedDB, LocalStorage, HTTP
└─────────────────┘
```
### 6. Server Role Abstraction
Single codebase handles both MODEL and ROUTER modes:
```typescript
// serverStore.ts
get isRouterMode() {
return this.role === ServerRole.ROUTER;
}
// Components conditionally render based on mode
{#if isRouterMode()}
<ModelsSelector />
{/if}
```
### 7. Modality Validation
Prevents sending attachments to incompatible models. The
`use-chat-screen-active-model` hook derives the active model's capabilities
from `modelsStore.props`:
```typescript
// use-chat-screen-active-model hook
const hasVisionModality = $derived.by(() => modelsStore.props.modelSupportsVision(activeModelId));
const hasAudioModality = $derived.by(() => modelsStore.props.modelSupportsAudio(activeModelId));
```
### 8. Persistent Storage Strategy
Data is persisted across sessions using two storage mechanisms:
```mermaid
flowchart TB
subgraph Browser["Browser Storage"]
subgraph IDB["IndexedDB (Dexie)"]
C[Conversations]
M[Messages]
end
subgraph LS["LocalStorage"]
S[Settings Config]
O[User Overrides]
T[Theme Preference]
end
end
subgraph Stores["Svelte Stores"]
CS[conversationsStore] --> C
CS --> M
SS[settingsStore] --> S
SS --> O
SS --> T
end
```
- **IndexedDB**: Conversations and messages (large, structured data)
- **LocalStorage**: Settings, user parameter overrides, theme, model favorites, recent picks, hidden models and model list open state (small key-value data)
- **Memory only**: Server props, model list (fetched fresh on each session), Hugging Face Hub details (cached per repo for the session)
### 9. Remote Assets Under Cross-Origin Isolation
llama-server serves the UI with `Cross-Origin-Embedder-Policy: require-corp` and `Cross-Origin-Opener-Policy: same-origin`, so a cross-origin subresource has to be allowed by the target origin. The UI handles that per asset kind:
- **Hub avatars** are requested in CORS mode (`crossorigin="anonymous"`), which the Hub allows
- **Hub metadata** is fetched with `fetch()`, which is a CORS request by default, and only while the Hub setting is on
- **MCP server icons and web-search result favicons** have no CORS guarantee, so they go through the server's `/cors-proxy` when the user started llama-server with `--ui-mcp-proxy`; without it they fall back to the bundled glyph
- **Per-row Hub lookups** wait until the row is near the viewport (the `nearViewport` action), so a long model list does not fire one request per row on mount
---
## Testing
### Test Types
| Type | Tool | Location | Command |
| ------------- | ------------------ | ---------------- | ------------------- |
| **Unit** | Vitest | `tests/unit/` | `npm run test:unit` |
| **UI/Visual** | Storybook + Vitest | `tests/stories/` | `npm run test:ui` |
| **E2E** | Playwright | `tests/e2e/` | `npm run test:e2e` |
| **Client** | Vitest | `tests/client/`. | `npm run test:unit` |
### Running Tests
```bash
# All tests
npm run test
# Individual test suites
npm run test:e2e # End-to-end (requires llama-server)
npm run test:client # Client-side unit tests
npm run test:server # Server-side unit tests
npm run test:ui # Storybook visual tests
```
### Storybook Development
```bash
npm run storybook # Start Storybook dev server on :6006
npm run build-storybook # Build static Storybook
```
### Linting and Formatting
```bash
npm run lint # Check code style
npm run format # Auto-format with Prettier
npm run check # TypeScript type checking
```
---
## Project Structure
```text
tools/ui/
├── src/
│ ├── lib/
│ │ ├── components/ # UI components (app/, ui/)
│ │ ├── hooks/ # Svelte hooks
│ │ ├── stores/ # State management
│ │ ├── services/ # API and database services
│ │ ├── types/ # TypeScript interfaces
│ │ └── utils/ # Utility functions
│ ├── routes/ # SvelteKit routes
│ └── styles/ # Global styles
├── static/ # Static assets
├── tests/ # Test files
└── .storybook/ # Storybook configuration
```
---
## Related Documentation
- [llama.cpp Server README](../server/README.md) - Full server documentation
- [Multimodal Documentation](../../docs/multimodal.md) - Image and audio support
- [Function Calling](../../docs/function-calling.md) - Tool use capabilities