* ui : move Models, MCP and Settings to a sidebar footer The sidebar's actions split in two: New chat, Search and Projects stay at its top, and Models, MCP and Settings move to a footer at its bottom, under a thin rule, in the expanded sidebar and its collapsed strip alike. On a phone the footer keeps clear of the home indicator, and Models stays desktop-only as before. The search dialog still lists every action. Assisted-by: Claude (Opus 5.5 Medium) * ui : group the sidebar footer in an inset surface The design system gains inset-group-surface, for rows grouped inside a glass panel: white on the light glass and a faint tint of the text colour on the dark one, with the sidebar's own ambient edge, no inner highlight, a faint wide shade under it and the card radius. It is matte-glass pointed at its own component tokens, like the sidebar and chat form surfaces, and reads as part of the panel: no tight contact shadow or solid border like a button, no inset shade like a field. The sidebar footer uses it in place of the rule above Models, MCP and Settings. DESIGN.md and the Surfaces page document it. Assisted-by: Claude (Opus 5.5 Medium) * ui : give the sidebar's logo button the add button's key look Whenever the sidebar shows, open or peeking, its logo button takes the same key as the chat form's add button (matte-key, white in light mode, a light tint in dark), in place of its own muted fill. While the sidebar is hidden the logo has no fill and no shadow, and hover and focus never change the fill. It keeps its round shape and its own size. Assisted-by: Claude (Opus 5.5 Medium) * ui : put search in a round key at the top of the phone sidebar On a phone the sidebar's Search row moves to a round key at the right end of its top row, across from the logo, with the same key look as the logo and the chat form's add button. It opens search as the row did. Desktop keeps the Search row. Assisted-by: Claude (Opus 5.5 Medium) * ui : size the phone sidebar for a thumb and list its footer with the actions On a phone the sidebar's actions and rows are 48px tall at 17px, flush one under the other with 16px corners, its section headings 15px and grey, and its groups 28px apart. MCP and Settings join the top actions there: the footer is a desktop pattern. Assisted-by: Claude (Opus 5.5 Medium) * ui : mark the selected sidebar row with a tinted inset surface Assisted-by: Claude (Opus 5.5 Medium) * ui: keep the phone sidebar's round keys white on the grey page Assisted-by: Claude (Opus 5.5 Medium) * ui : open Settings and MCP as pages on a phone, dialogs elsewhere On a phone the sidebar opens them at #/settings and #/mcp-servers, with the menu key and a title like any phone page. Pinned under the status bar, Settings' section tabs fill the screen above them in the page's colour. Assisted-by: Claude (Opus 5.5 Medium) * ui : open a phone sidebar's pages behind it, settled before they slide back A tap in the phone sidebar now opens its page while the current one is still aside and slides back once it has rendered, instead of swapping pages mid-slide. The slid page's fixed bars are pinned again as the page under them changes, the lock ends with the slide itself, and a phone's new chat shows its form and greeting at once, without the move and entry they have on a desktop. Assisted-by: Claude (Opus 5.5 Medium) * ui: style the sidebar footer group and the selected row with master's classes until the design system lands Assisted-by: Claude (Opus 5.5 Medium) * ui: add GlassKey with master's styles for the phone sidebar's round keys Assisted-by: Claude (Opus 5.5 Medium) * ui: center the top edge of a new chat's form on the screen The form rested 42px below the middle up to 2xl and 6px above it past that; one drop, from the resting card's height, now puts its top edge on the middle at every width from md up. Assisted-by: Claude (Opus 5.5 Medium) * ui: round the sidebar's action rows concentric with the footer group The sidebar stacks four surfaces: the page, the sidebar, the footer group inset in it and a row in the group. app.css now names the group's radius and padding, and derives the row radius from them (16px less 4px), so the footer's rows run parallel to its corners; the actions at the top of the sidebar share that radius. The footer moves to its own component, the README documents the pattern, and a client test holds the rule on the rendered footer. A selected action row was bg-accent, the same grey as a hovered row in light and darker than the footer's fill in dark, so it read as hovered or not at all; it is now raised and edged, while a hover only tints. Assisted-by: Claude (Opus 5.5 Medium) * ui: import demo chats through the Settings page on a phone A phone opens Settings as a page now, not a dialog, so the demo harness waited for a dialog that never came and every phone demo with traces failed; it now imports through whichever of the two opens. Assisted-by: Claude (Opus 5.5 Medium) * ui: count the sidebar's rows from their items in the surface test The test expected three footer rows and two top actions, which the stacked PRs change as they add Projects, Library and Servers; it now expects as many rows as the sidebar has items. Assisted-by: Claude (Opus 5.5 Medium) * ui: add a line on Meta+Enter off macOS too The chat form takes Meta+Enter as Ctrl+Enter and leaves the newline to the editor, but the editor bound only Mod-Enter, which is Ctrl off macOS, so on Linux Meta+Enter neither sent nor added a line. This failed the client test on CI's Linux runners, on master too. Assisted-by: Claude (Opus 5.5 Medium)
771 lines
32 KiB
Markdown
771 lines
32 KiB
Markdown
# llama-ui
|
|
|
|
A modern, feature-rich web interface for llama-server built with SvelteKit. This UI provides an intuitive chat interface with advanced file handling, conversation management, and comprehensive model interaction capabilities.
|
|
|
|
Llama UI supports two server operation modes:
|
|
|
|
- **MODEL mode** - Single model operation (standard llama-server)
|
|
- **ROUTER mode** - Multi-model operation with dynamic model loading/unloading
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
- [Features](#features)
|
|
- [Getting Started](#getting-started)
|
|
- [Tech Stack](#tech-stack)
|
|
- [Build Pipeline](#build-pipeline)
|
|
- [Architecture](#architecture)
|
|
- [Data Flows](#data-flows)
|
|
- [Architectural Patterns](#architectural-patterns)
|
|
- [Testing](#testing)
|
|
|
|
---
|
|
|
|
## Features
|
|
|
|
### Chat Interface
|
|
|
|
- **Streaming responses** with real-time updates
|
|
- **Reasoning content** - Support for models with thinking/reasoning blocks
|
|
- **Dark/light theme** with system preference detection
|
|
- **Responsive design** for desktop and mobile
|
|
|
|
### File Attachments
|
|
|
|
- **Images** - JPEG, PNG, GIF, WebP, SVG (with PNG conversion)
|
|
- **Documents** - PDF (text extraction or image conversion for vision models)
|
|
- **Audio** - MP3, WAV for audio-capable models
|
|
- **Text files** - Source code, markdown, and other text formats
|
|
- **Drag-and-drop** and paste support with rich previews
|
|
|
|
### Conversation Management
|
|
|
|
- **Branching** - Branch messages conversations at any point by editing messages or regenerating responses, navigate between branches
|
|
- **Regeneration** - Regenerate responses with optional model switching (ROUTER mode)
|
|
- **Import/Export** - JSON format for backup and sharing
|
|
- **Search** - Find conversations by title or content
|
|
|
|
### Advanced Rendering
|
|
|
|
- **Syntax highlighting** - Code blocks with language detection
|
|
- **Math formulas** - KaTeX rendering for LaTeX expressions
|
|
- **Markdown** - Full GFM support with tables, lists, and more
|
|
|
|
### Multi-Model Support (ROUTER mode)
|
|
|
|
- **Models manager** - one table for every model the server can serve, split into loaded, downloading, favorites, local and hidden sections, with search, context/capability/modality filters, sorting, load/unload, delete from disk and hide. It opens from the sidebar, from a model row or from the selector.
|
|
- **Model selector** with Loaded/Available groups
|
|
- **Automatic loading** - Models load on selection
|
|
- **Modality validation** - Prevents sending images to non-vision models
|
|
- **LRU unloading** - Server auto-manages model cache
|
|
- **Downloads** - In-flight downloads stay visible above the selector and can be paused, resumed or cancelled from the manager
|
|
- **Hub metadata** - Avatars, context length and chat template come from the Hugging Face Hub only while the `Use Hugging Face Hub API for models metadata` setting is on. It is off by default, and the UI then shows what the server reports for `/v1/models`
|
|
|
|
### Keyboard Shortcuts
|
|
|
|
| Shortcut | Action |
|
|
| ------------------ | -------------------- |
|
|
| `Shift+Ctrl/Cmd+O` | New chat |
|
|
| `Shift+Ctrl/Cmd+E` | Edit conversation |
|
|
| `Shift+Ctrl/Cmd+D` | Delete conversation |
|
|
| `Ctrl/Cmd+K` | Search conversations |
|
|
| `Ctrl/Cmd+B` | Toggle sidebar |
|
|
|
|
### Developer Experience
|
|
|
|
- **Request tracking** - Monitor token generation with `/slots` endpoint
|
|
- **Storybook** - Component library with visual testing
|
|
- **Hot reload** - Instant updates during development
|
|
|
|
---
|
|
|
|
## Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
- **Node.js** 18+ (20+ recommended)
|
|
- **npm** 9+
|
|
- **llama-server** running locally (for API access)
|
|
|
|
### 1. Install Dependencies
|
|
|
|
```bash
|
|
cd tools/ui
|
|
npm ci
|
|
```
|
|
|
|
### 2. Start llama-server
|
|
|
|
In a separate terminal, start the backend server:
|
|
|
|
```bash
|
|
# Single model (MODEL mode)
|
|
./llama-server -m model.gguf
|
|
|
|
# Multi-model (ROUTER mode)
|
|
./llama-server --models-dir /path/to/models
|
|
```
|
|
|
|
### 3. Start Development Servers
|
|
|
|
```bash
|
|
npm run dev
|
|
```
|
|
|
|
This starts:
|
|
|
|
- **Vite dev server** at `http://localhost:5173` - The main UI frontend app
|
|
- **Storybook** at `http://localhost:6006` - Component documentation
|
|
|
|
The Vite dev server proxies API requests to `SERVER_ORIGIN` (with fallback to default llama-server `9931` port):
|
|
|
|
```typescript
|
|
// vite.config.ts proxy configuration
|
|
proxy: {
|
|
'/v1': SERVER_ORIGIN,
|
|
'/props': SERVER_ORIGIN,
|
|
'/models': SERVER_ORIGIN,
|
|
'/tools': SERVER_ORIGIN,
|
|
'/slots': SERVER_ORIGIN,
|
|
'/cors-proxy': SERVER_ORIGIN
|
|
},
|
|
```
|
|
|
|
### Development Workflow
|
|
|
|
1. Open `http://localhost:5173` in your browser
|
|
2. Make changes to `.svelte`, `.ts`, or `.css` files
|
|
3. Changes hot-reload instantly
|
|
4. Use Storybook at `http://localhost:6006` for isolated component development
|
|
|
|
---
|
|
|
|
## Tech Stack
|
|
|
|
| Layer | Technology | Purpose |
|
|
| ----------------- | ------------------------------- | -------------------------------------------------------- |
|
|
| **Framework** | SvelteKit + Svelte 5 | Reactive UI with runes (`$state`, `$derived`, `$effect`) |
|
|
| **UI Components** | shadcn-svelte + bits-ui | Accessible, customizable component library |
|
|
| **Drawers** | vaul-svelte | Draggable drawer overlays (mobile pickers, model pane) |
|
|
| **Styling** | TailwindCSS 4 | Utility-first CSS with design tokens |
|
|
| **Database** | IndexedDB (Dexie) | Client-side storage for conversations and messages |
|
|
| **Build** | Vite | Fast bundling with static adapter |
|
|
| **Testing** | Playwright + Vitest + Storybook | E2E, unit, and visual testing |
|
|
| **Markdown** | remark + rehype | Markdown processing with KaTeX and syntax highlighting |
|
|
|
|
### Key Dependencies
|
|
|
|
```json
|
|
{
|
|
"svelte": "^5.0.0",
|
|
"bits-ui": "^2.8.11",
|
|
"dexie": "^4.0.11",
|
|
"pdfjs-dist": "^5.4.54",
|
|
"highlight.js": "^11.11.1",
|
|
"rehype-katex": "^7.0.1"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Build Pipeline
|
|
|
|
### Development Build
|
|
|
|
```bash
|
|
npm run dev
|
|
```
|
|
|
|
Runs Vite in development mode with:
|
|
|
|
- Hot Module Replacement (HMR)
|
|
- Source maps
|
|
- Proxy to llama-server
|
|
|
|
### Production Build
|
|
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
The build process:
|
|
|
|
1. **Vite Build** - Bundles all TypeScript, Svelte, and CSS
|
|
2. **Static Adapter** - Outputs to `../../build/tools/ui/dist` (llama-server's static file directory)
|
|
3. **Post-Build Script** - Cleans up intermediate files
|
|
4. **Custom Plugin** - Creates `index.html` with:
|
|
- Inlined favicon as base64
|
|
- GZIP compression (level 9)
|
|
- Deterministic output (zeroed timestamps)
|
|
|
|
```text
|
|
tools/ui/ → build → build/tools/ui/dist/
|
|
├── src/ ├── index.html (served by llama-server)
|
|
├── static/ └── (favicon inlined)
|
|
└── ...
|
|
```
|
|
|
|
### SvelteKit Configuration
|
|
|
|
```javascript
|
|
// svelte.config.js
|
|
adapter: adapter({
|
|
pages: '../../build/tools/ui/dist', // Output directory
|
|
assets: '../../build/tools/ui/dist', // Static assets
|
|
fallback: 'index.html', // SPA fallback
|
|
strict: true
|
|
}),
|
|
output: {
|
|
bundleStrategy: 'inline' // Single-file bundle
|
|
}
|
|
```
|
|
|
|
### Integration with llama-server
|
|
|
|
llama-ui is embedded directly into the llama-server binary:
|
|
|
|
1. `npm run build` outputs `index.html` to `build/tools/ui/dist/`
|
|
2. llama-server compiles this into the binary at build time
|
|
3. When accessing `/`, llama-server serves the bundled HTML
|
|
|
|
This results in a **single portable binary** with the full Llama UI included.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
Llama UI follows a layered architecture with unidirectional data flow:
|
|
|
|
```text
|
|
Routes → Components → Hooks → Stores → Services → Storage/API
|
|
```
|
|
|
|
### High-Level Architecture
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph Routes["📍 Routes"]
|
|
R1["/ (Welcome)"]
|
|
R2["/chat/[id]"]
|
|
R3["/mcp-servers"]
|
|
R4["/search"]
|
|
R5["/settings"]
|
|
RL["+layout.svelte"]
|
|
end
|
|
|
|
subgraph Components["🧩 Components"]
|
|
C_Screen["ChatScreen"]
|
|
C_Form["ChatForm"]
|
|
C_Messages["ChatMessages"]
|
|
C_Sidebar["ChatSidebar"]
|
|
C_Models["ModelsSelector"]
|
|
C_Settings["ChatSettings"]
|
|
C_Mcp["McpServers"]
|
|
end
|
|
|
|
subgraph Hooks["🔌 Hooks"]
|
|
H1["use-chat-screen-active-model"]
|
|
H2["use-processing-state"]
|
|
H3["use-context-gauge"]
|
|
H4["use-models-selector"]
|
|
H5["use-tools-panel"]
|
|
end
|
|
|
|
subgraph Stores["🗄️ Stores"]
|
|
S1["chatStore"]
|
|
S2["conversationsStore"]
|
|
S3["modelsStore"]
|
|
S4["mcpStore"]
|
|
S5["agenticStore"]
|
|
S6["serverStore"]
|
|
S7["settingsStore"]
|
|
S8["toolsStore"]
|
|
end
|
|
|
|
subgraph Services["⚙️ Services"]
|
|
SV1["ChatService"]
|
|
SV2["ModelsService"]
|
|
SV3["PropsService"]
|
|
SV4["DatabaseService"]
|
|
SV5["MCPService"]
|
|
SV6["ToolsService"]
|
|
SV7["SandboxService"]
|
|
end
|
|
|
|
subgraph Storage["💾 Storage"]
|
|
ST1["IndexedDB"]
|
|
ST2["LocalStorage"]
|
|
end
|
|
|
|
subgraph APIs["🌐 llama-server"]
|
|
API1["/v1/chat/completions"]
|
|
API2["/props"]
|
|
API3["/models/*"]
|
|
API4["/tools"]
|
|
end
|
|
|
|
R1 & R2 --> C_Screen
|
|
RL --> C_Sidebar
|
|
C_Screen --> C_Form & C_Messages & C_Settings
|
|
C_Screen --> H1 & H2 & H3
|
|
C_Models --> H4
|
|
C_Mcp --> S4
|
|
C_Screen --> S1 & S2 & S3
|
|
C_Models --> S3
|
|
H1 --> S3
|
|
S1 --> SV1 & SV4
|
|
S2 --> SV4
|
|
S3 --> SV2 & SV3
|
|
S4 --> SV5
|
|
S5 --> SV1 & SV5 & SV6 & SV7
|
|
SV4 --> ST1
|
|
SV1 --> API1
|
|
SV2 --> API3
|
|
SV3 --> API2
|
|
SV6 --> API4
|
|
```
|
|
|
|
### Layer Breakdown
|
|
|
|
#### Routes (`src/routes/`)
|
|
|
|
- **`/`** - Welcome screen, creates new conversation
|
|
- **`/chat/[id]`** - Active chat interface
|
|
- **`/mcp-servers`** - MCP server management
|
|
- **`/search`** - Conversation search
|
|
- **`/settings`** - Settings (optional `[[section]]`)
|
|
- **`+layout.svelte`** - Sidebar, navigation, global initialization
|
|
|
|
#### Components (`src/lib/components/`)
|
|
|
|
Components are organized in `app/` (application-specific) and `ui/` (shadcn-svelte primitives).
|
|
|
|
**Chat Components** (`app/chat/`):
|
|
|
|
| Component | Responsibility |
|
|
| ------------------ | --------------------------------------------------------------------------- |
|
|
| `ChatScreen/` | Main chat container, coordinates message list, input form, and attachments |
|
|
| `ChatForm/` | Message input textarea with file upload, paste handling, keyboard shortcuts |
|
|
| `ChatMessages/` | Message list with branch navigation, regenerate/continue/edit actions |
|
|
| `ChatAttachments/` | File attachment previews, drag-and-drop, PDF/image/audio handling |
|
|
| `ChatSettings/` | Parameter sliders (temperature, top-p, etc.) with server default sync |
|
|
| `ChatSidebar/` | Conversation list, search, import/export, navigation |
|
|
|
|
**Dialog Components** (`app/dialogs/`):
|
|
|
|
| Component | Responsibility |
|
|
| ------------------------------ | -------------------------------------------------------- |
|
|
| `DialogChatSettings` | Full-screen settings configuration |
|
|
| `DialogManageModels` | Models manager: browse, load, download and delete models |
|
|
| `DialogModelInformation` | Model details (context size, modalities, parallel slots) |
|
|
| `DialogChatAttachmentsPreview` | Full preview for images, PDFs (text or page view), code |
|
|
| `DialogConfirmation` | Generic confirmation for destructive actions |
|
|
| `DialogConversationRename` | Edit conversation title |
|
|
|
|
**Server/Model Components** (`app/server/`, `app/models/`):
|
|
|
|
| Component | Responsibility |
|
|
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
|
| `ServerErrorSplash` | Error display when server is unreachable |
|
|
| `ModelsSelector/` | Model dropdown and mobile sheet: favorites, loaded and local sections, rows windowed and folded into families |
|
|
| `ModelsManager/` | Models manager table and model pane: sections, filters, sorting, quants folded per repo, downloads |
|
|
| `ModelAvatar` | Org avatar of a model, with the quantizer badge; follows the family grouping of the list |
|
|
| `ModelId`, `ModelBadge` | Model name, aliases, tags, quantization and draft sidecar badges |
|
|
| `ModelContext`, `ModelCapabilities` | Context window and capability icons, filled from the Hub when it is enabled |
|
|
|
|
**Shared UI Components** (`app/misc/`):
|
|
|
|
| Component | Responsibility |
|
|
| ----------------------------------------- | ----------------------------------------------------------------------- |
|
|
| `MarkdownContent` | Markdown rendering with KaTeX, syntax highlighting, copy buttons |
|
|
| `SyntaxHighlightedCode` | Code blocks with language detection and highlighting |
|
|
| `ActionIcon`, `DropdownMenuActions` | Reusable action button and action menu |
|
|
| `BadgesModality`, `BadgeInfo` | Status and capability badges |
|
|
| `CollapsibleSection`, `CollapsibleRegion` | Section header with a trigger that expands in place |
|
|
| `GroupedList` | Grouped rows with foldable groups, per-group windows and show-more rows |
|
|
| `ScrollCarousel`, `TruncatedText` | Horizontal scroll strip and single-line truncation with tooltip |
|
|
|
|
#### Hooks (`src/lib/hooks/`)
|
|
|
|
Hooks are the thin view-layer between components and stores: they own UI concerns (scroll, drag-and-drop, keyboard shortcuts, pickers, selection) and translate store state into view state.
|
|
|
|
| Hook | Responsibility |
|
|
| ------------------------------- | ----------------------------------------------------------------------------------------------- |
|
|
| `use-chat-screen-active-model` | Active model resolution + modality capability detection |
|
|
| `use-processing-state` | View over `chatStore.processing` for streaming progress/tokens |
|
|
| `use-context-gauge` | View over `contextStatsStore` for the context usage gauge |
|
|
| `use-models-selector` | Model selector state: hidden models, family grouping, windowed rows, selection and load on pick |
|
|
| `use-tools-panel` | Tools panel state |
|
|
| `use-reasoning-menu` | Reasoning-effort menu state |
|
|
| `use-attachment-menu` | Attachment menu + modality flags |
|
|
| `use-draft-messages` | Per-chat draft message/files persistence |
|
|
| `use-chat-form-pickers` | Chat form pickers (commands, mentions) |
|
|
| `use-debounced-search` | Shared debounced async search for pickers |
|
|
| `use-picker-navigation` | Picker keyboard navigation |
|
|
| `use-chat-message-edit-context` | Message edit context (content + extras) |
|
|
| `use-chat-screen-drag-and-drop` | Drag-and-drop state machine |
|
|
| `use-chat-screen-file-upload` | File upload queue + capability validation |
|
|
| `use-stick-to-bottom` | Follows new content while the reader sits at the bottom of a scroller |
|
|
| `use-marquee-selection` | Shift+click / marquee range selection |
|
|
| `use-keyboard-shortcuts` | Global keyboard shortcuts |
|
|
| `use-settings-navigation` | Settings section navigation |
|
|
| `use-pwa` | PWA install/update + version mismatch detection |
|
|
|
|
#### Stores (`src/lib/stores/`)
|
|
|
|
Stores own reactive application state as Svelte 5 runes. Larger stores are split into directories and compose focused sub-stores behind a narrow host interface (see Architectural Patterns).
|
|
|
|
| Store | Responsibility |
|
|
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `chatStore` | Chat lifecycle, streaming, abort control, error handling; composes `processing`, `activity`, `streams`, `flows` |
|
|
| `conversationsStore` | Conversation CRUD, message branching, navigation, import/export; composes `preferences` |
|
|
| `modelsStore` | Model list, selection, loading/unloading (ROUTER), recent picks, favorites, hidden models and list open state; composes `props`, `status` |
|
|
| `mcpStore` | MCP host role: multi-server lifecycle, tool routing, server icons; composes `health`, `resources` |
|
|
| `agenticStore` | Multi-turn agentic loop orchestration, tool execution; composes `gates` |
|
|
| `serverStore` | Server connection state, `/props`, role detection, modalities |
|
|
| `settingsStore` | User preferences, theme, parameter sync with server defaults |
|
|
| `toolsStore` | Tool registry: server + MCP tools, enabled set for the LLM |
|
|
| `permissionsStore` | Persisted tool permission grants |
|
|
| `contextStatsStore` | Context window usage for the active conversation |
|
|
| `draftMessagesStore` | Per-chat draft message/files |
|
|
| `deviceStore` | Browser environment signals (mobile, OS, theme) |
|
|
| `versionStore` | Build version information |
|
|
|
|
#### Services (`src/lib/services/`)
|
|
|
|
Services are a stateless protocol layer: static methods, pure I/O, no reactive state. Stores consume them for all API and storage access.
|
|
|
|
| Service | Responsibility |
|
|
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
|
| `ChatService` | `/v1/chat/completions` streaming + SSE parsing, message format conversion |
|
|
| `ModelsService` | `/models`, `/models/load`, `/models/unload` |
|
|
| `PropsService` | `/props`, `/props?model=` |
|
|
| `HuggingFaceService` | Hugging Face Hub metadata: model details, avatars, README and file tree, cached per repo and off unless the Hub setting is on |
|
|
| `DatabaseService` | IndexedDB operations via Dexie |
|
|
| `MCPService` | MCP protocol: transports, connect, list/execute tools, prompts, resources |
|
|
| `ToolsService` | Server tool list/execute/stream (`/tools`) |
|
|
| `SandboxService` | Browser JS execution in a sandboxed worker |
|
|
| `ParameterSyncService` | Syncs settings with server defaults |
|
|
| `ConversationTransferService` | Conversation import/export JSONL + ZIP format |
|
|
| `MigrationService` | Non-destructive localStorage/IndexedDB migrations |
|
|
| `RouterService` | Dynamic route URL construction |
|
|
|
|
---
|
|
|
|
## Data Flows
|
|
|
|
### MODEL Mode (Single Model)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant UI
|
|
participant Stores
|
|
participant DB as IndexedDB
|
|
participant API as llama-server
|
|
|
|
Note over User,API: Initialization
|
|
UI->>Stores: initStores() (awaited by route loads)
|
|
Stores->>Stores: run migrations
|
|
Stores->>DB: load conversations (background)
|
|
Stores->>API: GET /props
|
|
API-->>Stores: server config
|
|
Stores->>API: GET /v1/models
|
|
API-->>Stores: single model (auto-selected)
|
|
|
|
Note over User,API: Chat Flow
|
|
User->>UI: send message
|
|
Stores->>DB: save user message
|
|
Stores->>API: POST /v1/chat/completions (stream)
|
|
loop streaming
|
|
API-->>Stores: SSE chunks
|
|
Stores-->>UI: reactive update
|
|
end
|
|
Stores->>DB: save assistant message
|
|
```
|
|
|
|
### ROUTER Mode (Multi-Model)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant User
|
|
participant UI
|
|
participant Stores
|
|
participant API as llama-server
|
|
|
|
Note over User,API: Initialization
|
|
Stores->>API: GET /props
|
|
API-->>Stores: {role: "router"}
|
|
Stores->>API: GET /models
|
|
API-->>Stores: models[] with status
|
|
|
|
Note over User,API: Model Selection
|
|
User->>UI: select model
|
|
alt model not loaded
|
|
Stores->>API: POST /models/load
|
|
loop poll status
|
|
Stores->>API: GET /models
|
|
end
|
|
Stores->>API: GET /props?model=X
|
|
end
|
|
Stores->>Stores: validate modalities
|
|
|
|
Note over User,API: Chat Flow
|
|
Stores->>API: POST /v1/chat/completions {model: X}
|
|
loop streaming
|
|
API-->>Stores: SSE chunks + model info
|
|
end
|
|
```
|
|
|
|
---
|
|
|
|
## Architectural Patterns
|
|
|
|
### 1. Reactive State with Svelte 5 Runes
|
|
|
|
All stores use Svelte 5's fine-grained reactivity:
|
|
|
|
```typescript
|
|
// Store with reactive state
|
|
class ChatStore {
|
|
#isLoading = $state(false);
|
|
#currentResponse = $state('');
|
|
|
|
// Derived values auto-update
|
|
get isStreaming() {
|
|
return $derived(this.#isLoading && this.#currentResponse.length > 0);
|
|
}
|
|
}
|
|
|
|
// Exported reactive accessors
|
|
export const isLoading = () => chatStore.isLoading;
|
|
export const currentResponse = () => chatStore.currentResponse;
|
|
```
|
|
|
|
### 2. Unidirectional Data Flow
|
|
|
|
Data flows in one direction, making state predictable:
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph UI["UI Layer"]
|
|
A[User Action] --> B[Component]
|
|
end
|
|
|
|
subgraph State["State Layer"]
|
|
B --> C[Store Method]
|
|
C --> D[State Update]
|
|
end
|
|
|
|
subgraph IO["I/O Layer"]
|
|
C --> E[Service]
|
|
E --> F[API / IndexedDB]
|
|
F -.->|Response| D
|
|
end
|
|
|
|
D -->|Reactive| B
|
|
```
|
|
|
|
Components dispatch actions to stores, stores coordinate with services for I/O, and state updates reactively propagate back to the UI.
|
|
|
|
### 3. Per-Conversation State
|
|
|
|
Enables concurrent streaming across multiple conversations. Loading is tracked
|
|
per conversation by the activity ledger (`chatStore.activity`), while streaming
|
|
state and abort controllers live in per-conversation maps:
|
|
|
|
```typescript
|
|
class ChatStore {
|
|
chatStreamingStates = new SvelteMap<string, { response: string; messageId: string }>();
|
|
abortControllers = new SvelteMap<string, AbortController>();
|
|
}
|
|
```
|
|
|
|
### 4. Message Branching with Tree Structure
|
|
|
|
Conversations are stored as a tree, not a linear list:
|
|
|
|
```typescript
|
|
interface DatabaseMessage {
|
|
id: string;
|
|
parent: string | null; // Points to parent message
|
|
children: string[]; // List of child message IDs
|
|
// ...
|
|
}
|
|
|
|
interface DatabaseConversation {
|
|
currentNode: string; // Currently viewed branch tip
|
|
// ...
|
|
}
|
|
```
|
|
|
|
Navigation between branches updates `currentNode` without losing history.
|
|
|
|
### 5. Layered Service Architecture
|
|
|
|
Stores handle state; services handle I/O:
|
|
|
|
```text
|
|
┌─────────────────┐
|
|
│ Stores │ Business logic, state management
|
|
├─────────────────┤
|
|
│ Services │ API calls, database operations
|
|
├─────────────────┤
|
|
│ Storage/API │ IndexedDB, LocalStorage, HTTP
|
|
└─────────────────┘
|
|
```
|
|
|
|
### 6. Server Role Abstraction
|
|
|
|
Single codebase handles both MODEL and ROUTER modes:
|
|
|
|
```typescript
|
|
// serverStore.ts
|
|
get isRouterMode() {
|
|
return this.role === ServerRole.ROUTER;
|
|
}
|
|
|
|
// Components conditionally render based on mode
|
|
{#if isRouterMode()}
|
|
<ModelsSelector />
|
|
{/if}
|
|
```
|
|
|
|
### 7. Modality Validation
|
|
|
|
Prevents sending attachments to incompatible models. The
|
|
`use-chat-screen-active-model` hook derives the active model's capabilities
|
|
from `modelsStore.props`:
|
|
|
|
```typescript
|
|
// use-chat-screen-active-model hook
|
|
const hasVisionModality = $derived.by(() => modelsStore.props.modelSupportsVision(activeModelId));
|
|
const hasAudioModality = $derived.by(() => modelsStore.props.modelSupportsAudio(activeModelId));
|
|
```
|
|
|
|
### 8. Persistent Storage Strategy
|
|
|
|
Data is persisted across sessions using two storage mechanisms:
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph Browser["Browser Storage"]
|
|
subgraph IDB["IndexedDB (Dexie)"]
|
|
C[Conversations]
|
|
M[Messages]
|
|
end
|
|
subgraph LS["LocalStorage"]
|
|
S[Settings Config]
|
|
O[User Overrides]
|
|
T[Theme Preference]
|
|
end
|
|
end
|
|
|
|
subgraph Stores["Svelte Stores"]
|
|
CS[conversationsStore] --> C
|
|
CS --> M
|
|
SS[settingsStore] --> S
|
|
SS --> O
|
|
SS --> T
|
|
end
|
|
```
|
|
|
|
- **IndexedDB**: Conversations and messages (large, structured data)
|
|
- **LocalStorage**: Settings, user parameter overrides, theme, model favorites, recent picks, hidden models and model list open state (small key-value data)
|
|
- **Memory only**: Server props, model list (fetched fresh on each session), Hugging Face Hub details (cached per repo for the session)
|
|
|
|
### 9. Remote Assets Under Cross-Origin Isolation
|
|
|
|
llama-server serves the UI with `Cross-Origin-Embedder-Policy: require-corp` and `Cross-Origin-Opener-Policy: same-origin`, so a cross-origin subresource has to be allowed by the target origin. The UI handles that per asset kind:
|
|
|
|
- **Hub avatars** are requested in CORS mode (`crossorigin="anonymous"`), which the Hub allows
|
|
- **Hub metadata** is fetched with `fetch()`, which is a CORS request by default, and only while the Hub setting is on
|
|
- **MCP server icons and web-search result favicons** have no CORS guarantee, so they go through the server's `/cors-proxy` when the user started llama-server with `--ui-mcp-proxy`; without it they fall back to the bundled glyph
|
|
- **Per-row Hub lookups** wait until the row is near the viewport (the `nearViewport` action), so a long model list does not fire one request per row on mount
|
|
|
|
### 10. Nested Surfaces
|
|
|
|
The sidebar stacks four surfaces, outermost first:
|
|
|
|
| Layer | Example | Corners |
|
|
| ----- | ---------------------------------- | ---------------------------------- |
|
|
| Page | the body background | none |
|
|
| Panel | the sidebar | `rounded-2xl` |
|
|
| Group | the sidebar's footer, inset in it | `rounded-(--surface-group-radius)` |
|
|
| Row | Models, MCP or Settings in a group | `rounded-(--surface-row-radius)` |
|
|
|
|
A row keeps its corners concentric with its group's: its radius is the group's less the padding between their edges, so the curves run parallel instead of pinching at the corners. `app.css` holds the three tokens (`--surface-group-radius`, `--surface-group-padding` and the derived `--surface-row-radius`); change the group's radius or padding there and the rows follow. A group pads its rows with `p-(--surface-group-padding)`. `SidebarNavigationActions` rounds every row from the token, so the actions at the top of the sidebar (New chat, Search) match the footer's rows; a phone sizes its rows for a thumb instead. A selected row is raised and edged (`bg-background`, `shadow-sm`, `ring-1`), where a hovered one is only tinted, so the two never look alike. The group's hairline border is left out of the sum. `tests/client/nested-surfaces.svelte.test.ts` checks the rule on the rendered footer.
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
### Test Types
|
|
|
|
| Type | Tool | Location | Command |
|
|
| ------------- | ------------------ | ---------------- | ------------------- |
|
|
| **Unit** | Vitest | `tests/unit/` | `npm run test:unit` |
|
|
| **UI/Visual** | Storybook + Vitest | `tests/stories/` | `npm run test:ui` |
|
|
| **E2E** | Playwright | `tests/e2e/` | `npm run test:e2e` |
|
|
| **Client** | Vitest | `tests/client/`. | `npm run test:unit` |
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# All tests
|
|
npm run test
|
|
|
|
# Individual test suites
|
|
npm run test:e2e # End-to-end (requires llama-server)
|
|
npm run test:client # Client-side unit tests
|
|
npm run test:server # Server-side unit tests
|
|
npm run test:ui # Storybook visual tests
|
|
```
|
|
|
|
Locally, the browser tests (e2e, client and Storybook) run in your installed Google Chrome with its sandbox on, since Chrome updates itself and Playwright's bundled Chromium lags behind security fixes. CI (`CI` set) uses the bundled Chromium from `npx playwright install`.
|
|
|
|
### Storybook Development
|
|
|
|
```bash
|
|
npm run storybook # Start Storybook dev server on :6006
|
|
npm run build-storybook # Build static Storybook
|
|
```
|
|
|
|
### Linting and Formatting
|
|
|
|
```bash
|
|
npm run lint # Check code style
|
|
npm run format # Auto-format with Prettier
|
|
npm run check # TypeScript type checking
|
|
```
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```text
|
|
tools/ui/
|
|
├── src/
|
|
│ ├── lib/
|
|
│ │ ├── components/ # UI components (app/, ui/)
|
|
│ │ ├── hooks/ # Svelte hooks
|
|
│ │ ├── stores/ # State management
|
|
│ │ ├── services/ # API and database services
|
|
│ │ ├── types/ # TypeScript interfaces
|
|
│ │ └── utils/ # Utility functions
|
|
│ ├── routes/ # SvelteKit routes
|
|
│ └── styles/ # Global styles
|
|
├── static/ # Static assets
|
|
├── tests/ # Test files
|
|
└── .storybook/ # Storybook configuration
|
|
```
|
|
|
|
---
|
|
|
|
## Related Documentation
|
|
|
|
- [llama.cpp Server README](../server/README.md) - Full server documentation
|
|
- [Multimodal Documentation](../../docs/multimodal.md) - Image and audio support
|
|
- [Function Calling](../../docs/function-calling.md) - Tool use capabilities
|