Files
llama.cpp/tools/ui/demo
Aleksander Grygier 0943ca5472 ui: add Playwright harness for recording demo videos (#31318)
* ui: add Playwright harness for recording demo videos

Scripted walkthroughs under tools/ui/demo are recorded with Playwright and
exported as H.264 mp4 for pull requests (demo/pr) and docs (demo/docs).
Fixture MCP servers and seeded settings keep recordings deterministic, a
drawn cursor and captions make the flow readable, and DEMO_UI_DIR points
the recording at another checkout such as a PR worktree.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record demo scripts from PR descriptions and a configurable storage

Demo scripts no longer live in the repo. `npm run demo:pr -- <number>`
records the `ts demo` block from a PR description against the PR head,
uploads the mp4, a GIF preview and the script to DEMO_STORAGE, and links
the preview in a `## Demo` section of the PR description.
`npm run demo:docs` records the docs scripts kept in DEMO_STORAGE.
DEMO_STORAGE (hf:// buckets or file://) and the public URL are read
from tools/ui/.env.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: import generated conversations into demo recordings

Demos can now start with existing chats: `test.use({ traces })` imports
llama.app JSONL traces through Settings > Import/Export before the video
starts. Built-in traces are generated deterministically, other names are
downloaded from DEMO_TRACES_DATASET, and `demo:traces` pushes the built-in
ones there. Adds `demo.scroll` for wheel scrolling.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: keep demo captions clear of the chat form and isolate concurrent runs

Captions move to the top of the frame, since at the bottom they covered
the chat form a demo is often about. Each recording run gets its own
Playwright output folder, so a run starting elsewhere no longer clears
another run's video before it is saved.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record demos at 2x and 60 fps with human-like timing

Recording goes through Chromium's screencast instead of Playwright's
video, which encoded VP8 at about 1 Mbit/s at 1x. Frames are captured at
the device pixel ratio (DEMO_SCALE, default 2) and encoded into a 60 fps
H.264 mp4, starting once the page has settled.

The cursor now glides faster along a slight arc with Fitts-like
durations, settles briefly before each click, and typing and pauses vary
a little. The randomness is seeded per demo, so re-recording gives the
same motion.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: document the demo harness prerequisites

The demo README now lists the tools the harness needs and what each is
for, and walks through creating the Hugging Face bucket and the optional
traces dataset, with the matching .env settings.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: version linked demo URLs so re-recorded previews refresh

GitHub's image proxy caches a preview by URL, so a demo re-recorded at
the same PR head kept showing the old GIF. The Demo section now links
the files with a content hash in the query string.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: label PRs that feature a video demo

Adds a `video demo` label and a workflow that applies it when a PR
description has a demo written by the UI demo pipeline (the managed
demo:start/demo:end markers, or a preview linking to a demo video),
and removes it when that section is gone.

Assisted-by: Claude (Opus 5.5 Medium)

* ci: label video demos with gh instead of github-script

The label workflow now runs the gh CLI in a shell step, like
pr-draft-label.yml, so it needs no action outside the repository's
allowed list. The video link check only counts videos uploaded for the
PR itself, so a description linking other PRs' demos isn't labeled.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: zoom demos in on details with a two-finger pinch

demo.zoom pinches in on a control or a point, holds, and pinches back out
to 1x, so a detail too small to read in the frame can be shown up close.
It drives Chromium's own pinch gesture, so the page's fixed bars stay put,
and works at desktop and phone sizes alike.

The pinch is evened out where Chromium misses: one anchored near the top
does not zoom at all, so it starts lower, and one near the bottom
overshoots its ratio, so a small pinch from the middle corrects it. The
way out reads the real scale until the page is back at 1x, which the next
gesture needs.

The overlay also draws a fingertip per touch, so a pinch on a touch
device shows as two fingers.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: wait out a cold dev server before a demo starts

A fresh checkout's dev server optimizes its dependencies on the first
load and reloads the page, which can take well past an action's timeout,
so a demo failed before it started. The demo now waits up to 90 seconds
for the app; the recording only starts once it is ready.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: record a video per test in a PR demo, shown in a table

A large PR can split its demo into several shorter videos: each test()
in the ts demo block is recorded as its own video (<sha>-1.mp4, ...),
and the Demo section lays their previews out in a table, each column
headed by its test's title.

GIF previews now keep their longer side at DEMO_PREVIEW_WIDTH and shrink
until they are under 5 MB, since GitHub's image proxy refuses anything
larger and shows a broken image instead.

Importing traces works on a phone viewport too, where Settings sits in
the sidebar under the page until the menu key slides it aside.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: fix demo trace imports on desktop and lay landscape demos out two to a row

Importing traces failed on a desktop whose collapsed sidebar sits under
the page: it now expands the sidebar from the page's menu key on a phone
or from the sidebar's logo on a desktop, whichever makes Settings
reachable. Afterwards it forgets the expanded sidebar, so a demo starts
with it collapsed, as on a first visit.

A PR demo with several videos shows up to four portrait (phone) videos
to a row, or two landscape ones, so each preview stays readable.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: only record other people's PR demos at a reviewed head commit

demo:pr runs the PR's code and demo script on this machine, with the
gh and hf logins, so a PR by someone else now needs --trust <sha>
naming the head commit that was read, and is refused again after new
pushes. Recordings also launch Chromium with its sandbox on, which
Playwright turns off by default.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-03 12:30:49 +01:00
..

Demo videos

Scripted Playwright walkthroughs of the UI, exported as mp4 with an animated GIF preview, for pull requests and documentation. The harness lives here; the demo scripts don't:

  • PR demos live in the PR description, as a ts demo code block.
  • Docs demos live in the storage target, under scripts/docs/.
demo/
├── cli.ts      demo:pr, demo:docs and demo:traces
├── storage.ts  storage targets (hf://, file://)
├── helpers.ts  `test` with a `demo` fixture: cursor, captions, human-like timing
├── recorder.ts screencast recording and mp4 export
├── fixtures/   fixture MCP servers, seeded settings and demo conversations
└── videos/     output (git-ignored)

Setup

Tools

Tool Used for
Node 22.18+ runs demo/cli.ts directly (TypeScript type stripping)
npx playwright install chromium the browser demos record in, with its sandbox on
ffmpeg mp4 encoding and the GIF preview
gh, logged in reads the PR description and writes the ## Demo section, so you need edit rights on the PR
hf, logged in with a write token (hf auth login) uploads to the bucket and the traces dataset
llama-server build/bin/llama-server, or one already running at DEMO_SERVER_ORIGIN (default :9931). It runs in router mode, so demos can pick any model in the local cache; scripts that send messages need at least one.

gh is only needed for demo:pr, and hf only with the bucket or dataset below. Without them, the video stays in demo/videos/.

Videos are recorded through Chromium's screencast at DEMO_SCALE (2x by default, so 2880x1800) and 60 fps, starting once the page has settled.

Hugging Face bucket (videos, previews, scripts)

The bucket must be public, since GitHub loads the preview from it:

hf buckets create <owner>/llama-ui-demos

Then set it in tools/ui/.env (see .env.example):

DEMO_STORAGE='hf://buckets/<owner>/llama-ui-demos'
DEMO_STORAGE_PUBLIC_URL='https://huggingface.co/buckets/<owner>/llama-ui-demos/resolve'

Layout: videos/pr/<number>/<sha>.{mp4,gif,demo.ts}, videos/docs/<name>.{mp4,gif} and scripts/docs/*.demo.ts (the docs scripts). Without DEMO_STORAGE_PUBLIC_URL, videos are uploaded but not linked in the PR. DEMO_STORAGE='file:///some/dir' stores them in a local folder instead.

Hugging Face dataset (conversations, optional)

Only needed for demos that import conversations other than the built-in ones, or to publish the built-in ones with npm run demo:traces. It can be private: it is read with your hf login.

hf repos create <owner>/llama-ui-demo-traces --repo-type dataset --private
DEMO_TRACES_DATASET='<owner>/llama-ui-demo-traces'

Other storage

Only Hugging Face buckets and file:// folders are supported for now. S3, R2 or similar would be one more class in storage.ts (upload and downloadDir) plus the matching DEMO_STORAGE_PUBLIC_URL.

Recording a pull request

Add the script to the PR description:

<details>
<summary>Demo script</summary>

```ts demo
test('demo', async ({ demo, page }) => {
	await demo.click(page.getByRole('button', { exact: true, name: 'Settings' }));
	await demo.caption('What this shows');
});
```

</details>

Then run:

npm run demo:pr -- 31303

A PR by someone else runs its code and demo script on your machine, with your gh and hf logins, so demo:pr refuses it until you've read its diff and demo block and pass the head commit you read: npm run demo:pr -- 31303 --trust <sha>. If the PR gets new commits, it refuses again.

It checks out the PR head in a temporary worktree, records the script against it, uploads videos/pr/<number>/<sha>.{mp4,gif,demo.ts} and adds or updates a ## Demo section in the PR description with the preview linking to the video.

A large PR can show its changes in several shorter videos: each test() in the block is recorded as its own video (<sha>-1.mp4, <sha>-2.mp4, ...), and the ## Demo section shows their previews in a table, each headed by its test's title: up to four portrait (phone) videos to a row, or two landscape ones. Helpers and test.use at the top of the block are shared by all of them.

test('Header and safe areas', async ({ demo, page }) => {
	// ...
});

test('Sidebar gestures', async ({ demo, page }) => {
	// ...
});
``` GitHub has no API for attachments, so for an inline player drag `demo/videos/pr-<number>.mp4` into the description instead.

## Recording docs

```bash
npm run demo:docs

Records every scripts/docs/*.demo.ts from the storage target and uploads videos/docs/<name>.{mp4,gif}.

Writing a demo

Scripts get test and expect from helpers.ts without importing them. demo.click, demo.hover and demo.type glide a visible cursor to the target before acting, with timing varied like a person's (seeded per demo, so re-takes match); demo.scroll turns the mouse wheel over a target; demo.zoom pinches in on a control or a point to show a detail too small to read in the frame, holds, and pinches back out (with the fingertips drawn on a touch device); demo.caption shows a caption at the top of the frame. Use test.use({ settings, startPath }) to seed other settings or start on another route.

Conversations

Demos that need existing chats import them with test.use({ traces: ['long-chat'] }), through Settings > Import/Export before the video starts. Traces use the export format (llama.app JSONL), so the Hub shows them as Llama traces:

  • fixtures/traces.ts generates the built-in ones: long-chat (36 tall exchanges) and agentic-debugging (long reasoning, shell output and code blocks).
  • Any other name is downloaded as <name>.jsonl from DEMO_TRACES_DATASET, e.g. an exported conversation uploaded there.

npm run demo:traces writes the built-in traces to demo/traces/ and pushes them to DEMO_TRACES_DATASET.

Trying a script locally

To try a script locally, put it in demo/ as <name>.demo.ts, importing from ./helpers.ts, and run npm run demo:record -- demo/<name>. npm run demo:codegen opens Playwright codegen against :5180 for a first draft.

The config starts, or reuses when already running, the fixture MCP servers on :8765, llama-server on :9931 (router mode, --tools all) and the UI's vite dev server.