From f9a5c231ed856143e309cf0da5199553ea16f1be Mon Sep 17 00:00:00 2001
From: Lumpiasty <arek.dzski@gmail.com>
Date: Thu, 10 Sep 2026 17:06:01 +0200
Subject: [PATCH] scripts: add the RX 580 benchmark harness

Encodes the production config, fixed corpus slices, repeat/median discipline
and the noise floor, so the measurement method does not have to be
rediscovered each time. Runs the corpus prefill test through llama-server and
the llama-bench sweep as a controlled cross-check, with interleaved A/B.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
---
 scripts/rx580-bench/README.md    | 200 ++++++++++++++++++++++++++
 scripts/rx580-bench/bench.sh     | 167 +++++++++++++++++++++
 scripts/rx580-bench/lbsweep.sh   |  69 +++++++++
 scripts/rx580-bench/ppbench.py   | 239 +++++++++++++++++++++++++++++++
 scripts/rx580-bench/run.sh       |  91 ++++++++++++
 scripts/rx580-bench/summarize.py |  95 ++++++++++++
 6 files changed, 861 insertions(+)
 create mode 100644 scripts/rx580-bench/README.md
 create mode 100755 scripts/rx580-bench/bench.sh
 create mode 100755 scripts/rx580-bench/lbsweep.sh
 create mode 100755 scripts/rx580-bench/ppbench.py
 create mode 100755 scripts/rx580-bench/run.sh
 create mode 100755 scripts/rx580-bench/summarize.py

diff --git a/scripts/rx580-bench/README.md b/scripts/rx580-bench/README.md
new file mode 100644
index 000000000..f029c6a8f
--- /dev/null
+++ b/scripts/rx580-bench/README.md
@@ -0,0 +1,200 @@
+# rx580-bench
+
+Prompt-processing benchmark harness for the RX 580 (Polaris) serving pod.
+
+It exists so that no future session has to rediscover how to measure this
+machine. It encodes the production config, fixed corpus slices, the repeat
+and median discipline, and the noise floor.
+
+## What it measures
+
+Two things, deliberately:
+
+1. **Corpus prompt processing** (the number we actually care about). Real
+   Polish prose from `pan-tadeusz.txt` pushed through `llama-server`'s
+   `/completion` endpoint with `n_predict: 1`, `cache_prompt: false`. This
+   mimics agentic tool-result parsing: a big blob of real text arriving cold.
+   Reported as `timings.prompt_per_second` at prompt lengths of about 4k, 16k
+   and 32k tokens.
+2. **`llama-bench` sweep** (the controlled cross-check). Synthetic random
+   tokens, `pp2048`, `pp8192`, `tg32`. This is what prior work on this box
+   recorded, so it is the continuity metric.
+
+They do not measure the same thing and they do not agree. `llama-bench`
+pp2048 is a cold depth-0 batch; the corpus numbers include the cost of a
+growing KV cache, so they fall off with prompt length. Both are useful.
+
+## How to run it
+
+One command from the workstation:
+
+```
+./scripts/rx580-bench/run.sh --build /root/arms/new-clean/build --label new-clean
+```
+
+Interleaved A/B, which is the only valid way to compare two builds:
+
+```
+./scripts/rx580-bench/run.sh \
+  --build   /root/arms/new-clean/build --label   new-clean \
+  --build-b /root/arms/new-fork/build  --label-b new-fork \
+  --rounds 3
+```
+
+Summarize whatever has accumulated:
+
+```
+./scripts/rx580-bench/run.sh --summarize
+```
+
+`run.sh` copies the harness onto the pod and execs it there. The wrapper is
+thin on purpose: all the methodology lives in the pod-side scripts, so it can
+also be driven directly on the pod if kubectl is inconvenient.
+
+The run is detached on the pod and then tailed, so Ctrl-C on the wrapper does
+not orphan a half-finished run. `bench.sh` restarts `llama-swap` from an EXIT
+trap on success, failure and interrupt alike.
+
+### Pod access
+
+```
+export KUBECONFIG=/home/user/Projects/klaster/talos/generated/kubeconfig
+kubectl -n llama exec -i deploy/supervisord -- sh -c '<command>'
+```
+
+The pod has `curl` and `python3` but **no `jq`**; the harness uses python3 for
+all JSON.
+
+## Files
+
+| file | side | what |
+|---|---|---|
+| `run.sh` | workstation | one-command wrapper; installs and execs the rest |
+| `bench.sh` | pod | orchestrator: stops llama-swap, waits for idle GPU, runs both harnesses, restarts llama-swap via trap |
+| `ppbench.py` | pod | the corpus prompt-processing harness |
+| `lbsweep.sh` | pod | the llama-bench sweep |
+| `summarize.py` | pod | turns `results.txt` into a median/min-max table with delta significance |
+
+Results append to `/root/bench/results.txt` as parseable `RESULT` /
+`SUMMARY` / `LBRESULT` lines. Server logs land in `/root/bench/server-*.log`.
+
+## The config it assumes
+
+Production serving config, matching `/root/config.yaml`:
+
+```
+-t 6 -ngl 99 --n-cpu-moe 40 -b 2048 -ub 2048 -fa 1 --no-mmap
+--ctx-size 40960 --no-warmup
+```
+
+Model:
+
+```
+/root/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF/snapshots/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
+```
+
+Qwen3.6-35B-A3B, 20.8 GiB, 40 layers, 256 experts of which 8 are used.
+Machine: AMD RX 580 8 GB (Polaris/GCN, RADV, no fp16 accel), PCIe 3.0 x16,
+12 CPU cores, 62 GB RAM.
+
+The `llama-bench` equivalent of that config is:
+
+```
+-t 6 -ngl 99 -ncmoe 40 -b 2048 -ub 2048 -fa 1 -mmp 0 -p 2048,8192 -n 32 -r 3
+```
+
+## The corpus slices are constants, not recalibrated
+
+`pan-tadeusz.txt` is 447334 chars / 482907 bytes of Polish text. Against this
+model's tokenizer it runs **2.6702 chars/token**, and `/completion` reports
+`prompt_n` identical to `/tokenize` (BOS offset 0).
+
+The slices are `corpus[0:NCHARS]` and are baked into `ppbench.py`:
+
+| target tokens | chars | utf-8 bytes |
+|---|---|---|
+| 4096 | 10776 | 11608 |
+| 16384 | 43858 | 47175 |
+| 32768 | 87165 | 93843 |
+
+These hit the target `prompt_n` exactly. Every build is therefore measured on
+byte-identical input. `ppbench.py` warns loudly if the reported `prompt_n`
+ever stops matching the target, which is the signal that the table is stale.
+Only recalibrate (`ppbench.py <build> <label> --calibrate`) if the model file
+or the corpus changes.
+
+## Reading the numbers
+
+* `prompt_n` is the token count actually prefilled. It must equal the target.
+* `prompt_ms` is wall-clock prefill time.
+* `tps` / `prompt_per_second` is `prompt_n / prompt_ms * 1000`. Higher better.
+* `pp2048` / `pp8192` are llama-bench prefill throughput at depth 0.
+* `tg32` is token generation at DEFAULT depth (n_ctx 32). **It is NOT a valid
+  decode metric on this box.** At that size decode is launch/barrier bound, not
+  compute bound, and measures a regime serving never reaches: it read -36%
+  between two upstream commits that are at parity in real use (the cause was
+  the view-alias fix in ggml_vk_graph_optimize). For decode use **tg256 with an
+  explicit depth**, e.g. `-p 0 -n 256 -d 2048`. tg is also NOT monotonic in
+  n_ctx here - on one binary n_ctx 256 measured slower than both 32 and 544 -
+  so only ever compare same-depth cells.
+
+## Noise floor: deltas under about 5 percent are unproven
+
+This is the single most important thing in this directory.
+
+`llama-bench`'s within-run error bars understate **cross-invocation** variance
+by roughly **14x**. The same config measured `276.7 +/- 0.9` in one invocation
+and `296.4 +/- 0.6` in another, 7 percent apart, with error bars that claimed
+0.3 percent precision.
+
+Therefore:
+
+* Any comparison you want to draw a conclusion from must live **inside a
+  single `llama-bench` invocation**, using comma-separated sweeps.
+* Two different *builds* cannot share an invocation, so run them
+  **interleaved A/B/A/B/A/B across at least 3 rounds** and compare medians.
+  That is what `--build-b` does.
+* State explicitly that any cross-build delta under about 5 percent is
+  unproven. `summarize.py` labels them `unproven` for you.
+
+## Traps
+
+* **`-tb` is not a `llama-bench` flag.** It is a `llama-server` flag. Passing
+  it makes `llama-bench` print usage and exit silently, mid-sweep, which looks
+  exactly like a run that produced nothing. Always check the row count.
+* **A ubatch larger than the prompt never fills.** With `-ub 2048`, prompt
+  lengths must be multiples of 2048 or the sweep measures nothing meaningful.
+* **Nothing else may touch the GPU.** `bench.sh` stops `llama-swap` and
+  refuses to start if any `llama-*` process is still alive.
+* **Never leave the pod not serving.** `bench.sh` restarts `llama-swap` from
+  an EXIT trap. If you bypass the script, restart it by hand:
+  `supervisorctl -c /root/supervisord.conf start llama-swap`.
+* **Do not run `llama-cli` detached with closed stdin** on this box; it loops
+  forever. Use the server harness or `llama-bench -d` for depth timing.
+* `--no-mmap` is deprecated upstream in favour of `--load-mode` / `-lm none`,
+  but still accepted and still what production passes.
+
+## Known-good reference values
+
+If a fresh run disagrees with these by much more than the noise floor,
+something is wrong (something else on the GPU, a thermal problem, a bad
+build) before you believe you found a speedup.
+
+Model Qwen3.6-35B-A3B-UD-Q4_K_XL, production config, current upstream
+(`f3f1a8f27`) and the deployed fork, measured 2026-09-08/09:
+
+| metric | clean upstream | fork |
+|---|---|---|
+| corpus pp @ 4k | about 240 t/s | about 255 t/s |
+| corpus pp @ 16k | about 206 t/s | about 225 t/s |
+| corpus pp @ 32k | about 171 t/s | about 195 t/s |
+| llama-bench pp2048 | see results.txt | about 285 t/s |
+| llama-bench tg32 | | not a valid metric, see above |
+
+Older reference points for the same model/config: `pp2048` depth-0 about
+285 t/s. The historical "tg32 about 19 t/s" figure is retained only as trivia -
+do not treat it as a target, and see the tg32 warning above.
+
+Model load with `--no-mmap` takes about 26 s warm, longer cold.
+A full 3-round A/B run of both harnesses takes roughly 100 minutes;
+add about 16 minutes per extra arm per round.
diff --git a/scripts/rx580-bench/bench.sh b/scripts/rx580-bench/bench.sh
new file mode 100755
index 000000000..cb9dbc0d9
--- /dev/null
+++ b/scripts/rx580-bench/bench.sh
@@ -0,0 +1,167 @@
+#!/bin/sh
+#
+# RX 580 prompt-processing benchmark orchestrator. Runs ON THE POD.
+#
+# Stops llama-swap, waits for the GPU to go idle, runs the real-corpus
+# prompt-processing harness and the llama-bench sweep for every arm, appends
+# parseable results, and ALWAYS restarts llama-swap on exit (success, error
+# or Ctrl-C).
+#
+# Usage:
+#   bench.sh --build DIR --label NAME
+#            [--build-b DIR --label-b NAME]   second arm, interleaved
+#            [--arm DIR:LABEL ...]            extra arms, repeatable (N-way)
+#            [--rounds N]                     default 3 multi-arm, 1 otherwise
+#            [--reps N]                       corpus repeats per size, default 3
+#            [--sizes 4096,16384,32768]
+#            [--corpus-only | --lb-only]
+#            [--results PATH]                 default /root/bench/results.txt
+#            [--keep-swap-down]               do not restart llama-swap at exit
+#
+set -e
+
+BENCH_DIR=/root/bench
+SUPCONF=/root/supervisord.conf
+RESULTS="$BENCH_DIR/results.txt"
+MODEL=/root/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF/snapshots/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
+
+ARMS=""            # space separated DIR|LABEL entries
+BUILD_A=""; LABEL_A=""
+BUILD_B=""; LABEL_B=""
+ROUNDS=""
+REPS=3
+SIZES=4096,16384,32768
+DO_CORPUS=1
+DO_LB=1
+KEEP_SWAP_DOWN=0
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --build)     BUILD_A="$2"; shift 2 ;;
+    --label)     LABEL_A="$2"; shift 2 ;;
+    --build-b)   BUILD_B="$2"; shift 2 ;;
+    --label-b)   LABEL_B="$2"; shift 2 ;;
+    --arm)       ARMS="$ARMS ${2%%:*}|${2#*:}"; shift 2 ;;
+    --rounds)    ROUNDS="$2"; shift 2 ;;
+    --reps)      REPS="$2"; shift 2 ;;
+    --sizes)     SIZES="$2"; shift 2 ;;
+    --results)   RESULTS="$2"; shift 2 ;;
+    --corpus-only) DO_LB=0; shift ;;
+    --lb-only)     DO_CORPUS=0; shift ;;
+    --keep-swap-down) KEEP_SWAP_DOWN=1; shift ;;
+    -h|--help)   sed -n '2,20p' "$0"; exit 0 ;;
+    *) echo "unknown arg: $1" >&2; exit 2 ;;
+  esac
+done
+
+if [ -n "$BUILD_A" ] && [ -z "$LABEL_A" ]; then
+  echo "error: --build needs --label" >&2; exit 2
+fi
+if [ -n "$BUILD_B" ] && [ -z "$LABEL_B" ]; then
+  echo "error: --build-b needs --label-b" >&2; exit 2
+fi
+# --build/--label and --build-b/--label-b are sugar for the first two arms.
+if [ -n "$BUILD_A" ]; then ARMS="$BUILD_A|$LABEL_A $ARMS"; fi
+if [ -n "$BUILD_B" ]; then ARMS="$ARMS $BUILD_B|$LABEL_B"; fi
+ARMS=$(echo $ARMS)
+if [ -z "$ARMS" ]; then
+  echo "error: need at least one arm (--build/--label or --arm DIR:LABEL)" >&2
+  exit 2
+fi
+NARMS=$(echo "$ARMS" | wc -w)
+if [ -z "$ROUNDS" ]; then
+  if [ "$NARMS" -gt 1 ]; then ROUNDS=3; else ROUNDS=1; fi
+fi
+for e in $ARMS; do
+  d=${e%%|*}
+  if [ ! -x "$d/bin/llama-server" ] || [ ! -x "$d/bin/llama-bench" ]; then
+    echo "error: $d lacks bin/llama-server or bin/llama-bench" >&2
+    exit 2
+  fi
+done
+
+cat <<'WARN'
+-------------------------------------------------------------------------
+RX 580 benchmark. Read this before trusting any number.
+
+  * Cross-invocation variance on this box is about 7 percent. llama-bench
+    within-run error bars understate it by roughly 14x. Any cross-build
+    delta under about 5 percent is UNPROVEN noise.
+  * Comparisons you care about must live inside ONE llama-bench invocation
+    (comma-separated sweeps), or be interleaved across at least 3 rounds.
+    That is what multiple arms plus --rounds does.
+  * Nothing else may touch the GPU while this runs. llama-swap is stopped
+    for the duration and restarted on exit, including on error or Ctrl-C.
+  * Traps: -tb is NOT a llama-bench flag. A ubatch larger than -p never
+    fills, so prompt lengths must be multiples of the 2048 ubatch. A bad
+    flag makes llama-bench print usage and exit silently mid-sweep.
+-------------------------------------------------------------------------
+WARN
+
+echo "[bench] $NARMS arm(s), $ROUNDS round(s): $ARMS"
+
+restore_swap() {
+  rc=$?
+  if [ "$KEEP_SWAP_DOWN" -eq 0 ]; then
+    echo ""
+    echo "[bench] restarting llama-swap"
+    supervisorctl -c "$SUPCONF" start llama-swap || true
+    supervisorctl -c "$SUPCONF" status llama-swap || true
+  else
+    echo "[bench] --keep-swap-down: llama-swap left STOPPED"
+  fi
+  exit $rc
+}
+trap restore_swap EXIT INT TERM
+
+echo "[bench] stopping llama-swap"
+supervisorctl -c "$SUPCONF" stop llama-swap || true
+
+wait_gpu_idle() {
+  i=0
+  while [ $i -lt 60 ]; do
+    if ! pgrep -f 'bin/llama-server|bin/llama-bench|bin/llama-cli' >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 2
+    i=$((i+1))
+  done
+  echo "error: something is still on the GPU:" >&2
+  pgrep -af 'bin/llama-server|bin/llama-bench|bin/llama-cli' >&2
+  exit 3
+}
+
+echo "[bench] waiting for the GPU to go idle"
+wait_gpu_idle
+echo "[bench] GPU idle"
+
+run_corpus() {
+  [ "$DO_CORPUS" -eq 1 ] || return 0
+  echo "===== corpus $2 ====="
+  wait_gpu_idle
+  python3 "$BENCH_DIR/ppbench.py" "$1" "$2" \
+    --sizes "$SIZES" --reps "$REPS" --results "$RESULTS"
+}
+
+run_lb() {
+  [ "$DO_LB" -eq 1 ] || return 0
+  echo "===== llama-bench $2 ====="
+  wait_gpu_idle
+  RESULTS="$RESULTS" MODEL="$MODEL" sh "$BENCH_DIR/lbsweep.sh" "$1" "$2"
+}
+
+# Interleave: every arm is measured once per round, so slow drift in the
+# machine hits all arms roughly equally instead of biasing whichever ran first.
+r=1
+while [ "$r" -le "$ROUNDS" ]; do
+  echo ""
+  echo "########## ROUND $r / $ROUNDS ##########"
+  for e in $ARMS; do run_corpus "${e%%|*}" "${e#*|}"; done
+  for e in $ARMS; do run_lb     "${e%%|*}" "${e#*|}"; done
+  r=$((r+1))
+done
+
+echo ""
+echo "ALL_RUNS_DONE"
+echo "[bench] raw results appended to $RESULTS"
+python3 "$BENCH_DIR/summarize.py" "$RESULTS" || true
diff --git a/scripts/rx580-bench/lbsweep.sh b/scripts/rx580-bench/lbsweep.sh
new file mode 100755
index 000000000..ec3ab26d8
--- /dev/null
+++ b/scripts/rx580-bench/lbsweep.sh
@@ -0,0 +1,69 @@
+#!/bin/sh
+#
+# llama-bench sweep, the controlled cross-check for the corpus harness.
+# Runs ON THE POD. Normally invoked by bench.sh.
+#
+# Usage: lbsweep.sh <build_dir> <label>
+# Env overrides: RESULTS, MODEL, LB_P, LB_N, LB_R
+#
+# Everything that must be compared lives inside ONE llama-bench invocation
+# (-p takes a comma-separated list), because cross-invocation variance on this
+# box is about 7 percent while the within-run error bars are about 0.3 percent.
+#
+# Flag traps, learned the hard way:
+#   * -tb is NOT a llama-bench flag. It exists on llama-server only.
+#   * -ncmoe is the llama-bench spelling of --n-cpu-moe.
+#   * -mmp 0 is the llama-bench equivalent of the server's --no-mmap.
+#   * A bad flag makes llama-bench print usage and exit silently, which looks
+#     exactly like a sweep that produced no rows. Always check row count.
+#   * A ubatch larger than -p never fills, so keep -p a multiple of -ub.
+#
+set -e
+BUILD_DIR="$1"
+LABEL="$2"
+if [ -z "$BUILD_DIR" ] || [ -z "$LABEL" ]; then
+  echo "usage: $0 <build_dir> <label>" >&2
+  exit 2
+fi
+
+RESULTS="${RESULTS:-/root/bench/results.txt}"
+MODEL="${MODEL:-/root/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF/snapshots/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf}"
+LB_P="${LB_P:-2048,8192}"
+LB_N="${LB_N:-32}"
+LB_R="${LB_R:-3}"
+
+STAMP=$(date -u +%Y-%m-%dT%H:%M:%S)
+export LD_LIBRARY_PATH="$BUILD_DIR/bin"
+export LABEL BUILD_DIR STAMP RESULTS
+
+OUT=$("$BUILD_DIR/bin/llama-bench" \
+  -m "$MODEL" \
+  -t 6 -ngl 99 -ncmoe 40 -b 2048 -ub 2048 -fa 1 -mmp 0 \
+  -p "$LB_P" -n "$LB_N" -r "$LB_R" -o json 2>/dev/null)
+
+if [ -z "$OUT" ]; then
+  echo "ERROR: llama-bench produced no output for $LABEL (bad flag?)" >&2
+  exit 3
+fi
+
+printf '%s' "$OUT" | python3 -c '
+import json, sys, os
+label = os.environ["LABEL"]
+build = os.environ["BUILD_DIR"]
+stamp = os.environ["STAMP"]
+results = os.environ["RESULTS"]
+rows = json.load(sys.stdin)
+if not rows:
+    sys.stderr.write("ERROR: llama-bench returned zero rows\n")
+    sys.exit(3)
+out = open(results, "a")
+for r in rows:
+    test = "pp%d" % r["n_prompt"] if r["n_prompt"] else "tg%d" % r["n_gen"]
+    line = ("LBRESULT label=%s build=%s test=%s tps=%.2f stddev=%.2f "
+            "samples=%s time=%s"
+            % (label, build, test, r["avg_ts"], r["stddev_ts"],
+               r["samples_ts"], stamp))
+    out.write(line + "\n")
+    print(line)
+out.close()
+'
diff --git a/scripts/rx580-bench/ppbench.py b/scripts/rx580-bench/ppbench.py
new file mode 100755
index 000000000..6e4d5ccde
--- /dev/null
+++ b/scripts/rx580-bench/ppbench.py
@@ -0,0 +1,239 @@
+#!/usr/bin/env python3
+"""
+Real-corpus prompt-processing benchmark for llama-server on the RX 580 pod.
+
+Runs ON THE POD. Normally invoked by bench.sh, not directly.
+
+Starts llama-server from a given build directory with the production config,
+waits for /health, POSTs fixed slices of the Pan Tadeusz corpus to /completion,
+records timings.prompt_n / prompt_ms / prompt_per_second, then shuts the server
+down cleanly.
+
+Prompt slices are FIXED CONSTANTS (see SLICES below), calibrated once against
+the Qwen3.6-35B-A3B tokenizer, so that every build is measured on byte-identical
+input. Do not re-calibrate for a normal run; use --calibrate only if the model
+or the corpus changes.
+
+Usage:
+  ppbench.py <build_dir> <label> [--sizes 4096,16384,32768] [--reps 3]
+             [--results PATH] [--calibrate]
+"""
+
+import json
+import os
+import signal
+import subprocess
+import sys
+import time
+import urllib.request
+from statistics import median
+
+BENCH_DIR = "/root/bench"
+CORPUS = os.path.join(BENCH_DIR, "pan-tadeusz.txt")
+DEFAULT_RESULTS = os.path.join(BENCH_DIR, "results.txt")
+
+MODEL = ("/root/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-GGUF/"
+         "snapshots/a483e9e6cbd595906af30beda3187c2663a1118c/"
+         "Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf")
+
+PORT = 8099
+BASE = "http://127.0.0.1:%d" % PORT
+
+# Production serving config. Keep in sync with /root/config.yaml (llama-swap).
+# Traps encoded here on purpose:
+#   -b / -ub 2048 : a ubatch larger than the prompt never fills, so every
+#                   prompt length below must stay a multiple of 2048.
+#   --no-mmap     : deprecated upstream in favour of --load-mode / -lm none,
+#                   but still accepted; it is what production passes today.
+SERVER_ARGS = ["-t", "6", "-ngl", "99", "--n-cpu-moe", "40",
+               "-b", "2048", "-ub", "2048", "-fa", "1", "--no-mmap",
+               "--ctx-size", "40960", "--no-warmup",
+               "--port", str(PORT), "-m", MODEL]
+
+# Calibrated once on 2026-09-08 against Qwen3.6-35B-A3B-UD-Q4_K_XL:
+# Polish text of pan-tadeusz.txt runs 2.6702 chars/token, and /completion
+# reports prompt_n identical to /tokenize (BOS offset 0).
+# Each slice is corpus[0:NCHARS] read as UTF-8 text; NBYTES is the resulting
+# UTF-8 byte length, recorded so the slice can be reproduced with byte tools.
+# These produce EXACTLY the target prompt_n. Do not edit without recalibrating.
+SLICES = {
+    4096:  {"chars": 10776, "bytes": 11608},
+    16384: {"chars": 43858, "bytes": 47175},
+    32768: {"chars": 87165, "bytes": 93843},
+}
+
+DEFAULT_SIZES = [4096, 16384, 32768]
+
+
+def post(path, payload, timeout=2400):
+    req = urllib.request.Request(
+        BASE + path,
+        data=json.dumps(payload).encode("utf-8"),
+        headers={"Content-Type": "application/json"},
+        method="POST")
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.loads(r.read().decode("utf-8"))
+
+
+def health_ok():
+    try:
+        with urllib.request.urlopen(BASE + "/health", timeout=5) as r:
+            return json.loads(r.read().decode("utf-8")).get("status") == "ok"
+    except Exception:
+        return False
+
+
+def start_server(build_dir, logpath):
+    env = dict(os.environ)
+    env["LD_LIBRARY_PATH"] = os.path.join(build_dir, "bin")
+    binary = os.path.join(build_dir, "bin", "llama-server")
+    if not os.path.exists(binary):
+        raise RuntimeError("no llama-server at %s" % binary)
+    log = open(logpath, "wb")
+    p = subprocess.Popen([binary] + SERVER_ARGS, stdout=log,
+                         stderr=subprocess.STDOUT, env=env,
+                         start_new_session=True)
+    deadline = time.time() + 1200
+    while time.time() < deadline:
+        if p.poll() is not None:
+            raise RuntimeError("llama-server exited early rc=%s, see %s"
+                               % (p.returncode, logpath))
+        if health_ok():
+            return p
+        time.sleep(2)
+    stop_server(p)
+    raise RuntimeError("llama-server not healthy within 1200s, see %s" % logpath)
+
+
+def stop_server(p):
+    if p is None or p.poll() is not None:
+        return
+    try:
+        os.killpg(os.getpgid(p.pid), signal.SIGTERM)
+    except Exception:
+        p.terminate()
+    for _ in range(120):
+        if p.poll() is not None:
+            return
+        time.sleep(1)
+    try:
+        os.killpg(os.getpgid(p.pid), signal.SIGKILL)
+    except Exception:
+        p.kill()
+    p.wait()
+
+
+def n_tokens(text):
+    """Exact token count via /tokenize. Cheap: no prefill."""
+    return len(post("/tokenize", {"content": text}, timeout=300)["tokens"])
+
+
+def calibrate(corpus, targets):
+    """Recompute the SLICES table. Only needed if model or corpus changes."""
+    probe = corpus[:2000]
+    tk = n_tokens(probe)
+    r = post("/completion", {"prompt": probe, "n_predict": 1,
+                             "cache_prompt": False, "temperature": 0})
+    offset = r["timings"]["prompt_n"] - tk
+    cpt = len(probe) / float(tk)
+    sys.stderr.write("calibrate: %.4f chars/token, prompt_n offset %d\n"
+                     % (cpt, offset))
+    for target in targets:
+        want = target - offset
+        lo, hi = 1, len(corpus)
+        guess = min(len(corpus), max(1, int(want * cpt)))
+        best = None
+        for _ in range(60):
+            got = n_tokens(corpus[:guess])
+            if got == want:
+                best = guess
+                break
+            if got < want:
+                lo = guess + 1
+            else:
+                hi = guess - 1
+            if lo > hi:
+                break
+            guess = (lo + hi) // 2
+        if best is None:
+            best = guess
+        nbytes = len(corpus[:best].encode("utf-8"))
+        sys.stderr.write("calibrate: %d tokens -> chars=%d bytes=%d\n"
+                         % (target, best, nbytes))
+
+
+def main():
+    if len(sys.argv) < 3:
+        sys.stderr.write(__doc__)
+        return 2
+    build_dir = os.path.abspath(sys.argv[1])
+    label = sys.argv[2]
+    sizes = list(DEFAULT_SIZES)
+    reps = 3
+    results_path = DEFAULT_RESULTS
+    do_calibrate = "--calibrate" in sys.argv
+    args = sys.argv[3:]
+    for i, a in enumerate(args):
+        if a == "--sizes":
+            sizes = [int(x) for x in args[i + 1].split(",")]
+        elif a == "--reps":
+            reps = int(args[i + 1])
+        elif a == "--results":
+            results_path = args[i + 1]
+
+    corpus = open(CORPUS, encoding="utf-8").read()
+    logpath = os.path.join(BENCH_DIR, "server-%s-%d.log" % (label, int(time.time())))
+
+    p = None
+    try:
+        sys.stderr.write("[%s] starting llama-server from %s\n" % (label, build_dir))
+        t0 = time.time()
+        p = start_server(build_dir, logpath)
+        sys.stderr.write("[%s] healthy after %.1fs\n" % (label, time.time() - t0))
+
+        if do_calibrate:
+            calibrate(corpus, sizes)
+            return 0
+
+        stamp = time.strftime("%Y-%m-%dT%H:%M:%S")
+        out = open(results_path, "a")
+        for target in sizes:
+            if target not in SLICES:
+                sys.stderr.write("no calibrated slice for %d tokens, skipping\n" % target)
+                continue
+            nchars = SLICES[target]["chars"]
+            prompt = corpus[:nchars]
+            rates, mss, ns = [], [], set()
+            for rep in range(1, reps + 1):
+                r = post("/completion", {"prompt": prompt, "n_predict": 1,
+                                         "cache_prompt": False, "temperature": 0})
+                t = r["timings"]
+                rates.append(t["prompt_per_second"])
+                mss.append(t["prompt_ms"])
+                ns.add(t["prompt_n"])
+                line = ("RESULT corpus label=%s build=%s target=%d chars=%d "
+                        "prompt_n=%d rep=%d prompt_ms=%.2f tps=%.2f time=%s"
+                        % (label, build_dir, target, nchars, t["prompt_n"], rep,
+                           t["prompt_ms"], t["prompt_per_second"], stamp))
+                out.write(line + "\n")
+                out.flush()
+                sys.stderr.write(line + "\n")
+            if ns != {target}:
+                sys.stderr.write("WARNING: prompt_n %s != target %d; slice table "
+                                 "is stale, rerun with --calibrate\n" % (sorted(ns), target))
+            s = ("SUMMARY corpus label=%s target=%d prompt_n=%s median_tps=%.2f "
+                 "min_tps=%.2f max_tps=%.2f median_ms=%.1f time=%s"
+                 % (label, target, sorted(ns), median(rates), min(rates),
+                    max(rates), median(mss), stamp))
+            out.write(s + "\n")
+            out.flush()
+            sys.stderr.write(s + "\n")
+        out.close()
+    finally:
+        stop_server(p)
+        sys.stderr.write("[%s] server stopped\n" % label)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/scripts/rx580-bench/run.sh b/scripts/rx580-bench/run.sh
new file mode 100755
index 000000000..d936de0f9
--- /dev/null
+++ b/scripts/rx580-bench/run.sh
@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+#
+# One-command entry point for the RX 580 prompt-processing benchmark.
+#
+# Runs from your workstation. Copies the pod-side harness into the pod and
+# execs it there over kubectl. All the real work happens on the pod; this is
+# a thin wrapper so there is exactly one command to remember.
+#
+# Examples:
+#   ./scripts/rx580-bench/run.sh --build /root/llama.cpp/build --label fork-before
+#   ./scripts/rx580-bench/run.sh --build /root/arms/new-clean/build --label new-clean \
+#                               --build-b /root/arms/new-fork/build --label-b new-fork
+#   ./scripts/rx580-bench/run.sh --summarize
+#   ./scripts/rx580-bench/run.sh --install-only
+#
+# Every argument other than the wrapper-only flags below is passed straight
+# through to bench.sh on the pod. See bench.sh --help.
+#
+set -euo pipefail
+
+KUBECONFIG_PATH="${KUBECONFIG:-/home/user/Projects/klaster/talos/generated/kubeconfig}"
+NS="${RX580_NS:-llama}"
+DEPLOY="${RX580_DEPLOY:-deploy/supervisord}"
+POD_BENCH_DIR=/root/bench
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+export KUBECONFIG="$KUBECONFIG_PATH"
+
+kx() { kubectl -n "$NS" exec -i "$DEPLOY" -- sh -c "$1"; }
+
+install_harness() {
+  echo "[run] installing harness into $POD_BENCH_DIR"
+  kx "mkdir -p $POD_BENCH_DIR"
+  for f in bench.sh ppbench.py lbsweep.sh summarize.py; do
+    kx "cat > $POD_BENCH_DIR/$f" < "$HERE/$f"
+  done
+  kx "chmod +x $POD_BENCH_DIR/bench.sh $POD_BENCH_DIR/lbsweep.sh $POD_BENCH_DIR/ppbench.py $POD_BENCH_DIR/summarize.py"
+  # The corpus is large and rarely changes; only push it if missing.
+  if ! kx "test -s $POD_BENCH_DIR/pan-tadeusz.txt" 2>/dev/null; then
+    if [ -f "$HERE/pan-tadeusz.txt" ]; then
+      echo "[run] uploading corpus"
+      kx "cat > $POD_BENCH_DIR/pan-tadeusz.txt" < "$HERE/pan-tadeusz.txt"
+    else
+      echo "[run] ERROR: $POD_BENCH_DIR/pan-tadeusz.txt missing on the pod and" >&2
+      echo "      no local copy at $HERE/pan-tadeusz.txt to upload." >&2
+      exit 4
+    fi
+  fi
+}
+
+ARGS=()
+INSTALL_ONLY=0
+SUMMARIZE=0
+for a in "$@"; do
+  case "$a" in
+    --install-only) INSTALL_ONLY=1 ;;
+    --summarize)    SUMMARIZE=1 ;;
+    *) ARGS+=("$a") ;;
+  esac
+done
+
+install_harness
+
+if [ "$INSTALL_ONLY" -eq 1 ]; then
+  echo "[run] harness installed; not running anything"
+  exit 0
+fi
+
+if [ "$SUMMARIZE" -eq 1 ]; then
+  kx "python3 $POD_BENCH_DIR/summarize.py ${ARGS[*]:-}"
+  exit 0
+fi
+
+if [ ${#ARGS[@]} -eq 0 ]; then
+  kx "sh $POD_BENCH_DIR/bench.sh --help"
+  exit 2
+fi
+
+# A full 3-round A/B run takes hours; run detached on the pod and tail it, so a
+# dropped kubectl connection cannot orphan a half-finished run with llama-swap
+# still down. bench.sh restarts llama-swap from its own EXIT trap either way.
+STAMP="$(date -u +%Y%m%d-%H%M%S)"
+LOG="$POD_BENCH_DIR/run-$STAMP.log"
+echo "[run] starting detached run on the pod, log: $LOG"
+kx "cd $POD_BENCH_DIR && nohup sh $POD_BENCH_DIR/bench.sh ${ARGS[*]} > $LOG 2>&1 & echo started"
+
+echo "[run] tailing until ALL_RUNS_DONE (safe to Ctrl-C: the pod keeps running)"
+kx "i=0; while [ \$i -lt 100000 ]; do
+      if grep -qE 'ALL_RUNS_DONE|Traceback|error:' $LOG; then break; fi
+      sleep 15; i=\$((i+1));
+    done; cat $LOG"
diff --git a/scripts/rx580-bench/summarize.py b/scripts/rx580-bench/summarize.py
new file mode 100755
index 000000000..be51ce714
--- /dev/null
+++ b/scripts/rx580-bench/summarize.py
@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""
+Summarize a results.txt produced by bench.sh into a per-label table.
+
+Usage: summarize.py [results.txt]
+
+Prints median and min-max of every metric for every label, and flags
+cross-label deltas against the ~5 percent noise floor of this machine.
+"""
+
+import re
+import sys
+from statistics import median
+
+NOISE_FLOOR_PCT = 5.0
+
+RE_CORPUS = re.compile(
+    r"^RESULT corpus label=(\S+) build=\S+ target=(\d+) chars=\d+ "
+    r"prompt_n=(\d+) rep=\d+ prompt_ms=([\d.]+) tps=([\d.]+)")
+RE_LB = re.compile(
+    r"^LBRESULT label=(\S+) build=\S+ test=(\S+) tps=([\d.]+)")
+
+
+def main():
+    path = sys.argv[1] if len(sys.argv) > 1 else "/root/bench/results.txt"
+    # data[metric][label] = list of tps
+    data = {}
+    order = []
+    labels = []
+    for line in open(path):
+        m = RE_CORPUS.match(line)
+        if m:
+            label, target, _pn, _ms, tps = m.groups()
+            metric = "pp%s-corpus" % target
+        else:
+            m = RE_LB.match(line)
+            if not m:
+                continue
+            label, test, tps = m.groups()
+            metric = "%s-llama-bench" % test
+        if metric not in data:
+            data[metric] = {}
+            order.append(metric)
+        data[metric].setdefault(label, []).append(float(tps))
+        if label not in labels:
+            labels.append(label)
+
+    if not data:
+        print("no parseable results in %s" % path)
+        return 1
+
+    def sort_key(m):
+        n = re.search(r"(\d+)", m)
+        return (0 if "corpus" in m else 1, int(n.group(1)) if n else 0)
+    order.sort(key=sort_key)
+
+    w = max(len(l) for l in labels) + 2
+    head = "metric".ljust(20) + "".join(l.ljust(max(w, 24)) for l in labels)
+    print("")
+    print("t/s, median (min-max), n samples")
+    print(head)
+    print("-" * len(head))
+    for metric in order:
+        row = metric.ljust(20)
+        for label in labels:
+            vals = data[metric].get(label)
+            if not vals:
+                row += "-".ljust(max(w, 24))
+            else:
+                cell = "%.1f (%.1f-%.1f) n=%d" % (
+                    median(vals), min(vals), max(vals), len(vals))
+                row += cell.ljust(max(w, 24))
+        print(row)
+
+    if len(labels) >= 2:
+        base = labels[0]
+        print("")
+        print("deltas vs %s (noise floor %.0f%%, anything under it is UNPROVEN)"
+              % (base, NOISE_FLOOR_PCT))
+        for other in labels[1:]:
+            print("  %s vs %s:" % (other, base))
+            for metric in order:
+                a = data[metric].get(base)
+                b = data[metric].get(other)
+                if not a or not b:
+                    continue
+                ma, mb = median(a), median(b)
+                pct = (mb - ma) / ma * 100.0
+                verdict = "SIGNIFICANT" if abs(pct) >= NOISE_FLOOR_PCT else "unproven"
+                print("    %-20s %+6.1f%%  %s" % (metric, pct, verdict))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())