docs(readme): document GCN mask_opt and fix Vulkan serving guidance

Add the flash-attn mask_opt change (auto-on for GCN large head sizes,
lossless, +12% pp @ 32k on the RX 580) and correct the earlier pinning
advice: pinning helps in isolated llama-bench but fails in a long-running
server on RADV, so --no-mmap is the serving path. Add a recommended
Polaris serving command.

Assisted-by: Claude

README.md

@@ -42,35 +42,46 @@ Branches: [`fable5/host-register`](https://github.com/thecodacus/llama.cpp/tree/
 
 ### Vulkan (older AMD, e.g. RX 580 / Polaris)
 
-On the Vulkan backend the two optimizations behave differently from CUDA:
+On the Vulkan backend the CUDA-oriented flags above behave differently, and this fork adds a
+Polaris-specific flash-attention fix. Findings on an **RX 580 8GB** (Polaris / GCN, PCIe 3.0 x16,
+no fp16, no matrix cores) with **Qwen3.5-35B-A3B Q4_K_M**, `-b 2048 -ub 2048`:
 
-- **Pinning (`GGML_CUDA_REGISTER_HOST=1`) is the win.** It brings the H2D expert uploads up to full PCIe DMA rate.
-- **`GGML_SCHED_PREFETCH_EXPERTS=1` regresses here - do not use it.** The prefetch runs through a second backend instance that shares the same Vulkan device queue, so uploads do not actually overlap compute; it only adds overhead.
-- **Tune `--n-cpu-moe` to context length.** Keeping some expert layers resident in spare VRAM helps short prompts, but at long context the growing KV cache needs that VRAM back, so residency backfires. Use a low value (more resident) for short prompts and a high value (all experts on host) for long context.
-- Keep flash attention on (`-fa 1`, the default) and do not quantize the KV cache on this hardware (`-ctk/-ctv q8_0` is slower - the dequant overhead in the scalar FA kernel outweighs the bandwidth saving).
+- **Flash-attention `mask_opt` is now enabled for GCN large head sizes (this fork's own change).**
+  Upstream disables it on GCN; it is a **lossless** win in high-context prefill - it skips
+  fully-masked causal blocks and the per-block mask add on fully-visible ones, which is real work on
+  a card whose attention is compute-bound (no matrix cores). Auto-on, no flag. On Qwen3.5-35B
+  (head_dim 256): pp2048 **+8% @ 16k, +12% @ 32k**, growing with depth; perplexity bit-identical.
+- **For a long-running server, load with `--no-mmap`, not pinning.** `GGML_CUDA_REGISTER_HOST=1`
+  (pinning) gives ~+17% in an isolated `llama-bench` run, but in a server the RADV host-pointer
+  import fails and the fallback pre-stage buffer allocation fails for large / co-resident models, so
+  it silently reverts to slow staging (and can trip warnings/OOM). `--no-mmap` (weights in RAM) is
+  both faster and clean there. Pinning is still fine for one-off `llama-bench` numbers.
+- **`GGML_SCHED_PREFETCH_EXPERTS=1` regresses - do not use it** on Vulkan (its second backend shares
+  one device queue, so uploads never overlap compute).
+- **`-b 2048 -ub 2048` is the biggest prefill lever** (the default `-ub 512` roughly halves pp).
+- **Tune `--n-cpu-moe` to context length.** Keep some expert layers resident in spare VRAM for short
+  prompts (e.g. `ncmoe 28` on the 35B, ~+5% over all-host); at long context the KV cache needs that
+  VRAM, so raise it (`ncmoe 40`, all experts on host). Keep flash attention on (`-fa 1`).
 
-Measured on an **RX 580 8GB** (Polaris, PCIe 3.0 x16, no fp16 / no matrix cores) with **Qwen3.5-35B-A3B Q4_K_M**, `-b 2048 -ub 2048`:
-
-```bash
-# baseline (pinning off):
-./build/bin/llama-bench -m MODEL -ngl 99 -ncmoe 99 -p 2048 -n 0 -r 2 -b 2048 -ub 2048
-
-# pinned, all experts on host:
-GGML_CUDA_REGISTER_HOST=1 \
-./build/bin/llama-bench -m MODEL -ngl 99 -ncmoe 99 -p 2048 -n 0 -r 2 -b 2048 -ub 2048
-
-# pinned + a few expert layers kept resident (best for short prompts):
-GGML_CUDA_REGISTER_HOST=1 \
-./build/bin/llama-bench -m MODEL -ngl 99 -ncmoe 28 -p 2048 -n 0 -r 2 -b 2048 -ub 2048
-```
+Prefill throughput (isolated `llama-bench`, pinned unless noted):
 
 | Config | pp2048 (t/s) |
 | --- | ---: |
-| baseline (unpinned) | ~252 |
-| pinned, `-ncmoe 99` | ~294 |
-| pinned, `-ncmoe 28` | ~308 |
+| baseline, unpinned, `ncmoe 40` | ~252 |
+| pinned, `ncmoe 40` | ~294 |
+| pinned, `ncmoe 28` | ~308 |
+| server default (`--no-mmap`, `ncmoe 40`) | ~285 |
+| + `mask_opt`, @ 32k context | **+12%** |
 
-Result: **~252 -> ~308 t/s** prefill (**+22%**), token-identical. At long context the bottleneck shifts to attention compute (GPU-bound), where these MoE-transfer optimizations no longer apply; prefer a high `--n-cpu-moe` there for VRAM headroom.
+**Recommended RX 580 / Polaris serving command** (per model):
+
+```bash
+llama-server -hf <repo>:<quant> --no-mmap -ngl 99 --n-cpu-moe 40 -b 2048 -ub 2048 -fa 1
+```
+
+Lower `--n-cpu-moe` (e.g. 28) if the model plus your context budget leave spare VRAM; keep it high
+for long-context / agentic use. At long context the bottleneck is attention compute (GPU-bound), so
+`mask_opt` (above) is where the remaining prefill wins come from, not the MoE-transfer path.
 
 ## Recent API changes