From 372f14e0b2d71fa8b6062e1a8d803a073e02438b Mon Sep 17 00:00:00 2001 From: Lumpiasty Date: Sat, 21 Nov 2026 19:40:51 +0100 Subject: [PATCH] docs: runbook and FORK.md status for the 2026-11-21 rebase onto e6d624b9a Co-Authored-By: Claude Sonnet 5.5 --- FORK.md | 3 +++ docs/fork-rebase-runbook.md | 29 +++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/FORK.md b/FORK.md index b844fcc69..bc9a59ce4 100644 --- a/FORK.md +++ b/FORK.md @@ -6,6 +6,9 @@ which also holds the operations reference (pod access, files, recovery). Status at the time of writing (2026-11-21): +- The branch `fable5/vram-ondemand-rebase-20261121` is the same fork rebased onto upstream `e6d624b9a` + (757 upstream commits). It is not built, validated or deployed: production still runs `adc2aa3ec` + from the branch below. The hand ports are listed in the runbook (section 3, "2026-11-21"). - Branch `fable5/vram-ondemand-rebase-20261024` on upstream `3f03b1772`. The fork commits are `git log --oneline origin/master..HEAD`; that list includes documentation and script commits (this file among them), so count it rather than trusting a number written here. Section 2 maps diff --git a/docs/fork-rebase-runbook.md b/docs/fork-rebase-runbook.md index 8c4d757ff..1cfd1d060 100644 --- a/docs/fork-rebase-runbook.md +++ b/docs/fork-rebase-runbook.md @@ -131,6 +131,35 @@ upstream's `if (moe_cache)` breakdown block outside our `if (sched)` guard; make (upstream `4e201388c` uses a fixed deadline: keep our order "check `!running`, return null", then `wait_until(lock, deadline)`). +**What the 2026-11-21 rebase (upstream `e6d624b9a`, 757 commits past `3f03b1772`) changed** (nothing +built or run locally; every item below is by reading only and needs the pod build): +- Upstream refactors that moved fork code (hand ports, not conflicts of one hunk): + - `clip.cpp`/`clip.h` are gone (#31865). The encoder release/restore (F5) is now + `mtmd_model_release_device`/`mtmd_model_restore_device` on `mtmd_model` (`mtmd-model.cpp`), called + from `mtmd_release_device`/`mtmd_restore_device` under `compute_mutex` after syncing the shared + scheduler; the page-cache drop of F10 moved to `mtmd_model_loader::load_tensors`. + - `server_prompt_cache` moved to `server-slot-sched.{h,cpp}` and `get_available_slot()` was replaced + by `server_slot_sched::select()`. The disk prompt cache (F11) lives there now + (`server_prompt_disk_cache`), is called from the task dispatch after `update_prompt_cache()`, and + uses `common_state_data` and `shared_ptr`. Only prompts made with + the server lora config are saved, and a restored prompt gets that config in `prompt.lora`. + - Upstream's `llama_memory_bufs` replaced the raw `ctxs_bufs` vectors of the KV cache and the recurrent + memory: the null-buffer guard moved into `llama_memory_bufs::memory_breakdown()`, and the release + code indexes `ctxs_bufs.bufs[i]`. Those two did not compile after a clean pick. + - The MoE cache is now a ClassLRU (`moe_cache_lru`, `g.lru`) with `no_alloc` as a constructor + argument and `cpu_share`. `invalidate()` rebuilds `g.lru`, refills every slot map with + `g.lru.empty` (not -1), clears the CPU maps and `cpu_experts`. + - `server.cpp` was split into `server_run_instance` and friends: the self-pipe shutdown thread + (F7) is started in `server_run_instance` only, just before `server_install_signal_handlers()`. + - `server_queue` got `n_inflight` (#32133). `should_sleep` checks it first, so a response that is still + open also blocks the want-lock yield. This is a decision, not an accident: check it with `--arbiter`. +- Upstream added its own tiled-transpose concat (#30591) with no size gate. The fork path still runs + first (`ne0 >= 32`); the gate and the `GGML_VK_CONCAT_FASTPATH` knob were added to upstream's selector + (`ggml_vk_concat_is_transposed`), otherwise decode would take upstream's tiled path again (F3). +- Memory types with no release override (stay in VRAM when cold, not used in production): + `llama_kv_cache_dsa`, `_dsa_iswa`, `_dsv4`, `_msa`, `llama_memory_hybrid_iswa`. The KV rotation + tensors (`attn_rot_bufs`, #31073) are small and are not released either. + **Semantic re-checks to do for every rebase, clean or not** (details in FORK.md): - F1: `sched_copy_experts` and its `st` struct unchanged in meaning. - F2: `use_mask_opt` still contains upstream's terms plus our `>= 256`.