docs: runbook and FORK.md status for the 2026-11-21 rebase onto e6d624b9a

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
LumpiastyandClaude Sonnet 5.5 committed 2026-11-21 19:40:51 +01:00
1 parent 3ab97b345f
commit 3bf6b55b39
2 files changed
+32

No files matched your search

+3
View File
@@ -6,6 +6,9 @@ which also holds the operations reference (pod access, files, recovery).
Status at the time of writing (2026-11-21):
- The branch `fable5/vram-ondemand-rebase-20261121` is the same fork rebased onto upstream `e6d624b9a`
(757 upstream commits). It is not built, validated or deployed: production still runs `adc2aa3ec`
from the branch below. The hand ports are listed in the runbook (section 3, "2026-11-21").
- Branch `fable5/vram-ondemand-rebase-20261024` on upstream `3f03b1772`. The fork commits are
`git log --oneline origin/master..HEAD`; that list includes documentation and script commits
(this file among them), so count it rather than trusting a number written here. Section 2 maps
+29
View File
@@ -131,6 +131,35 @@ upstream's `if (moe_cache)` breakdown block outside our `if (sched)` guard; make
(upstream `4e201388c` uses a fixed deadline: keep our order "check `!running`, return null", then
`wait_until(lock, deadline)`).
**What the 2026-11-21 rebase (upstream `e6d624b9a`, 757 commits past `3f03b1772`) changed** (nothing
built or run locally; every item below is by reading only and needs the pod build):
- Upstream refactors that moved fork code (hand ports, not conflicts of one hunk):
- `clip.cpp`/`clip.h` are gone (#31865). The encoder release/restore (F5) is now
`mtmd_model_release_device`/`mtmd_model_restore_device` on `mtmd_model` (`mtmd-model.cpp`), called
from `mtmd_release_device`/`mtmd_restore_device` under `compute_mutex` after syncing the shared
scheduler; the page-cache drop of F10 moved to `mtmd_model_loader::load_tensors`.
- `server_prompt_cache` moved to `server-slot-sched.{h,cpp}` and `get_available_slot()` was replaced
by `server_slot_sched::select()`. The disk prompt cache (F11) lives there now
(`server_prompt_disk_cache`), is called from the task dispatch after `update_prompt_cache()`, and
uses `common_state_data` and `shared_ptr<const common_prompt_checkpoint>`. Only prompts made with
the server lora config are saved, and a restored prompt gets that config in `prompt.lora`.
- Upstream's `llama_memory_bufs` replaced the raw `ctxs_bufs` vectors of the KV cache and the recurrent
memory: the null-buffer guard moved into `llama_memory_bufs::memory_breakdown()`, and the release
code indexes `ctxs_bufs.bufs[i]`. Those two did not compile after a clean pick.
- The MoE cache is now a ClassLRU (`moe_cache_lru`, `g.lru`) with `no_alloc` as a constructor
argument and `cpu_share`. `invalidate()` rebuilds `g.lru`, refills every slot map with
`g.lru.empty` (not -1), clears the CPU maps and `cpu_experts`.
- `server.cpp` was split into `server_run_instance` and friends: the self-pipe shutdown thread
(F7) is started in `server_run_instance` only, just before `server_install_signal_handlers()`.
- `server_queue` got `n_inflight` (#32133). `should_sleep` checks it first, so a response that is still
open also blocks the want-lock yield. This is a decision, not an accident: check it with `--arbiter`.
- Upstream added its own tiled-transpose concat (#30591) with no size gate. The fork path still runs
first (`ne0 >= 32`); the gate and the `GGML_VK_CONCAT_FASTPATH` knob were added to upstream's selector
(`ggml_vk_concat_is_transposed`), otherwise decode would take upstream's tiled path again (F3).
- Memory types with no release override (stay in VRAM when cold, not used in production):
`llama_kv_cache_dsa`, `_dsa_iswa`, `_dsv4`, `_msa`, `llama_memory_hybrid_iswa`. The KV rotation
tensors (`attn_rot_bufs`, #31073) are small and are not released either.
**Semantic re-checks to do for every rebase, clean or not** (details in FORK.md):
- F1: `sched_copy_experts` and its `st` struct unchanged in meaning.
- F2: `use_mask_opt` still contains upstream's terms plus our `>= 256`.