docs: runbook and FORK.md status for the 2026-11-21 rebase onto e6d624b9a
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3ab97b345f
commit
3bf6b55b39
2 files changed
+32
No files matched your search
@@ -6,6 +6,9 @@ which also holds the operations reference (pod access, files, recovery).
|
||||
|
||||
Status at the time of writing (2026-11-21):
|
||||
|
||||
- The branch `fable5/vram-ondemand-rebase-20261121` is the same fork rebased onto upstream `e6d624b9a`
|
||||
(757 upstream commits). It is not built, validated or deployed: production still runs `adc2aa3ec`
|
||||
from the branch below. The hand ports are listed in the runbook (section 3, "2026-11-21").
|
||||
- Branch `fable5/vram-ondemand-rebase-20261024` on upstream `3f03b1772`. The fork commits are
|
||||
`git log --oneline origin/master..HEAD`; that list includes documentation and script commits
|
||||
(this file among them), so count it rather than trusting a number written here. Section 2 maps
|
||||
|
||||
@@ -131,6 +131,35 @@ upstream's `if (moe_cache)` breakdown block outside our `if (sched)` guard; make
|
||||
(upstream `4e201388c` uses a fixed deadline: keep our order "check `!running`, return null", then
|
||||
`wait_until(lock, deadline)`).
|
||||
|
||||
**What the 2026-11-21 rebase (upstream `e6d624b9a`, 757 commits past `3f03b1772`) changed** (nothing
|
||||
built or run locally; every item below is by reading only and needs the pod build):
|
||||
- Upstream refactors that moved fork code (hand ports, not conflicts of one hunk):
|
||||
- `clip.cpp`/`clip.h` are gone (#31865). The encoder release/restore (F5) is now
|
||||
`mtmd_model_release_device`/`mtmd_model_restore_device` on `mtmd_model` (`mtmd-model.cpp`), called
|
||||
from `mtmd_release_device`/`mtmd_restore_device` under `compute_mutex` after syncing the shared
|
||||
scheduler; the page-cache drop of F10 moved to `mtmd_model_loader::load_tensors`.
|
||||
- `server_prompt_cache` moved to `server-slot-sched.{h,cpp}` and `get_available_slot()` was replaced
|
||||
by `server_slot_sched::select()`. The disk prompt cache (F11) lives there now
|
||||
(`server_prompt_disk_cache`), is called from the task dispatch after `update_prompt_cache()`, and
|
||||
uses `common_state_data` and `shared_ptr<const common_prompt_checkpoint>`. Only prompts made with
|
||||
the server lora config are saved, and a restored prompt gets that config in `prompt.lora`.
|
||||
- Upstream's `llama_memory_bufs` replaced the raw `ctxs_bufs` vectors of the KV cache and the recurrent
|
||||
memory: the null-buffer guard moved into `llama_memory_bufs::memory_breakdown()`, and the release
|
||||
code indexes `ctxs_bufs.bufs[i]`. Those two did not compile after a clean pick.
|
||||
- The MoE cache is now a ClassLRU (`moe_cache_lru`, `g.lru`) with `no_alloc` as a constructor
|
||||
argument and `cpu_share`. `invalidate()` rebuilds `g.lru`, refills every slot map with
|
||||
`g.lru.empty` (not -1), clears the CPU maps and `cpu_experts`.
|
||||
- `server.cpp` was split into `server_run_instance` and friends: the self-pipe shutdown thread
|
||||
(F7) is started in `server_run_instance` only, just before `server_install_signal_handlers()`.
|
||||
- `server_queue` got `n_inflight` (#32133). `should_sleep` checks it first, so a response that is still
|
||||
open also blocks the want-lock yield. This is a decision, not an accident: check it with `--arbiter`.
|
||||
- Upstream added its own tiled-transpose concat (#30591) with no size gate. The fork path still runs
|
||||
first (`ne0 >= 32`); the gate and the `GGML_VK_CONCAT_FASTPATH` knob were added to upstream's selector
|
||||
(`ggml_vk_concat_is_transposed`), otherwise decode would take upstream's tiled path again (F3).
|
||||
- Memory types with no release override (stay in VRAM when cold, not used in production):
|
||||
`llama_kv_cache_dsa`, `_dsa_iswa`, `_dsv4`, `_msa`, `llama_memory_hybrid_iswa`. The KV rotation
|
||||
tensors (`attn_rot_bufs`, #31073) are small and are not released either.
|
||||
|
||||
**Semantic re-checks to do for every rebase, clean or not** (details in FORK.md):
|
||||
- F1: `sched_copy_experts` and its `st` struct unchanged in meaning.
|
||||
- F2: `use_mask_opt` still contains upstream's terms plus our `>= 256`.
|
||||
|
||||
Reference in new issue
Block a user