server: fix the off-by-two in the SWA reuse threshold (#30580)

The first evaluated token attends to the positions above
pos - n_swa, so a SWA state starting at pos - n_swa + 1 holds the
whole window, but the reuse threshold asked for two more cells.
A saved SWA state keeps only the cells inside its window, so a
state read back from a slot file or from the RAM prompt cache was
always rejected and the slot rolled back to a checkpoint.

The SWA threshold now matches the window, while a state without SWA
keeps the exact bound. The test harness gains override_kv, used to
give the SWA test model a window smaller than the test context.

[no release]
This commit is contained in:
Pascal authored and GitHub committed 2026-10-15 16:25:19 +03:00
1 parent a6b91d8c5e
commit 61ad80bf93
3 files changed
+95 -2

No files matched your search

+6 -2
View File
@@ -3784,8 +3784,12 @@ private:
// ref: https://github.com/ggml-org/llama.cpp/pull/24110
const bool has_new_tokens = (n_past < slot.task->n_tokens());
// the largest pos_min required for a checkpoint to be useful
const auto pos_min_thold = std::max(0, pos_next - n_swa - (has_new_tokens ? 0 : 1));
// the first evaluated token attends to the positions > pos_eval - n_swa, so a SWA state
// starting at pos_eval - n_swa + 1 is enough, while a state without SWA must end before pos_eval
const llama_pos pos_eval = pos_next - (has_new_tokens ? 0 : 1);
// a state or a checkpoint is useful when its pos_min is below this threshold
const auto pos_min_thold = std::max(0, n_swa > 0 ? pos_eval - n_swa + 2 : pos_eval);
if (n_past > 0 && n_past <= slot.prompt.n_tokens()) {
const auto pos_min = llama_memory_seq_pos_min(llama_get_memory(ctx_tgt), slot.id);