memory: refuse rollbacks past the snapshots of the last ubatch (#31368)

* memory: refuse rollbacks past the snapshots of the last ubatch

The recurrent memory and the DSV4 cache accepted any rollback up to
n_rs_seq, whether or not the last ubatch wrote that snapshot. Each now
records the deepest rollback its last ubatch guarantees and seq_rm
refuses anything past it:

- recurrent: n - 1 for a ubatch of n <= n_rs_seq + 1 tokens, 0 for a
  longer one, which only guarantees the final state
- DSV4: min(n, n_rs_seq), the snapshots are rebuilt from the state
  before the ubatch

The dead rollback block of the rollback test becomes the assertion it
asked for, and rollback_bound checks the exact bound of each memory
after ubatches of 1, 2, n_rs_seq + 1 and n_rs_seq + 2 tokens.

* tests: address review from ggerganov

Detect architectures with llama_arch_from_string in comp_padding and
arch_reserves_single_seq, as in rollback_bound.
This commit is contained in:
Pascal authored and GitHub committed 2026-10-30 10:11:58 +02:00
1 parent e0f142d5dd
commit ab3768cdc2
7 files changed
+128 -33

No files matched your search

+4 -2
View File
@@ -148,6 +148,7 @@ void llama_memory_recurrent::clear(bool data) {
cells[i].seq_id.clear();
cells[i].src = -1;
cells[i].tail = -1;
cells[i].rs_max = 0;
}
head = 0;
@@ -194,7 +195,7 @@ bool llama_memory_recurrent::seq_rm(llama_seq_id seq_id, llama_pos p0, llama_pos
if (tail_id >= 0) {
auto & cell = cells[tail_id];
// partial rollback via per-token snapshot index (bounded by n_rs_seq)
// partial rollback via per-token snapshot index (bounded by the snapshots of the last ubatch)
if (0 < p0 && p0 <= cell.pos && p1 > cell.pos) {
// a cell shared by several sequences cannot move back for only one of them
if (cell.seq_id.size() > 1) {
@@ -208,7 +209,7 @@ bool llama_memory_recurrent::seq_rm(llama_seq_id seq_id, llama_pos p0, llama_pos
const llama_pos rollback = cell.pos - (p0 - 1);
// pending rollback is single-use
const bool pending = rs_idx[seq_id] != 0;
if (!pending && rollback >= 1 && rollback <= (llama_pos) n_rs_seq) {
if (!pending && rollback >= 1 && rollback <= (llama_pos) cell.rs_max) {
set_rs_idx(seq_id, (uint32_t) rollback);
cell.pos = p0 - 1;
return true;
@@ -672,6 +673,7 @@ bool llama_memory_recurrent::find_slot(const llama_ubatch & ubatch) {
__func__, last_pos, cell.pos, ubatch.seq_id[i][0], n_seq_tokens);
}
cell.pos = last_pos;
cell.rs_max = n_seq_tokens <= n_rs_seq + 1 ? n_seq_tokens - 1 : 0;
cell.seq_id.clear();
for (int32_t j = 0; j < ubatch.n_seq_id[i]; ++j) {
const llama_seq_id seq_id = ubatch.seq_id[i][j];