With LLAMA_USERSTART_MERGE on (the default), F14 skipped every user-message
batch break when the rest of the prompt fits in one ubatch. On a fresh
prompt that also dropped the break at the first user message, so no context
checkpoint was left at the end of the system prompt, and a new chat with the
same system prompt re-processed the system prompt from scratch (a
recurrent/hybrid state cannot be cut back without a checkpoint).
Keep that one break in the merged path: the batch ends right before the
first user message, and the next batch starts there and gets the checkpoint
(is_user_start, near-end rule). The batch only reaches the first user
message when the prompt is processed from before it: checkpoints past the
restore point are erased on reuse, so this is a fresh prompt or one that
diverged inside the system prompt. Follow-up turns start past it and keep
the merge (no break at the last user message). Boundaries below 8 tokens
(only a BOS before the first user message) are skipped.
Cost: on a fresh prompt the system prompt runs as its own batch again (the
upstream layout for turn 1), measured before as part of turn 1's
6.08 vs 3.99 s TTFT (FORK.md F14). The unmerged path is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ