server + ui: fix stream routes for model names containing a slash (#26137)

* server + ui: refactor resumable stream routes to query string conv_id

The conversation id can embed a model name containing slashes
(ggml-org/...) in router mode, which the decoded path splits before the
:conv_id param is captured, so stop and resume never matched the
session. Move the id to the conv_id query string on the public routes
and on the internal router -> child hop, where slashes survive
encoding. Handlers are unchanged since query and path params land in
the same map. Add a regression test with a slashed model name.

* server: move stream route docs to server-stream.h

Address review: ngxson wants the main server.cpp registration code kept
clean and simple, with route-level explanations living in the header.
Move the query string rationale and the lookup ownership note next to
the handler declarations in server-stream.h, and shorten the wiring
comment to a pointer.

* server: cancel a pending request when its stream is stopped during model load

The conversation was registered in the conv map only after the blocking
autoload wait, so a stop issued while the model loaded found nothing to
cancel and the request went on to generate an orphan once the load
ended. Register the conversation before the wait and give the entry a
ticket: a stop erases the entry, and the parked request checks its
ticket after the wait and aborts with 400 instead of starting. A newer
request on the same conversation replaces the entry, so only the
stopped request is cancelled. Add a regression test that stops during
the load window.

* server + ui: resume a stream after a page reload during model load

A pending request died with the client socket when the page was
reloaded while its model was loading, so no session ever existed and
the conversation had nothing to recover. A session request that waited
for a load now detaches from the client socket and reaches the child
regardless, the session buffer receives the generation, and the resume
route answers 503 while the owner is loading so the client retries
instead of dropping its state. The WebUI persists the pending stream at
send time, quietly polls on 503, and attaches once the session exists.
Add a regression test that drops the client during the load window.

* ui: show the model load progress again after a page refresh

The resume wait was invisible, so a conversation refreshed while its
model was loading showed nothing until the first byte. On a 503 from
the resume probe, mark the conversation as loading again so the
assistant row persisted at send time renders the processing info, and
target the model frozen in the persisted stream state for the
progress, since the row has no model yet and the dropdown may not be
restored.

* fix CI

* fix CI bis
This commit is contained in:
Pascal
2026-07-27 07:34:47 +02:00
committed by GitHub
parent 88b47a755c
commit d73c1d6b22
11 changed files with 345 additions and 55 deletions
+5 -5
View File
@@ -136,13 +136,13 @@ Producer side: `server_res_generator` extends `server_res_spipe`, which keeps al
Lifetime safety: the session holds no back reference to the response, so `spipe` is a plain `unique_ptr` touched only by the http worker. `cancel` raises an atomic the producer polls; the producer finalizes the session from its destructor, which also runs `~server_response_reader::stop()` to cancel the generation at the queue level. A `DELETE` stops work by raising the flag and letting the worker unwind.
Consumer side: `GET /v1/stream/<conv_id>?from=N` opens a `text/event-stream` that replays buffered bytes from offset `N` and blocks for live bytes, so the browser reattaches like a fresh EventSource. An offset below the dropped prefix returns 400.
Consumer side: `GET /v1/stream?conv_id=<id>&from=N` opens a `text/event-stream` that replays buffered bytes from offset `N` and blocks for live bytes, so the browser reattaches like a fresh EventSource. An offset below the dropped prefix returns 400.
Routes:
- `GET /v1/stream/:conv_id?from=N`: replay or live reattach.
- `GET /v1/stream?conv_id=<id>&from=N`: replay or live reattach. The id travels in the query string because it can embed a model name containing slashes.
- `POST /v1/streams/lookup` with `{"conversation_ids": [...]}`: returns session status only for ids the caller already owns. There is no listing route, so live sessions cannot be enumerated (an earlier `GET /v1/streams` was removed for exactly this reason).
- `DELETE /v1/stream/:conv_id`: explicit Stop, idempotent (`evict_and_cancel`).
- `DELETE /v1/stream?conv_id=<id>`: explicit Stop, idempotent (`evict_and_cancel`).
Router mode binds the same paths to proxy handlers. A `conv_id -> child` map (`conv_models`), populated when a POST is routed, resolves the owning child in one lookup with no polling. The lookup groups ids per child; GET and DELETE proxy straight to the owner. This loopback REST hop is expected to move to a websocket IPC later, swapping only the transport.
@@ -166,8 +166,8 @@ graph TD
GC[GC thread] -- drop after TTL --> Sess
end
Sess -- read_from offset --> Cons[stream_pipe_consumer]
Cons -- "GET /v1/stream/:id?from=N" --> Client
DEL[DELETE /v1/stream/:id] -- evict_and_cancel --> Sess
Cons -- "GET /v1/stream?conv_id=id&from=N" --> Client
DEL[DELETE /v1/stream?conv_id=id] -- evict_and_cancel --> Sess
```
The diagram shows the buffer touch points. The live wire (chunks streamed to the original client during a normal generation) is the producer's default output, described under "Producer side" above.