Files
llama.cpp/tools/server/server-stream.h
T
Aleksander Grygier 3a7660371d ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)
* ui: move tools/ui to ui/apps/chat

The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's package @ggml-org/llama-chat

It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: call the built-in web UI Llama Chat in docs, help text and comments

The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.

Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name tools/ui as a former path of Llama Chat in the code-review skill

Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.

Assisted-by: Claude (Opus 5.5 Medium)

* ui: name Llama Chat's path in the security policy's exclusions

Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.

Assisted-by: Claude (Opus 5.5 Medium)
2026-11-16 17:23:36 +01:00

80 lines
3.0 KiB
C++

#pragma once
#include "server-http.h"
#include <atomic>
#include <cstddef>
#include <functional>
#include <memory>
#include <string>
// streaming buffer for one generation, survives HTTP disconnect. the producer appends SSE bytes,
// readers drain from any offset via read_from. keyed by conversation_id, one conv = one live session
struct stream_session;
using stream_session_ptr = std::shared_ptr<stream_session>;
// base of the producer/consumer pipe ends. virtual dtor so each runs its own teardown:
// the producer finalizes the session, the consumer leaves it untouched
struct stream_pipe {
virtual ~stream_pipe() = default;
bool is_cancelled() const;
protected:
explicit stream_pipe(stream_session_ptr session);
stream_session_ptr session_;
};
// producer end: writes chunks into the ring buffer and owns the session lifetime, finalizing it
// on destruction.
struct stream_pipe_producer : stream_pipe {
~stream_pipe_producer() override;
bool write(const char * data, size_t len);
static stream_pipe_producer * create(stream_session_ptr session);
private:
explicit stream_pipe_producer(stream_session_ptr session);
};
void server_stream_session_manager_start();
void server_stream_session_manager_stop();
// route handler factories wired under /v1/stream/* by server.cpp
// child-side handlers for the resumable stream routes. the conv id travels in the conv_id
// query string because it can embed a model name containing slashes (org/repo), which the
// decoded path would split before the param is captured
server_http_context::handler_t server_stream_make_get_handler();
// POST /v1/streams/lookup with body {"conversation_ids": [...]}: only answers for ids the
// caller already owns (Llama Chat passes the convs visible in its sidebar), the server never
// lists ids it has not been asked about, so a random caller cannot enumerate live sessions
server_http_context::handler_t server_stream_make_lookup_handler();
server_http_context::handler_t server_stream_make_delete_handler();
// extract the X-Conversation-Id header value (case-insensitive), empty when absent
std::string server_stream_conv_id_from_headers(const std::map<std::string, std::string> & headers);
// implement tee-style pipe (spipe) for "stream replay" functionality
struct server_res_spipe : server_http_res {
private:
// if set, the stream survives a client disconnect:
// connection kept alive, output is forwarded to spipe and reuse later
std::unique_ptr<stream_pipe_producer> spipe;
// if spipe is set, use this next_orig to implement tee-style pipe
std::function<bool(std::string &)> next_orig;
const server_http_req * req = nullptr;
// set once next_orig reports no more data, so on_complete() doesn't re-drain a finished stream
bool next_finished = false;
public:
void set_req(const server_http_req * req);
bool conn_alive();
bool should_stop();
void on_complete() override;
void set_next(std::function<bool(std::string &)> next_fn);
};