From 087f94d82e597a966c78d2c0afd2b34df561ea34 Mon Sep 17 00:00:00 2001 From: Pascal Date: Mon, 17 Aug 2026 21:32:15 +0200 Subject: [PATCH] doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (#26847) * doc: document MCP stdio servers and CORS defaults in the server README The MCP arguments were listed but nothing explained what an MCP server is or how to declare one. Cover the stdio transport, the config keys, the tool naming, and add a POSIX shell echo server as a minimal example. Also document the CORS behavior: the default reflected origin, the switch to localhost once tools are enabled, and the recommended setting per deployment. * doc: drop the inline MCP shell example from the server README The example parsed JSON-RPC by hand and sat in a page people copy paste from, into servers spawned with the privileges of llama-server. Point to the specification instead. Link the pull request that introduced the feature, and keep a short mcp.json snippet so the table of configuration keys has a declaration to refer to. --- tools/server/README.md | 52 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tools/server/README.md b/tools/server/README.md index 017981fd1..f1e1facee 100644 --- a/tools/server/README.md +++ b/tools/server/README.md @@ -343,6 +343,58 @@ The server includes a set of built-in tools that enable the LLM to access the lo To use this feature, start the server with `--tools all`. You can also enable only specific tools by passing a comma-separated list: `--tools name1,name2,...`. Run `--help` for the full list of available tool names. +### MCP servers + +Besides the built-in tools, the server can expose tools coming from MCP servers, added in [#26062](https://github.com/ggml-org/llama.cpp/pull/26062). Only the stdio transport is supported: such a server is a child process reading JSON-RPC messages on its stdin and writing replies on its stdout, so nothing has to be started or maintained outside `llama-server`. + +Servers are declared in a Cursor-compatible JSON file: + +```json +{ + "mcpServers": { + "example": { "command": "/path/to/server", "args": [] } + } +} +``` + +```sh +llama-server -m model.gguf --mcp-servers-config mcp.json +``` + +The same JSON can be passed inline with `--mcp-servers-json`. Each entry under `mcpServers` accepts: + +| Key | Explanation | +| --- | ----------- | +| `command` | executable to spawn, required, entries without it are skipped | +| `args` | array of arguments | +| `env` | object merged over the parent environment | +| `cwd` | working directory of the child process | +| `timeout_ms` | per-tool-call timeout (default: 30000) | + +Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `_` alongside the built-in ones: they show up in the Web UI and in `GET /tools`, and the model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available. + +The child process runs with the same privileges as the server, so only declare commands you trust. As with `--tools`, `--cors-origins` then defaults to `localhost`. + +Note: `--ui-mcp-proxy` is unrelated, it only lets the Web UI reach remote MCP servers from the browser. + +Any server written against the [MCP specification](https://modelcontextprotocol.io) works as is, whether it uses an official SDK or not: the transport is one JSON-RPC message per line on stdio, so a script wrapping an existing program is a valid server too. + +### CORS + +By default the server reflects any `Origin` header back with credentials allowed. This matches the old, always-on `*` behavior and is fine as long as the server only exposes stateless, read-only endpoints. + +Enabling `--tools` or `--agent` exposes file read/write over the API, so in that case `--cors-origins` defaults to `localhost` instead: only pages served from localhost can reach the server. Pass `--cors-origins` explicitly to override either default. + +Recommended `--cors-origins` setting, depending on where the server runs: + +| Deployment | Recommendation | +| ---------- | --------------- | +| Public | set an API key, put the server behind a reverse proxy, `--cors-origins` optional | +| Local network | set `--cors-origins` to your frontend's origin | +| Same machine | `--cors-origins localhost` (default once `--agent` is set) | + +Related flags: `--cors-origins`, `--cors-methods`, `--cors-headers`, `--cors-credentials` / `--no-cors-credentials`. Background and rationale: [#25655](https://github.com/ggml-org/llama.cpp/pull/25655). + ## Build `llama-server` is built alongside everything else from the root of the project