Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

server: add initial tool isolation support (via docker) (#26507)

Browse Source 
* server: add initial tool isolation support (via docker)

* add docs

* adapt get_info

* py: fix type check

* cont

* separate tools_io_sandbox / tools_io_docker

* rename sandbox --> isolate

* x-tool-docker --> x-tool-runtime

---------

Co-authored-by: Pascal <admin@serveurperso.com>
This commit is contained in:
Xuan-Son Nguyen
2026-08-08 16:35:53 +02:00
committed by GitHub
co-authored by Pascal
parent 69bf643791
commit dd2c7c4471
11 changed files with 613 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/arg.cpp
+10
View File
@@ -3308,6 +3308,16 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
params.server_tools = parse_csv_row(value);
}
).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_TOOLS"));
add_opt(common_arg(
{"--tools-runtime"}, "OPTION",
"experimental: run tools in a separate runtime environment (default: none, use host environment)\n"
"available options:\n"
" 'docker:<image>': spin up a new Docker container and reuse it for all invocations, clean up on server exit\n"
" 'docker-container:<id>': use an existing Docker container by ID, won't stop on server exit\n",
[](common_params & params, const std::string & value) {
params.server_tools_runtime = value;
}
).set_examples({LLAMA_EXAMPLE_SERVER}).set_env("LLAMA_ARG_TOOLS_RUNTIME"));
add_opt(common_arg(
{"--mcp-servers-config"}, "PATH",
"experimental: path to JSON file with MCP server definitions (Cursor-compatible format) - do not enable in untrusted environments (default: none)\n"