chat : reject JSON nested more than 128 levels before walking it (#30242)

* chat : reject JSON nested more than 128 levels before walking it

The schema builder, the jinja engine and the decision code walk request
JSON with recursion. A request nested deep enough overflows the stack of
the HTTP thread and llama-server dies (SIGBUS on macOS, SIGSEGV on Linux):
for a tool schema from a few hundred levels on macOS and about 10000 on
Linux.

Check the depth first and answer 400 past 128 levels: the chat inputs in
common_chat_templates_apply (tools, tool-call arguments,
chat_template_kwargs, json_schema), the json_schema of /completion, and the
whole /v1/systemone body. A gemma4 tool result that is too deep stays text.

* fix the root, not the call side

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
Yuri KhrustalevandXuan Son Nguyen authored and GitHub committed 2026-10-25 06:46:35 +01:00
1 parent 74bca39500
commit b3fa69955e
5 files changed
+135 -15

No files matched your search

+27
View File
@@ -464,6 +464,32 @@ static void test_rerank_template(testing & t) {
}
}
//
// server_decision_context: the recursive functions stop at RECURSIVE_LIMIT
//
static void test_decision_depth(testing & t) {
const auto prepare = [](size_t depth) {
server_decision_context ctx;
ctx.type = COMMON_DECISION_TYPE_H2O_LIGHTNING; // the state goes through decision_py_dump
std::string error;
try {
ctx.prepare_state(json::parse(std::string(depth, '[') + std::string(depth, ']')));
} catch (const std::invalid_argument & e) {
error = e.what();
}
return error;
};
t.test("state at the limit", [&](testing & t) {
const std::string error = prepare(128);
t.assert_true("accepted: " + error, error.empty());
});
t.test("deep state", [&](testing & t) {
const std::string error = prepare(1000);
t.assert_true("rejected as nested: " + error, error.find("nested more than 128 levels deep") != std::string::npos);
});
}
int main(int argc, char ** argv) {
testing t;
@@ -484,6 +510,7 @@ int main(int argc, char ** argv) {
t.test("server_pipe", test_server_pipe);
t.test("decision_escape", test_decision_escape);
t.test("rerank_template", test_rerank_template);
t.test("decision_depth", test_decision_depth);
return t.summary();
}