chat : fix muse-glimmer detection of tool calls after EOM (#26879)
* chat : fix muse-glimmer swallowing a trailing tool call into content
Muse Glimmer routinely answers the user and calls a tool in a single
generation. The template terminates a message with <|eom|> when more
messages follow in the same turn and <|eot|> only at the end of the turn,
so the answer is closed by <|eom|> and the call opens a fresh header:
<prose><|eom|><|start|>assistant to=<tool><|message|><atem:function_calls>...
The final-message rule read content with until("<|eot|>"), which assumed the
user-facing message is always last. There is no <|eot|> before the call, so
content ran to the end of the turn, absorbed the markup, and no tool_calls
were emitted - the tool never ran. On a tau2-bench telecom run this hit 43
turns across 19 of 114 tasks.
Stop the answer at <|eom|> and parse what follows as tool calls.
Adds models/templates/muse-glimmer.jinja and four parser tests: a plain
answer, the <|eom|> junction, markup quoted in an answer staying content,
and tool markup inside the to=self channel staying reasoning.
* address comment
This commit is contained in:
@@ -5843,6 +5843,52 @@ static void test_template_output_peg_parsers(bool detailed_debug) {
|
||||
.run();
|
||||
}
|
||||
|
||||
// Muse Glimmer format tests
|
||||
{
|
||||
auto tst = peg_tester("models/templates/muse-glimmer.jinja", detailed_debug);
|
||||
|
||||
const std::string call_markup =
|
||||
"<atem:function_calls>\n"
|
||||
"<atem:invoke name=\"special_function\">\n"
|
||||
"<atem:parameter name=\"arg1\">1</atem:parameter>\n"
|
||||
"</atem:invoke>\n"
|
||||
"</atem:function_calls>";
|
||||
|
||||
// A plain answer is unaffected
|
||||
tst.test(" to=user<|message|>Hello, world!\nWhat's up?<|eot|>")
|
||||
.reasoning_format(COMMON_REASONING_FORMAT_AUTO)
|
||||
.expect(message_assist)
|
||||
.run();
|
||||
|
||||
// "Inform then act": the model answers the user and calls a tool in ONE generation,
|
||||
// closing the answer with <|eom|>. The answer must stop there rather than swallow it.
|
||||
tst.test(" to=user<|message|>Hello, world!\nWhat's up?<|eom|>"
|
||||
"<|start|>assistant to=special_function<|message|>" +
|
||||
call_markup)
|
||||
.tools({ special_function_tool })
|
||||
.reasoning_format(COMMON_REASONING_FORMAT_AUTO)
|
||||
.expect(message_with_content_and_tool_call("Hello, world!\nWhat's up?", "special_function",
|
||||
"{\"arg1\":1}"))
|
||||
.run();
|
||||
|
||||
// Markup quoted in an answer has no preceding <|eom|>, so it stays content instead of
|
||||
// becoming an invocation the user never asked for
|
||||
tst.test(" to=user<|message|>You invoke it like this:\n" + call_markup + "<|eot|>")
|
||||
.tools({ special_function_tool })
|
||||
.reasoning_format(COMMON_REASONING_FORMAT_AUTO)
|
||||
.expect_content("You invoke it like this:\n" + call_markup)
|
||||
.run();
|
||||
|
||||
// Tool markup inside the analysis channel is reasoning, not a call
|
||||
tst.test(" to=self<|message|>I could use " + call_markup + " here<|eom|>"
|
||||
"<|start|>assistant to=user<|message|>Hello!<|eot|>")
|
||||
.tools({ special_function_tool })
|
||||
.reasoning_format(COMMON_REASONING_FORMAT_AUTO)
|
||||
.expect_reasoning("I could use " + call_markup + " here")
|
||||
.expect_content("Hello!")
|
||||
.run();
|
||||
}
|
||||
|
||||
// GPT-OSS format tests
|
||||
{
|
||||
auto tst = peg_tester("models/templates/openai-gpt-oss-120b.jinja", detailed_debug);
|
||||
|
||||
Reference in New Issue
Block a user