Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

chat : fix muse-glimmer detection of tool calls after EOM (#26879)

Browse Source 
* chat : fix muse-glimmer swallowing a trailing tool call into content

Muse Glimmer routinely answers the user and calls a tool in a single
generation. The template terminates a message with <|eom|> when more
messages follow in the same turn and <|eot|> only at the end of the turn,
so the answer is closed by <|eom|> and the call opens a fresh header:

    <prose><|eom|><|start|>assistant to=<tool><|message|><atem:function_calls>...

The final-message rule read content with until("<|eot|>"), which assumed the
user-facing message is always last. There is no <|eot|> before the call, so
content ran to the end of the turn, absorbed the markup, and no tool_calls
were emitted - the tool never ran. On a tau2-bench telecom run this hit 43
turns across 19 of 114 tasks.

Stop the answer at <|eom|> and parse what follows as tool calls.

Adds models/templates/muse-glimmer.jinja and four parser tests: a plain
answer, the <|eom|> junction, markup quoted in an answer staying content,
and tool markup inside the to=self channel staying reasoning.

* address comment
This commit is contained in:
ruanslv
2026-08-11 15:15:20 -05:00
committed by GitHub
parent 7b13a8404d
commit 0b1bad14ff
3 changed files with 261 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/chat.cpp
+4 -2
View File
@@ -3155,7 +3155,8 @@ static common_chat_params common_chat_params_init_muse_glimmer(const common_chat
auto analysis = p.ref("analysis");
auto recipient = p.optional(p.literal(" to=user"));
auto final_msg = p.rule("final", recipient + p.literal("<|message|>") + p.content(p.until("<|eot|>")));
auto final_msg = p.rule("final", recipient + p.literal("<|message|>") +
p.content(p.until_one_of({ "<|eot|>", "<|eom|>" })));
if (has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE) {
auto string_value = p.ac(
@@ -3211,7 +3212,8 @@ static common_chat_params common_chat_params_init_muse_glimmer(const common_chat
if (inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED) {
return p.zero_or_more(start + analysis) + start + tool_calls;
}
return p.zero_or_more(start + analysis) + start + (tool_calls | final_msg);
auto trailing_calls = p.optional(p.literal("<|eom|>") + start + tool_calls);
return p.zero_or_more(start + analysis) + start + (tool_calls | (final_msg + trailing_calls));
}
return p.zero_or_more(start + analysis) + start + final_msg;