server: expose the reasoning effort levels of the chat template (#31271)
* server: expose the reasoning effort levels of the chat template The capability probe renders a user turn for each candidate level (minimal, low, medium, high, xhigh, max) and for an unknown level. A level is honored when it renders without raising and differently from the unknown level, which shows the template fallback. Levels sharing a render are aliases and the highest one is kept. The caps serialize to JSON, and chat_template_caps in /props reports the list as reasoning_effort_levels, from lowest to highest. * jinja: keep the caps free of the json dependency The caps keep reporting their flags as a map, and the effort levels stay a plain member. common_chat_templates_get_caps() builds the JSON the server reports from both, so jinja depends on json only for the probe inputs.
This commit is contained in:
1 parent
a919179942
commit
1b4554eff1
9 files changed
+90
-13
No files matched your search
+7
-6
@@ -1940,12 +1940,13 @@ const common_chat_msg & common_chat_session::finish(const common_chat_input & ch
|
||||
return result;
|
||||
}
|
||||
|
||||
std::map<std::string, bool> common_chat_templates_get_caps(const common_chat_templates * chat_templates) {
|
||||
common_json common_chat_templates_get_caps(const common_chat_templates * chat_templates) {
|
||||
GGML_ASSERT(chat_templates != nullptr);
|
||||
GGML_ASSERT(chat_templates->template_default != nullptr);
|
||||
if (chat_templates->template_tool_use != nullptr) {
|
||||
// take the more expressive template when available
|
||||
return chat_templates->template_tool_use->caps.to_map();
|
||||
}
|
||||
return chat_templates->template_default->caps.to_map();
|
||||
// take the more expressive template when available
|
||||
const auto & caps = chat_templates->template_tool_use != nullptr ? chat_templates->template_tool_use->caps
|
||||
: chat_templates->template_default->caps;
|
||||
common_json result = caps.to_map();
|
||||
result["reasoning_effort_levels"] = caps.reasoning_effort_levels;
|
||||
return result;
|
||||
}
|
||||
Reference in new issue
Block a user