model : Granite Switch Decision (#30676)

* feat(convert): Add decision.chat to support decision models that can also chat

Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat(convert): Add conversion for Granite Switch as decision model

The template follows from the usage sample:
https://huggingface.co/ibm-granite/granite-switch-4.1-8b#guardian-and-decision-side-by-side

Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat(common): Add support for chat+decision models in common

Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat(common): Add granite switch decision model enum

Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat(server): Plumb through decision_chat bool to support decision+chat models

Branch: GraniteSwitchDecision
AI-usage: full (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat(server): Add Granite Switch decision support

Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* docs: Update server README for granite switch decisions / chat+decisions

Branch: GraniteSwitchDecision
AI-usage: draft (Claude + Sonnet 5.5)
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* Revert "feat(convert): Add decision.chat to support decision models that can also chat"

This reverts commit 952ef567647e53ac9beda7a1f79d6927240d068d.

* Revert "feat(common): Add support for chat+decision models in common"

This reverts commit 93ed782e3a8b2fd5968732c6525c7ad29ad4bea2.

* Revert "feat(server): Plumb through decision_chat bool to support decision+chat models"

This reverts commit 97e9af60ddaccbd82f1eb0a488adff43b16ee095.

* fix: If Granite Switch supports decisions, it always supports text too

Branch: GraniteSwitchDecision
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* docs: Remove reference to decision.chat in server README

Branch: GraniteSwitchDecision
AI-usage: none
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
This commit is contained in:
Gabe Goodhart authored and GitHub committed 2026-10-16 20:00:41 +02:00
1 parent 7182ef71e6
commit 23f7aabc16
8 files changed
+40 -5

No files matched your search

+2 -2
View File
@@ -87,7 +87,7 @@ void server_decision_context::init(const llama_model * model) {
bias[level] = std::strtof(val, nullptr);
}
if (model_type == COMMON_DECISION_TYPE_OPENJEV) {
if (model_type == COMMON_DECISION_TYPE_OPENJEV || model_type == COMMON_DECISION_TYPE_GRANITE_SWITCH) {
// one letter per option, each must be a single token
const std::string letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
for (const char c : letters) {
@@ -753,7 +753,7 @@ void server_decision_context::fill_task(
const std::string prompt = render(state, questions, question, variant, files.size());
if (type == COMMON_DECISION_TYPE_OPENJEV || type == COMMON_DECISION_TYPE_LEV || type == COMMON_DECISION_TYPE_NIMBLE || type == COMMON_DECISION_TYPE_PPLX_DECIDER || type == COMMON_DECISION_TYPE_VLLM_D3) {
if (type == COMMON_DECISION_TYPE_OPENJEV || type == COMMON_DECISION_TYPE_GRANITE_SWITCH || type == COMMON_DECISION_TYPE_LEV || type == COMMON_DECISION_TYPE_NIMBLE || type == COMMON_DECISION_TYPE_PPLX_DECIDER || type == COMMON_DECISION_TYPE_VLLM_D3) {
// lev reads the ratings of a noul question at its first labels, not at the digits
task.decision.labels.assign(labels.begin(), labels.begin() + n_outputs(question));
}