server: use port 9931 by default (#30159)

* server: use port 9931 by default

* revert unrelated changes
This commit is contained in:
Xuan-Son Nguyen authored and GitHub committed 2026-10-09 10:40:03 +02:00
1 parent b013e56a71
commit b42b7e6d30
40 files changed
+68 -66

No files matched your search

+3 -3
View File
@@ -282,7 +282,7 @@ This table can be generated with:
# Usage - need tool-aware Jinja template
First, start a server with any model, but make sure it has a tools-enabled template: you can verify this by inspecting the `chat_template` or `chat_template_tool_use` properties in `http://localhost:8080/props`).
First, start a server with any model, but make sure it has a tools-enabled template: you can verify this by inspecting the `chat_template` or `chat_template_tool_use` properties in `http://localhost:9931/props`).
Here are some models known to work (w/ chat template override when needed):
@@ -336,7 +336,7 @@ To get the official template from original HuggingFace repos, you can use [scrip
Test in CLI (or with any library / software that can use OpenAI-compatible API backends):
```bash
curl http://localhost:8080/v1/chat/completions -d '{
curl http://localhost:9931/v1/chat/completions -d '{
"model": "gpt-3.5-turbo",
"tools": [
{
@@ -366,7 +366,7 @@ curl http://localhost:8080/v1/chat/completions -d '{
}'
curl http://localhost:8080/v1/chat/completions -d '{
curl http://localhost:9931/v1/chat/completions -d '{
"model": "gpt-3.5-turbo",
"messages": [
{"role": "system", "content": "You are a chatbot that uses tools/functions. Dont overthink things."},