Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

server: (router) lazy-load startup_models after main setup (#27424)

Browse Source 
* server: (router) lazy-load startup_models after main setup

* only allow is_first_load to populate it

* nits

* nits 2
This commit is contained in:
Xuan-Son Nguyen
2026-08-20 15:22:16 +02:00
committed by GitHub
parent 8a832e4bf3
commit 9855ad69d3
4 changed files with 55 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/README.md
+1 -1
View File
@@ -1757,7 +1757,7 @@ The precedence rule for preset options is as follows:
3. **Global options** defined in the preset file (`[*]`)
We also offer additional options that are exclusive to presets (these aren't treated as command-line arguments):
- `load-on-startup` (boolean): Controls whether the model loads automatically when the server starts
- `load-on-startup` (boolean): Controls whether the model loads automatically when the server starts. Only applies at startup: if the model list is reloaded later (for example after editing the preset file), a newly added model is listed but not loaded
- `stop-timeout` (int, seconds): After requested unload, wait for this many seconds before forcing termination (default: 10)
- `dedup-cache-models` (boolean): When the preset uses `hf-repo` pointing to a model that is already downloaded, hide the corresponding cached model entry from `GET /models` (the preset entry remains visible). Set it in the `[*]` section to apply to all presets.