Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

common: add system-level config file (#26118)

Browse Source 
* common: Add CLI > ENV > models-presets > INI precedence

1. CLI flags have the highest precedence
2. ENV vars have the second-highest precedence
3. System and User configs have the lowest precedence
   - Linux/BSD/Mac
     - /etc/llama.cpp/config.ini < ${XDG_CONFIG_HOME:-~/.config}/llama.cpp/config.ini
   - Windows
     - %PROGRAMDATA%\llama.cpp\config.ini < %APPDATA%\llama.cpp\config.ini

* fix UB

* use common_get_env

* ignore_unknown_keys

* nits

* add docs

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
Johnathan Craig Maudlin
2026-08-13 00:02:27 +02:00
committed by GitHub
co-authored by Xuan Son Nguyen
parent 84e908c625
commit 8e7f22b67e
6 changed files with 136 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/preset.md
+16 -1
View File
@@ -4,7 +4,7 @@
The INI preset feature, introduced in [PR#17859](https://github.com/ggml-org/llama.cpp/pull/17859), allows users to create reusable and shareable parameter configurations for llama.cpp.
### Using Presets with the Server
## Using Presets with the Server
When running multiple models on the server (router mode), INI preset files can be used to configure model-specific parameters. Please refer to the [server documentation](../tools/server/README.md) for more details.
@@ -93,3 +93,18 @@ llama-server -hf user/repo:gpt-oss-120b-hf
```
Please make sure to provide the correct `hf-repo` for each child preset. Otherwise, you may get error: `The specified tag is not a valid quantization scheme.`
## System-level config
The system-level config, added in PR [#26118](https://github.com/ggml-org/llama.cpp/pull/26118), allows sharing the same set of options among multiple tools and examples. Unlike the sections above, it is not limited to the server.
These files are loaded on startup if present. A later file overrides an earlier one:
1. System-wide: `/etc/llama.cpp/config.ini` (or `%PROGRAMDATA%\llama.cpp\config.ini` on Windows)
2. User-level: `$XDG_CONFIG_HOME/llama.cpp/config.ini`, `~/.config/llama.cpp/config.ini` by default (or `%APPDATA%\llama.cpp\config.ini` on Windows)
The config file is applied first, then its options are overridden by ENV variables, CLI arguments and model presets (in router mode).
Note:
- Only the `[*]` and default sections are used; options written before any section header belong to "default. Named sections are ignored
- Tool-specific options can be specified, but will be ignored (with a warning) if the example doesn't support it<br/>Example: if you specify `port = 1234`, only `llama-server` will use it, other examples will ignore it
- `model` or `hf-repo` are not recommended to be configured system-level, because it may introduce conflicts<br/>Example: a `hf-repo` in the config file still takes effect when you pass `-m` on the command line, so you may load a different model than expected