Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

common : skip device_info loop if it's not going to be printed (#26692)

Browse Source 
The device_info loop iterates over the discovered devices and gets
the available and total memory counts. With the CUDA backend (and
possibly others too) this requires creating a GPU context, which,
in case of CUDA, results in a 550 MB VRAM allocation.

For this information to be used in any way, the log verbosity must
be set to LOG_LEVEL_TRACE. If it's not, including in the default
configuration, the contexts get created, memory sizes get queried,
then the log function quietly discards the data.

In certain cases the user may not want to use any GPU resources.
The device_loop iteration is the only place touching the GPU that
cannot be skipped.

Fix by checking the verbosity level and skipping the loop if there
would be no output.
This commit is contained in:
Bartosz Taudul
2026-08-23 14:39:16 +02:00
committed by GitHub
parent b0539c43ed
commit ba8e0eddfb
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
common/common.cpp
+3 -2
View File
@@ -402,10 +402,11 @@ void common_params_print_info(const common_params & params, bool print_devices)
#endif #endif
COM_TRC("%s: build %d (%s) with %s for %s%s\n", __func__, llama_build_number(), llama_commit(), llama_compiler(), llama_build_target(), build_type); COM_TRC("%s: build %d (%s) with %s for %s%s\n", __func__, llama_build_number(), llama_commit(), llama_compiler(), llama_build_target(), build_type);
COM_INF("%s: verbosity = %d (adjust with the `-lv N` CLI arg)\n", __func__, common_log_get_verbosity_thold()); const int verbosity = common_log_get_verbosity_thold();
COM_INF("%s: verbosity = %d (adjust with the `-lv N` CLI arg)\n", __func__, verbosity);
// device enumeration creates a primary context on CUDA backends, skip it when the caller does not own any device // device enumeration creates a primary context on CUDA backends, skip it when the caller does not own any device
if (print_devices) { if (print_devices && verbosity >= LOG_LEVEL_TRACE) {
COM_TRC("%s", "device_info:\n"); COM_TRC("%s", "device_info:\n");
for (size_t i = 0; i < ggml_backend_dev_count(); ++i) { for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
auto * dev = ggml_backend_dev_get(i); auto * dev = ggml_backend_dev_get(i);