sycl: Add optional USM system allocations (#22526)

This introduces an optional feature to allocate large GPU buffers (≥ 1GB)
using USM system allocations if supported by the device. It allows using
buffers from the system allocator then letting the system manage memory
migrations between host and device as necessary.

This feature is disabled by default and requires the GGML_SYCL_USM_SYSTEM
environment variable to enable. If USM system allocations are not supported
by the device or the system, we fallback to regular allocations.

This feature can allow VRAM overcommit. For example, the test below fails
on B580 due to lack of memory for allocation, but it passes when enabling
USM system allocations:

  ./examples/sycl/test.sh -m Qwen3.5-27B-Q3_K_M.gguf -lv 4

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
This commit is contained in:
Francois Dugast
2026-06-17 08:54:21 +03:00
committed by GitHub
parent 74ade52741
commit 9b260fc9ef
3 changed files with 69 additions and 25 deletions
+1
View File
@@ -230,6 +230,7 @@ struct sycl_device_info {
size_t total_vram;
sycl_hw_info hw_info;
optimize_feature opt_feature;
bool usm_system_support; // support for USM system allocations
};