Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

sycl: Increase minimum buffer size for USM system allocations (#25525)

Browse Source 
Raise the threshold for minimum buffer size from 1 GiB to 4 GiB, based
on real-world experiments of overcommitting device memory with model
weights larger than available VRAM, for example Qwen3.5-35B-A3B-Q8
running on a B70.

Also add a debug message to better track USM system allocations.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
This commit is contained in:
Francois Dugast
2026-07-15 10:28:24 +03:00
committed by GitHub
parent 32b741c336
commit 0e148a573f
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-sycl/ggml-sycl.cpp
+2 -1
View File
@@ -843,7 +843,7 @@ static const char * ggml_backend_sycl_buffer_type_get_name(ggml_backend_buffer_t
} }
static bool check_usm_system(int device, size_t size) { static bool check_usm_system(int device, size_t size) {
bool use_usm_system = g_ggml_sycl_usm_system && size >= MEM_SIZE_1G; bool use_usm_system = g_ggml_sycl_usm_system && size >= ((size_t)4 * MEM_SIZE_1G);
if (use_usm_system && !ggml_sycl_info().devices[device].usm_system_support) { if (use_usm_system && !ggml_sycl_info().devices[device].usm_system_support) {
GGML_LOG_INFO("Device does not support USM system allocations\n"); GGML_LOG_INFO("Device does not support USM system allocations\n");
@@ -882,6 +882,7 @@ ggml_backend_sycl_buffer_type_alloc_buffer(ggml_backend_buffer_type_t buft,
void * dev_ptr; void * dev_ptr;
if (use_usm_system) { if (use_usm_system) {
GGML_SYCL_DEBUG("[SYCL] allocating %lu Bytes with USM system\n", size);
dev_ptr = (void *)aligned_malloc_host(alignment, aligned_size); dev_ptr = (void *)aligned_malloc_host(alignment, aligned_size);
if (!dev_ptr) { if (!dev_ptr) {
GGML_LOG_ERROR("%s: can't allocate %lu Bytes of memory on host\n", __func__, size); GGML_LOG_ERROR("%s: can't allocate %lu Bytes of memory on host\n", __func__, size);