With -lm none the GGUF is read() into our own buffers, but the kernel keeps
the file in page cache afterwards, charged to the pod's cgroup: about 20 GB
after a 35B load. On 2026-10-24 the Talos OOM controller killed the pod right
after a load under exactly that pressure. Those pages are pure waste once the
data is copied.
llama_file::drop_cache() wraps posix_fadvise(POSIX_FADV_DONTNEED) (Linux only,
errors logged at debug). load_all_data() drops each copied tensor's range right
after reading it, so the cache never grows past about one tensor, and drops the
whole file at the final cleanup to catch the metadata and partial pages at
tensor edges. Both only when the loader holds no mapping of the files
(mappings.empty(): not mmap, not mmap+mlock, no lazy tensors), so nothing that
is mapped is ever advised away. Split GGUFs are covered (every file in files).
clip.cpp always copies the mmproj via ifstream, so it drops the whole file
after loading it.
Measured on the pod, gemma-4-E2B UD-Q4_K_XL (2.5 GiB) + mmproj BF16 (0.9 GiB),
CPU only, -lm none, cold cache, same binary with the drops disabled as the
control (4 rounds, rotated order):
- file pages resident after load (mincore): 2431-2444 / 940 MiB -> 0 / 0
- peak model-file residency during load: 2.5 GiB -> 1.2-1.5 GiB (largest tensor)
- pod cgroup file after load: +3.37 GiB -> +0; host MemFree after load
0.6 GB -> 2.5-2.8 GB (MemAvailable counts clean cache, so it cannot show this)
- load time: 5.87 s -> 6.82 s median (+0.95 s); a post-load-only drop is
6.56 s (+0.68 s). The cost is synchronous page freeing that reclaim would
otherwise do later; -lm dio (direct IO) costs the same (+0.9 s) and does not
cover metadata or the mmproj.
- greedy output byte-identical; -lm mmap keeps the model file fully resident
and mapped (2498 MiB, no refaults); -lzm on skips the drop (file mapped).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>