Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

mtmd: add --mmproj-device argument (#23255)

Browse Source 
* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
John-Henry Lim
2026-08-20 18:45:37 +02:00
committed by GitHub
co-authored by Xuan Son Nguyen
parent 07822bddf8
commit 681c29d36a
12 changed files with 41 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/mtmd/mtmd.cpp
+2
View File
@@ -456,6 +456,7 @@ static clip_flash_attn_type mtmd_get_clip_flash_attn_type(enum llama_flash_attn_
mtmd_context_params mtmd_context_params_default() {
mtmd_context_params params {
/* use_gpu */ true,
/* device */ nullptr,
/* print_timings */ true,
/* n_threads */ 4,
/* image_marker */ nullptr,
@@ -564,6 +565,7 @@ struct mtmd_context {
clip_context_params ctx_clip_params {
/* use_gpu */ ctx_params.use_gpu,
/* device */ ctx_params.device,
/* flash_attn_type */ mtmd_get_clip_flash_attn_type(ctx_params.flash_attn_type),
/* image_min_tokens */ ctx_params.image_min_tokens,
/* image_max_tokens */ ctx_params.image_max_tokens,