mtmd: add --mmproj-device argument (#23255)

* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This commit is contained in:
John-Henry Lim
2026-08-20 18:45:37 +02:00
committed by GitHub
co-authored by Xuan Son Nguyen
parent 07822bddf8
commit 681c29d36a
12 changed files with 41 additions and 11 deletions
+1
View File
@@ -86,6 +86,7 @@ int main(int argc, char ** argv) {
mtmd_context_params mtmd_params = mtmd_context_params_default();
mtmd_params.use_gpu = params.mmproj_use_gpu;
mtmd_params.device = params.mmproj_device;
mtmd::context_ptr mctx(mtmd_init_from_file(params.mmproj.path.c_str(), model, mtmd_params));
if (!mctx) {
LOG_ERR("failed to load mmproj %s\n", params.mmproj.path.c_str());