Files
llama.cpp/docs/multimodal/minicpmv4.7.md
T
69f201a205 model : support MiniCPM-V 4.7 (#29416)
* mtmd : add MiniCPM-V 4.7 support

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* model : allow mrope time from an extra position slot

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* Update conversion/minicpm.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* Slim down comments

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix for "do not hand-wrap comments"

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* fix ci

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* rm 3d repo for pr one

Signed-off-by: tc-mb <tianchi_cai@icloud.com>

* gguf: add rope.section_order metadata

* fix comments

* handle grid layout

* allow compat

---------

Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-10 20:41:59 +02:00

2.5 KiB

MiniCPM-V 4.7

Prepare models and code

Download MiniCPM-V-4.7 PyTorch model from huggingface to "MiniCPM-V-4.7" folder.

The model must be the standard transformers checkpoint (no trust_remote_code for the text and vision graph used here); the architecture in config.json is MiniCPMV4_7ForConditionalGeneration with a qwen3_5_text (or qwen3_5_moe_text) text model and a SigLIP-based vision tower plus a window-attention vit_merger, same as MiniCPM-V 4.6.

If the checkpoint ships no MTP weights, pass --no-mtp to skip the nextn layers.

Build llama.cpp

If there are differences in usage, please refer to the official build documentation

Clone llama.cpp:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

Build llama.cpp using CMake:

cmake -B build
cmake --build build --config Release

Usage of MiniCPM-V 4.7

MiniCPM-V 4.7 is converted directly through convert_hf_to_gguf.py. The same script is invoked twice on the original Hugging Face directory: once to produce the language-model GGUF and once with --mmproj to produce the multimodal projector GGUF.

# language model
python ./convert_hf_to_gguf.py ../MiniCPM-V-4.7 --outfile ../MiniCPM-V-4.7/ggml-model-f16.gguf --no-mtp

# multimodal projector (vision tower + window-attention vit_merger + DownsampleMLP merger)
python ./convert_hf_to_gguf.py ../MiniCPM-V-4.7 --mmproj --outfile ../MiniCPM-V-4.7/mmproj-model-f16.gguf

# optional: quantize to Q4_K_M
./build/bin/llama-quantize ../MiniCPM-V-4.7/ggml-model-f16.gguf ../MiniCPM-V-4.7/ggml-model-Q4_K_M.gguf Q4_K_M

The default projector merges 16x (4x4 patches into one token). To keep 4x more visual tokens, copy the model dir and set "downsample_mode": "4x" in the copy's preprocessor_config.json before running the --mmproj conversion; the loader reads clip.vision.projector.scale_factor to pick the graph.

Inference on Linux or Mac

# run in single-turn mode
./build/bin/llama-mtmd-cli -m ../MiniCPM-V-4.7/ggml-model-f16.gguf --mmproj ../MiniCPM-V-4.7/mmproj-model-f16.gguf -c 4096 --jinja --image xx.jpg -p "What is in the image?"

# run in conversation mode
./build/bin/llama-mtmd-cli -m ../MiniCPM-V-4.7/ggml-model-Q4_K_M.gguf --mmproj ../MiniCPM-V-4.7/mmproj-model-f16.gguf --jinja

The chat template enables thinking by default. Pass --chat-template-kwargs '{"enable_thinking": false}' to llama-server to turn it off.