b013e56a7147dbff5ec03175828442a94086507c
* ui : add the drawer, sheet and grouped list primitives Add the drawer and sheet overlay components, the toggle and toggle group, the shared searchable input, the collapsible sections and the grouped list, and the near viewport helper they position with. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : add the shared model components and data layer Add the model row pieces shared by the selector and the manager, the models store with its download status feed, the huggingface and migration services, and the model utils, enums and constants they read through. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : rework the models selector around its providers Group the selector components under their own folder, rework the list around the provider grouping, add the mobile trigger and the download item, and derive the selection state in one hook. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : add the models manager Add the manager table with its repo, quant and status rows, the filters and the toolbar, the row actions in a drawer, the downloads section, the manage models dialog and the stories and tests. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : rework the chat form actions and the mobile experience Move the add actions into a drawer and a sheet, float the model actions in a bar on a phone, open the context panel in a drawer, and derive the attachment, reasoning and tools menus in hooks. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : move the mcp servers to the sidebar rail and tidy the dialogs Move the mcp servers dialog to the sidebar rail with its menu entries, even out the dialogs on a phone, and group the mcp and settings stores behind their own modules. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : polish the shell The secondary button gets its own look back and the chat add button its own light surface. Pressable elements get a pointer cursor again, the font rendering smooths on the app shell, and the agent skills stay out of prettier's way. The root layout props probe rework that came with this polish reads the providers api url and stays with the providers change instead. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : wrap the model id classes Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * ui : load the model the new chat CTA picks Start a new chat selected the model but left it unloaded, so the chat opened against a server with nothing in memory. Both CTAs share a load helper now. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash * refactor: Post-review fixes
ggml-openvino: update to 2026.4.1, optimize performance, expand ops, improve device listing. (#29852)
llama.cpp
LLM inference in C/C++
ggml / ops / maintainer PRs / dev stats / lib llama API / llama-server REST API
Quick start
A few options to get llama.cpp installed on your machine:
# curl
curl -LsSf https://llama.app/install.sh | sh
# powershell
irm https://llama.app/install.ps1 | iex
- Visit https://llama.app and follow the instructions
- Run with Docker - see our Docker documentation
- Download pre-built binaries from the releases page
- Build from source by cloning this repository - check out our build guide
Once installed:
# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
|
|
|
Description
The main goal of llama.cpp is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on
a wide range of hardware - locally and in the cloud.
- Plain C/C++ implementation without any dependencies
- Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks
- AVX, AVX2, AVX512 and AMX support for x86 architectures
- RVV, ZVFH, ZFH, ZICBOP and ZIHINTPAUSE support for RISC-V architectures
- 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use
- Custom CUDA kernels for running LLMs on NVIDIA GPUs (support for AMD GPUs via HIP and Moore Threads GPUs via MUSA)
- Vulkan and SYCL backend support
- CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity
The llama.cpp project is build on top of the ggml library.
Supported backends
| Backend | Target devices |
|---|---|
| BLAS | All |
| BLIS | All |
| CANN | Ascend NPU |
| CUDA | Nvidia GPU |
| HIP | AMD GPU |
| Hexagon | Snapdragon |
| IBM zDNN | IBM Z & LinuxONE |
| MUSA | Moore Threads GPU |
| Metal | Apple Silicon |
| OpenCL | Adreno GPU |
| OpenVINO [In Progress] | Intel CPUs, GPUs, and NPUs |
| RPC | All |
| SYCL | Intel GPU |
| VirtGPU | VirtGPU APIR |
| Vulkan | GPU |
| WebGPU | All |
| ZenDNN | AMD CPU |
Documentation
Tools
Development
- How to build
- Running on Docker
- Build on Android
- Multi-GPU usage
- Performance troubleshooting
- GGML tips & tricks
- XCFramework
- Completions
- Models
- Release process
Contributing
- Contributors can open PRs
- Collaborators will be invited based on contributions
- Maintainers can push to branches in the
llama.cpprepo and merge PRs into themasterbranch - Any help with managing issues, PRs and projects is very appreciated!
- Read the CONTRIBUTING.md for more information
Acknowledgements
- yhirose/cpp-httplib - Single-header HTTP server, used by
llama-server- MIT license - nothings/stb - Single-header image format decoder, used by multimodal subsystem - Public domain
- nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
- mackron/miniaudio - Single-header audio format decoder, used by multimodal subsystem - Public domain
- sheredom/subprocess.h - Single-header process launching solution for C and C++ - Public domain
Languages
C++
54.9%
C
16.7%
Python
7%
Cuda
5.6%
TypeScript
3.9%
Other
11.7%