281e7dad7cfe4539b13d0218f4b0fd5a03f3abc9
* ui: let a menu action open a submenu Group and project pickers open as flyouts from a chat's menus. Master's menu list has no submenus, so bring them back next to its tooltips, with a close callback for picks made inside them. Assisted-by: Claude (Opus 5.5 Medium) * ui : store groups and let conversations join them Groups live in their own IndexedDB table (schema version 2) with a name. groupsStore creates, renames and deletes them; deleting a group keeps its chats, out of any group. A conversation records the group it is sorted into, and a group picked before the first message is kept pending and applied when the conversation is created. Assisted-by: Claude (Opus 5.5 Medium) * ui : show each group as a sidebar section Every group is a sidebar section next to Pinned and Recent, folding, showing its first six chats and dragged into place like them. A chat dropped on a group joins it and leaves Pinned; dropped on Recent, it leaves its group. Recent is called Ungrouped once a group exists, and its menus create a group in a small naming dialog; so does a right-click anywhere in the list that no row or heading takes, down to the empty space under the last section. A group's heading starts a new chat in it, and its menus rename or delete it; deleting keeps its chats, back in Ungrouped. Assisted-by: Claude (Opus 5.5 Medium) * ui : move chats into groups from their menus A chat's menus, in the sidebar and on right-click, move it into a group or take it out of its group. Desktop menus show a searchable group flyout whose New group entry names a group in the group dialog and moves the chat into it; a phone gets a picker dialog instead, which creates a group from what is typed. Assisted-by: Claude (Opus 5.5 Medium) * ui: open the group picker from a phone's hold menu The hold menu is anchored to the row, so it showed the desktop flyout, which ran off the left edge of the screen and whose New group closed without naming one. On a phone, Move to group now opens the picker dialog, as intended. Assisted-by: Claude (Opus 5.5 Medium) * ui: open a new chat in a tab of its own With conversation tabs on, New chat from the sidebar, a group's heading or the shortcut replaced the open chat's tab. It now adds a tab, as the tab bar's "+" does, and the open chat keeps its own. Assisted-by: Claude (Opus 5.5 Medium) * ui: list a phone's groups under the picker's search On a phone the picker fills the screen, and its grid spread the rows over the height, so the list sat in the middle. It is now a column: the list starts under the search and scrolls in the rest of the screen. Assisted-by: Claude (Opus 5.5 Medium) * ui: fix gaps in chat groups found in review - A phone shows a group heading's New chat and menu, and the Ungrouped heading's menu, which were reachable only by hover or right-click. - A pinned chat moved into a group from its menu leaves Pinned, as a drag onto the group already did, so the move shows. - Only the sidebar's New chat, a group's New chat and the new-chat shortcut open a new tab; stepping past the last chat, search and the models drawer show the new-chat screen in the current tab again. - An export carries each chat's group, and an import sorts the chat into the group of that id or name, or creates it; a group the file does not carry is dropped. - The group picked with a group's New chat survives leaving the new-chat tab and coming back. - A phone's menus skip any submenu in the shared menu list, not only the group one. - Deleting a group forgets its section's place and fold. - Groups load newest first by creation, so a rename no longer moves one after a reload. - groupsStore looks groups up in a map and answers which live group a chat is in, for the three places that checked it by hand. Assisted-by: Claude (Opus 5.5 Medium) * ui: keep a forked chat in its source chat's group A fork copied the working directory but not the group, so it landed in Ungrouped. Assisted-by: Claude (Opus 5.5 Medium) * ui: move a chat with its whole fork family between groups Moving a chat into a group, or out of one, takes its parents and its children along, so a fork tree never splits across sidebar sections. A pinned member that joins a group leaves Pinned with the rest. Group sections render the tree with its depth, as Ungrouped does. * ui: show the group name prompt as a small centered card on phones The new and rename group prompt opened full screen on a phone, like a large dialog. It is now built on the alert dialog, as the chat rename prompt is, so a phone shows it as a card in the middle of the screen. Its Cancel is a plain button, so Enter in the name field saves. Assisted-by: Claude (Opus 5.5 Medium) * ui: space a phone's group sections like Ungrouped On a phone a group or Pinned section put its first chat right under the heading, where Ungrouped leaves a row gap. They now leave the same gap, and the sections sit a little further apart than the rows inside them. Assisted-by: Claude (Opus 5.5 Medium) --------- Co-authored-by: Pascal <admin@serveurperso.com>
common/json-schema : fix allOf with plain types (e.g. integer) generating an empty object grammar (#31834)
llama.cpp
LLM inference in C/C++
ggml / ops / maintainer PRs / dev stats / lib llama API / llama-server REST API
Quick start
A few options to get llama.cpp installed on your machine:
# curl
curl -LsSf https://llama.app/install.sh | sh
# powershell
irm https://llama.app/install.ps1 | iex
- Visit https://llama.app and follow the instructions
- Run with Docker - see our Docker documentation
- Download pre-built binaries from the releases page
- Build from source by cloning this repository - check out our build guide
Once installed:
# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
|
|
|
Description
The main goal of llama.cpp is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on
a wide range of hardware - locally and in the cloud.
- Plain C/C++ implementation without any dependencies
- Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks
- AVX, AVX2, AVX512 and AMX support for x86 architectures
- RVV, ZVFH, ZFH, ZICBOP and ZIHINTPAUSE support for RISC-V architectures
- 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and 8-bit integer quantization for faster inference and reduced memory use
- Custom CUDA kernels for running LLMs on NVIDIA GPUs (support for AMD GPUs via HIP and Moore Threads GPUs via MUSA)
- Vulkan and SYCL backend support
- CPU+GPU hybrid inference to partially accelerate models larger than the total VRAM capacity
The llama.cpp project is build on top of the ggml library.
Supported backends
| Backend | Target devices |
|---|---|
| BLAS | All |
| BLIS | All |
| CANN | Huawei Ascend NPU |
| CUDA | Nvidia GPU |
| HIP | AMD GPU |
| Hexagon | Snapdragon |
| IBM zDNN | IBM Z & LinuxONE |
| MUSA | Moore Threads GPU |
| Metal | Apple Silicon |
| OpenCL | Adreno GPU |
| OpenVINO [In Progress] | Intel CPUs, GPUs, and NPUs |
| RPC | All |
| SYCL | Intel GPU |
| VirtGPU | VirtGPU APIR |
| Vulkan | GPU |
| WebGPU | All |
| ZenDNN | AMD CPU |
Documentation
Tools
Development
- How to build
- Running on Docker
- Build on Android
- Multi-GPU usage
- Performance troubleshooting
- GGML tips & tricks
- XCFramework
- Completions
- Models
- Release process
Contributing
- Contributors can open PRs
- Collaborators will be invited based on contributions
- Maintainers can push to branches in the
llama.cpprepo and merge PRs into themasterbranch - Any help with managing issues, PRs and projects is very appreciated!
- Read the CONTRIBUTING.md for more information
Acknowledgements
- yhirose/cpp-httplib - Single-header HTTP server, used by
llama-server- MIT license - nothings/stb - Single-header image format decoder, used by multimodal subsystem - Public domain
- nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
- mackron/miniaudio - Single-header audio format decoder, used by multimodal subsystem - Public domain
- sheredom/subprocess.h - Single-header process launching solution for C and C++ - Public domain
Languages
C++
54.9%
C
16.7%
Python
7%
Cuda
5.6%
TypeScript
3.9%
Other
11.7%