This website requires JavaScript.
efb3036c18
sycl: add fused top-k MoE (#25217 )
2026-07-13 01:56:41 -05:00
e474bba7af
sycl: add Q2_K to DMMV reorder path (#25064 )
Todd Malsbary and GitHub
2026-07-12 23:53:39 -07:00
38fd5c9993
ui: Remove recommended MCP Servers + improve MCP Servers Settings UI/UX (#25535 )
Aleksander Grygier and GitHub
2026-07-13 08:45:04 +02:00
99f3dc3229
server: honour per-request reasoning_budget_tokens in chat completions (#23116 )
2026-07-13 01:58:44 +02:00
34558825a2
vendor : update cpp-httplib to 0.50.1 (#25576 )
Alessandro de Oliveira Faria (A.K.A.CABELO) and GitHub
2026-07-12 20:10:03 -03:00
8014d2cf97
server: Don't consider models with --no-mmproj-auto as multimodal (#25590 )
Sebastian Dröge and GitHub
2026-07-13 01:48:13 +03:00
4114ba18b2
mtmd: fix silent prompt truncation on embedded NUL (#25548 )
2026-07-13 00:47:25 +02:00
0c4fa7a989
server : evict checkpoints within min-step of each other (#25472 )
Aldehir Rojas and GitHub
2026-07-12 15:59:14 -05:00
6b4dc2116a
server : fix image blocks in tool_result being dropped during Anthropic OpenAI conversion (#22536 )
quei and GitHub
2026-07-12 23:43:51 +08:00
8a545c7820
ggml-vulkan: implement host buffer pinning for faster H2D uploads
Lumpiasty
2026-07-12 15:51:29 +02:00
e3546c7948
Fix conditional to display 'LLAMA_SPLIT_MODE_TENSOR not implemented for architecture' message (#24926 )
kdkd and GitHub
2026-07-11 13:03:24 -05:00
d72bfa38f7
gguf : reject empty metadata keys (#24917 )
Rohit Mahesh and GitHub
2026-07-11 13:02:44 -05:00
3cec3bcd16
cuda: Don't crash when querying memory on device with no free memory. (#25157 )
cphlipot and GitHub
2026-07-11 10:13:43 -07:00
13f2b28b09
DeepseekV4: clear cache only for seq rather than full (#25521 )
Aman Gupta and GitHub
2026-07-11 23:35:45 +08:00
c92e806d1c
server: allow stream for exec_shell_command (#25526 )
Xuan-Son Nguyen and GitHub
2026-07-11 12:42:55 +02:00
ea1f7bbb5d
server: refactor server_stream (#25541 )
2026-07-11 12:41:47 +02:00
00f5442cc4
ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (#24231 )
2026-07-11 11:39:07 +02:00
76f2798059
Vulkan: route large matmuls to medium tile on Adreno (#24877 )
Raman Shinde and GitHub
2026-07-11 13:58:29 +05:30
1d1d9a9ed7
opencl: add int8 dp4 dense and MoE prefill optimization for Adreno GPUs (#25537 )
2026-07-10 23:05:58 -07:00
4f37f51972
server: accept null sampling params (#25538 )
Pascal and GitHub
2026-07-10 22:07:29 +02:00
c749cb0417
llama : make tensor-split regex patterns static (#24710 )
eduardopessin and GitHub
2026-07-10 18:04:12 +01:00
67776eaee5
hexagon: improve ARGSORT performance for small tensors (#25512 )
Max Krasnyansky and GitHub
2026-07-10 09:06:06 -07:00
22b69b6e92
arg: prevent duplicate spec model downloads (#25527 )
Xuan-Son Nguyen and GitHub
2026-07-10 16:53:26 +02:00
3e706dd55f
mtmd: deepseek-ocr v1 multi-tile (#24717 )
2026-07-10 16:05:49 +02:00
07d9378286
feat: pre-select models in the webui using alias (#25492 )
2026-07-10 13:04:00 +00:00
9f623c683d
ui: use server modalities in non-router mode (#24874 )
Josh Leverette and GitHub
2026-07-10 08:03:52 -05:00
a935fbffe1
server: remove loading.html (#25500 )
Xuan-Son Nguyen and GitHub
2026-07-10 14:42:17 +02:00
0badc06ab5
sync : ggml
Georgi Gerganov
2026-07-10 13:10:49 +03:00
ac17f8ac1c
ggml : use ggml_vqtbl1q_u8 for 32-bit compat (whisper/0)
Georgi Gerganov
2026-07-10 11:06:42 +03:00
c4ae9a88f8
server: improve tools, remove apply_diff (#25498 )
Xuan-Son Nguyen and GitHub
2026-07-10 11:52:59 +02:00
1b9691bcd5
cli: fix crash on wrong server base url (#25497 )
marcoStocchi and GitHub
2026-07-10 11:52:20 +02:00
c7af942e8f
ui: prevent tooltip from flickering open and closed on hover (#25503 )
Pascal and GitHub
2026-07-10 11:49:52 +02:00
8f114a9b57
sync : ggml (#25517 )
Georgi Gerganov and GitHub
2026-07-10 10:28:39 +03:00
d46786f296
ui: export full message tree instead of active path only (#25501 )
Pascal and GitHub
2026-07-10 09:10:45 +02:00
2ed3c1abbb
llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4 (#25370 )
2026-07-10 09:06:58 +02:00
082b326fc7
ggml-et: Initial ET backend (#24179 )
2026-07-10 12:38:34 +08:00
961e4b26a7
llama-batch: add unit test (#25471 )
Aman Gupta and GitHub
2026-07-10 11:04:31 +08:00
049326a000
opencl: cluster-parallel decode FA for Adreno (#25473 )
Hongqiang Wang and GitHub
2026-07-09 11:13:48 -07:00
074944998d
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776 )
2026-07-09 20:07:12 +02:00
3de7dd4c8f
cli: add --output option (#25484 )
Xuan-Son Nguyen and GitHub
2026-07-09 19:37:39 +02:00
fb30ba9a6c
hexagon: tiling, tracing and optimizations for unary ops (#25474 )
2026-07-09 22:45:47 +05:30
82fce65d8b
server : move chat-template thinking probe inside the init try/catch (#24093 )
Jesse LaRose and GitHub
2026-07-09 12:37:39 -04:00
5c3a586860
ggml : fix conv 2d dw (#25490 )
Georgi Gerganov and GitHub
2026-07-09 17:56:32 +03:00
c15c5c77a4
meta: add hard emphasis on agents not writing descriptions/comments (#25480 )
Piotr Wilkin (ilintar) and GitHub
2026-07-09 15:18:07 +02:00
f84a519403
Refactor: Consistently use smart pointers in test-backend-ops (#25440 )
Oliver Simons and GitHub
2026-07-09 15:00:17 +02:00
683f0c72e5
Only index by compile times + always multiply/add (#25445 )
Oliver Simons and GitHub
2026-07-09 13:23:57 +02:00
259f2e2a53
llama-bench : init params.offline (#25476 )
Adrien Gallouët and GitHub
2026-07-09 11:56:56 +02:00
92b187c97e
metal : add CONV_2D_DW (depthwise convolution) support (#21565 )
2026-07-09 18:29:15 +09:00
ccb0c34223
ggml-hip: enable -funsafe-math-optimizations (#24668 )
2026-07-09 01:02:26 -07:00
2021515a1a
cuda: align snake fusion matcher with the other backends (#25460 )
Pascal and GitHub
2026-07-09 10:00:06 +02:00
64c8b7db72
server : respect min-step when splitting prompt batches (#25420 )
Aldehir Rojas and GitHub
2026-07-09 01:23:30 -05:00
f2d1c2f398
hexagon: add VISION RoPE support (#25216 )
Aparna M P and GitHub
2026-07-09 10:25:00 +05:30
32e41fa5b4
ggml-webgpu: tune subgroup split (d_split) in flash_attn_vec (#25418 )
Masashi Yoshimura and GitHub
2026-07-09 08:34:19 +09:00
92366df30d
opencl: Q6_K GEMM/GEMV fix for ne01 of weights that are not multiples of 128. (#25464 )
2026-07-08 15:52:21 -07:00
5e7f6271c0
docs(readme): add usage + benchmark instructions for the MoE-offload optimizations
Anirban Kar
2026-07-08 23:56:46 +05:30
a646006f09
vulkan: disable FA mask_opt on GCN to improve performance (#24362 )
Ruben Ortlam and GitHub
2026-07-08 19:01:25 +02:00
167d057604
opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433 )
2026-07-08 09:44:55 -07:00
1ee093937f
llama-batch: fix allowed decreasing pos in a seq (#25449 )
Aman Gupta and GitHub
2026-07-09 00:24:34 +08:00
0bbc87b163
vulkan: for small AMD GPUs, reduce submission threshold based on CU count (#25240 )
Ruben Ortlam and GitHub
2026-07-08 18:15:18 +02:00
81ff7abe50
hexagon: new vtcm layouts and improved pipelines for MUL_MAT, MUL_MAT_ID and FLASH_ATTN_EXT (#25425 )
Max Krasnyansky and GitHub
2026-07-08 07:38:27 -07:00
c264f65ff9
cli : move to HTTP-based implementation (#24948 )
2026-07-08 14:52:43 +02:00
07e012afdc
Make hip quality check run on all changes (#25403 )
Oliver Simons and GitHub
2026-07-08 14:38:51 +02:00
ed8c26150e
cuda : add support for f16->f16 GGML_OP_SET_ROWS (#25367 )
fairydreaming and GitHub
2026-07-08 13:24:20 +02:00
90e0f5cfcb
llama: refactor fused ops (#24646 )
Aman Gupta and GitHub
2026-07-08 18:18:09 +08:00
bbebeec4a8
server-stream: follow-up on SSE Replay Buffer (#23226 ) (#25047 )
2026-07-08 11:02:50 +02:00
230ea9d214
llama-batch: add n_keep_tail in split_equal for recurrent models (#25278 )
Aman Gupta and GitHub
2026-07-08 15:55:19 +08:00
f296fdfbed
common: auto-create prompts-log-dir at argument parsing, so all tools using the flag benefit (#25322 )
rankaiyx and GitHub
2026-07-08 15:45:28 +08:00
f1161b15f2
ui: Context usage gauge and panel (#25340 )
2026-07-08 09:22:35 +02:00
da46e59cbf
llama-eval : fix crash when answer is None in HTML dump (#25435 )
Georgi Gerganov and GitHub
2026-07-08 10:00:03 +03:00
0512ef1e5a
metal : add set_rows with src0 f16 (#25434 )
2026-07-08 08:49:07 +02:00
4a7ee3126d
fix: OOB reads in UGM tokenizer (precompiled_charsmap handling) (#18750 )
2026-07-08 13:02:09 +08:00
57b50e1f6b
ggml : fix A indexing in simd_gemm scalar tail-column path (#25390 )
tyronecai and GitHub
2026-07-08 13:00:05 +08:00
68a521b591
ggml : add support for CPU f16->f16 GGML_OP_SET_ROWS (#25344 )
2026-07-08 05:46:28 +02:00
931ca30bef
opencl: fix potential crash in aos reconstruct (#25383 )
lhez and GitHub
2026-07-07 20:34:29 -07:00
bec4772f6a
Add Q2_0 quantization: type definition and CPU backend (#24448 )
Pasha Khosravi and GitHub
2026-07-07 12:05:47 -07:00
c198af4dc2
spec : fix naming, spacing (#25410 )
Georgi Gerganov and GitHub
2026-07-07 18:52:30 +03:00
3899b39ce2
CUDA: Fuse MMVQ post-scale for NVFP4 (#24481 )
2026-07-07 17:12:19 +02:00
f5525f7e7a
server : fix draft model fit vs load inconsistency (#25056 )
Alex and GitHub
2026-07-07 10:20:42 -04:00
5eca4e3cab
server : add timings and progress to /responses API stream (#25348 )
Thomas LECONTE and GitHub
2026-07-07 16:13:03 +02:00
6c487e2f79
server: enforce prompt cache RAM limit (#25070 )
Thiago Padilha and GitHub
2026-07-07 10:24:35 -03:00
c1a411fb1b
common : add missing <fstream> include in common.h (#25220 )
zhangrunda and GitHub
2026-07-07 21:23:53 +08:00
33ca0dcb9d
ggml-hip : add -fno-finite-math-only alongside -ffast-math (#25373 )
asf0 and GitHub
2026-07-07 05:27:50 -06:00
024c46ae4e
llama: fix quantized kv-cache for dsv4 (#25202 )
Aman Gupta and GitHub
2026-07-07 17:46:57 +08:00
108f186d17
[SYCL] fix unsupported UT cases of CONT & CPY (#25231 )
Neo Zhang and GitHub
2026-07-07 17:20:52 +08:00
47e1de77aa
[SYCL] support op col2im_1d (#25264 )
Neo Zhang and GitHub
2026-07-07 16:07:46 +08:00
55edb2de44
[SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236 )
Neo Zhang and GitHub
2026-07-07 15:48:50 +08:00
d209086157
sycl : set K_QUANTS_PER_ITERATION to 1 on DMMV path (#25063 )
Todd Malsbary and GitHub
2026-07-07 00:43:41 -07:00
95e5254c0a
[SYCL] fix unsupport ACC UT cases for noncontiguous (#25124 )
Neo Zhang and GitHub
2026-07-07 15:40:38 +08:00
9e5ef0dbb1
sycl : enhance argsort to support all UT cases (#25125 )
Neo Zhang and GitHub
2026-07-07 15:39:29 +08:00
3d4cbdf18a
sycl : use sycl func to fix AOT double type issue (#25081 )
Neo Zhang and GitHub
2026-07-07 15:38:33 +08:00
26145b3db7
sycl : rename the env vars from "disable" to "enable" (#25042 )
Neo Zhang and GitHub
2026-07-07 15:33:51 +08:00
1a7c25bfdb
ggml : make ggml_time_init idempotent (#24422 )
An Long and GitHub
2026-07-07 16:29:17 +09:00
defa95c306
speculative : fix out-of-bounds read in ngram-map on prompt shrink (#23936 )
o7si and GitHub
2026-07-07 15:25:04 +08:00
a8cfdbb9e4
vulkan : check src0 type in GGML_OP_SET_ROWS to avoid failures due to unimplemented f16 support (#25351 )
2026-07-07 06:56:02 +02:00
6f8895feec
opencl: general flash attention decode performance optimizations (#25366 )
2026-07-06 19:57:52 -07:00
ee445f93d8
common: Set optimal default thread count for ppc ( linux as well as AIX) (#25237 )
shalinib-ibm and GitHub
2026-07-07 03:05:20 +05:30
f36e5c348b
metal: add col2im_1d op (f32/f16/bf16) (#25176 )
2026-07-06 20:47:36 +02:00
74976e1aef
CUDA: remove -sm row, refactor cuBLAS (#24216 )
Johannes Gäßler and GitHub
2026-07-06 20:04:53 +02:00
9abce7473a
server: fix deadlock in load_models() when erasing a finished download (#25358 )
Pascal and GitHub
2026-07-06 19:26:06 +02:00
cb295bf596
CUDA: extend K-type validation to V-types for flash attention (#24403 )
Alexey Kopytko and GitHub
2026-07-06 23:26:50 +09:00
Lumpiasty