This website requires JavaScript.
80b91cbf76
server: on-demand mmproj - free encoder VRAM on the text path
Lumpiasty
2026-07-26 14:56:56 +02:00
da76fcb326
mtmd: release/restore the encoder weights from VRAM on demand
Lumpiasty
2026-07-26 14:56:43 +02:00
6d476d6ccc
llama: free backend compute scratch on cold; guard memory_breakdown
Lumpiasty
2026-07-26 14:56:05 +02:00
a8dc3ebddb
ggml: add ggml_backend_free_scratch to drop Vulkan compute prealloc
Lumpiasty
2026-07-26 14:55:50 +02:00
5ddcf40fc1
llama: evict recurrent/SSM state on device release
Lumpiasty
2026-07-26 14:55:39 +02:00
fd7cbd5f4f
server: also free the compute-graph scheduler on KV eviction
Lumpiasty
2026-07-26 00:32:33 +02:00
4d576de9dd
server: restore weights/KV on wake, not at decode, so KV eviction is safe
Lumpiasty
2026-07-26 00:07:01 +02:00
f707a2430d
llama: guard buffer iteration against released device buffers
Lumpiasty
2026-07-25 14:36:17 +02:00
2af8f5ae6e
server: coordinate model load with the VRAM arbiter to avoid load-time OOM
Lumpiasty
2026-07-25 14:36:16 +02:00
189856ff9b
server: keep the sleep-wake path active when the VRAM arbiter is enabled
Lumpiasty
2026-07-25 02:32:16 +02:00
62873a34bb
server: optionally evict the KV cache too (Phase 2 of VRAM sharing)
Lumpiasty
2026-07-25 02:02:18 +02:00
0b0cc69635
server: on-demand VRAM sharing to time-share one GPU between models
Lumpiasty
2026-07-24 22:32:29 +02:00
a9c986086b
common : fractional -ncmoe for tensor-granularity expert placement
Lumpiasty
2026-07-23 18:36:51 +02:00
34ee143caa
docs(readme): document GCN mask_opt and fix Vulkan serving guidance
Lumpiasty
2026-07-22 21:26:18 +02:00
c1ba5f0f6a
ggml-vulkan: enable flash-attn mask_opt for GCN large head sizes
Lumpiasty
2026-07-22 20:40:34 +02:00
d7e32623cd
docs(readme): document Vulkan behavior of the MoE-offload flags
Lumpiasty
2026-07-21 17:17:23 +02:00
0abb8e7ece
ggml-vulkan: don't assert compute_ctx empty before perf timestamp
Lumpiasty
2026-07-21 17:06:16 +02:00
cfdc33ec83
ggml-vulkan: pre-stage host weights when import is unavailable
Lumpiasty
2026-07-17 03:04:16 +02:00
4fb182ac30
ggml-vulkan: tiled transpose fast-path for concat with transposed source
Lumpiasty
2026-07-15 23:44:42 +02:00
bc4fa113de
ggml-vulkan: pin mmap CPU weights for faster H2D uploads
Lumpiasty
2026-07-15 20:43:18 +02:00
daf1998ac0
docs(readme): add usage + benchmark instructions for the MoE-offload optimizations
Anirban Kar and Lumpiasty
2026-07-08 23:56:46 +05:30
31ba631b76
ggml : size prefetch slots per layer and fix fallback use-after-free
thecodacus and Lumpiasty
2026-07-02 12:42:55 +00:00
9a34ee7dbe
ggml : overlap offloaded expert weight uploads with compute
thecodacus and Lumpiasty
2026-07-02 12:16:39 +00:00
1f1f1a8e3e
llama : pin mmap-backed CPU weights for faster H2D uploads
thecodacus and Lumpiasty
2026-07-02 09:44:53 +00:00
d2a818231e
common: add subproc.h wrapper, disabled on android/ios (#26102 )
Xuan-Son Nguyen and GitHub
2026-07-26 20:54:25 +02:00
af285020e9
mtmd: add GLM-5.2-Vision (#26126 )
2026-07-26 14:43:51 -04:00
b1d4c65524
model: Add MiniMax-M3 (MSA: MiniMax Sparse Attention) support (#24908 )
2026-07-26 19:43:45 +02:00
42fc243060
opencl: fix fused RMS norm mul view offset (#26085 )
yzyyzyhhh and GitHub
2026-07-26 23:01:08 +08:00
45d48465f7
server: on-demand mmproj - free encoder VRAM on the text path
Lumpiasty
2026-07-26 14:56:56 +02:00
d32c33dab4
mtmd: release/restore the encoder weights from VRAM on demand
Lumpiasty
2026-07-26 14:56:43 +02:00
c4c97f0595
llama: free backend compute scratch on cold; guard memory_breakdown
Lumpiasty
2026-07-26 14:56:05 +02:00
994fd757fc
ggml: add ggml_backend_free_scratch to drop Vulkan compute prealloc
Lumpiasty
2026-07-26 14:55:50 +02:00
c2c8d377cb
llama: evict recurrent/SSM state on device release
Lumpiasty
2026-07-26 14:55:39 +02:00
ff067f76dd
ui: fix context gauge card regressions and land at the conversation end (#26099 )
Pascal and GitHub
2026-07-26 06:51:10 +02:00
7cdd557f76
ggml-webgpu: Fix WASM compilation with OpenMP (#25943 )
Reese Levine and GitHub
2026-07-25 17:37:18 -07:00
8bb909374d
common : use-after-free when loading LoRA adapter fails (#25611 )
Nicky Mouha and GitHub
2026-07-25 19:10:32 -04:00
20455a4ad3
server: support MCP stdio (#26062 )
2026-07-26 01:08:49 +02:00
355303edab
hexagon: partial im2col support (#26007 )
Todor Boinovski and GitHub
2026-07-25 15:47:29 -07:00
961cd62ebc
server: also free the compute-graph scheduler on KV eviction
Lumpiasty
2026-07-26 00:32:33 +02:00
9573505011
server: restore weights/KV on wake, not at decode, so KV eviction is safe
Lumpiasty
2026-07-26 00:07:01 +02:00
c812c543f8
common : skip empty implicit default preset (#25643 )
2026-07-25 21:15:27 +02:00
abc348790e
server: add format arg to datetime tool (#26117 )
Xuan-Son Nguyen and GitHub
2026-07-25 21:15:15 +02:00
2cfc7670ed
server : add missing task parameters(adaptive_target, adaptive_decay) in generation_settings (#25830 )
Tekin Ertekin and GitHub
2026-07-25 21:53:08 +03:00
720d7fa409
vendor : update cpp-httplib to 0.51.0 (#26067 )
Adrien Gallouët and GitHub
2026-07-25 18:16:29 +02:00
deee33503b
llama: guard buffer iteration against released device buffers
Lumpiasty
2026-07-25 14:36:17 +02:00
f2d79d89f5
server: coordinate model load with the VRAM arbiter to avoid load-time OOM
Lumpiasty
2026-07-25 14:36:16 +02:00
fb92d8f187
Update ggml/src/gguf.cpp : Defined virtual keyword for destructor of gguf_writer_base (#25867 )
Yongmin Yoo 유용민 and GitHub
2026-07-25 21:32:37 +09:00
910196f6b3
common : add support for multiple end sequences in the reasoning budget sampler (#25544 )
Aldehir Rojas and GitHub
2026-07-25 04:58:09 -05:00
d67c0b4107
tests: synchronize save-load-state generation (#26056 )
helanfxz and GitHub
2026-07-25 16:23:31 +08:00
bf68c5446e
server: keep the sleep-wake path active when the VRAM arbiter is enabled
Lumpiasty
2026-07-25 02:32:16 +02:00
7398c40eda
server: optionally evict the KV cache too (Phase 2 of VRAM sharing)
Lumpiasty
2026-07-25 02:02:18 +02:00
be7f3b3172
server: on-demand VRAM sharing to time-share one GPU between models
Lumpiasty
2026-07-24 22:32:29 +02:00
555881ebc8
ui: reduce per-token render cost when streaming (#26053 )
2026-07-24 16:09:46 -04:00
96013c5112
ui: remove render effects (#26083 )
Pascal and GitHub
2026-07-24 21:43:23 +02:00
88bfee1429
model: add GLM 5.2 Indexer support (#25407 )
2026-07-24 20:55:56 +02:00
95a923a64c
ui: fix MCP server display name conflicts in tools lists (#26011 )
Pascal and GitHub
2026-07-24 19:28:14 +02:00
27209a598d
server: support "reasoning_effort": "none" in OAI API (#26045 )
Nigel Bosch and GitHub
2026-07-24 12:19:10 -05:00
298219f985
llama: various bug fixes (#26051 )
Xuan-Son Nguyen and GitHub
2026-07-24 18:56:42 +02:00
fa72aeccb2
HIP: remove rocWMMA FlashAttention (#26046 )
Johannes Gäßler and GitHub
2026-07-24 17:53:54 +02:00
ed7adbfefd
opencl: cache compiled cl_program binaries on disk (#26050 )
Hongqiang Wang and GitHub
2026-07-24 08:14:33 -07:00
56a83860dd
opencl: do not treat NULL-mask flash attention as causal (#25771 )
kumaal and GitHub
2026-07-24 20:12:01 +05:00
77095ee0cb
skill: create add-new-model and code-review (#26042 )
2026-07-24 17:04:17 +02:00
54ce507b6f
UI: Fix settings precedence, Factory < Admin (--ui-config-file) < Users (Settings panel) (#26002 )
Pascal and GitHub
2026-07-24 15:09:55 +02:00
8f5ab832ca
cohere2 moe template parser: enforce JSON schema for text responses if a response schema is provided (#26018 )
Matt Thompson and GitHub
2026-07-24 03:54:47 -07:00
0cea36222f
vendor: update subprocess.h (#26061 )
Xuan-Son Nguyen and GitHub
2026-07-24 08:02:23 +02:00
0a50d9909a
hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA) (#26049 )
Max Krasnyansky and GitHub
2026-07-23 19:13:03 -07:00
2b94398ed7
common : fractional -ncmoe for tensor-granularity expert placement
Lumpiasty
2026-07-23 18:36:51 +02:00
c0bc8591e8
hexagon: fix Windows crash when op_poll is enabled (#26029 )
adgup-qti and GitHub
2026-07-23 21:38:10 +05:30
1425386fd9
CUDA: fix external compilation of q1_0 MMQ (#25778 )
Johannes Gäßler and GitHub
2026-07-23 14:45:51 +02:00
e6dd0e29a6
args: refactor mlock/mmap/directio into load-mode (#20834 )
Aaron Teo and GitHub
2026-07-23 20:32:56 +08:00
da296d6e72
contrib: fix leftovers from the AI usage policy update (#26030 )
Pascal and GitHub
2026-07-23 12:32:23 +02:00
c588c4f476
metal : add f16 type support to leaky relu (#25981 )
Ilia Ilmer and GitHub
2026-07-22 23:45:46 -04:00
d941f6e1c9
conversion: fix non-MoE NomicBert GGUF conversion error (#25996 )
Shahir BIn Zulfiker and GitHub
2026-07-23 09:01:35 +06:00
4310aa4f87
contrib: allow all AI-generated code in general (#26012 )
Xuan-Son Nguyen and GitHub
2026-07-23 00:29:03 +02:00
cf512566dc
ui: Add a "Default" option for the reasoning selector (#25846 )
Pascal and GitHub
2026-07-22 23:09:49 +02:00
e5eb5edb58
docs(readme): document GCN mask_opt and fix Vulkan serving guidance
Lumpiasty
2026-07-22 21:26:18 +02:00
4442815c02
ggml-vulkan: enable flash-attn mask_opt for GCN large head sizes
Lumpiasty
2026-07-22 20:40:34 +02:00
891760faba
docs(readme): document Vulkan behavior of the MoE-offload flags
Lumpiasty
2026-07-21 17:17:23 +02:00
1bc7c581d8
ggml-vulkan: don't assert compute_ctx empty before perf timestamp
Lumpiasty
2026-07-21 17:06:16 +02:00
3972da9cc9
ggml-vulkan: pre-stage host weights when import is unavailable
Lumpiasty
2026-07-17 03:04:16 +02:00
42e52045dc
ggml-vulkan: tiled transpose fast-path for concat with transposed source
Lumpiasty
2026-07-15 23:44:42 +02:00
1b8abd1c23
ggml-vulkan: pin mmap CPU weights for faster H2D uploads
Lumpiasty
2026-07-15 20:43:18 +02:00
c3c913ba79
docs(readme): add usage + benchmark instructions for the MoE-offload optimizations
Anirban Kar and Lumpiasty
2026-07-08 23:56:46 +05:30
c440f0b925
ggml : size prefetch slots per layer and fix fallback use-after-free
thecodacus and Lumpiasty
2026-07-02 12:42:55 +00:00
02cff16681
ggml : overlap offloaded expert weight uploads with compute
thecodacus and Lumpiasty
2026-07-02 12:16:39 +00:00
5a0c899424
llama : pin mmap-backed CPU weights for faster H2D uploads
thecodacus and Lumpiasty
2026-07-02 09:44:53 +00:00
1a064ab092
CUDA: Improve NVFP4 W4A4 activation quantization (#25730 )
Oliver Simons and GitHub
2026-07-22 19:28:02 +02:00
0278d8362d
hexagon: activation ops update (#25974 )
2026-07-22 09:25:04 -07:00
e0833bf686
mtmd: use RAII for setting and resetting non-causal attention (#25723 )
Niklas Wenzel and GitHub
2026-07-22 18:10:03 +02:00
61328e6a91
feat(ui): add symbolic math support to JS sandbox via nerdamer (#25948 )
2026-07-22 23:52:55 +08:00
e8e6c7af24
minor: fix reasoning preserve var for DS4 [no ci] (#25999 )
Piotr Wilkin (ilintar) and GitHub
2026-07-22 14:32:54 +02:00
6d5a910c50
common: infer the speculative type from the draft repo sidecars (#25989 )
Pascal and GitHub
2026-07-22 13:06:35 +02:00
f534da26e4
Fix DeepSeek4 crafted template (#25414 )
Piotr Wilkin (ilintar) and GitHub
2026-07-22 12:54:40 +02:00
3ce7da2c85
ggml: enable PowerPC backend variants on AIX (#25983 )
shalinib-ibm and GitHub
2026-07-22 14:56:40 +05:30
b4d6c7d8ff
ci : fix SYCL package shared library lookup (#25987 )
KyleHagy and GitHub
2026-07-22 02:20:40 -07:00
7347430f44
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847 )
m1el and GitHub
2026-07-22 03:24:44 -05:00
c5a4a0bb83
cuda: GET_ROWS quants (#25962 )
Pascal and GitHub
2026-07-22 08:42:47 +02:00
67b9b0e7f6
llama-arch: fix DeepSeek4 APE tensor op (#25945 )
helanfxz and GitHub
2026-07-22 10:55:44 +08:00
1f66c3ce1c
Add support for Laguna XS.2 & M.1 (#25165 )
Joe Rowell and GitHub
2026-07-22 03:54:08 +02:00
66e4bf7e59
convert: fix handle HunyuanVL XD-RoPE config (#25514 )
wendadawen and GitHub
2026-07-22 06:42:35 +08:00
Lumpiasty
Anirban KarandLumpiasty