This website requires JavaScript.
cb48bf2240
docs: FORK.md F11 - 35B results, the two leg-c anomalies explained
fable5/vram-ondemand-rebase-20261024
fable5/disk-prompt-cache
Lumpiasty and Claude Opus 5.5
2026-10-25 20:57:42 +01:00
18691e0090
docs: FORK.md F11 disk prompt cache, the CPU_REPACK release fix; runbook notes
Lumpiasty and Claude Opus 5.5
2026-10-25 19:00:21 +01:00
2c6c517ba5
scripts: fork-validate.sh --diskcache, the disk prompt cache
Lumpiasty and Claude Opus 5.5
2026-10-25 14:28:21 +01:00
2874dcb107
server: disk prompt cache, save slots on shutdown and restore after a restart
Lumpiasty and Claude Opus 5.5
2026-10-25 14:28:11 +01:00
dd851fd0c9
llama: read a released KV/recurrent state from its host shadow
Lumpiasty and Claude Opus 5.5
2026-10-25 14:27:50 +01:00
f24c3582c2
llama: do not release CPU extra buffers (CPU_REPACK) on device release
Lumpiasty and Claude Opus 5.5
2026-10-25 14:27:41 +01:00
71d4cbe195
docs: FORK.md and runbook after the 2fc9f9c35 deploy
Lumpiasty and Claude Opus 5.5
2026-10-25 14:12:52 +01:00
2fc9f9c351
docs: FORK.md F10 page-cache drop, -lm dio dead end; runbook note
Lumpiasty and Claude Opus 5.5
2026-10-25 13:14:08 +01:00
cce0ac1f09
llama: drop the model file's page cache after a copying load
Lumpiasty and Claude Opus 5.5
2026-10-25 13:13:00 +01:00
d7b3510499
docs: runbook - new fork-validate checks, gdb gone, scratch lost on restart
Lumpiasty and Claude Opus 5.5
2026-10-25 02:52:32 +02:00
47b892271c
docs(readme): short fork summary, FORK.md is the source of truth
Lumpiasty and Claude Opus 5.5
2026-10-25 02:51:27 +02:00
36514dba27
docs: FORK.md - review fixes, new checks, llama-swap restart behaviour
Lumpiasty and Claude Opus 5.5
2026-10-25 02:50:36 +02:00
e5fc474b87
scripts: fork-validate.sh - --moe-exact, greedy identity for the readback skip
Lumpiasty and Claude Opus 5.5
2026-10-25 02:47:16 +02:00
ceab1a91f8
scripts: fork-validate.sh - check that a cold server gives its VRAM back
Lumpiasty and Claude Opus 5.5
2026-10-25 02:43:51 +02:00
80e662a2a1
scripts: fork-validate.sh - A/B ordinary handoff latency against --ref
Lumpiasty and Claude Opus 5.5
2026-10-25 00:39:13 +02:00
2512290f94
scripts: fork-validate.sh - gate --perf on medians, not means
Lumpiasty and Claude Opus 5.5
2026-10-25 00:38:17 +02:00
62d442e7bf
scripts: rx580-bench - decode at tg256 depth 4096, not tg32
Lumpiasty and Claude Opus 5.5
2026-10-25 00:33:22 +02:00
f664b0f20e
llama: env knob for the routing-ids readback skip
Lumpiasty and Claude Opus 5.5
2026-10-25 00:31:25 +02:00
f5c0d8f28a
server: fix stale VRAM-arbiter comments and the sleep log line
Lumpiasty and Claude Opus 5.5
2026-10-25 00:30:20 +02:00
bfbaf15b2a
docs: FORK.md and rebase runbook for the RX 580 fork
Lumpiasty and Claude Opus 5.5
2026-10-25 00:18:02 +02:00
149a8cb0ae
scripts: add fork-validate.sh, post-rebase validation of the fork
Lumpiasty and Claude Opus 5.5
2026-10-24 20:08:11 +02:00
e085c347d2
ggml-vulkan: env knob for the concat transpose fast path
Lumpiasty and Claude Opus 5.5
2026-10-24 20:08:11 +02:00
6c480f7e70
server: release the GPU token promptly on shutdown
Lumpiasty and Claude Opus 5.5
2026-10-22 15:04:13 +02:00
11044b35e3
server: run the shutdown handler off the signal-interrupted thread
Lumpiasty and Claude Opus 5.5
2026-10-22 14:26:30 +02:00
54a7f534b6
scripts: rx580-bench - use -lm none, document the MoE cache option
Lumpiasty and Claude Opus 5.5
2026-10-09 00:35:00 +02:00
7ac2014a80
llama: release/restore the MoE cache with the device weights
Lumpiasty and Claude Opus 5.5
2026-10-09 00:34:09 +02:00
53f9077719
llama: skip the compute-buffer size check when the context is cold
Lumpiasty and Claude Opus 5
2026-09-11 03:26:15 +02:00
878d1338f4
server: replace the VRAM doorbell with a want-lock
Lumpiasty and Claude Opus 5
2026-09-10 17:45:18 +02:00
4c4486f8e9
scripts: add the RX 580 benchmark harness
Lumpiasty and Claude Opus 5
2026-09-10 17:06:01 +02:00
5e429d4a9c
ggml-vulkan: skip the concat transpose fast path below one tile
Lumpiasty and Claude Opus 5
2026-09-10 17:06:01 +02:00
b7c131219c
docs(readme): Polaris MoE profile - GCN mask_opt, the ids-readback win, tuning and dead ends
Lumpiasty
2026-09-09 00:41:59 +02:00
5e99ac1982
server: on-demand mmproj - free encoder VRAM on the text path
Lumpiasty
2026-07-26 14:56:56 +02:00
0a0f3ab6d4
mtmd: release/restore the encoder weights from VRAM on demand
Lumpiasty
2026-07-26 14:56:43 +02:00
ad27d49218
llama: free backend compute scratch on cold; guard memory_breakdown
Lumpiasty
2026-07-26 14:56:05 +02:00
9b555f1aad
ggml: add ggml_backend_free_scratch to drop Vulkan compute prealloc
Lumpiasty
2026-07-26 14:55:50 +02:00
9857a45f0c
llama: evict recurrent/SSM state on device release
Lumpiasty
2026-07-26 14:55:39 +02:00
bf2c1a6992
server: also free the compute-graph scheduler on KV eviction
Lumpiasty
2026-07-26 00:32:33 +02:00
2feb6aeb27
server: restore weights/KV on wake, not at decode, so KV eviction is safe
Lumpiasty
2026-07-26 00:07:01 +02:00
8d80f0673f
llama: guard buffer iteration against released device buffers
Lumpiasty
2026-07-25 14:36:17 +02:00
b9df854de8
server: coordinate model load with the VRAM arbiter to avoid load-time OOM
Lumpiasty
2026-07-25 14:36:16 +02:00
958257cd60
server: keep the sleep-wake path active when the VRAM arbiter is enabled
Lumpiasty
2026-07-25 02:32:16 +02:00
17165fe2a7
server: optionally evict the KV cache too (Phase 2 of VRAM sharing)
Lumpiasty
2026-07-25 02:02:18 +02:00
481be8b284
server: on-demand VRAM sharing to time-share one GPU between models
Lumpiasty
2026-07-24 22:32:29 +02:00
3df719e1e8
ggml-vulkan: tiled transpose fast-path for concat with transposed source
Lumpiasty
2026-07-15 23:44:42 +02:00
ff2a96d2d4
vulkan: record why flash-attn shmem staging stays off for GCN
Lumpiasty and Claude Opus 5
2026-07-27 23:56:14 +02:00
fb6a8a0761
ggml-vulkan: enable flash-attn mask_opt for GCN large head sizes
Lumpiasty
2026-07-22 20:40:34 +02:00
c72cb424e1
llama: skip the routing-ids readback in sched_copy_experts when the batch uses every expert
Lumpiasty and Claude Opus 5.5
2026-10-09 00:30:07 +02:00
3f03b17724
metal : remove mul_mm_id amax rescaling (#30924 )
Georgi Gerganov
2026-10-24 18:40:19 +03:00
9a0fde9967
rpc : add bounds check in rdma_recv (#30399 )
Katara
2026-10-24 16:34:46 +01:00
e168e041ec
ui: Models Manager as Route + UI improvements (#30988 )
Aleksander Grygier
2026-10-24 17:34:11 +02:00
ea1d7d66f3
fix(ui): align hover card and context popup styling with dropdown menu (#30978 )
Aleksander Grygier
2026-10-24 17:33:27 +02:00
71255fefc0
ci : pass per-runner results dir to self-hosted jobs on T4 runners (#31100 )
Georgi Gerganov
2026-10-24 17:17:04 +03:00
73adf89e38
minimax-m3 : forward quant scales to the graph (fixes #31063 ) (#31097 )
Satyam Thakur and origin-labs-ai
2026-10-24 19:39:02 +05:30
df9beac715
tests : fix test-llama-context for large n_batch (#31083 )
Alexey Kopytko
2026-10-24 20:14:58 +09:00
4d32b29aaa
ci : exclude test-speculative-rejection-sampling from macos-latest-arm64 (#31088 )
Georgi Gerganov
2026-10-24 14:14:31 +03:00
3032209758
llama : deny -sm tensor for arches with moe amax rescaling (#31081 )
Georgi Gerganov
2026-10-24 13:48:32 +03:00
701fd25b3a
ggml : allow MUL_MAT_ID -1 index to skip the computation (#26631 )
Xuan-Son Nguyen and yomaytk
2026-10-24 12:15:36 +02:00
3642238cc1
tests : add speculative rejection draft sampling test (#30213 )
Daniel Bevenius
2026-10-24 11:47:49 +02:00
6b9ba4f2e8
ci : fix data race between runners sharing the model cache (#31046 )
Georgi Gerganov
2026-10-24 09:21:24 +03:00
8e5a58df87
vulkan: ggml src1 precision support (#30656 )
Ruben Ortlam
2026-10-24 06:05:05 +02:00
1ddffe9f70
HIP: add byte permute fallback when __builtin_amdgcn_perm is unavailable (#30915 )
Arseniy Obolenskiy
2026-10-24 06:04:05 +02:00
7894e8b924
mtmd: move build_inp_temporal_merge to base class (#31065 )
Xuan-Son Nguyen
2026-10-24 03:38:25 +02:00
edc353842c
mtmd : lfm2 resize matches torchvision (#30943 )
Yuri Khrustalev and Xuan Son Nguyen
2026-10-23 20:21:06 -04:00
9ae73e68f5
mtmd : add Soprano TTS support (#28607 )
Hans
2026-10-24 07:09:50 +08:00
0ae0e19766
hexagon: add SGN op (#30346 )
kurquhar and Max Krasnyansky
2026-10-23 14:58:44 -07:00
21ec307146
common : allow concurrent downloads across processes (#29312 )
Adrien Gallouët
2026-10-23 23:52:56 +02:00
9145ddb955
hip: fix bf16 WMMA operand vector type for gfx11/gfx12 (#30885 )
WalterCool
2026-10-23 18:31:28 -03:00
1a24df1639
ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size (#30837 )
thelittlefireman
2026-10-23 21:37:20 +02:00
9ff0d7e90f
CUDA: fix build after chunked CUDA graphs (#30804 ) (#31053 )
Sreeram and Johannes Gäßler
2026-10-23 11:47:20 -07:00
06230cb877
allows the usage of 4-10 gpus for stepfun and laguna in tensor parallel (#24554 )
Christian Loehnert
2026-10-23 11:56:38 -06:00
5c31142682
qwen4exp : build the QSA selection without the kq_mask (#30965 )
Alexey Kopytko
2026-10-24 02:49:08 +09:00
da2ee1e7a6
vendor : update LibreSSL to 4.4.0 [no ci] (#31049 )
Adrien Gallouët
2026-10-23 19:12:35 +02:00
280141e6d4
CUDA: chunk graphs to amortize graph launch (#30804 )
Aman Gupta
2026-10-24 00:26:16 +08:00
36e36e63a5
hexagon: add CEIL op (#31006 )
Max Krasnyansky
2026-10-23 07:46:53 -07:00
e1e3b2c409
ui : bump packages to clear npm audit (#31038 )
Aleksander Grygier
2026-10-23 16:33:13 +02:00
4231b23dc2
ggml : fix time overflow in ggml_time_us (#30969 )
Kartik Gulia
2026-10-23 18:26:59 +05:30
f00c9083b2
ci : add behavior check workflow (#31036 )
Sigbjørn Skjæret
2026-10-23 14:53:44 +02:00
f5027fb213
CANN: missing const qualifiers added for consistency (#30933 )
Raul Torres
2026-10-23 08:34:20 +01:00
f64a49819c
sycl : pass device into mul_mat_uses_onednn (#31014 )
Titaniumtown
2026-10-22 23:37:53 -07:00
1ab09d632e
Extend probabilistic sampling for DFlash2 drafts (#30663 )
Pranesh Gonegandla and Pranesh Gonegandla
2026-10-23 05:45:36 +00:00
f9599d0cc8
vocab : fix Nemotron-3.5-Embed tokenization (#30974 )
2026-10-23 14:31:30 +09:00
643b0cdacb
hexagon: Op Fusion refactoring, optimizations and new fusions like BINARY_NX (back-to-back ADDs, etc) (#30843 )
Max Krasnyansky
2026-10-22 21:05:44 -07:00
0a9ef00678
sycl: Replace host waits in fattn-mkl with explicit dependencies (#30684 )
Todd Malsbary
2026-10-22 19:43:12 -07:00
7377bada8c
sycl: Add graph record and replay (#28725 )
Todd Malsbary
2026-10-22 19:39:52 -07:00
184f653602
server: Substitute rerank placeholder from the original template only. (#30864 )
pzhu
2026-10-23 06:38:03 +08:00
d814cbb88c
server: realtime API: send complete after committed audio drains (#30994 )
Xuan-Son Nguyen and Pascal
2026-10-23 00:37:47 +02:00
970b51ce7c
opencl: add depthwise conv2d (#30900 )
kurquhar and Li He
2026-10-22 10:54:24 -07:00
a08ab75ae7
mtmd : add MiniCPM-OCR 1.0 support (#30979 )
tc-mb
2026-10-23 01:22:15 +08:00
19ab5a5e65
clef : include <algorithm> for std::find and std::copy (#30981 )
Martin Kleinberger
2026-10-22 17:14:47 +01:00
36cbf415f1
Fix !30793
Gaurav Garg
2026-10-22 19:30:57 +05:30
5c19a5a756
glm5 : optimize inputs (#30932 )
Georgi Gerganov
2026-10-22 16:59:29 +03:00
d9ee007a06
model : fix qwen4exp layer inputs for DFlash drafters (#30927 )
Georgi Gerganov
2026-10-22 16:49:38 +03:00
d81d0d8fb9
Multi-column on Intel XMX matrix engines refactored (#29864 )
budiony
2026-10-22 16:39:01 +03:00
5751c977b7
server: release the GPU token promptly on shutdown
fable5/vram-ondemand-rebase-20261009
Lumpiasty and Claude Opus 5.5
2026-10-22 15:04:13 +02:00
d77256999f
vulkan: replace std::aligned_alloc with posix_memalign for termux compatibility (#30828 )
Bill Sideris
2026-10-22 15:40:13 +03:00
64543a435b
server: run the shutdown handler off the signal-interrupted thread
Lumpiasty and Claude Opus 5.5
2026-10-22 14:26:30 +02:00
0693dbfb91
nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type (#23343 )
martin-cimmino and Martin Cimmino
2026-10-22 13:34:47 +02:00
a5d22276be
server : support typed content input for RERANK endpoint (#29625 )
Tim Wang and timothywang21
2026-10-22 07:27:24 -04:00
5f5acfe250
ci : disable slow WebGPU tests (#30963 )
Georgi Gerganov
2026-10-22 13:56:20 +03:00
1f4c2eeaa5
ci : be able to reconfigure HTTP port for server pytest workers (#30972 )
Georgi Gerganov
2026-10-22 13:56:03 +03:00