Commit Graph
525 Commits
Author SHA1 Message Date
LumpiastyandClaude Opus 5.5 2c6c517ba5 scripts: fork-validate.sh --diskcache, the disk prompt cache
Runs beside production. The model is the small hybrid (Qwen3.5-0.8B), CPU only
(-dev none -ngl 0, no VRAM), with chat requests and greedy decoding. A
reference server that never restarts runs turn 1 and turn 2. The candidate
saves turn 1 on SIGTERM and restarts. Turn 2 must then be restored from disk
with the same prompt_n as the reference (only the suffix processed) and the
same answer. Cases:
- restore plain, with the arbiter env warm, and forced cold by a fake waiter
  on want.lock (the save from a released state). Each case records the exit
  time and SIGTERM-to-token-free, also once without the cache;
- no restore for another conversation, or with another n_ctx, KV type or
  model path (other key);
- a truncated file is removed at startup and a byte-flipped one at load,
  never used;
- the size limit evicts the oldest file;
- a write error (RLIMIT_FSIZE 1 MiB, SIGXFSZ ignored: EFBIG) is logged, the
  exit stays 0 with no file left, and the next start serves;
- the same restore on the GPU (-ngl 99, arbiter env, warm and cold) when
  >= 1.5 GiB of VRAM is free, else SKIP.
start_server gains SRV_WRAP, a command prefix that execs the server.

First run (2026-10-25, beside production): 11 PASS, 0 FAIL; the GPU leg
SKIPped (1219 MiB free). The first run found the CPU_REPACK release crash
(exit 139 in the cold case).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 18:55:57 +01:00
LumpiastyandClaude Opus 5.5 71d4cbe195 docs: FORK.md and runbook after the 2fc9f9c35 deploy
Feedback from the 2026-10-25 deploy, which used only these docs:
- status: production is 2fc9f9c35; F10 deployed and verified on a 35B
  GPU load; --moe-exact and median --perf have run on the GPU; last
  full fork-validate run (37 PASS) and its timings; the first-candidate
  -arm VRAM spike seen twice is an open question.
- runbook: a no-rebase redeploy path; --arbiter beside production only
  if 1536 MiB VRAM is free, else in the swapdown window; full --moecache
  model path; page-cache eviction before and after phase 2; deploy
  steps reordered so the memory check follows stopping llama-swap;
  in-place rebuild and system_fingerprint check; mtime check through the
  versioned symlinks; rollback pointer numbering and one format; disk
  space; nothink alias and a PPM image helper for post-deploy requests;
  power1_cap via sysfs; restart and OOM checks; harmless UI warning.
- fork-validate.sh header: note the --arbiter VRAM requirement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 14:12:52 +01:00
LumpiastyandClaude Opus 5.5 e5fc474b87 scripts: fork-validate.sh - --moe-exact, greedy identity for the readback skip
F1 (the routing-ids readback skip in sched_copy_experts) had no automated
check. --moe-exact MODEL reads expert_count / expert_used_count from the GGUF
header, computes the skip threshold ceil(4 * n_expert / n_expert_used) tokens
per ubatch (128 on the 35B), and runs one greedy request with a prompt of
about 2x that in a single ubatch, every expert on the host, so the prefill
takes the skip and each decode step takes the readback. The candidate's 64
tokens must equal the reference's byte for byte; with the LLAMA_MOE_IDS_SKIP
knob in the candidate, also the candidate with the skip forced off (=0), which
proves the skip exact even when the reference carries it, and =log must show
skips, all at >= 4 * n_expert ids. Without the knob skip-engaged SKIPs with
the batch-size argument. It needs the GPU to itself (same guard as --perf,
now shared as gpu_exclusive); FV_TEST_SHARED_GPU=1 bypasses the guard for
script tests against stub binaries only.

Also unset inherited LLAMA_MOE* variables.

Not run on the GPU yet (needs a swapdown window). Dry-run on the pod
against stub servers and the real 35B GGUF header: threshold 128, all-PASS
when outputs match, FAIL on a differing reference, SKIP without the knob,
FAIL for a dense model and while llama-swap runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 02:47:16 +02:00
LumpiastyandClaude Opus 5.5 ceab1a91f8 scripts: fork-validate.sh - check that a cold server gives its VRAM back
Nothing checked that a release actually frees VRAM. --arbiter now takes a
baseline of mem_info_vram_used before any scratch server starts, then with A
warm and alone on the scratch arena makes it release through a fake waiter
on want.lock. v-vram-release requires the card-wide delta to drop by at
least 70% of what A added warm; v-vram-restore requires it back (and the
output equal to the reference) after the next request. Production's own
share is re-read at every point from the DRM fdinfo of every process the
script did not start (deduplicated by drm-client-id); if it moved by more
than 64 MiB the check SKIPs instead of blaming the candidate.

Run beside production on the pod (reb1024 vs itself): warm +546 MiB, cold
+4 MiB, restored +546 MiB, production steady at 5053 MiB; 16 PASS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 02:43:51 +02:00
LumpiastyandClaude Opus 5.5 80e662a2a1 scripts: fork-validate.sh - A/B ordinary handoff latency against --ref
--arbiter only reported the candidate's handoff latency, while the fork's
process rule (two production deploys lost) is to A/B the ordinary handoff,
not only the pathological cases. After the arbiter cases, a-handoff-latency
starts fresh A/B server pairs of each build on the scratch arena and runs
blocks ref, cand, cand, ref of 16 alternating handoffs each. It FAILs if the
candidate median exceeds the reference median by more than max(25%, 200 ms)
and reports both medians, maxima and per-block medians.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:39:13 +02:00
LumpiastyandClaude Opus 5.5 2512290f94 scripts: fork-validate.sh - gate --perf on medians, not means
--perf averaged the two per-arm llama-bench means of each build, so one
catastrophic sample (CPU contention from the ~60 other pods on the node) moved
the result, against the fork's own rule of medians, never means. llama-bench
now writes -o jsonl (every repetition sample) next to the markdown table, and
the gates use the median of all samples of a build pooled over its two arms
(4 per prefill cell, 6 for decode); the table shows min-max and n.

--perf-report TSV re-aggregates a saved <scratch>/perf.tsv without a GPU.

Checked offline on the pod: the 2026-09 A/B samples in results.txt give
pp2048 reference 250.24 (old mean) vs 247.76 (new median), candidate 280.15
vs 276.93; a synthetic 2026-10-24-like set with one 180 t/s candidate sample
read -9.3% (FAIL) the old way and -0.2% (PASS) with medians.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:38:17 +02:00
LumpiastyandClaude Opus 5.5 62d442e7bf scripts: rx580-bench - decode at tg256 depth 4096, not tg32
The llama-bench sweep measured decode as tg32 at the default depth and the
README called it the continuity metric, while tg32 at n_ctx 32 is launch /
barrier bound on this card and is not a valid decode metric (it once read
-36% between builds at parity at tg256). lbsweep.sh now runs prefill
(-p $LB_P -n 0, depth 0) and decode (-p 0 -n 256 -d 4096, LB_N / LB_D) as
two invocations and labels depth cells (tg256@d4096). summarize.py marks
old depth-less tg rows as invalid-metric history. The README examples and
the reference values are updated to the 2026-10-24 production numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 00:33:22 +02:00
LumpiastyandClaude Opus 5.5 149a8cb0ae scripts: add fork-validate.sh, post-rebase validation of the fork
Checks the op kernels, the concat fast path, the VRAM arbiter incl. every
shutdown path that has caused a production incident, and prefill/decode
against a reference build. Prints PASS/FAIL per check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:08:11 +02:00
LumpiastyandClaude Opus 5.5 54a7f534b6 scripts: rx580-bench - use -lm none, document the MoE cache option
Upstream #28334 removed --mmap/--no-mmap/--mlock and llama-bench's -mmp, so the
harness flags no longer parse: the server would refuse to start and llama-bench
would silently produce no rows. Switch both harnesses to -lm none, the same
load mode, and note --moe-cache-mib as a server-only option worth measuring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
2026-10-24 18:33:19 +02:00
LumpiastyandClaude Opus 5 4c4486f8e9 scripts: add the RX 580 benchmark harness
Encodes the production config, fixed corpus slices, repeat/median discipline
and the noise floor, so the measurement method does not have to be
rediscovered each time. Runs the corpus prefill test through llama-server and
the llama-bench sweep as a controlled cross-check, with interleaved A/B.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
2026-10-24 18:33:19 +02:00
Christian Kastner 707616feb5 ci: Fix --strict mode for API check (#30623)
abi-compliance-checker was running in strict mode unconditionally,
rather than honoring the CLI flag.
2026-10-20 13:38:45 +03:00
Max Krasnyansky 8466395353 hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (#30854)
* hex-build: bump toolchain, enable v85

* hex-build: bump toolchain to v0.8 in the CI and build scripts

* hex-build: add a little python script for swiv needed on some iot devices

* hex-build: fix flake8 warnings

* hex-arch: update version cap to v85

* hex-inspect: flexible TOOLS_ROOT

* hex-build: update to snake_case naming

* hex-build: update leftover old toolchain refs

* hex-target: update build and run scripts to support ubuntu with adb transport
2026-10-19 23:49:59 -07:00
7adf160c41 server: add --connect, allow p2p access to server outside LAN (#28277)
* init

* wip

* wire it up

* make it clear that it's experimental

* add props.connect_code

* nits

* fix windows build

* server: do not advertise a connect code without the tunnel

--connect-code on its own had no effect other than being served in
/props, so the Web UI showed a remote access code that no tunnel was
serving. Clear it when --connect is off, and say so once at startup.

The teardown comment claimed that terminate() closes the child stdout.
It does not: the log thread leaves fgets() only once every writer of
that pipe is gone, so a descendant of llama-connect would block the
join forever. Describe the real contract instead.

Also restore the --reasoning-preserve default in the generated docs.

* Update tools/connect/CMakeLists.txt

Co-authored-by: Adrien Gallouët <angt@huggingface.co>

* server: dial llama-connect on the bound port and a reachable host

Pass ctx_http.port so --port 0 and the GCP override work, pick the
dial host from --host (wildcards map to loopback, IPv6 is bracketed)
and refuse --connect when only unix sockets are bound.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
2026-10-20 00:23:14 +02:00
Alessandro de Oliveira Faria (A.K.A.CABELO) 7951f601e8 vendor : update cpp-httplib to 0.62.0 (#30785) 2026-10-19 20:23:23 +08:00
Georgi Gerganov 7db0d3d781 pi : turn .pi/gg into a system prompt addendum (#30762)
.pi/gg/SYSTEM.md repeated the full agent ruleset. Move it to .pi/gg/APPEND_SYSTEM.md and keep only the llama.cpp-specific rules, so it can be appended to a main system prompt with --append-system-prompt.

Drop the .pi/SYSTEM.md symlink setup from wc2wt.sh and pr2wt.sh. Worktrees no longer need it now that the folder holds an addendum instead of a full prompt.

Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (RTX 5090 32GB, DDR5 128GB)
2026-10-19 09:11:13 +03:00
Sandro Hanea 83f9389419 ci : added ABI reports as github artifact (#30619) 2026-10-17 11:41:53 +02:00
Max Krasnyansky b797b9ca81 hexagon: tensor repacking rework and support for tiled and flat quantized and fp weights (#30635)
* hex-repack: Remove need_repack hack and streamline the rules -- breaks GET_ROWS tests for K-Quants, will fix in the next round

Add support for GET_ROWS with tiled Q5_K
Add support for raw k-quant tensors

* hex-get-rows: vectorize raw k-quant ukernels

* hex-binary: refactor dma-loop thread functions (no need for macros, etc)

* hex-quant: move/cleanup flat and tiled dequant ukernels

* hex-repack: add support for repacking/tiling F16/F32/BF16 2D weights

* hex-get-rows: remove redundant cast

* hex-repack: proper filtering for BF16 that were leaking through during the tests

* hex-repack: streamling tiling rules (broken MM for now)

* hex-repack: fix mmid after tiling updates

* hex-repack: rename repacked to tiled in all flags/funcs/kernels/vars

* hex-mm: fix valid_rows calc in various kernels

* hex-mm: remove unused hvx_mm_nx_2d_flat

* hex-mm: cleanup and optimize hvx-tiled vec_dot_f16

* hex-mm: another pass at hvx-tiled f16 matmul

* hex-get-rows: optimize small src1

* hex-get-rows: one more pass at recovering get-rows perf regression

* hex-repack: export GGML_HEXAGON_RT_REPACK via run.py

* hex-mm-tiled: fix accuracy regression with hmx-tiled-f16 matmul

* hex-mm: fold tiled_f16 checks into the general flow

* hex-get-rows: remove dead checks for buffer_is_hexagon and tiling during supports_op

* hex-quant: proper tail handling in new dequant ukernels

* hex-get-rows: remove unused code and cleanup support checks

* hex-mm: cleanup ne1 padding/tiling checks

* hex-mm: cleanup fusion checks for tiled tensors

* hex-repack: add a little can_repack helper to reduce duplication

* hex-repack: never repack host buffers

* hex-repack: factor out repack functions into separate .h and .cpp

* hex-tensor: cleanup tensor ifaces for tiled/padded sizes

* hex-repack: update k-quant tiling comments and cleanup #defines

* hex-dcache: factor out l2/dcache dirty tracking and flushing

* hex-tensor: update docs and remove #defs duplication

* hex-tensor: remove unused/incorrect get_data interface

* hex-get-rows: cleanup vtcm layout build

* tests: disable huge MM tests

* hex-dcache: add missing header
2026-10-15 20:30:11 -07:00
Georgi Gerganov a11b6530b5 scripts : add ci-status.sh to list active CI runs (#30586)
Small helper around the GitHub Actions API: lists the workflow runs that are not
finished yet (in_progress, queued, pending, waiting, requested), grouped by status.

Supports selecting runs by status, by branch (server-side, to reduce API traffic)
or by a regex on the branch name, and can cancel the selected runs via --kill.

Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
2026-10-15 16:28:09 +03:00
Georgi Gerganov c0cb53ac4e sync : ggml 2026-10-15 16:20:14 +03:00
Sigbjørn Skjæret 711780ffc7 ci : bump ty to 0.0.88 (#30587)
* bump ty to 0.0.88

* arange -> linspace.tolist() to pass expected type
2026-10-15 14:15:55 +02:00
Adrien Gallouët 3eb101360f vendor : update cpp-httplib to 0.61.0 (#30494)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-14 08:28:24 +02:00
Sandro Hanea 10da8c463f unicode : add opt-in NFC normalization for Unigram tokenizers (#30390)
* unicode : add opt-in NFC normalization for Unigram tokenizers

Note: used help from Codex, Sol 6.1 Medium

* Fixing initial bot comments + linter error (using existing out instead of print)

* removed test_nfc_tokenizer
2026-10-14 08:33:24 +03:00
Pranav DhinakarandMax Krasnyansky ec0bfac179 hexagon: add configurable HTP DCVS/HMX power mode (#30423)
* hexagon: add configurable HTP DCVS/HMX power mode

* hexagon: address PR review feedback (drop CLI flag, align QAIRT power modes, comment/log cleanup)

* hexagon: add --hex-power-mode flag to run.py

Hooks GGML_HEXAGON_POWER_MODE into the unified run.py runner, following
the existing --hex-nhvx pattern. The env var must be set before the
process touches the Hexagon backend registry (see comment in
ggml_hexagon_init), so this is exposed as a runner flag rather than a
common/arg.cpp CLI option.

* hexagon: keep HTP awake for DCVS power profiles

* Apply suggestion from @max-krasnyansky

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-10-13 07:47:55 -07:00
8bb3cf0d21 hexagon: optimize CPY/CONCAT — round 3, short-rows edition (#30341)
* hexagon: move the converting CPY in blocks of rows per DMA descriptor so short rows are not DMA-latency bound

* hex-cpy: cleanup cpy cursor handling

* hex-cpy-concat: introduce threaded dma/hvx gather-row helper

Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com>

* hex-cpy: remove leftover rejections for extended buffers

* hex-cpy: relax the gate for fast-path

* hex-cpy: remove scalar divs and vtcm access and use explicit gather_sync

* hex-cpy: add proper support for mdev work splitting to fastpath

* hex-cpy: optimize dense gather-row tasks

* hex-cpy: unroll gather-rows loops

* hex-cpy: add fastpath for f16

* hex-cpy-concat: support for colapsing 3/4D dims

* hex-cpy: double-buffer support

* hex-cpy: factor out gather-rows into more of an op by itself

* hex-cpy-concat: dedup kernel-params

* hex-cpy-concat: step-down nr to fit instead of hard fail

* hex-gather: replace div with a mask

* hex-gather-rows: cleanup dma pipeline and remove the stalls

* hex-gather: specialize compute funcs

* hex-gather: reduce scalar reg.spills and func calls

* hex-gather: do not recompute rows-per-thread on the npu

* hex-build: add support for building/installing just hexagon

* hex-cpy-concat: remove redundant barrier in concat and move cpy barrier under fence

* hex-gather: annotations for dma-out

* hex-mdev: make all key vtcm_acquire sync state atomic (fixes random hangs in mdev setups)

---------

Co-authored-by: Yiwei Shao <yiwei@aizip.ai>
Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com>
2026-10-12 07:39:55 -07:00
Xuan-Son Nguyen 10a60cf303 vendor: apply deep nested json patch from upstream (#30253) 2026-10-10 00:07:44 +02:00
Xuan-Son Nguyen b42b7e6d30 server: use port 9931 by default (#30159)
* server: use port 9931 by default

* revert unrelated changes
2026-10-09 10:40:03 +02:00
Pascal 2c564f43df ci: fix HIP quality check by ignoring the 320/256 FA kernel spill (#30212) 2026-10-09 10:29:22 +02:00
Adrien Gallouët 03aa006acb vendor : update cpp-httplib to 0.60.1 (#30134)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-08 10:57:47 +02:00
Max KrasnyanskyandJhen-Jie Hong 9c2e0e491a hexagon: enable alloc_buffer_n (#30126)
* hex-bufs: add support for alloc_buffer_n

* hex-bufs: add support for splitting large tensors into separate buffers

* hex-bufs: update GGML_HEXAGON_MBUF to accept three values dyn,static,total

* hex-bufs: bump dyn. default to 512MB since 128MB causes perf regressions with big MOEs

* hex-run: add --no-embd-offload option to simplify command lines on devices that need it

* Update scripts/snapdragon/run.py

Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>

---------

Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>
2026-10-07 18:30:06 -07:00
Alessandro de Oliveira Faria (A.K.A.CABELO) b7dafa01e5 vendor : update cpp-httplib to 0.60.0 (#30081) 2026-10-07 11:18:33 +02:00
Daniel Bevenius d7a695ef67 scripts : limit apiabi checks to libllama and libmtmd (#30038)
Refs: https://github.com/ggml-org/llama.cpp/pull/29997#issuecomment-5998572938
2026-10-06 11:42:59 +02:00
Georgi Gerganov c250304960 ci : skip container re-tagging when Require Docker is disabled (#30008)
Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
2026-10-05 21:22:08 +03:00
8345f33395 hexagon: matmul and flash-atten scalability updates (#29974)
* hexagon: head-parallel flash_attn partitioning for row-split multicore

In row-split mode each core computes its output row shard of every
MUL_MAT, but flash_attn was previously partitioning by Q tokens
(flat qrow split) instead of by heads. This forced every core to
read the full KV cache (all n_kv_heads), negating the memory
bandwidth benefit of multicore on flash_attn.

Change both HMX and HVX flash_attn kernels to partition by KV heads
when n_kv_heads is divisible by n_cores: core i processes heads
[i*n_kv_heads/N, (i+1)*n_kv_heads/N) exclusively, reading only its
head shard of the KV cache. Falls back to the original token-block
split when n_kv_heads % n_cores != 0 (e.g. Gemma-4 with 2 KV heads
on 4 cores).

Controlled by GGML_HEXAGON_FA_HEAD_SPLIT (default 1 = on).
The flag is packed into bit 1 of the existing is_dst_fp32 kparams
byte to stay within the 128-byte kernel_params blob limit.

Measured gains at 4c row-split (PP t/s, ubatch=1024):
  Qwen3-0.6B:    6977 -> 11026  (+58%)
  llama-3.2-3B:  3717 ->  5522  (+49%)
  Qwen3.5-4B:    2739 ->  2855   (+4%)
  Gemma-4 MoE:   no change (MoE FFN dominates, fallback path)

TG is unchanged (flash_attn is a small fraction of decode time
relative to the matmul+barrier cost per layer).

* hex-fa: cleanup kern_params and head-split selection

* hex-fa: add -fa-head-split option to run.py

* hex-mdev: update matmul solver to account for reduced work in row-split scenarios

* hex-mmid: better work splitting by expers in multi-dev scenarios

* hex-fa: update HMX gating based on the model/n-hvx/ctx-len sweep

* hex-fa: precompute softcap/scale on the host

* hexagon: flatten matmul into 2d to use HMX in multi-sequence

* hex-mm: cleanup kparams and use collapse to 3/4D -> 2D mapping

* hex-mm: fix typo in collapse fallback

* hex-mm: another pass at consistent naming for act tensors

* hex-mm: add support for colapsing dims in fused matmuls

* hex-build: fix WoS build errors

* hex-mm: make sure to enforce dst stride in can_collapse

* hex-fa: add a onliner commit for head-split check

* hex-fa: remove unused local head_split var

* hex-fa: tighten up can_split checks

* hex-mm: update unfused paths to use act instead src1

* hex-mm: make sure to check all dsts for splitting

* hexagon: fix the second weight chunk address in the batched HMX matmul prologue

* hexagon: F16 activation and ragged N in the HMX matmul

* hex-mm: tighten the ragged/split checks in mdev cases

* hex-mm: enable MM fusion for F16 activations

* hex-mm: pass tiled sizes to the solver in fused paths

* hex-mmid: remove scalar divs from expert mapping loops

* hex-mmid: proper cacheline safety enforcement for mdev splits

* hex-mm: improve solver for mdev split scanarios and tail handling

* hex-mm: remove redundant checks

* hex-mm: fix fused HMX MUL_MAT_NX drops the final partial tile for quantized weights

* hex-mm: better handling of ragged shapes (removes scalar memset of vtcm)

---------

Co-authored-by: ebateni <ebateni@qti.qualcomm.com>
Co-authored-by: Jhen-Jie Hong <iainst0409@gmail.com>
Co-authored-by: Yiwei Shao <yiwei@aizip.ai>
2026-10-05 08:55:21 -07:00
Georgi Gerganov d812350493 llama.cpp : bump version to 0.6.0 (#29997)
* llama.cpp : bump version to 0.6.0

* scripts : update summary prompt (#0)
2026-10-05 18:13:51 +03:00
Georgi Gerganov c06f84160a sync : ggml 2026-10-05 17:37:50 +03:00
Georgi Gerganov 994e8f2222 ci : add "Require Docker" flag to make-release workflow (#29989)
Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-10-05 16:04:47 +03:00
Alessandro de Oliveira Faria (A.K.A.CABELO) eec18f5d32 vendor : update cpp-httplib to 0.59.0 (#29886) 2026-10-03 19:08:47 +02:00
Ehsan BateniandMax Krasnyansky a4d880fd5c Hexagon: optimize ALLREDUCE with support for safe scatter mode (#29757)
* hex-allreduce: add support for safe scatter mode

* hex-allreduce: pare down excessive comments

* hex-allreduce: re-write to remove register spills

---------

Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
2026-09-30 12:56:44 -07:00
thelittlefiremanandCarl Philipp Klemm 748d4225b9 ggml-cuda: HIP: optimize packed byte subtraction (__vsubss4 -> __vsub4) (#29478)
* ggml-cuda: HIP: optimize non-saturating packed byte subtraction (`__vsubss4`)

* CI: ignore 1 spilled vgpr in fattn_vec

---------

Co-authored-by: Carl Philipp Klemm <carl@uvos.xyz>
2026-09-29 19:59:19 +02:00
Trivikram Reddy 1c4729414d hex-scripts: show trace events smaller than 100nsec in perfetto (#29614) 2026-09-28 15:02:58 -07:00
uvos f916130d00 ci : ignore more vgpr spills in > 256 DQK fattn kernels (#29571) 2026-09-28 14:52:07 +02:00
uvos 2ebd9ae621 HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes (#28907)
* HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes

* CI: hip-quality-check: ignore spills for very large mfma mma kernels
2026-09-27 13:06:56 +03:00
Trivikram Reddy 4e7481175c hexagon: find software divide calls using binary inspection tool (#29449)
* hex-scripts: fix table alignment

* hex-scripts: find sw div calls using binary inspection tool
2026-09-25 19:43:38 -07:00
Alessandro de Oliveira Faria (A.K.A.CABELO) 171e8846b4 vendor : update cpp-httplib to 0.58.0 (#29407) 2026-09-26 01:55:18 +02:00
Georgi Gerganov bced4595b8 sync : ggml (#29396)
* ggml : bump version to 0.25.2 (ggml/1642)

* ggml : fix ubsan error in `ggml_graph_nbytes` (ggml/1644)

* ggml : bump version to 0.25.3 (ggml/1645)

* sync : ggml
2026-09-24 22:44:05 +03:00
Georgi Gerganov 013b31c03c scripts : make-release-desc - link previous release in changelog title (#29336)
make-release-desc.sh now emits "Changelog since [vX.Y.Z](<repo>/releases/tag/vX.Y.Z)"
instead of a plain version string, so the release notes link back to the previous release.

The repo URL is derived from the origin remote (SSH or HTTPS); if it cannot be
resolved (local run without origin), the title falls back to plain text.

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-09-24 07:46:38 +03:00
Georgi Gerganov 177cd8cc70 sync : ggml 2026-09-23 20:29:29 +03:00
Georgi Gerganov 183d2a04c2 make-release : update summary prompt 2026-09-23 11:47:24 +03:00
Georgi Gerganov 45062d4056 sync : ggml 2026-09-23 11:47:24 +03:00
Max Krasnyansky 58367713a6 hexagon: new HMX-optimized GATED_DELTA_NET (#29199)
* hex-gdn: start putting together HMX support for GDN

* hex-gdn: working hmx but not-pipelined and slow for now

* hex-gdn: re-write vtcm layout handling and prep for pipelining

* hex-gdn: starting to pipeline hmx and dmas

* hex-gdn: add hvx threading for most pipeline stages

* hex-gdb: add detailed trace events

* hex-gdn: vectorize expfs and use aligned hvx reads/writes

* hex-gnd: vectorize the rest of expf

* hex-gdn: optimize tail processing (pad partial chunks)

* hex-gdb: avoid float up/down casts in hot loops

* hex-fa: remove float up/down casts from inner loops

* hex-gdn: do exp() in f16 to improve HVX utilization

* hex-gdn: optimize tiler

* hex-hmx: bump hmx-queue to 128 and dispatch all GDN gemms at once

* hex-gdn: further pipeline improvements

* hex-gdn: optimize gdn prep stage

* hex-gdn: yet more tweaks to optimize GND_SOLVE task and pipeline

* hex-gdn: improve accuracy and optmize gdn-prep further

* hex-gdn: fix rebase conflict

* hex-bufs: revert max_bufsize enforcement, it is enough to just enforce max_vmem

* hex-scripts: improved inspect script to avoid false alarms in reg spill detector

* hex-fa: improve inline softmax with in-reg VKQ32 accum

* hex-fa: minor improvement for dma pipeline in hvx kernel

* hex-fa: reduce ddr reads by 20-30% during token gen

* hex-gdn: proper alignment for hvx vtcm spads
2026-09-21 14:49:52 -07:00