Commit Graph
1 Commits
Author SHA1 Message Date
Piotr WilkinandClaude Opus 5.5 66b3c3f72f upstream-carry #32740: ggml: compact bias on the existing FLASH_ATTN_EXT for CUDA, ROCm and Vulkan
Temporary carry of the open upstream PR
https://github.com/ggml-org/llama.cpp/pull/32740 at head 808fd1c4e,
its four commits (1fa93f902, d29f9d489, ff91e4f84, 808fd1c4e) squashed.
Inkling's per-query bias table is applied inside flash attention
instead of a dense [n_kv, n_tokens, n_head] bias tensor, which cuts
Inkling's compute buffers and long-context prefill time on the RX 580.
Drop this commit when the PR is merged upstream (FORK.md, "Temporary
upstream carries").

One conflict, ggml-vulkan.cpp use_mask_opt: kept the fork's GCN
`HSK >= 256 || HSV >= 256` clause and its comment (F2) and upstream's
`!use_sparse`, added the PR's `!bias &&`.

Original commit trailer: Assisted-by: Codex

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZz44SLQvTXMyWGio6t9DZ
2026-11-22 20:38:33 +01:00