With LLAMA_SLEEP_EVICT_KV the KV cache device buffers are freed when a model
goes cold. update_slots() touches the KV before the decode-time vram_ensure_warm
(e.g. SWA models create a checkpoint that reads the KV via ggml_backend_tensor_get),
so the KV must already be resident by then. Move the restore to the sleep-wake
handler (handle_sleeping_state(false)), which runs before update_slots, fixing a
GGML_ASSERT(buffer) crash on iSWA models (gemma) when KV eviction is enabled.
Assisted-by: Claude
release_device_weights()/release_device_buffers() leave a null buffer in
ctxs_bufs while the device memory is released for on-demand VRAM sharing.
The memory-breakdown / total_size / clear paths iterated these and called
ggml_backend_buffer_get_size()/get_type()/clear() on the null buffer, tripping
GGML_ASSERT(buffer) and aborting on shutdown of a cold model. Skip null buffers
in llama_model::memory_breakdown and llama_kv_cache::{memory_breakdown,
total_size,clear}.
Assisted-by: Claude
Before uploading a model's weights, acquire the shared VRAM token (ring the
doorbell so any resident model releases first). Previously load uploaded weights
to VRAM before the arbiter was active, so loading a large model while another
(e.g. the warm 4B task model) held VRAM could exceed the budget and OOM.
- add vram_arena_open() (idempotent flock/doorbell setup) and
vram_acquire_for_load(), called from load_model() before
common_init_from_params().
- a coordinated load now stays warm holding the token and serves its first
request without a re-warm (drop the init-time go_cold cold-start).
Assisted-by: Claude
The request handler only calls wait_until_no_sleep() (which wakes a server out
of its sleeping state) when sleep_idle_seconds >= 0. But the VRAM arbiter's
cross-process doorbell can put a server to sleep even when idle-sleep is
disabled, so without this a doorbell-slept server would never wake and requests
to it would hang until timeout.
Do not bypass the wake path when LLAMA_SLEEP_VRAM_ONLY is set, so the arbiter no
longer depends on --sleep-idle-seconds being configured.
Assisted-by: Claude
Extend on-demand device residency to the KV cache so that when a model's KV
plus another model would not fit in VRAM, the KV can also be evicted to a host
shadow (D2H on release, H2D on restore) instead of only the weights.
- llama_memory_i: add release_device_buffers()/restore_device_buffers()
(default no-op). Implemented in llama_kv_cache (D2H shadow of the live
ctxs_bufs, freed and reallocated like the weights); llama_memory_hybrid and
llama_kv_cache_iswa delegate to their child caches.
- llama_context::release_device(evict_kv): also evict the memory's device
buffers when requested; restore_device() rebuilds them. Public API
llama_context_release_device gains an evict_kv flag.
- server: LLAMA_SLEEP_EVICT_KV=1 enables it. Off by default (weights-only),
since the KV shadow adds a D2H/H2D copy of the live cache each cycle.
Validated on RX 580 (Vulkan), 4B @ 32k ctx: weights-only cold VRAM 1750 MB
(KV stays); weights+KV cold VRAM 726 MB (KV freed, ~1 GB reclaimed). KV
survives the round-trip: prompt cache reused after the cycle (prompt_n 4 vs
42), correct output.
Assisted-by: Claude
Add release/restore of a model's GPU weight buffers (keeping a host shadow
and the KV cache) so several always-loaded llama-server processes can
time-share a single GPU without reloading or losing the prompt cache.
- llama-model: release_device_weights()/restore_device_weights() capture a
compact host shadow (stable iteration order, view-skipping) and free then
realloc the device weight buffers; weights_resident() query.
- llama-context: release_device()/restore_device() wrappers; decode() auto-
restores; public C API llama_context_release_device/restore_device.
- server: LLAMA_SLEEP_VRAM_ONLY makes idle-sleep release only the VRAM weights
(not a full unload/reload). A cross-process flock token in LLAMA_VRAM_ARENA
enforces "resident iff holds token"; an inotify doorbell forces the holder
to release on contention. The warden thread only touches the task queue, so
releases run on the loop thread and never race a decode.
Validated on RX 580 (Vulkan): two models share 8GB, never both resident,
correct output under contention, KV cache preserved (no re-prefill).
Assisted-by: Claude
Extends --n-cpu-moe to accept a fractional layer count. The integer part
offloads whole layers as before; the fractional part offloads a subset of the
boundary layer expert tensors (gate, then up), keeping down_proj resident.
This realizes ATSInfer tensor-granularity static placement, giving sub-layer
control over expert VRAM residency. Placement-only, lossless.
Assisted-by: Claude
Add the flash-attn mask_opt change (auto-on for GCN large head sizes,
lossless, +12% pp @ 32k on the RX 580) and correct the earlier pinning
advice: pinning helps in isolated llama-bench but fails in a long-running
server on RADV, so --no-mmap is the serving path. Add a recommended
Polaris serving command.
Assisted-by: Claude
mask_opt was disabled on AMD GCN for head sizes <= 256, but it is
beneficial there in high-context prefill: on fully-visible causal
blocks it skips the per-block mask load+add, and it skips fully-masked
blocks entirely. The attention op on GCN is compute-bound on the
softmax path (no matrix cores), so this cuts real work.
Verified lossless (perplexity bit-identical with it on vs off) and a
measured prefill win on Qwen3.5-35B-A3B (head_dim 256) on an RX 580:
pp2048 unchanged at short context, +8% @ 16k, +12% @ 32k, growing with
depth. Enable for GCN when HSK/HSV >= 256; the existing large-mask
conditions keep it off for decode.
Assisted-by: Claude
Pinning helps on Vulkan too, but GGML_SCHED_PREFETCH_EXPERTS regresses
there (second backend shares one device queue, no overlap). Add an RX 580
Polaris benchmark and the context-dependent --n-cpu-moe guidance.
Assisted-by: Claude
Pinned async host uploads can record into compute_ctx before graph
compute, so the perf logger's opening timestamp must append after them
instead of asserting the context is expired. Without this, profiling
(GGML_VK_PERF_LOGGER) crashes whenever host-register pinning is on.
Assisted-by: Claude
VK_EXT_external_memory_host import fails on RADV for file-backed mmap pages,
so register_host_buffer no-op'd and every MoE-expert upload paid a slow
single-threaded pageable memcpy into the staging buffer each eval (~3 GB/s
effective on the 35B, ~25% of PCIe bandwidth).
When the import fails, fall back to a one-time copy of the region into a
host-visible Vulkan buffer registered in device->pinned_memory under the
mmap's address range. The existing pinned fast path then DMAs straight from
that buffer each eval at full PCIe bandwidth - no per-eval CPU memcpy and no
blocking sync. Costs one-time host-visible memory equal to the CPU-resident
weight region.
Assisted-by: opencode
The generic concat shader reads a transposed source (nb[1]==type_size) with an
uncoalesced stride, which is catastrophically slow on discrete GPUs (~5.6ms for
a 16MB concat on the RX 580 vs ~130us of memory bandwidth), a ~9% prefill
hotspot on Qwen3.5-4B (delta-net state concat).
Add a fast path for concat along dim 0 where one source is stored transposed and
the other source + dst are contiguous along dim 0: copy the contiguous source
with copy.comp and transpose the other source into the matching dst sub-region
with the existing tiled copy_transpose shader (shared-memory 32x32 transpose,
coalesced read+write). Reuses pipeline_cpy_* / pipeline_cpy_transpose_* with
custom push constants + doffset, no new shader. Falls back to the generic path
otherwise (gated on type/shape/contiguity + 16-bit doffset bound).
Assisted-by: opencode
Export register_host_buffer/unregister via the backend reg so the existing
GGML_CUDA_REGISTER_HOST path in llama-model-loader pins mmap'd expert
weights on Vulkan. Imports host pages through VK_EXT_external_memory_host
into device->pinned_memory, letting the existing pinned fast path in
ggml_vk_buffer_write_2d_async DMA straight from system RAM instead of
bouncing through the staging buffer + blocking host memcpy.
A single Vulkan buffer cannot cover a multi-GB mmap (capped at
device->max_buffer_size), so the region is imported in page-aligned
chunks; a bound check makes tensors straddling a chunk boundary fall back
to staging instead of reading out of bounds. Opt-in via
GGML_CUDA_REGISTER_HOST=1 or GGML_VK_REGISTER_HOST=1, no-op otherwise.
Assisted-by: opencode
Profiling showed the 2-slot rotation stalls ~5.5ms per layer waiting for
the down_exps upload: 3 tensors per MoE layer need 3 slots for a full
layer of lookahead. Default is now 3 (one layer), configurable via the
env var value, degrading to however many slots fit on OOM.
Also fixes a use-after-free: on a failed slot regrow the repointed
staging tensors kept dangling pointers into freed device memory, which
graph reuse carried into later evals. Staging repoints are now restored
right after kernel launch, slots are sized once from the graph max, and
allocation happens before freeing.
Qwen3.6-35B-A3B pp2048 on RTX 3060: 1643 -> 1880 t/s (mainline: 1143).
At large batch sizes virtually every expert is used, so the per-layer
routing-ids readback that mainline waits on buys nothing while forcing a
full device sync per expert tensor (3x per MoE layer). Above a batch
threshold, upload the full expert tensors through a second backend
instance on the same device (own stream) with two event-ordered staging
slots, so uploads for tensor N+1 overlap compute of tensor N.
Qwen3.6-35B-A3B pp2048 on RTX 3060 (-ncmoe 26, ub 2048): 1383 -> 1663 t/s.
Generation output verified token-identical; decode path unaffected.
Opt-in via GGML_SCHED_PREFETCH_EXPERTS=1.
Wire the existing GGML_CUDA_REGISTER_HOST path back up: after model load,
cudaHostRegister the mmap pages backing weights kept in system memory.
Recovers pageable-copy losses when MoE experts are streamed to the GPU
during prefill (n-cpu-moe): Qwen3.6-35B-A3B pp2048 1144 -> 1385 t/s on
RTX 3060. Opt-in via GGML_CUDA_REGISTER_HOST=1, unchanged otherwise.
* Squash history before conflict-resolution during rebase on master
WIP commit
Add 32-byte loads, restore per-block amax
Use nvfp4x4 intrinsic when available
Fuse per-channel amax and quantization kernels
Do pointer arithmetic only once on x
Remove unnecessary ternary in the load
We assert on host side that ne00 is 64-aligned
Add back scale-search, but optimize it with intrinsics
Code cleanup
Make scale in MMQ-epilogue NVFP4-specific/restrictive for now
Remove unneeded include, add comment
Fix trailing whitespace
Guard __builtin_align__(32) struct to NVIDIA
Seems like HIP doesn't have this available, see https://github.com/ggml-org/llama.cpp/actions/runs/29438651734/job/87431623001
* compiler massaging to avoid unnecessary LDCs
* kvalues_mxfp4 -> kvalues_nvfp4 in quantize_mmq_nvfp4
* Always pass in src1_scale.ptr
* Extract ggml_cuda_is_aligned helper
* hex-geglu: optimized all-in-one geglu microkernel
* hex-geglu: enable non-contiguous src and strided DMA
* hex-act: enable non-contiguous srs and strided DMA for rest of ACT ops
* hex-act: generalize GLU per-thread functions via DEFINE_GLU_PER_THREAD macro
* hexagon: move UNARY_SILU and UNARY_GELU to unary-ops
* hex-act: replace the generic ops_context scratchpad usage with a local htp_vtcm_layout computation per act op.
---------
Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>
* feat(ui): add symbolic math support to JS sandbox via nerdamer
Preload nerdamer (with decimal.js) in the sandboxed worker,
exposing the `nerdamer` global for symbolic computation:
simplify, expand, factor, diff, integrate, solve, laplace,
ilt, limit, partfrac, gcd/lcm, roots, coefficients, and more.
Mirrors the math.js integration pattern from the
feature/sandbox-symbolic-math branch, but uses nerdamer
for a lighter, more focused symbolic math engine.
* Update sandbox-harness.ts
* docs(ui): update sandbox tool description with detailed nerdamer usage guide
* Clarify nerdamer usage in sandbox tool description
Updated the description of the sandbox tool to clarify usage of nerdamer.
* ui: build nerdamer sandbox prelude from vendored source
Replace the vendored all.min.js with the readable nerdamer-prime
source and its two bundled deps (big-integer, decimal.js), licenses
included. A vite plugin bundles and minifies them at build time with
the upstream esbuild flags, exposed as virtual:nerdamer and imported
lazily on first sandbox use. The vendors package.json pins commonjs
so the project level type: module does not break esbuild format
detection. The harness gains a CSP removing network egress from the
worker, and browser tests cover the prelude, exact arithmetic, the
fetch block and the timeout.
Upstream snapshot: together-science/nerdamer-prime@1936145
* feat(ui): make symbolic math (nerdamer) a user-toggleable setting
- Add SYMBOLIC_MATH_ENABLED setting key and registry entry (checkbox, default false)
- Convert SANDBOX_TOOL_DEFINITION to buildSandboxToolDefinition(includeSymbolicMath)
so the tool description includes/excludes nerdamer API docs dynamically
- Cache sandbox harness per variant ('nerdamer' / 'plain') for instant toggle
- Deprecate SANDBOX_TOOL_DEFINITION constant alias for backward compatibility
- Update tools store to pass symbolic math config into tool definition
* docs(ui): tell LLM to list nerdamer functions first, do not guess
* test(ui): enable symbolic math in sandbox tests via settingsStore config
* style(ui): fix formatting for tools.svelte.ts
---------
Co-authored-by: Pascal <admin@serveurperso.com>
With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars
and no --spec-type given, the draft resolved to a full model while the
sidecar was the intended draft.
When the speculative types are still at their default, discover the
sidecars of the draft repo, pick the first available following the
existing mtp > dflash > eagle3 priority, and set the corresponding
type, so this now works without any extra flag:
llama-server -hf repo:Q3_K_M -hfd repo:Q8_0
An explicit --spec-type disables the inference, and a draft repo
without sidecars keeps resolving to a full model as before.
* chat: fix DS4 template to explicitly follow reference behavior
* Support DeepSeekv4 flag (`drop_reasoning`).
* fix: hook DS3.2 parser for DS4 as well
* fix: add tool result reordering
* fix: post-merge
* ggml: enable PowerPC backend variants on AIX
Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. This reuses the existing PowerPC backend implementations without changing their behavior.
Also fix a missing semicolon in the PowerPC Q0 matmul implementation.
* Fix missing semicolon in sgemm.cpp
* webgpu : add CONV_2D_DW (depthwise conv2d) kernel
Implement GGML_OP_CONV_2D_DW for the WebGPU backend,
ported from the Vulkan backend's conv2d_dw.comp.
Assisted-by: Claude Opus-4.8
* Remove unnecessary comments in webgpu support
* update supported ops tables, triggered by adding webgpu CONV_2D_DW
* cuda: add k-quant support to GET_ROWS
Device-side embedding lookups require GET_ROWS to handle the k-quants
used by common GGUF recipes (Q4_K_M stores token_embd as q6_K). Without
it the backend rejects the op and the scheduler falls back to the host,
copying the full embedding matrix back on every token in single-device
graphs.
Factor the super-block dequantizers out of the dequantize_block kernels
in convert.cu into shared device functions in dequantize.cuh and reuse
them from a new k_get_rows_kq kernel : one thread block dequantizes one
(dst row, super-block) pair with the existing thread layouts, 32 threads
for q4_K and 64 for the other k-quants.
Covers q2_K to q6_K in get_rows_cuda and supports_op. i-quants are left
as a TODO.
* cuda: add i-quant support to GET_ROWS
Extends the shared super-block dequantizers to the nine i-quants and
reuses them from k_get_rows_kq with the 32-thread layout of the matching
convert.cu kernels. supports_op gates the k-quant and i-quant path on
ne0 being a multiple of QK_K, which iq4_nl does not guarantee on its
own (QK4_NL sub-blocks). mxfp4 is left as a TODO.
* cuda: add mxfp4 support to GET_ROWS
Moves the mxfp4 dequantizer into the shared super-block helpers and
reuses it from k_get_rows_kq with the 32-thread layout of the matching
convert.cu kernel. mxfp4 joins the ne0 % QK_K gate in supports_op since
its 32-value sub-blocks do not guarantee QK_K-aligned rows on their own.
This closes GET_ROWS type coverage on CUDA: every quantized GGML type
now takes the direct device path.
* cuda: gate the GET_ROWS row size only for 32-value sub-block types
Address review from @pwilkin: the i-quant commit replaced the return
shared by the whole supported type cascade, so f16/f32/bf16/i32 and the
legacy quants also inherited the ne0 % QK_K == 0 gate and any row size
that is not a multiple of 256 fell back to the scheduler. Split the
cascade: unconditional support is restored everywhere, the gate stays
only on iq4_nl and mxfp4 whose 32-value sub-blocks do not guarantee the
QK_K super-blocks the kernel iterates on.
The Qwen3-VL learned position embedding is interpolated to the runtime patch
grid with the default bilinear+antialias (align_corners=False) sampling, while
the transformers reference uses align_corners=True (torch.linspace(0, side-1, T)).
The mismatch scales grounding coordinates about the image center, growing with
image size and per-axis for non-square images (see #16880).
With -hfd pointing to a repo shipping speculative sidecars, the draft
resolved to the main model of that repo, since find_best_model()
excludes sidecar files, and the explicit draft plan suppressed the
sidecar discovery on the -hf repo.
The draft plan already discovers its sidecars, they were just never
consumed. Wire them as the draft, following the fallback pattern of
the main plan, so this now works as expected:
llama-server -hf repo -hfd repo --spec-type draft-dflash
* server: return 400 instead of 500 on validation error with X-Conversation-Id
set_req() attaches the spipe as soon as the header is present, before the request
body is parsed. When params validation throws, set_next() never runs and next_orig
stays empty, so on_complete() called it and crashed with std::bad_function_call,
turning the prepared 400 JSON into a generic 500.
on_complete() now treats an empty next_orig as "streaming never started" and evicts
the session installed by set_req(), so a failed request leaves nothing behind for
discovery or replay. This also covers valid requests that carry the header but do
not stream, which previously left an empty finalized session in the map until the
GC TTL.
* ui: do not send the backend_sampling placeholder
On a fresh profile the syncable settings hold the empty string placeholder meaning
"let the server decide". Every neighbor field goes through the hasValue() guard
that filters it, except backend_sampling, which sent the placeholder verbatim and
made every default settings completion fail validation.
Guard the field with hasValue() like its neighbors. hasValue(false) is true, so an
explicit false still reaches the server and the intent of #18781 (send both true
and false) is preserved. Only the placeholder is filtered.
* Refactor vk_queue to use per-instance mutexes and unique handles
* integrates VK_KHR_internally_synchronized_queues, abstracting the queue submission into a polymorphic interface that completely bypasses host-side mutex locking when driver-side synchronization is supported
* fix compilation error
* fix duplicate pNext chain for VkPhysicalDeviceInternallySynchronizedQueuesFeaturesKHR
* add fallback defines for VK_KHR_internally_synchronized_queues
* add null checks for queues in vk_device_struct destructor
* use unique_ptr for outer queues to enforce exclusive ownership and optimize lifetime
* use static constexpr for eInternallySynchronizedKHR
* add lock guard to ggml_vk_create_aliased_queue for thread safety
* initialize sync_query_features.internallySynchronizedQueues to VK_FALSE
* reuse sync_query_features for internallySynchronizedQueues and simplify chaining
* refactor internallySynchronizedQueues detection
* fix internallySynchronizedQueues query guard
* use eInternallySynchronizedKHR constant
* fix self-referential alias for eInternallySynchronizedKHR
* use macro for eInternallySynchronizedKHR fallback
* fix internallySynchronizedQueues query timing in ggml-vulkan.cpp to prevent device creation mismatch
* reset sync_query_features.pNext before reusing in device creation chain, also removed the redundant second probe call
* refactor internally synchronized queues detection to use chained feature query and avoid redundant API calls
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* Update ggml/src/ggml-vulkan/ggml-vulkan.cpp
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
* rename sync_enable_features to internally_synchronized_queues_features
* queue_flags is still computed before has_internally_synchronized_queues is set
* fix trailing whitespace
* replace eInternallySynchronizedKHR macro with static constexpr
* preserve source queue semantics in single-queue aliased transfer queue
* vulkan: fix cmd_pool access via pointer for compute_queue unique_ptr
* vulkan: lock queue during debug label emission when not internally synchronized
---------
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
k_get_rows_float did a scalar one-element-per-thread copy and recomputed the
row-invariant work (index load, fast_div_modulo, src/dst row pointers) for
every element. Hoist that out of the per-element loop, and add a vectorized
path (k_get_rows_float_vec) that copies one int4 (16 B) per thread for the
contiguous same-type (no-cast) case.
The vectorized path is gated at compile time (is_same<src0_t, dst_t>) and at
runtime on 16-byte alignment of the base pointers and all row strides and on
ne00 % VEC == 0. Vectorizing divides the block count by VEC, so a small
single-row gather can drop below the device CU count and regress; an
occupancy gate keeps those on the block-rich scalar path.
On Strix Halo (gfx1151) the DeltaNet recurrent-state gather (ne00=524288)
drops 18.6us -> 13.0us (rocprofv3 HW timestamps), faster than the Vulkan
backend, with no regression on the small conv-state gather; total get_rows
-27%. test-backend-ops GET_ROWS passes (47/47).
Assisted-by: Claude Opus 4.8
* feat: WIP
* feat: Replace conversation rename flow with unified AlertDialog component
* feat: Add radio group component and consolidate title generation settings
* refactor: Remove JS Sandbox global toggle and migrate legacy user state
* chore: Formatting
* refactor: Cleanup
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
* refactor: Cleanup
* refactor: Marquee selection hook
* feat: UI improvements
* refactor: Bulk db operations
* fix: optimize bulk conversation deletion to handle ancestor chains
* refactor: remove pairedKey mechanism from settings system
* fix: remove redundant onclick handler from dialog cancel button
* chore: pin @lucide/svelte to exact version
* feat: Run JavaScript tool disabled by default
* fix: correct active conversation deletion tracking in bulk delete
* feat: improve shift-key multi-selection support in sidebar via keyboard
* refactor: Retrieve JS Tool enabling via Developer Settings
* nits: sync, dialog wording, cycle guard, and lockfile follow-ups
- Restore titleGenerationUseLLM registry entry so it syncs across devices again
- Mention fork cascade in the bulk delete confirmation dialog
- Clear newParent on cycle guard break so children never point at a deleted conversation
- Align @lucide/svelte in package-lock.json with the exact pin in package.json
---------
Co-authored-by: Pascal <admin@serveurperso.com>
Edge paragraph margins are now zeroed at the source in
markdown-content.css, but the user bubble and the system message still
carried the -my-4 compensation for them. The uncompensated negative
margins shrank the wrapper 2rem below its content, collapsing
single-line user bubbles into a scrollable sliver and skewing the
system message expand threshold.
* ui: fix Show tool call in progress toggle ignored
The showToolCallInProgress setting was disconnected from the render
path during the agentic content rework: getDefaultExpanded() returns
a hardcoded false for tool call sections and an unconditional effect
auto-expands the currently executing tool call regardless of the
setting.
Drive default expansion of all tool call section types from the
setting and remove the now redundant auto-expand effect. Manual
toggling still takes precedence over the default in both directions.
* ui: rename Show tool call in progress to Always show tool call content
The previous name suggested symmetry with Show thought in progress,
which only applies while inference is running, but tool call content
stays expanded after completion. Rename the label, the settings key
and the syncable server key to alwaysShowToolCallContent. The synced
parameter never worked under its previous name so no migration is
needed.
The agentic gate counted MCP servers, builtin and custom tools but not
frontend tools, so with the JS sandbox as the only tool source the
agentic flow was skipped, no tools field reached the server and the
chat template rendered without the tool system prompt.
The sandbox is fully client-side: frontendTools derives from the
Developer settings toggle alone, counting it in the gate restores that
single source of truth.
Lumpiasty
Anirban KarandLumpiasty