* server: add read_image tool (#25875)
Adds a server-tool that allows vision models to analyze server-side images.
This tool is reading a single file for now:
The image data is base64 encoded and passed to the UI, which
decodes it, fills the <img> tag and removes the data URI before
passing the tool result back to the model.
* cleanup read_image tool: move magic strings to constants
* Add dedicated constants file: tools/ui/src/lib/constants/read-image.ts
with PREFIX_IMAGE, PREFIX_SIZE, PREFIX_MIME constants
* Use ATTACHMENT_SAVED_REGEX from agentic.ts in ChatMessageToolCallBlockReadImage.svelte
* Use NEWLINE constant from code.ts instead of hardcoded '\n'
* Use PREFIX_SIZE in regex pattern for size parsing
* Add SERVER_TOOL_READ_IMAGE_PREFIX_* constants in C++ server-tools.cpp
to match the TypeScript PREFIX_* constants for consistency
* server: rename read_image tool to read_media for images and audio
* Rename server_tool_read_image to server_tool_read_media in C++
* Rename enum BuiltInTool.READ_IMAGE to READ_MEDIA
* Rename UI constants, parser, and Svelte component files
* Update display label from 'Read image' to 'Read media'
* ui: consolidate audio data URI handling into shared utility
* Extract getAudioInputFormat to a shared utility (was duplicated inline)
* Store raw base64 in base64Data on the message object
* Use base64Data to construct data URIs for audio rendering
* Update agentic store to build INPUT_AUDIO parts from base64Data
* server: read_media: restrict audio to wav/mp3 and minor fixes
* Server get_mime_from_extension now only advertises audio/wav and
audio/mpeg (the only formats the model's input_audio API accepts)
* Case-insensitive extension matching (fixes .MP3, .Wav, etc.)
* Unknown extensions return an error instead of a multi-MB data URI
that inflates model context with garbage
* Updated tool description to document supported formats
* Frontend AUDIO_MIME_TO_EXTENSION trimmed to match server
* fix a missing import in tools/ui/src/lib/stores/agentic.svelte.ts
* server: read_media: add to --tools help text and README tool list
* ui: fix indentation in ChatMessageToolCallBlockDefault.svelte
* server: read_media tool: fix a cast to use the correct type
* server: read_media: multiple fixes
* server-tools.cpp import cctype, remove UTF-8 char, check mime before reading file
* ui: add MimeTypePrefix.AUDIO and use it in agentic.svelte.ts
* server: make read_media inherit from read_file and add uses_cwd
* ui: fix formating issues
* rm from server
* move it to frontend-only tool
* correct partial commit
* rm unused
* ui: address review from allozaur
Replace the magic strings, regexes and number in the read_media parser
and service with named constants. Path splitting reuses
FILE_PATH_SEPARATOR_REGEX, the size header regex moves to
READ_MEDIA_SIZE_REGEX derived from PREFIX_SIZE, and
FILE_EXTENSION_SEPARATOR lands next to it in constants/code.ts.
---------
Co-authored-by: ckrafft <ckrafft@epyc>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pascal <admin@serveurperso.com>
* server : extend file_glob_search for UI pickers
* ui : add per-conversation working directory with picker
* ui : add path navigation and search scope to cwd picker
Treat path-like queries (starting with / or ~) as directory navigation
instead of glob-matching the whole query: search the parent for the last
segment, and descend into an exactly-typed directory by listing its
children. Show the effective search scope in the footer and auto-search
on open so the current directory and its siblings appear immediately.
Assisted-by: Claude
* db : persist per-call tool cwd on tool result messages
* ui : abbreviate tool paths under home with a tilde
* ui : show the per-call cwd on exec shell rows
* ui : clarify the synthetic cwd message for the model
* ui : reuse the trailing cwd row on a repeated pick
* ui : don't jump when a cwd row is injected mid-chat
* chore: Formatting
* refactor: Cleanup comments
* ui : unify working directory naming and add a synthetic-message flag
* ui : render synthetic cwd rows without a scroll jump
* ui : decouple the working directory picker into utils and sub-components
* ui : add get_info tool call block
* chore: Formatting
* refactor: Cleanup
* refactor: Cleanup
* refactor: Cleanup
* fix: UI
* server : harden file_glob_search listing (kind enum, timeout, symlink guard, absolute base)
* ui : use persisted isSynthetic flag for cwd rows, drop legacy formats
* ui : cache picker search, fail visibly on native resolve
* ui : escape glob metacharacters in picker search glob
* ui : simplify auto-scroll pin
* chore: Format
* fix: Use `SvelteMap`
* refactor: Post-review fixes
* ui: accept Windows roots in the working directory picker
recognize a drive root (C:) and a UNC share (//host/share) as path
navigation, alongside the POSIX root and ~, so a query like D:\repos
lists that directory instead of glob-matching it under the home dir
split below the root, so a bare drive resolves to its root rather than
to a drive-relative prefix
rewrite backslashes into forward slashes only when the query carries a
Windows root, since a backslash is a legal POSIX filename character
paths keep travelling with forward slashes, which is what the server
returns and what Windows accepts
---------
Co-authored-by: Pascal <admin@serveurperso.com>
* feat: Add shimmer text animation for processing state indicators
* feat: Redesign CollapsibleContentBlock component with improved UX
* feat: Add conditional setting display support with dependsOn field
* feat: Add showAgenticTurnStats setting for per-turn statistics
* feat: Update ChatMessageAgenticContent with improved UI and new features
* feat: Enhance file read tool UI/UX
* feat: Refine styling of collapsible content and code preview blocks
* feat: add terminal variant to CollapsibleContentBlock
* feat: add built-in tools UI registry
* feat: extract ChatMessageReasoningBlock and ChatMessageToolCallBlock
* refactor: simplify ChatMessageAgenticContent to use extracted blocks
* fix: correct markdown content block margin spacing
* fix: reorganize SettingsChatFields layout and reset button positioning
* fix: use direct map access in agentic store session methods
* refactor: remove reasoning preview/throttle system from CollapsibleContentBlock
* feat: add auto-scroll to reasoning block and remove showThoughtInProgress
* feat: add ChatMessageToolCallDateTime component and support for new tool types
* feat: improve auto-scroll reliability in reasoning block with RAF coalescing and MutationObserver
* feat: show MCP server favicon for tools without a built-in icon
* feat: add search-results parsing utilities and tests
* feat: add ChatMessageToolCallSearchResults component
* feat: integrate search results rendering into ChatMessageAgenticContent
* feat: display tool call input alongside output in ChatMessageToolCallBlock
* style: use muted foreground color in reasoning block content
* chore: Format
* feat: Refine reasoning block layout and make pending thoughts display configurable
* feat: Stream tool call code blocks with auto-scroll and handle partial JSON
* feat: add streaming permission gate infrastructure
* feat: wire permission gate into the agentic loop
* fix: bail out on abort and skip already-approved tool calls
* fix: clear partial tool calls on abort and savePartialResponse
* test: cover partial tool call cleanup end-to-end
* refactor: Remove streaming permission gate logic
* fix: Correct autoscroll and streaming gates for tool calls and reasoning blocks
* refactor: Chat Message Assistant componentization
* fix: Show health metadata for disabled MCP servers and promote connections on enable
* fix: Inherit global enabled state for missing MCP per-chat overrides
* refactor: Cleanup
* refactor: Split ChatMessageToolCallBlock into dedicated components
* feat: Add live streaming and auto-scroll for tool execution output
* feat: Add line numbers and change markers to file edit diffs
* chore: Formatting
* feat: Add type definitions and utilities for recommended MCP servers
* feat: Add recommended MCP servers configuration and storage key
* feat: Add McpServerCardCompact component for recommended servers
* feat: Add recommended servers section to Add New Server dialog
* feat: Update McpServerForm to support authorization requirements
* feat: Add select-none classes for text selection prevention
* feat: Add recommended MCP server icon assets
* refactor: Store dismissed MCP recommendations as a boolean flag
* feat: Render tool results as JSON or Markdown based on detected content type
* feat: UI improvement
* feat: Render search block early and update heading to show execution state
* fix: Prevent non-web-search tools from triggering the search UI block
* refactor: Cleanup
* refactor: Extract hardcoded icon size classes into shared constants
* refactor: Extract hardcoded tool result separator into a shared constant
* refactor: Tool Calls UI/logic
* refactor: Cleanup
* refactor: Cleanup
* refactor: Cleanup
Aleksander Grygier