UI/jpeg exif orientation (#24196)

* ui: bake jpeg exif orientation into uploaded images

stb_image in mtmd ignores exif metadata, so rotated smartphone photos
reach the model with raw pixel orientation. The webui now reads the
exif orientation tag at send time and feeds it into the existing
capImageDataURLSize canvas pass: the browser applies the rotation when
decoding, so capped images come out upright for free, and images under
the cap threshold get a single plain redraw when orientation > 1.

At most one re-encode ever happens per image. Upright jpegs with
capping disabled pass through untouched, bit perfect.

Adds jpeg-orientation.ts with a minimal exif parser working on a
bounded base64 prefix (both endianness, returns 1 on any malformed
input) and unit tests against handcrafted jpeg byte streams.

* ui: move jpeg exif constants into lib/constants

* ui: add browser test for jpeg orientation and capping

Covers capImageDataURLSize end to end in chromium with real Pillow
generated jpeg fixtures across exif orientations 1/3/5/6/8: upright
quadrant colors checked pixel-wise, expected dimensions with and
without capping, no orientation tag left in the output, and strict
passthrough when nothing needs rewriting.
This commit is contained in:
Pascal
2026-06-12 10:20:27 +02:00
committed by GitHub
parent 88a39274ec
commit 6471e3c090
6 changed files with 419 additions and 6 deletions
+5 -5
View File
@@ -34,7 +34,6 @@ import type {
import { modelsStore } from '$lib/stores/models.svelte';
import { settingsStore } from '../stores/settings.svelte';
import { capImageDataURLSize } from '../utils/cap-img-size';
import { MEGAPIXELS_TO_PIXELS } from '$lib/constants/image-size';
function getAudioInputFormat(mimeType: string): AudioInputFormat {
const normalizedMimeType = mimeType.trim().toLowerCase();
@@ -961,10 +960,11 @@ export class ChatService {
for (const image of imageFiles) {
const maxImageResolution = settingsStore.getConfig(SETTINGS_KEYS.MAX_IMAGE_RESOLUTION);
let base64Url = image.base64Url;
if (maxImageResolution > 1 / MEGAPIXELS_TO_PIXELS) {
base64Url = await capImageDataURLSize(image.base64Url, maxImageResolution);
}
// Caps the resolution and bakes the jpeg exif orientation in one pass,
// untouched images pass through as is
const base64Url = await capImageDataURLSize(image.base64Url, maxImageResolution);
contentParts.push({
type: ContentPartType.IMAGE_URL,
image_url: { url: base64Url }