Lumpiasty/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

UI/jpeg exif orientation (#24196)

Browse Source 
* ui: bake jpeg exif orientation into uploaded images

stb_image in mtmd ignores exif metadata, so rotated smartphone photos
reach the model with raw pixel orientation. The webui now reads the
exif orientation tag at send time and feeds it into the existing
capImageDataURLSize canvas pass: the browser applies the rotation when
decoding, so capped images come out upright for free, and images under
the cap threshold get a single plain redraw when orientation > 1.

At most one re-encode ever happens per image. Upright jpegs with
capping disabled pass through untouched, bit perfect.

Adds jpeg-orientation.ts with a minimal exif parser working on a
bounded base64 prefix (both endianness, returns 1 on any malformed
input) and unit tests against handcrafted jpeg byte streams.

* ui: move jpeg exif constants into lib/constants

* ui: add browser test for jpeg orientation and capping

Covers capImageDataURLSize end to end in chromium with real Pillow
generated jpeg fixtures across exif orientations 1/3/5/6/8: upright
quadrant colors checked pixel-wise, expected dimensions with and
without capping, no orientation tag left in the output, and strict
passthrough when nothing needs rewriting.
This commit is contained in:
Pascal
2026-06-12 10:20:27 +02:00
committed by GitHub
parent 88a39274ec
commit 6471e3c090
6 changed files with 419 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/ui/src/lib/utils/cap-img-size.ts
+17 -1
View File
@@ -1,11 +1,19 @@
import { MEGAPIXELS_TO_PIXELS } from '$lib/constants/image-size';
import { BASE64_IMAGE_URI_REGEX } from '$lib/constants/uri-template';
import { getJpegOrientationFromDataURL, isJpegMimeType } from './jpeg-orientation';
import { MimeTypeImage } from '$lib/enums';
/**
* Converts an Image base64 data URL to another Image data URL with capped dimensions to reduce file size.
*
* For JPEGs the EXIF orientation is baked into the pixels in the same canvas
* pass, the browser applies the rotation when decoding so naturalWidth and
* naturalHeight already describe the upright image. Backends decoding with
* stb_image ignore EXIF, see ggml-org/llama.cpp#20870. Images that need
* neither capping nor rotation pass through untouched, so at most one
* re-encode ever happens.
* @param base64UrlImage - The Image base64 data URL to convert
* @param maxMegapixels - The maximum image size in megapixels for the output Image
* @param maxMegapixels - The maximum image size in megapixels for the output Image, 0 disables capping
* @returns Promise resolving to Image data URL
*/
export function capImageDataURLSize(
@@ -26,6 +34,10 @@ export function capImageDataURLSize(
return reject(new Error(`Unsupported image MIME type: ${mimeType}`));
}
const orientation = isJpegMimeType(mimeType)
? getJpegOrientationFromDataURL(base64UrlImage)
: 1;
const img = new Image();
img.onload = () => {
@@ -46,6 +58,10 @@ export function capImageDataURLSize(
const scaleFactor = Math.sqrt(maxPixels / totalPixels);
canvas.width = Math.floor(targetWidth * scaleFactor);
canvas.height = Math.floor(targetHeight * scaleFactor);
} else if (orientation > 1) {
// No capping needed but the pixels still need the rotation baked in
canvas.width = targetWidth;
canvas.height = targetHeight;
} else {
return resolve(base64UrlImage);
}