Allow an image to indicate its own density and correct its intrinsic size #5574

noamr · 2020-05-24T18:41:03Z

…size.

See https://discourse.wicg.io/t/proposal-exif-image-resolution-auto-and-from-image/4326/24

This proposal allows using EXIF or other metadata to determine an image's default resolution, to be applied regardless of the resolution specified by srcset / image-set. This resolution is written with image-resolution in mind, with image-resolution: from-image representing this EXIF image resolution, similar to how image-orientation: from-image represents EXIF orientation.

The use-case is, for example, allowing serving low-density placeholder in poor network conditions without affecting layout.

The change to the HTML spec defines how resolution is expressed in EXIF, and it would take more work to define it in other image metadata formats such as XMP.

To mitigate backwards compatibility issues, the spec change requires five different metadata attributes to be present and valid - horizontal resolution, vertical resolution, preferred width, preferred height, and resolution unit. It was confirmed using Chrome histogram that this would affect a very small percentage of websites (see conversation on PR).

At least two implementers are interested (and none opposed):
- Already implemented in Chromium, Safari and Firefox. See WPT Dashboard.
Tests are written:
- https://github.com/web-platform-tests/wpt/tree/master/density-size-correction
Implementation bugs are filed:
- Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1069755
- Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1680387
- Safari: https://bugs.webkit.org/show_bug.cgi?id=212405

(See WHATWG Working Mode: Changes for more details.)

/acknowledgements.html ( diff )
/images.html ( diff )
/index.html ( diff )
/infrastructure.html ( diff )

annevk · 2020-05-26T09:37:29Z

Where would the proposal be standardized?

noamr · 2020-05-26T09:45:21Z

Where would the proposal be standardized?

I don't understand the question. Isn't here where proposals are standardized?

noamr · 2020-05-26T09:46:25Z

(Filing this after a conversation with @tabatkins and with @yoavweiss, where Tab had proposed to file this as an HTML PR).
After incubation on WICG (https://discourse.wicg.io/t/proposal-exif-image-resolution-auto-and-from-image/4326/24)
Working in parallel on Chromium/WebKit implementations and on WPT, but wanted to see how people feel about the proposal first.

annevk · 2020-05-26T09:55:12Z

Yes, HTML is such a place, but you're not adding the specifics here, right? HTML cannot forever depend on something only defined in a proposal. Some standard needs to eventually own that EXIF data definition.

noamr · 2020-05-26T10:01:24Z

Yes, HTML is such a place, but you're not adding the specifics here, right? HTML cannot forever depend on something only defined in a proposal. Some standard needs to eventually own that EXIF data definition.

Agreed. The EXIF data is already standardized (resolution/size). The use of these properties in practice is specific to HTML, same as for EXIF orientation. I'd be happy to put that definition in the HTML spec or in any other place - probably wherever image-orientation EXIF should reside. Any suggestions?

noamr · 2020-05-26T10:04:47Z

The EXIF standard is here: http://cipa.jp/std/documents/download_e.html?DC-008-Translation-2019-E. However it's not really a living standard. It contains the definitions of size/resolution etc, but as said before, not the choice to abide to them only if they match, which is HTML specific.

annevk · 2020-05-26T10:08:08Z

I guess at some point HTML should own it or we should split out a lot of the image infrastructure into its own standard. CSS and everyone else using images are also expected to use HTML's algorithms to obtain images. (See #4474.)

noamr · 2020-05-26T10:17:39Z

So should I add that in as an algorithm to HTML as part of this patch? What would be the way forward?

annevk · 2020-05-26T10:28:21Z

Yeah, that seems preferable if that's all there is to it.

noamr · 2020-05-26T13:11:46Z

Yeah, that seems preferable if that's all there is to it.

Available in new PR changes.

noamr · 2020-05-27T08:37:55Z

@annevk: updated the PR with detailed EXIF algorithm, as per our IRC discussion

These test include images that have EXIF values that specify density-corrected size And they assert that the intrinsic size for those images is corrected by that EXIF. WhatWG PR: whatwg/html#5574

source

See whatwg/html#5574

…rigin images, a=testonly Automatic update from web-platform-tests Test density size correction for cross-origin images See whatwg/html#5574 -- wpt-commits: 2c19d6ee62676ac90146e7311ba13d8eaeb241fd wpt-pr: 29411

yoavweiss · 2021-07-08T08:32:13Z

@domenic - any other blockers here?

…and other issues

domenic · 2021-07-08T18:38:53Z

Are there tests that only EXIF is supported, and other formats like XMP (which aren't in the spec) don't work?

eeeps · 2021-07-08T21:16:15Z

@domenic no, but I'm not sure ① how one would create such a file, ② if there are any other metadata formats besides XMP that contain fields which map to the EXIF ResolutionUnit, XResolution, YResolution, PixelXDimension, and PixelYDimension fields, used here. I've asked the exiftool folks.

noamr · 2021-07-11T18:37:24Z

@domenic no, but I'm not sure ① how one would create such a file, ② if there are any other metadata formats besides XMP that contain fields which map to the EXIF ResolutionUnit, XResolution, YResolution, PixelXDimension, and PixelYDimension fields, used here. I've asked the exiftool folks.

Arguably, Exif embedded in XMP is still Exif. The spec doesn't limit to Binary EXIF, and I'm not sure it should go further than it does into image-codec specifics at this point in time. The line "Let exifTagMap be the EXIF tags obtained from req's image data, as defined by the relevant codec." leaves it to the codec to decide whether EXIF is binary or XMP-embedded.

The Mac/WebKit implementation supports Exif embedded in XMP in PNG images, as the CF image decoders (Mac OS APIs) do that by default. I think we should add tests for that, which would currently pass on Mac, rather than add tests to assert that Exif embedded in XMP doesn't work.

lrosenthol · 2021-07-12T09:19:51Z

Arguably, Exif embedded in XMP is still Exif. The spec doesn't limit to Binary EXIF, and I'm not sure it should go further than it does into image-codec specifics at this point in time.

Correct. That was the agreement early on in this thread/discussion - that how & where a processor locates the EXIF was undefined. One reason for doing so is that the location of EXIF data varies from file format to file format as noted by @noamr above.

…rigin images, a=testonly Automatic update from web-platform-tests Test density size correction for cross-origin images See whatwg/html#5574 -- wpt-commits: 2c19d6ee62676ac90146e7311ba13d8eaeb241fd wpt-pr: 29411

svgeesus · 2021-10-16T18:36:39Z

One reason for doing so is that the location of EXIF data varies from file format to file format as noted by @noamr above.

Note that PNG used not to have a standard way to embed EXIF (hence people putting it in XMP) but then eXIfwas added to the extension chunks register and now, there is an issue to add it directly to the PNG spec.

As part of that, it would be good to point to the HTML LS on how browsers handle the EXIF data.

See whatwg/html#5574 Currently applies to JPEG only. Use the EXIF resolution/size information we read from the JPEG image to determine a density-corrected preferred size, which overrides the intrinsic size of the image. We only apply this intrinsic size when an image has both resolution and size information in its exif. When asking an Image for display size, the preferred size is returned, and the source rect for reading from the image is scaled when applicable. Added many WPT reference tests. Bug: 1069755 Change-Id: I01a38c72bb46974a0c112175934cd3c2c48607a9 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2292219 Commit-Queue: Stephen Chenney <[email protected]> Reviewed-by: Stephen Chenney <[email protected]> Cr-Commit-Position: refs/heads/master@{#817600} GitOrigin-RevId: 86ae1272b4a60b42439bc115777bb916fa3a6720

Using wpt-import in Chromium 1ca167279b1e149a0fa86adc0102e8f3be8ae343. With Chromium commits locally applied on WPT: 7c0edf3443 "Add test for crbug.com/1138917" 86ae1272b4 "Read image resolution from EXIF, apply to intrinsic size See whatwg/html#5574" Note to sheriffs: This CL imports external tests and adds expectations for those tests; if this CL is large and causes a few new failures, please fix the failures by adding new lines to TestExpectations rather than reverting. See: https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md Directory owners for changes in this CL: [email protected]: external/wpt/resources NOAUTOREVERT=true [email protected] No-Export: true Cq-Include-Trybots: luci.chromium.try:linux-wpt-identity-fyi-rel,linux-wpt-payments-fyi-rel Change-Id: Iaf79b62b9fcf5f5ceaeb388b2bb3f741a16ccca8 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2477373 Reviewed-by: WPT Autoroller <[email protected]> Commit-Queue: WPT Autoroller <[email protected]> Cr-Commit-Position: refs/heads/master@{#817734} GitOrigin-RevId: 63931a18b27df4e94ebec891d27cc34b7fe46194

noamr force-pushed the intrinsic-density-correction branch from 2ace940 to 238836b Compare May 25, 2020 05:43

annevk added addition/proposal New features or enhancements needs implementer interest Moving the issue forward requires implementers to express interest topic: img labels May 26, 2020

noamr force-pushed the intrinsic-density-correction branch from 238836b to 0f651cb Compare May 26, 2020 13:10

noamr force-pushed the intrinsic-density-correction branch 6 times, most recently from d5179c1 to 75383fa Compare May 27, 2020 08:14

noamr force-pushed the intrinsic-density-correction branch 4 times, most recently from 5dc5560 to 6a6a6d8 Compare May 27, 2020 10:12

noamr mentioned this pull request May 27, 2020

Add tests for density-corrected size web-platform-tests/wpt#23789

Closed

annevk reviewed May 28, 2020

View reviewed changes

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

noamr and others added 9 commits June 27, 2021 07:31

Nit req/img

cd6d4db

Declare an img pointing to img's req

84fce1d

Use better terms for cross-origin

71af446

More nits

fc718c4

Fix typo px -> pt

00b5724

Fix source formatting

9f5e047

Apply review comments

98ccb1d

Apply review comments

550cd15

CR comments

da9f571

noamr force-pushed the intrinsic-density-correction branch from 7fe14ad to da9f571 Compare June 27, 2021 04:33

zcorpan added a commit to web-platform-tests/wpt that referenced this pull request Jun 29, 2021

Test density size correction for cross-origin images

2c19d6e

See whatwg/html#5574

Fix wrong variable references (img does not exist at the call sites) …

441ca72

…and other issues

domenic approved these changes Jul 8, 2021

View reviewed changes

domenic merged commit 355a0ca into whatwg:main Jul 8, 2021

hamishwillee mentioned this pull request Jul 12, 2021

FF90: Add info about EXIF intrinsic size (basic) mdn/content#6798

Merged

hamishwillee mentioned this pull request Jul 13, 2021

More information on how EXIF image sizing actually works mdn/content#6836

Open

nyanpasu64 mentioned this pull request Jul 15, 2021

Encoding display DPI information in images, so they show at the correct size in browsers? ShareX/ShareX#5702

Closed

eeeps mentioned this pull request Jul 17, 2021

Bits per pixel calculation is not intrinsic-density-aware GoogleChrome/lighthouse#12729

Closed

svgeesus mentioned this pull request May 24, 2023

Natural image resolution, CSS. web compat, and pHYs w3c/png#97

Open

jonsneyers mentioned this pull request Apr 29, 2024

cjpegli: Pass on the DPI resolution into a JFIF marker libjxl/libjxl#3533

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow an image to indicate its own density and correct its intrinsic size #5574

Allow an image to indicate its own density and correct its intrinsic size #5574

noamr commented May 24, 2020 •

edited by pr-preview bot

Loading

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 26, 2020 •

edited

Loading

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 26, 2020

annevk commented May 26, 2020 •

edited

Loading

noamr commented May 26, 2020

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 27, 2020

yoavweiss commented Jul 8, 2021

domenic commented Jul 8, 2021

eeeps commented Jul 8, 2021 •

edited

Loading

noamr commented Jul 11, 2021 •

edited

Loading

lrosenthol commented Jul 12, 2021

svgeesus commented Oct 16, 2021

Allow an image to indicate its own density and correct its intrinsic size #5574

Allow an image to indicate its own density and correct its intrinsic size #5574

Conversation

noamr commented May 24, 2020 • edited by pr-preview bot Loading

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 26, 2020 • edited Loading

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 26, 2020

annevk commented May 26, 2020 • edited Loading

noamr commented May 26, 2020

annevk commented May 26, 2020

noamr commented May 26, 2020

noamr commented May 27, 2020

yoavweiss commented Jul 8, 2021

domenic commented Jul 8, 2021

eeeps commented Jul 8, 2021 • edited Loading

noamr commented Jul 11, 2021 • edited Loading

lrosenthol commented Jul 12, 2021

svgeesus commented Oct 16, 2021

noamr commented May 24, 2020 •

edited by pr-preview bot

Loading

noamr commented May 26, 2020 •

edited

Loading

annevk commented May 26, 2020 •

edited

Loading

eeeps commented Jul 8, 2021 •

edited

Loading

noamr commented Jul 11, 2021 •

edited

Loading