art with code

2012-09-01

Hi-res trouble

Retina displays, mobile phones, zoomable browsers. Whatever you call it, the days when you could assume a single image pixel to be displayed on a single display pixel are gone.

Nowadays a normal web site has images that are too big for mobiles and too small for hi-res desktops. The mobile shows your 800x600 image on 588x441 display pixels and the hi-res desktop shows it on 1600x1200 display pixels.

And it's not just images. The <canvas> element suffers from the same issues. Safari renders the <canvas> at display resolution and scales down when zooming out, Chrome uses a layout resolution <canvas> that gets scaled to display resolution. By comparison, SVG scales gracefully.

The issue with <canvas> is that if you think of it as a pixel drawing surface, you think layout pixels should be drawing surface pixels. If you think of it as a vector drawing surface with some pixel manipulation commands, you think display pixels should be drawing surface pixels.

What I'd like to have as a web developer is a way to not care. I'd just make my website look good on tablets and desktops using <img src="foo.bar"> like always and use getImageDataHD for hi-res <canvas> pixel manipulation. For mobiles I'd have to make a custom UI as always, but I'd like to use the same photos as on the large-screen site.

Technology-wise I'd want the browser to load enough of each image to fill all the display pixels with at least one image pixel. And stop loading there. For huge images, I'd like the browser to load only the portion of the image that's visible on the screen.

And this all should happen with a single HTTP request per image, have no bandwidth overhead and require no server-side support.

The single request requirement means that either the image element or the image filename need to contain enough information to load the image at the optimal size. The no bandwidth overhead requirement means that the loaded file should contain only the optimal size image. The requirement for no server-side support means that the image should be a static file or a directory.

The <picture> element and <img> srcset-attribute are ways to add more information to the image element. The problem with them is that I want to use my regular <img src="foo.bar"> and not type more stuff.</picture>

Making a custom image file format with multiple image sizes would get you <img src="foo.bar">. On the downside you either end up with bandwidth overhead from loading several versions of the image, or require two HTTP requests: one to load the list of image sizes, second to load the wanted image.

With server-side support the browser can send display resolution in the request and the server will pick the appropriate image to send back. That needs server-side support though.

If you have the image stored as a directory with multiple sizes of the image and tiled versions for large images, and you name the image versions by their display resolution, the browser can directly request the optimal resolution image. By having the image name contain the maximum image size, the browser can limit itself to loading only up to that resolution. For backward compatibility, the <img> src-attribute can point to a valid image inside the directory. If you implement that in JavaScript you end up with an extra request for the <img> src-attribute though.


About Me

My photo

Built art installations, web sites, graphics libraries, web browsers, mobile apps, desktop apps, media player themes, many nutty prototypes, much bad code, much bad art.

Have freelanced for Verizon, Google, Mozilla, Warner Bros, Sony Pictures, Yahoo!, Microsoft, Valve Software, TDK Electronics.

Ex-Chrome Developer Relations.