Sunday, January 16, 2011

The Problem with a "Retina" Display on the iPad 2: Lack of Content

Update: This post was mentioned on Crunchgear and AppleInsider, and it seems many of the commenters didn't understand the point of the original post.

At this point I wish I'd never mentioned video in the post (or had relegated it to less than a paragraph).

My point was not that you won't be able to watch DVD resolution content on some future 7,680 x 4,320 pixel HD TV (1080p x 16) - the point was that additional resolution is only worth paying for if you're going to use it (and to answer the question of many commenters - yes, I have heard of upscaling video).

Of course user interface elements and text will look amazing on the super high-res iPad 2 (or iPad 3). But almost no content (forget about video for a second) is currently created with a 3 megapixel, 260 dpi display in mind. This is especially true of most graphics and photos used on the web. So, the home page of the New York Times will have print-quality text (in Safari for iPad 2) while all the rest of the graphics will be of much lower resolution and have to be upscaled (the main home page image is only 330 pixels wide). The effect will be a bit jarring. On the current iPhone 4 I can tell at a glance that an app hasn't been updated for the new retina display - the app icon even looks out of place on the home screen surrounded by high-res icons from other apps.

And yes, yes, I know that it's possible to use the web on a 30" monitor that has a resolution greater than this hypothetical iPad 2. But this works because a 27" iMac (for example) uses the additional pixels to create a physically larger display, not to increase the pixel density. Graphics that were designed on a 20" monitor and assumed a ~ 100 dpi display look fine on a 27" monitor - they are the same physical size on screen and are not pixel-doubled.

And what about iOS apps? I write offline travel apps for the iPhone, and because they don't require an internet connection, I include all the images in the app. The next update of my Prague guide will contain more than 10,000 images (at 300x200). Can I use these in a universal app that runs on an iPad 2? Sure, but the result won't be very satisfactory - blurry, massively upscaled images on the same screen as razor-sharp text and UI elements just won't look good. I'll be forced to include much higher resolution images in the app, and that will mean leaving most of the images out of the app entirely (I can't expect users to download a 5GB travel guide).

So, again, yes - I do understand the concept of upscaling. The point wasn't that you can't use the current web or a 2011 iOS app on the 2020 gigapixel 100" wall-mounted iPad 11. The point was that users are going to expect all the elements on screen to be of the same resolution, and that means the creation of a massive amount of new content (thus the "Lack of Content" part of my post title).

And did I mention that I understand the concept of upscaling video? Because I do.

The original post:

The iPad 2 will be announced soon, and thus there are a lot of rumors floating around as to the second-generation iPad's technical specs. Some of the new features are almost inevitable (front and rear cameras, more RAM) and some are less likely (SD card slot, new processor). But the real wildcard is the iPad 2's display. Both the iPod Touch and iPhone got 960x640 "retina" (i.e. more than 300dpi) displays in their latest versions, and there is a growing consensus on gadget blogs that the iPad 2 will have a similar "ultra high-res" display. Even John Gruber of Daring Fireball isn't dismissing the possibility of a 2048x1536 display on the iPad 2. We're talking about an iPad with nearly the resolution of a 30" desktop display.

Let's assume for the sake of argument that all of the usual problems with such a display have been solved (battery life, cost, speed, etc.). Even if the iPad 2 comes out in a 16GB WiFi version for $499 with a 2048x1536 display (four times the pixels of the original iPad, which is 1024x768), where is the content going to come from?

Take a look at the graphic I created showing the relative sizes of popular video formats in relation to this theoretical iPad 2 "retina" display. Forget about DVD (Standard Definition) video, which if displayed at native resolution would be little bigger than a postage stamp on this new display. 720p HD video (the highest resolution sold on iTunes) would fill only a quarter of the screen, and 1080p video would have black bars not only along the top and bottom, but also along the sides of the screen. Of course, Apple will scale up these videos to fill the screen, but most people don't expect pixel-doubling when watching HD video.

And what about on-line photos? Are web site owners now expected to post two or three megapixel photos on their sites to support Safari on iPad? Will the Photos application for iPad import full resolution photos when syncing to your computer? How big will games be if they fully support the 3,145,728 pixels of such a display?

I'm not arguing that Apple won't include such a display on the iPad 2, but if they do, I hope there's a 128GB version so that I can load more than one movie on it at a time (assuming a few apps, some photos, some music, and 1080p video, the 16GB version would hold about one 90 minute film).

I guess my next purchase is a RED camera to shoot home videos that will support the iPad 2's display.


Anonymous said...

720p video on a 9.7" 2048x1536 display will look identical to 720p video on a 9.7" 1024x768 display. Because both displays are 9.7".

Unknown said...

Great post. Also, have you ever downloaded a movie to the iPad? It would take an eternity to download a movie of this resolution to the iPad. Most movies are 3-4gb, and you would be very lucky over WiFi to get a DVD quality movie in 2 hours. In most cases it takes me >4 hours which is why I haven't done it very often. It might take less time to shoot the film than download if the resolution was at 2048x1536.

Hamranhansenhansen said...

> But almost no content (forget about
> video for a second) is currently created
> with a 3 megapixel, 260 dpi display in mind

That is fundamentally untrue. Other than video, ALL content is made for that display. We just think of it as "print" content. Books, magazines, business correspondence, photographic prints, and anything that comes out of a printer match that resolution. Even a lowly business letter has more and smaller dots on it than an iPad with Retina Display. Safari/WebKit will happily render the Web at that resolution.

When a screen gets up into 300 dpi territory, you stop treating it like a screen device (authoring in pixels) and start treating it like a print device (authoring in inches). iOS devices encourage this because they have fixed screen sizes in inches, so they are akin to a paper size:

• "Letter" 8.5x11 300 dpi
• "Tabloid" 11x17 300 dpi
• "iPhone" 2x3 160/320 dpi
• "iPad" 6x8 130/260 dpi

The truth is, if we were to conceptualize an electronic device to replace print, we would start with a 300 dpi display as the fundamental feature, because that is a fundamental "magic number" in print. Most artwork is made at that resolution, the resolution ruler is marked at multiples of 300, and it fools a 20/20 eye at typical reading distance into not being able to see the dots. The lower-resolution displays in iPhone original/3G/3GS and iPad 3G (and any future iPads with low-res display) are hacks … stepping stones to the Retina Displays that the devices are "supposed" to have to fulfill their function as paper replacements. It's a very elegant compromise because it is short-term and there is almost no pain to transition from the low-res to high-res screen, either for users or developers. The high-res screen emulates the low-res screen (4 pixels in place of 1) from day one and then apps gradually take advantage of the enhanced sharpness that has been made available to them. Soon, the low-res devices are retired as part of the regular product life cycle.

So an iPad with Retina Display is REQUIRED by book, magazine, and other "print" content. It's required for photography, where people are commonly making 12 megapixel or 18 megapixel images that have way more dots than an iPad with Retina Display. And even for video, always the lowest-res content, HD is now old hat. The next size up is "4K" (4000 pixels horizontally), and it is commonly made now, and is even supported by YouTube already. That has to be downscaled for iPad with Retina Display.

The only question on this is what year it will arrive. Can they get it done in the 2011 iPad, or 2012? Whenever it arrives, the content will exploit it immediately.

Anonymous said...

> The next update of my Prague guide
> will contain more than 10,000 images
> (at 300x200). Can I use these in a
> universal app that runs on an iPad 2?
> Sure, but the result won't be very
> satisfactory - blurry, massively upscaled
> images

The images will look IDENTICAL on all iPads, whether they have original or Retina Display. That is the key feature of doubling the resolution. What is shown in 1 pixel on the original display will be shown in 4 pixels that are each 25% of the size on the Retina Display. Because the Retina Display has exactly 4x the pixels in the same space, it can emulate the previous display by using 4 pixels in place of every 1.

If you run an iPhone app that has not been updated for Retina Display side-by-side on iPhone 3GS and 4, the app looks the same. Every 1 pixel of the app is rendered by 1 pixel of the iPhone 3GS, and every 1 pixel of the app is rendered by 4 pixels of the iPhone 4, but because the iPhone 4 pixels are 25% of the size of iPhone 3GS pixels, the result looks the same. So the additional sharpness is an optional feature for developers. That is why the bigger display on iPhone did not break apps.

In the future, you may switch to vector maps, and your storage would go down, and yet you could render at full resolution on iPad with Retina Display. CPU usage would go up, but it might be worth it for razor sharp maps.