Chasing Pixels - Finding Gems - Home
Seeing in Augmented Reality is Hard
JUN 09, 2017 18:18 PM
A+ A A-

Seeing in Augmented Reality is Hard

Smart glasses (or smartglasses, the industry can’t make up its mind on how to refer to them) are wearable computer glasses that add information alongside or in front of what the wearer is looking at. I believe we will have consumer quality (not obnoxious attention calling, geeky looking) augmented reality glasses, smart glasses, by 2020. But it’s not going to be easy getting there, which is why some of my friends (well, they’re my friends now) have bets with me on the date.
Augmented reality places much heavier demands on the display than does VR.
Wide range illumination. An augmented reality display has to work in low light indoors (and not blind you), and in bright light outdoors (and be clearly seen). In fact, in the taxonomy of augmented reality (which includes everything from HUDs to helmets, to smart glasses), there is the differentiation of indoors vs outdoors. We could call this high-dynamic illumination HDI to be differentiated from high-dynamic range (HDR) which is about contrast. [1]
Positional accuracy. If your augmented reality glasses are calling your attention to a coffee shop, the sign or logo of that coffee shop needs to be in close proximity to its actual location. That means the logo/sign has to float within the image as you move your head, which you do a lot walking, or driving. In simpler smart glasses, the images are basic directions (left arrow, right arrow, etc., and a bit of text), and the image area stays fixed (typically in in the left or right edge of the glasses).
Addressable occlusion. When that coffee shop logo pops up, can you see through it, or is it opaque? That’s known as addressable occlusion, where the augmented image from the image generator has a synchronous and geometric correlation to a segmented part (or parts) of the image on display which blocks out light directly behind the augmented imagery. Otherwise the image is ghosted (semi-transparent). Think about Pokémon—those characters are opaque, whereas text could be semi-transparent so as to not obstruct your view (think of a HUD in a car’s wind shield).
Depth of field. And where in the world, literally, is that coffee shop logo—is it close, or far away, and can you tell? The system needs to have an adjustable depth of field for the augmented reality image to correlate to the real world. If that isn’t done there will be an accommodation-convergence conflict. Visual fatigue and discomfort occur as the viewer attempts to adjust vergence and accommodation appropriately. (Vergence is the simultaneous movement of both eyes in opposite directions to obtain or maintain single binocular vision.)
Latency. Researchers have found that for virtual reality (where the observer cannot see the normal world), the end-to-end latency (or, perhaps more descriptively the difference between perception to motion) should be below 40 ms. For augmented reality, the requirements are even higher. The displacement of objects between two frames should not exceed 15 arcmin (0.25 degrees), which would require a maximal latency of 5 ms even when one rotates their head at a moderate speed of 50 degrees-per-second. Most researchers suggest that 10 ms will be acceptable for augmented reality.


And more. There are several other issues that exist in augmented reality (that don’t in VR) such as eye box, field of view, pixel pitch, proximity, and refresh rate which I’ll go into in another posting, and discuss in my book. Suffice it to say, augmented reality is possibly the most challenging display technology we have faced, maybe ever. 
We can experience augmented reality today with a smartphone (if you haven’t done this yet, I strongly encourage you to try it—there are several free apps available). In the industrial and scientific segment, you can experience augmented reality if you attend any conferences (Siggraph is coming up BTW).
Once you do try augmented reality you will immediately be struck by the difficulty of managing the image over the real world. You will also be delighted and have your imagination lit up as you realize the potential for augmented reality—seeing is truly believing in this case.

[1] Pedantic note, there is an acronym—HDRI, which stand for high-dynamic range illumination which refers to HDR in ray tracing.

[%= name %]
[%= createDate %]
[%= comment %]
Share this:
Please login to enter a comment: