Apple has often been accused of acting like it invented things that others have been doing for years. That complaint is not without merit, however Apple can lay claim to transforming existing things into mainstream successes, which takes no small amount of invention in its own right. Fingerprint authentication and contactless payments are just two recent examples, having both existed in Japan and on niche devices for over a decade before Apple raised them to global prominence with the iPhone.
Next up on Apple’s agenda is augmented reality, the act of superimposing digital data and visuals atop a live video feed of your surroundings — something that Google, Microsoft, and many others have been experimenting with for a long time. Apple is far from being able to claim it invented AR, but its new ARKit in iOS 11 is already showing signs to suggest that Apple will help bring AR into the mainstream faster and better than anyone else.
ARKit, Apple’s augmented reality (AR) technology, delivers immersive, engaging experiences that seamlessly blend virtual objects with the real world. In AR apps, the device’s camera presents a live, onscreen view of the physical world. Three-dimensional virtual objects are superimposed over this view, creating the illusion that they actually exist. The user can reorient their device to explore the objects from different angles and, if appropriate for the experience, interact with objects using gestures and movement
Designing an Engaging Experience
Use the entire display. Devote as much of the screen as possible to viewing and exploring the physical world and your app’s virtual objects. Avoid cluttering the screen with controls and information that diminish the immersive experience.
Create convincing illusions when placing realistic objects. Not all AR experiences require realistic virtual objects. Those that do should include objects that appear to inhabit the physical environment in which they’re placed. For best results, design detailed 3D assets with lifelike textures. Use the information ARKit provides to position objects on detected real-world surfaces, scale objects properly, reflect environmental lighting conditions on virtual objects, cast virtual object shadows on real-world surfaces, and update visuals as the camera’s position changes.
Anticipate that people will use your app in environments that aren’t optimal for AR. People may open your app in a location where there isn’t much room to move around or there aren’t large, flat surface areas. Try to anticipate scenarios that present challenges, and clearly communicate requirements or expectations to people up front. Consider offering varying sets of features for use in different environments.
Be mindful of the user’s comfort. Holding a device at a certain distance or angle for a prolonged period can be fatiguing. Consider how people must hold their device when using your app, and strive for an enjoyable experience that doesn’t cause discomfort. For example, by default, you could place objects at a distance that reduces the need to move the device closer to the object. A game could keep levels short and intermixed with brief periods of downtime.
If your app encourages user motion, introduce it gradually. In a game, the user shouldn’t need to move out of the way to avoid a virtual projectile as soon as they enter AR. Give them time to adapt to the experience first. Then, progressively encourage movement.
Be mindful of the user’s safety. Moving around too much can be dangerous if other people or objects are nearby. Consider ways of making your app safe to operate. A game could avoid encouraging large or sudden movements.
Use audio and haptic feedback to enhance the immersive experience. A sound effect or bump sensation is a great way to confirm that a virtual object has made contact with a physical surface or other virtual object. In an immersive game, background music can help envelop the user in the virtual world. For related guidance, see Audio and Haptic Feedback.
Wherever possible, provide hints in context. Placing a three-dimensional rotation indicator around an object, for example, is more intuitive than presenting text-based instructions in an overlay. Textual overlay hints may be warranted prior to surface detection, however, or if the user isn’t responding to contextual hints.
If you must display instructional text, use approachable terminology. AR is an advanced concept that may be intimidating to some users. To help make it approachable, avoid referring to technical, developer-oriented terms like ARKit, world detection, and tracking. Instead, use friendly, conversational terms that most people will understand.
Apple’s AR will immediately reach millions of people who already have the requisite hardware.
Considering how little time it took to develop two convincingly accurate AR measuring apps with the iOS 11 beta, and reading the comments from their makers, Apple also appears to have an advantage in the ease of development with ARKit.
For the people who don’t care about incremental changes in phone specs or design, the differentiator between devices has always been in the unique things that each one can do — or, failing that, the convenience and ease of use of common features.