Over the past several years, augmented reality (AR) technology has established a home in entertainment, marketing, education and many other industries. The use of AR apps in the enterprise will grow to $2.4 billion in 2019. On the flip side, augmented reality also brings a lot of challenges for designers. Today most experienced designers have got skills in designing web and mobile apps, but these skills aren’t always applicable for immersive AR experiences.

This article will look at how AR is affecting UX, and how UX designers can rise to the challenge of designing engaging augmented user interfaces.

What is AR?

Augmented Reality refers to technology that uses real-time inputs to create an output that combines both real-world data and some programmed elements. AR adds a programmed layer over actual reality to create a third, dynamic level of augmented experience. With AR apps, instead of just seeing information, users interact with it and receive live feedback on the action they have performed.

AR apps are already thriving in the Android and iOS ecosystems right on our smartphones and tablets. Examples of AR that the majority of users have at least heard of, if not used, are things like:

Pokémon Go: Players can collect game characters that can be uncovered by moving in the real world.

SnapChat lenses: SnapChat uses facial-recognition technology to enable users to enhance images with computer-generated effects.

Snapchat’s lenses feature

Microsoft HoloLens: Using tools like Microsoft’s HoloLens it’s possible to see and interact with complex models such as 3D model of a human heart.

Microsoft HoloLens

How To Design for AR

The field of designing AR user experiences is still in its infancy, and since there are no established UX best practices for it yet, I’d like to share my own personal approach to UX in AR apps…

1. AR Use-Case Needs to be Evaluated

The concept of “measure twice, cut once” is especially important in building AR apps. Before diving in, it’s important to have a clear answer on the question “What do I want to achieve with this AR app?” Your ultimate goal is to ensure that the AR experience is right for the project. Keep in mind the following:

• AR experiences should be tied to clear business and user objectives. You shouldn’t create an AR app just because it’s trendy—that’s almost a sure way to create a poor UX. Rather, the desired functionality needs to be evaluated to fit with the experience that the AR display medium can offer.
• As always, good user experience only comes from close attention to users’ needs. This means that if you’re going to design an AR experience, you should invest heavily in user research. Spend some time really getting feedback from your target audience, get to know how they do something in the real world without any kind of device, and how AR can help them do it better.

2. Consider the Environment in Which the Product Will be Used

Since you will integrate an AR design solution into the users’ environment, you want it to feel as natural as possible. The environment significantly affects AR design:

• In a private environment (e.g. home or work) the UX designer can count on long user sessions and a complex interaction model. The whole body can be involved in the interaction, and specific devices, such as a head mounted display, can be used for manipulation.
• In public environments, usually outdoors, it’s important to focus on short user sessions. Because regardless of how much people might enjoy the AR experience, they won’t want to walk around with their hands up holding a device for an extended period of time.

Thus, when designing an AR app, you first need to research the environmental conditions in which the app will be used and how it will affect the user:

• Identify interaction scenarios upfront, even before specifying technical requirements for the project.
• Collect all the details of the physical environment to be augmented. The more environmental conditions you identify before building a product, the better.

User testing should be a critical step in the review process. When the first working prototype of your AR app is ready, you should run comprehensive user tests on product use in real conditions. Your ultimate goal here is to make interaction with the product comfortable for users.

3. Make the Interaction with AR Simple

AR in an app should be a layer of added value that reduces the time needed to complete simple tasks. Keep in mind that with each product people are seeking out experiences, not technologies, and they won’t like a technology that isn’t friendly to use. Thus, when designing your AR solution I recommend the following approach:

• go to the environment where the user will perform the task;
• think through each step that your users will go through in order to accomplish the task;
• record each of those steps.

This information will help you conduct a task analysis. The analysis will help you make things more natural for the users. Consider the Google Translate app in the example below. The app uses the phone’s built-in camera to translates the captured text into another language. This example perfectly illustrates the value that AR technology can provide.

AR experiences should be designed to require as little physical input from users as possible. When users are looking through the device screen at an augmented picture, it’s going to be hard for them to input data at the same time.