How to design UX for an AR mobile app – a few lessons from our latest project

Introduction – the what & the why 

Apptension’s R&D team is used to tackling problems with unconventional and high-tech solutions daily. This time, we have decided to educate younger audiences about recycling and waste management with the use of Augmented Reality.  AR was our technology of choice for two reasons: we have plenty of experience working with it and even a boilerplate codebase in place to jumpstart projects. On top of that, Augmented Reality would add a level of playfulness to our app.

The R&D team reached out to me at the beginning of the project to kickstart the UX research and design for the app.

In this case study, you will find 

  • an outline of the process we followed,
  • some good practices,
  • a few handy tools and guides,
  • a set of reusable wireframes,
  • a sneak peek into our game
  • and a few other inspiring solutions that you should check out!

All of that will be focused around designing UX for AR. 

Process –  and good practices of designing UX for AR

Being an UX Designer, I have worked on a bunch of cutting-edge solutions – from voice interfaces to  humanoid robots, but never on an AR mobile app. I have never even had a chance to use an AR app cause I have a not-that fancy, 3-years old smartphone! What I am trying to say is that – it was challenging. – all I know now about designing for AR mobile apps I have learned “on the go”. I am hoping that reading this article and learning these things will decrease the learning curve as you will approach your first AR project. 

Interested in our Product Development service? Learn more here.

1. Persona – empathizing with your user

When the R&D team first reached out to me, one of the first things that popped into my head was “Omg, a game, in AR, for KIDS! So many challenges”. They specified the age group as 7-10 and, with that in mind, I started off by creating a persona.

Empathizing with users by creating a user persona is naturally the staple of UX design and we should all do it at the beginning of each project. While it might be sometimes overlooked as just a “formality”, I find personas particularly useful when designing for very specific target groups and unconventional technologies. 

My suggestion is – document your persona and do it well. I would highly recommend using MURAL for that purpose. They have a great template which ensures that your findings and reflections will not get lost among many other workshop or discovery notes, but will be structured and easily accessible by any of the team members, at any stage of product development. I found that when you have a comprehensive, well laid-out persona at hand, it is easier to always keep your end-user in mind. 

2. User Journey

Mapping out a user journey for all relevant scenarios is yet another crucial step enabling you and your team to think about a lot of aspects and generate a lot of ideas. And again, find a way of documenting the user journey that will work for you! I used a Storyboarding template from MURAL and it worked very well. 

I truly love post-its and I hope there are some sticky notes geeks out there who can relate! Post-its require concise thinking, their simplicity and universality reduce the pressure – after all, you are not creating a complex diagram, you are just scribbling away – which can actually encourage creative thinking! If you additionally have a digital space, where you can easily generate and organize your post-its, your life will be much easier! And it can be super helpful during the COVID-19 pandemic since you’ll avoid spreading tons of post-its all over the floor or dining table! (They don’t stick to the walls anyway, let’s not kid ourselves!)

3. Wireframes

While mapping out the user journey already required me to do a bit of research about designing for AR and watch a couple of videos, creating wireframes was when I really had to get into it and understand the specifics of this technology. Some things I learned from our developers and some I discovered through my research. 

Before continuing with 3 important lessons in regards to UX of AR mobile apps, I am going to quickly mention AR wireframing tools, with wiARframe being one of the most promising ones! I signed up for it, played around a little bit, watched a tutorial video, and came to the conclusion I would stick to my regular prototyping tool – Figma.

 

I might consider learning and using an AR-dedicated tool in the future. However, in this project, I could not justify asking my 5-person team to learn a brand new tool when Figma is a great tool that can work perfectly fine for the UX design of an AR mobile app. There even is this handy tutorial for how to use Figma for creating complex and convincing AR prototypes. 

Good UX practices when designing for AR

1. Use translucent UI elements

Safety has to come first! Since users often use AR apps while being on the move, it is essential not to block their view. UI elements should be see-through whenever possible. However, there is another important aspect to take into account here – readability! If you use a transparent UI element, the text or icons that you place on them will not be visible enough when the background is too dark or too light! How to combine these two? Use translucent UI elements. That way the user will see the background and the text! Just note not to use too much blur – the human eye automatically blurs the background when focusing on closer objects. 

Focus on the half-transparent box  in the middle – doesn’t your vision get blurry? 

Tip: By experimenting with some edge cases I found out that, when using black color for the UI (#333333) and white color for the text, 80% is a safe level of opacity! The contrast ratio stays acceptable and the shapes in the background are visible in all conditions. 

Opacity level vs text readability.

Tip: Always remember to assess your application’s accessibility against WCAG guidelines. I am sure you do not want to be excluding any user groups from your application. And nowadays in Europe and the USA you can even face legal consequences for doing that! Distinguishability is one of the WCAG requirements, with minimum contrast ratios for different elements. For example, in the graphic above, due to a relatively bright background, the variation with a transparent UI element (30% opacity) does not meet the minimum text to background contrast ratio!

WCAG online contrast checker.

Tip: I generally use this WCAG online tool to check the contrast ratio. However, in Figma itself, you have a free plugin for that as well. The plugin as many, many other handy tools were published by a member of and can be found within Figma Community. Our whole design team has recently joined the Figma community (see how to sign up here, it is free!) and we really find ourselves getting a lot of useful resources (like icons and design kits) from there. 

What does a low contrast ratio mean in practice? Users with poor eye-sight or even users with great eye-sight who are using the app in the sun, will not be able to read the instructions and will have no idea what to do or what your application does. 

2. Use large margins – One of those things no one tells you online, but it is so important to know! 

Doing wireframes for the first few screens I used translucent UI elements knowing that it was “a thing” but other than that, I paid attention to the usual accessibility rules – 31px padding around the screen edges, making sure any clickable elements have the surface of at least 44px², setting the font sizes for at least 12pt (I if you’re not fully familiar with the accessibility checks and guidelines yet, please find this really handy guide which summarizes them). However, I have overlooked one thing that is apparently a golden rule in AR mobile app development – large margins! 

Tip: When designing a cross-platform AR mobile app, a 31px margin between UI elements and screen edge will not be enough! Leave 5-10% of the screen empty around the edges, to avoid finding your application unusable either on Android or on iOS! That is due to the different placing of built-in UI elements in these two systems and their tendency to overlap with in-app UI elements. However, if you make sure to keep off the screen edges – you will be safe! Our developers at Apptension have learned it the hard way, but fortunately, they kindly shared their knowledge with me. 

3. Good UX of the “scanning the environment” part is tricky but crucial!  

If you are using a dedicated AR design and development tool such as Torch AR, this will be sorted for you. However, if you are going to use Figma and Unity as we did, you will have to design and implement that part yourself. 

If you are not entirely sure what I am talking about, please take a look at this short video. The video, and how setting an environment is done in Torch AR in general,  served as an inspiration to me when designing for our app. If you want to see another example that inspired me in terms of a set-up, but also UX of the app in general, take a look at this short video presenting a more complex AR mobile app. 

Environment scanning in a simple AR app built with Torch AR.

Watching these videos, you have probably noticed a few things – the process of scanning the environment is well explained, with descriptive captions, appropriate icons, and an additional aid in form of the dots that indicate a flat surface. However, you have, hopefully, also noticed a few threats as well, mainly – white text with no background (!) (what if someone was scanning a white table???) and the captions disappearing super quickly with no option to go back (definitely not a suitable solution for 7-10-year-olds kids who are not very fast readers yet!). 

With the observed good and bad practices in mind, I created the UX design for the process of scanning and saving the environment.

We are sharing the wireframes as a resource for you to use! Check them out here. The design is universal and you can use it regardless of what AR mobile app you are creating – a game, a medical app or anything else! Please let us know when you do use it, we would love to see your ideas and solutions! 

Accessible environment scanning – mockup.

More importantly, in the Figma project, along with the wireframes you can find a sneak peak to the UI design of the game. Take a look at how we have transformed UX designs into a final UI design of the game and stay tuned for more! The game will be available for free in just two weeks. 😊

If you need some know more about what our creative team can do for you, see our Product Design expertise.

Inspiring AR apps and games

While researching existing AR apps and games I have discovered some truly inspiring solutions that I now want to share with you:

WallaMe! – An amazing game for older kids and adults that uses AR and geo-location to enable sharing AR messages in the real world! WallaMe gives you a funny way of hiding private messages in public places and sharing them with your friends or with the public! Download it from Google Play or AppStore.

Avo – one of my personal favorites! This iOS AR game by Playdeo is cute, fun, engaging, and it is both for kids and for adults. Avo is a story of a young inventor Billie and a curious, mystery-solving avocado that she brought to life! The game has an amazing narrative and effects that make it feel almost like a cinematic experience. If you can’t download the game, you can watch Avo’s adventures and get inspired here!

Color Quest – also a great app, but this one is more aimed at kids! It is fun, engaging and has an educational twist to it! Color Quest lets you color a character and, after reading a fun health-related fact, you can see your new creation come to life and dance in front of you in AR! You can even take a picture or make a video of you and your favorite AR character! You can see a very short demo of the app here and download the app for free on Google Play or the App Store

Conclusions

The final product

To see the final version of the product with a beautiful visual design by our Art Director will be live in two weeks. As I am writing this a number of Beta users are testing the game and you can see a (little) sneak peek of the designs here.

Sneak peek into our gameplay design.

Impact

As the app is currently in beta testing, it is too early to talk about the impact of the product itself! However, I think it is fair to say that we have all learned a lot! The R&D team has definitely taken their AR development & design skills to the next level and as for me – I have simply started my adventure with UX design for AR mobile applications. I have learned a few tips and tricks that I have shared with you here! 

While I am really looking forward to hearing what users will say, I am happy that the main stakeholders of my work, the R&D team, were satisfied with the outcome! They reached out for help with designing the UX of the most problematic parts of the user journey and, according to their feedback, the solutions I provided were useful and served as a blueprint for UI designs. 

Reflections

AR apps have a huge potential to change the way we think about and use mobile applications. However, it is still a fairly new technology and I believe that the best design practices are yet to be established. It will take many hours of user testing, observations and much more feedback for us, product teams, to optimize the UX of these apps

Ready to take a step into tomorrow’s tech?

References

https://uxdesign.cc/ar-design-tools-a-few-ux-principles-96f989511783

https://www.interaction-design.org/literature/article/how-to-write-the-perfect-conclusion-to-your-ux-case-study

https://www.productplan.com/post-it-notes-for-product-managers/

https://books.google.pl/books?id=IfviDQAAQBAJ&lpg=SA3-PA65&ots=SONd9C1sx4&dq=human%20eye%20automatically%20blurs%20the%20background%20when%20focusing%20on%20closer%20objects&pg=SA3-PA65#v=onepage&q&f=false

https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2015%3A0615%3AFIN

About the author

Dagmara Kukawka

Dagmara Kukawka

Junior UX Researcher
Junior UX Researcher at Apptension. Born and bred in Poznań, spent her uni years in Scotland & Amsterdam. Loves swimming, independent movie theaters, techno and 5th league games.

Related Articles