How to Remotely Test Your Mobile App

The HubSpot usability team loves to get feedback on apps we’re developing to make our customer’s marketing lives easier. But testing in the mobile environment presents a unique set of challenges. We recently devised a system that allows us to get much more insight from our testing sprints with real mobile users, using real apps in their natural environment — in situ — and discovered a whole new set of questions we were able to ask as a result.

The problems with most prototyping tools

In the past, we used mobile prototypes on desktop computers for mobile testing. We’ve found that this strategy gives us a fairly decent understanding of how our customers use our mobile app. But even though mobile prototyping tools are relatively sophisticated, they were not entirely realistic. When you test with a mobile prototype, you compromise on a lot of important aspects of the user experience.

Prototypes can’t:

• Be tapped, swiped or zoomed, because there’s no finger interaction
• Scroll, which means you can’t see more than a screen’s worth of content
• Explore the whole in-phone context of the app, such as where the app lives, and how the user accesses it
• Recreate usage on a physical handheld device
• Be installed from an App Store

In addition, we wanted the least intrusive way to actually see how people were using these prototypes, and doing in-person testing simply would not scale.

Trick 1: The Remote Hack

Our team discovered the first hack from the folks at Mailchimp, who shared a method to use the camera on your laptop or iPad to record what was happening on your iPhone. This simple hack allows you to see someone’s phone from nearly the same perspective as they themselves see, and mimics real use amazingly well (see picture). This solved part one of our dilemma, and would enable us to test with anybody who used the HubSpot mobile app no matter where they were.

Trick 2: Use Proto.io or InVision

The next step was figuring out how to test with a mobile app that didn’t exist yet. Clickable mockups are reasonably useful for desktop testing, but presented an obstacle in the mobile world. After trying out several different options, we decided to use an interactive mobile prototype through the tool Proto.io because it did the best job of mimicking a fully functional mobile app that can be downloaded and used on a testing subject’s own phone. It made it easy for our team to create and customize prototypes for the varying devices and screen sizes of the testers, which made the testing experience the most realistic.

Another great app that does this is InVision — which our design team swears by for desktop app testing. It makes it easy and has taken our process to the next level. We are able to iterate faster after getting customer feedback using these tools.

(Note: Now that our mobile app exists and is no longer in mockup form, we use Testflight for our beta testing.)

Some unexpected insights

Once we had a system in place that would allow us to test mobile users using prototypes with some degree of accuracy and authenticity, we were off and running. Aside from the more general usability insights we gained from these in situ sessions, we also discovered that testing the users on actual phones made a huge difference in the scope of what we were able to discover during testing. We documented findings that would never have arisen in a less realistic testing environment.

For instance, we learned:

• What other apps people use: Seeing what other apps our users have on their home screens helped us understand what marketers are doing on a day-to-day basis on their phones, what other apps they’re trying and using, and what they like and don’t like about them.
• How people organize their apps: We got to see how our users group the HubSpot app with their other apps, whether in folders or just on separate screens. The folder names and the app with which they grouped us with were especially insightful.
• How people navigate between apps: Looking at how HubSpot users interact with their phones (such as using status bar to scroll to top, if they use the multi-tasking bar and so on), how they switch between apps with bookmarklets, copying and pasting etc. gave us a look into their basic actions with their phones.

Although we try not to, sometimes we think we know what we expect to find out from the research. But, the best part of testing your app is finding out something you weren’t expecting. Testing users in their own native environment (like their own mobile phones) not only helps you see how they’d actually complete tasks in your app, but how they think about mobile as a whole. For any tech company, the data is hugely valuable and it’s worth watching your users interact with devices on their own terms. Give it a shot!

What do you currently use to test mobile apps remotely?