Can your customers use your digital product? And once you know if they can, more importantly will they want to use it? Whether your digital product is still conceptual, newly released to the market, or has been out in the wild for some time, if you don’t know the answers to these questions currently, it’s time to ask them directly through user testing.

There are many methods for conducting user testing available today: guerrilla testing, lab testing, remote testing, card sorting, and more. With so many options, it can be a significant challenge to know what is right for your own project. As a product owner or manager, you may also be asking yourself: “What’s the best value for my company’s investment?”

Let’s explore two methods that have created exceptional results for our clients’ digital products and their respective customers: moderated vs. unmoderated usability testing. These insights will assist you in making a more informed decision to ensure an optimal customer experience. We will also touch on the two primary bottlenecks that challenge many companies: whether or not to employ user testing at all; and time and money.

When it comes to the overall success of your business, these common barriers to entry can be overcome by adopting this simple yet effective mantra: “Without customers, you have no business.” It’s a mantra worth repeating and remembering, because it’s 100% true.

Decide what to test

In order to establish a better basis for collecting insights to process from user testing, you’ll need to understand exactly what to test. If you’re not sure where to begin, we suggest initiating an empathy mapping workshop with key project stakeholders. This will promote a refined perspective on the pain points your audiences are encountering. These pain points are linked to key tasks performed, as well as their ultimate goal. Pain points can help shed light on friction points as well as opportunities for improvement that can be validated with customer insights.

Taking this process a step further, you could also use analytics to visualize patterns where customers are running into challenges. Strictly using analytics however would be challenging, because the numbers reveal only one side of the story. To truly understand what a problem is, you must gain perspective directly from your customers by empathizing with them.

Create a user testing script

Next, you’ll use these insights to generate a user testing script. Whether it’s using the current digital product or a conceptual one, think chronologically about the tasks you want testers to perform, as well as the steps involved. For each task, identify the user story, i.e., “As a [user], I would [perform an action] to [achieve a specific outcome].” To substantiate if the user story is attainable, create a corresponding task question for your user to perform. For example, you might ask testers “If you were going to check for new messages, where might you go to perform this task?”

To assist with this script in testing, you’ll want to ensure there is something to test against, i.e. a visual aid for your customers to engage with. If your digital product is still conceptual you could use sketches, wireframes, concept art, or simple prototypes. If your digital product is currently live or you have a minimal viable product, you additionally have the ability to choose whether you want to gather customer insights from the live digital product, or a conceptual solution based on the design system.

With a test script ready and a visual aid to assist with the testing process, you’re ready to make some decisions on what type of user testing is right for your project.

With either testing scenario, your comprehensive goal is to understand from a quantitative analysis whether or not the tasks are viable and qualitatively, if testers liked or disliked the experience of performing the task. 

Moderated User Testing 

Less expensive, but time intensive

When it comes to being prudent with finances, but getting direct feedback on your digital product from your customers, moderated testing is hard to beat. What you save in financial costs however will be offset by the timeline involved to conduct this kind of testing. This method requires substantial  planning and logistical coordination between both your staff and your customers.

Firstly, you are responsible for coordinating these meetings with your customers. You will need to contact them directly and ask them to participate in a user test. In most circumstances, you will also need to provide an incentive. 

Once meeting times have been scheduled, which will vary roughly from 30 minutes to an hour based on the length of your user testing script, you’ll need someone to run customers through each task. The goal is to be as unbiased as possible during this process. You don’t want to unintentionally lead them to the answer. You want customers to try completing each task on their own, only interjecting if they hit a stumbling block and ask for additional clarification.

If this process seems daunting, it’s possible to partner with a digital product agency such as Emerge to conduct these interviews. If in-person testing is not feasible, the testing can also be conducted remotely by utilizing video chat platforms with screen sharing capabilities. There may be a challenge with remote sessions if additional time orienting testers to this environment is required. Ideally, you want their camera on so you can see their facial expressions and share their screen if they are clicking on a prototype link.

Another aspect with this type of testing is that if you conduct it yourself, it is incumbent on you to parse the data from each session. You’ll need to prepare documentation to present back to your stakeholder team so they can understand how testing went. What was the breakdown of pass or fail from a quantitative standpoint? At what point did they fail, and qualitatively how did they rank their experience and what did they have to say about it? All of these factors are critical in being able to understand what constitutes a problem and what to do about it.

Unmoderated User Testing

More expensive, but less time intensive

There are many platforms now available to perform remote user testing on your behalf, such as Usertesting.com, Userlytics.com, and Trymyui.com. Depending on your specific needs, it’s best to go to each site and research the costs involved before you go further because those costs can add up quickly. But these platforms are great for automating the process of collecting insights.

How this is achieved is that these platforms have a subscriber base that is paid for each test that they take. You can create screeners to narrow down the types of test takers that may elect to take a test of yours, but it is good to keep in mind that these users may or may not be current customers of yours.

Some platforms allow you to specify test participants by email address, so it’s possible to use your current customers. But the participants will need to download and run user testing platform software to take the test, which can be a challenge for individuals who are less technically savvy. On desktop, this is not as big of a challenge as the software is relatively easy to install and use. For mobile users however, this quickly becomes cumbersome as users will need to go through a series of steps that include privacy prompts and jumping through specific installation hoops. By the time your users arrive at the test they may already be negatively biased based on how difficult this process was for them.

Within these platforms, you’ll take your testing script and use the platform’s specific test building tools to generate your test and submit it for testing. In the platforms mentioned above, this process is relatively easy to perform and get going. The only variable that changes depending on the platform used is cost. Some may charge per user, and some may change a flat monthly testing fee.

As for the time required to run testing and process results, once tests are submitted you can usually expect a turnaround time of 24 hours or less if using the platform’s paid subscribers. And, you’ll get a report generated for each test that illustrates pass and fail metrics as well as any likes or dislikes around each task performed along the way. The only downside is manually reviewing each user test to ensure the data was captured correctly and that test-takers didn’t run into any challenges. If you’re going this route, we suggest performing an initial “dry-run” with a single test taker before proceeding further.

Which usability test is right for my project?

While both moderated and unmoderated testing are equally viable in many project circumstances, it’s best to know what your project budget and timeline are. With some thoughtfulness ahead of execution, timelines can be padded to run moderated testing and gather significant insights that lead to successful outcomes, while an increase in project budgets to run unmoderated testing can be more challenging but worthwhile if timing is an issue.

With either case, think again about the mantra from above. “Without customers, you have no business.” Completing a project without user testing is always possible, but at what cost to your business? If the experience that is built doesn’t service your customers’ needs, they may venture elsewhere to achieve their goals. It’s a big risk to take, so it’s important to consider this carefully.

What if I need further help with testing?

Moderated and unmoderated user testing are both great ways to get invaluable user and customer feedback, but either can still be a daunting challenge for those who have never run such a process or are still uncertain which is best for their unique situation.

Concerning either circumstance, Emerge is here to help. We have an experienced team of design practitioners who specialize in both areas of testing focus that have led many successful user testing scenarios for our clients. Our approach is strategic and collaborative — aimed at rectifying the digital product challenges that simultaneously affect your business, your brand, and your customers. We’re always happy to consult, so if you want to talk with us about your unique situation we’d be happy to give you an unbiased understanding of what we can do for you. Schedule a call with us today to find out more.