A Beginners Guide to User Experience (UX) Testing

If you are reading this then you’ve probably just been given the job of improving a website, app, product, or maybe even a real-world service. Like with most things that are new, you aren’t sure where to start and might be a little overwhelmed with the amount of information Google is throwing back at your search queries.

What I hope to achieve in this article is give you the bare basics of user testing so that you can move forward and get a little experience of your own, get some positive results and then move deeper into the world of UX.

The first thing you need to do is:

Define the problem

One way of looking at this first point is asking yourself either;

• What currently isn’t working at the level we’d like with our product? Or
• What is the most complained about aspect of our product based on customer feedback?
• What is the first touch point our clients reach and what is our desired action for them?

The answer to this question will be the focus of your first user test(s). If you have multiple answers which you find hard to prioritise then you should flip a coin and focus on one first.

DO NOT focus on multiple problems with your early user testing. You’ll most likely not answer any of your questions if you do.

The reason why we identify a problem and focus on it to the exclusion of everything else is:

• The user tests will be more targeted and the chances of you getting a clear, actionable result will be higher.
• The tests will be shorter; thus, participants are more likely to remain engaged and provide valuable insights.
• If you are very specific about what you are testing, then you can ideally run the same test multiple times at regular intervals to ensure your product iterations are continuing to move you in the right direction.

“What if I can’t identify the problem?”

Firstly, this is not a silly question, many a smart UX professional has asked this but what they do next is what is important. They’ll generally do exploratory or discovery research, user testing will sometimes be a part of this process, to explore how customers are using their product and, in the process, identify problem areas. Exploratory research is beyond the scope of this article but if you need a kick start try this article (https://www.invisionapp.com/blog/strengthen-discovery-phase/).

Who are you targeting?

Once you’ve defined the problem you are going to focus on the next thing to consider is who should be participating in your studies. If you are a social media tool targeting university students, then you don’t want middle aged professionals giving you feedback on your product.

Why?

Because context is key. People from different locations, and in different stages of their life view the same thing very differently. Imagine if you optimised your tool for middle aged men and then marketed it to twenty something women. What would happen?

Nothing good, that’s what.

If your product is already in the wild and has some users, then you’re in luck. While it’s not guaranteed, there’s a strong chance that these people are your target market. So, recruit them to participate in your study. If your product isn’t live, or you cannot recruit real users then don’t be stingy on your recruitment. You may need to pay an incentive of $20 — $100 per participant, depending on your defined demographics, but you must ensure the right people are flowing through your tests.

Additionally, is money really the best thing you can offer? If you are recruiting current clients of yours then you’ll most likely be able to offer your product or service at a discounted, or even free, rate which can be more cost effective. For example, a $50 t-shirt is worth $50 to the participant but might only cost you $25. In this case it’s a no brainer, don’t offer them $50, instead offer the $50 t-shirt.

Knowing who you are targeting also helps a lot when writing tasks, questions and generally structuring your user tests.

What are you wanting to learn?

Are you looking for Quantitative or Qualitative Data?

Another way of saying this is do you want people’s opinions (qualitative) or do you want to see the results of their actions (quantitative). This somewhat relates to the previous point about recruitment as well.

Qualitative, or Qual, testing often only requires around 5 participants to run through a usability test. Why only 5? Read this article — https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ — not everyone agrees with its conclusions, but many do. These tests often include video and audio of the participant ‘thinking aloud’ and telling you about their experience.

Quantitative, or Quant, on the other hand requires many more participants. The resulting data from these tests often comes from tasks that the participants were asked to complete, and you’d be more interested in things like task success rates, page count, path analysis and time on task. Generally speaking you want to recruit over 50 participants for Quant testing.

You don’t necessarily need to choose one or the other, Quant or Qual, tools like Loop11 — user testing allow you to run user tests collecting both forms of data, however, focussing on one helps you to keep your testing focussed.

Qual testing is often associated with the discovery phase of testing which we identified earlier, whereas Quant testing is great at validating a product’s usability.

What are your key tasks and questions?

It goes without saying but just to be clear, your tasks and questions should aim to provide answers surrounding the focus of your testing. Answering where, why and how people are finding difficulties is key, and this should lead to clues regarding how to improve your users’ experience.

To give a point of reference, on average, Loop11 users will create 3 tasks for their usability studies and will generally follow each task with one or two questions aiming to gain further insight once the participant has moved on from the prior task.

Examples of a task may be to ask the participant to locate specific information, add an item to a shopping cart or to fill out a form. It’s relatively common to follow a task with a rating scale question looking to identify the perceived ease with which the participant associated completing the prior task. A second question may be open-ended asking why the participant gave the rating.

Ideally your tasks should all have a flow and/or be related to each other so that the participant’s journey represents a “real” experience as closely as possible. If you find yourself writing tasks which are completely unrelated to one another then perhaps you’ve fallen fowl of our first recommendation and have not split your problems out into separate user tests.

Frequently, UX professionals will add a few questions at the end of the survey asking anything from Net Promoter Score (NPS) to System Usability Scale(SUS) to demographic focussed questions. There is no right or wrong here, but only ask questions if you know how you’re going to use the resulting data. Don’t waste participants’ time.

Look at your test from a participant’s perspective

Once you’ve fleshed out the flow of your test, your tasks and questions in place, run through the study as though you were a participant and time how long it takes. Even better, get someone else to do it in front of you, ideally someone not intimate with your product. This way you’ll be able to get an unbiased idea of how long your study is and the sort of time commitment you are asking of the participants.

The longer your study becomes, the greater your incentive/payment will need to be, or the higher your dropout rate will be as participants get frustrated with the length of the study.

There is no defined rule here but try to think of it in relation to your target customer. If you are looking at C level executives, then they’re unlikely to be motivated by an offer of $15 for 15-minutes. So in this instance, shorten your study or increase your incentive.

How are you analysing your results?

Do you have benchmarks?

Thinking about your results before you go out and get them is one of the most important things you can do. It helps you craft your study, set better tasks and ask better questions.

Another important question that needs to be answered is; do you have benchmarks or are you actually running a study to create them?

Benchmarks are the key data points for which you will be measuring future UX studies against. If you don’t have any, then your first study or two should be about identifying the relevant benchmarks. Even if you are committed to a new product design you should really benchmark your old product, so you can have concrete data to measure your new design against.

Here is an article detailing how Médecins Sans Frontières (MSF) had great success benchmarking a new website against their old one.

User testing doesn’t need to be expensive and doesn’t need to take up a lot of your time. However, it does need to be done in a thoughtful and focussed manner. It’s also something that should be done at all phases of your project. As long as your company is serving customers, then you should be speaking to and observing them to see how you can improve your offering.

So, now that you’ve dipped your toe into the waters of user experience testing, go forth and test! There is a world of insights waiting for you just around the corner.

UX for beginners

User experience (UX) testing allows us to see things from the perspective of customers. Identify pain points in their experience or journey, understanding how certain features of your product appeal to their interests (or lack thereof). It also gives you the opportunity to get direct feedback on your product before you give the green light to head to market – an invaluable sense of clarity when you need it most.

Get started with Loop11

Eliminate common problems that plague the user experience of a product

It’s important to note that there isn’t a one-size-fits-all solution when it comes to UX. Problems can stem from issues such as:

1. Your landing page doesn’t have any social proof to persuade visitors into trying out your service.
2. The checkout process for filling in information takes too long, which makes people abandon their cart
3. The website is not mobile-friendly or your app has confusing navigation

It is important to prioritise each issue and deal with them one at a time. UX customer experience testing ensures these kinks in the chain are ironed out before you go live.

Our UX features

Consumers don’t always know what they want

UX testing is an essential part of the development process. It helps you find out why people aren’t using your product, how they feel about it after using it for a while, and what changes could make it more effective at achieving its goals.

Find out how Loop11 works

Using the right UX testers is imperative

If you are not recruiting the right kind of people, then your results will be skewed. This means that you’ll never really know if what you have created is truly something people want or need.

Get started with our free trial

✅ Outline key tasks and questions to focus on

✅ Understand the commitment you’re asking of your participants

✅ Only ask questions if you know how you’re going to use the resulting data

✅ Do you have benchmarks? Or is this going to create them?

The difference between qualitative and quantitative data

When you ask most people what kind of information they want, they’ll tell you they want to know how customers feel about their product or service.

However, this isn’t always the case. Sometimes we need to see if our changes actually made an impact on user behaviour (quantitative), or if there were any other reasons why customers behaved in a certain way (qualitative).

UX features tailored for your needs

How we can assist with your UX

Loop11 helps companies run better UX tests with less effort by providing services that are easy to use and affordable for all budgets. We can help you test websites, mobile apps or prototypes on real users from anywhere in the world.

Get started for free