Online Usability Testing

1. Set Your Goals
   Before running usability testing, it’s crucial to define clear goals to help you determine the proper testing methods, how many testers are needed, and how to analyse research findings. Having a well-defined plan is crucial to the success of your usability test. Without a clear direction, you risk running a test that’s cluttered and produces inaccurate insights that may lead to poor product decisions.
   
   
2. Check Your Resources
   Effective usability testing requires a great deal of planning and, if not done right, may quickly turn into a logistical nightmare that can block your product development process. To ensure that you’re on the right path, you must ask yourself these questions:
   • When will the project start and end?
   • How many team members are needed to carry out the testing?
   • How will you recruit test participants?
   • Do you need to conduct the research online or offline – or both?
   • What tools will you require?
   • How long will each step of the process take?
    
3. Choose Your Test Wisely
   Depending on the insights you want to uncover, different types of usability tests will produce varying results. Consider these factors in choosing which one will work for you best:
   • Quantitative or Qualitative: Quantitative data provides objective, numerical data from usability tests, while qualitative research aims to understand the motivations, thoughts, and behavior of your users. Combining both methods has proven to be the most effective.
   • Moderated or Unmoderated: Researchers can be present (moderated) or absent (unmoderated) during the test. Sometimes, having a moderator present may create cognitive biases.
   Start free trial
    
4. Pick Your Sample Size
   Once you’re transparent with what you’re hoping to achieve and how you plan to do it, you can determine the sample size for your usability testing. Keep in mind that ‘bigger is better’ may not always work best. For example, you may attempt to analyse results from 100 interviews and find it daunting to come up with valuable insights. But it’s only sometimes the case, especially with quantitative research, where a larger sample size can mean a wider pool of statistics to review. By carefully checking your goals and the test you want to run, you can plot the optimal sample size to get actionable insights from your data.
   
   
5. Ask the Right Questions
   Once you’ve determined what you want to discover and how to go about it, you can now quickly screen users to target for your testing by asking them these initial set of questions:
   • How old are you?
   • What’s your highest level of education?
   • Do you consider yourself tech-savvy?
   • Have you used a similar product, app, or website before?
   • Which device do you often use for doing tasks?
   The qualifying questions can be tweaked to suit your requirement. For example, after selecting your testers based on their answers, you may ask follow-up questions like:
   • Did you experience any issues in completing any of the tasks?
   • Which task was the most challenging?
   • What did you think of features A, B, and C?
   • Are there any features on a competitor’s product that are missing here?
   • Why didn’t you like feature X?
   You may also come up with questions they can rate on a scale of 1-5 or quickly answer with a ‘yes’ or ‘no.’ When asking questions, always remember to make it easier for them to relay their experience during the test. In short, K.I.S.S. (keep it short and simple)!  
   
   
6. Measure What Matters
   A cliche, but conducting usability tests is half of the challenge; the other half is extracting valuable insights that can help you answer your research questions and achieve your goals. So, before you start your usability studies, specify the data you want to gather. Here are some qualitative and quantitative metrics to consider:
   • Time-on-Task: Assess how long a participant took to complete a usability task
   • Likes, Dislikes, Recommendations: Learn how participants felt and what they thought of the usability of your design through qualitative feedback
   • Critical Errors: Learn what errors were encountered that stopped the participant from completing a task
   • Non-Critical Errors: Spot minor issues that may have deviated the participant into taking the wrong action, like missing a step or navigating to the wrong page
   • Error-free Rate: Know the percentage of participants who completed their tasks without errors and label it as your ‘North Star’ metric
   • Successful Task Completion: Know the percentage of tasks successfully completed by participants, regardless of how they achieve it
     Assessing these metrics can help you understand how users interact with your product, identify problems, and make intelligent decisions to improve usability. Thankfully, Loop11’s integration with GPT-4 makes the analysis of your results lighting fast!
   Start free trial
    
7. Analyse and Take Action
   At this point, you have already collected several data points from your usability testing. Great! Now, it’s time to sort out which issues you need to address and prioritise them accordingly so you can come up with a timely solution for each.To make your findings more actionable, the NNGroup suggests that you can rank your discoveries into five different levels of severity to allow you to focus on the most critical problems first:
   • Level 4 – Critical: Issues that fall under these prevent users from completing their tasks and negatively impact their experience. For example, a user can’t sign up for an account.
   • Level 3 – Serious: Issues here require urgent attention as they slow down the user experience. For example, a user can’t reset the password.
   • Level 2 – Medium: Issues here are moderately severe and may cause frustration to the user, though they may not happen frequently. For example, infinite scrolling.
   • Level 1 – Low: These cosmetic issues may not significantly impact the user experience but may affect the brand image. For example, outdated fonts or typos.
   • Level 0 – No Issue: A user may report an issue that isn’t really a problem or request a feature that’s not yet available. You may take note of these, but no action is necessary now.

• Think about the person using your product. Watch them use it and see how you can make their experience better.
• Test your product early and often so you can find problems that you can fix quickly.
• Ask users for feedback to give you good ideas about how to make your product better.
• Use the right tools to help you keep track of what tasks you want them to do If you follow these tips, you’ll do just fine!

With all these amazing benefits come hurdles one can face when doing an online usability test. The biggest challenge to date is not the limitations of time and money to run these tests but audience accuracy. How can one be sure that you have the correct user group who will take part in the test and that they have used your products before?

Avoiding dubious test participants is still a hit or miss. Some take a massive number of tests in exchange for cash, devaluing the input and polluting data in the process. But the worst offender of them all is a competitor stealing your idea when you’re doing remote, unmoderated tests, as you can’t stop testers from taking screenshots or sharing what they saw.

To avoid this problem, you must pay attention and ensure your test is set up correctly. Even if you carefully pick your user demographics, you may still end up with results that only apply to your product or users. This means wasting resources on the information you don’t need.

Luckily, you’ve come to the right place! Let us help you dodge these hurdles to truly make online usability testing work for you in creating that great user experience.