How many participants should be used for Online, Quantitative Usability Testing?
Qualitative usability testing has traditionally been based around small sample sizes of 5-20 participants. However, the growth of online testing tools and quantitative usability research is changing the game. Whist many experts agreed that for qualitative, lab-based testing small samples sizes of 5-20 participants is sufficient. Online user testing has created a new wave of analysis such as benchmarking, A/B testing, competitor comparison, validating and much more. These kinds of quantitative analyses require larger numbers of participants to validate the data.
For example: Let’s say your company is testing two different versions of wireframes so management can decide and approve one to implement and allocate resources to. It would not be very good practice to use 10 participants and get a 60%-40% success rate. It would be very hard to validate a study and implement a strategy based on 10 participants. However, if 500-1000 participants were used, that data would be a lot more accurate, and management would have valid data to approve of the findings.
Some specialists, such as Usability Sciences, recommend up to several thousand participants for more high level quantitative testing such as, click-stream data, or multiple cross-tabulations. They have broken down the % error for margin based certain numbers of participants. Read the full article here.
Happy Testing!
Give feedback about this article
Were sorry to hear about that, give us a chance to improve.
