The importance of planning in guerilla testing

When talking about user testing in the UX circles, there’s formal user testing, and then there’s guerilla testing. Formal user testing is the subject of many research papers and studies, but there is relatively little guideline on how guerilla testing should be conducted. Although many UX purists frown on the practice, guerilla testing has become a useful tool for the everyday UX practitioner. Whether they be struggling with tight UX budgets or looming project deadlines, guerilla testing can help fast track the research and testing phases of their UX design cycle. In fact some UX practitioners might even refer to the practice as the art of guerilla testing.

In essence, guerilla testing is user research done using a lean and agile approach. While this means making the user testing simple, short and relatively flexible, it doesn’t mean going about this in a totally unstructured, undocumented and unplanned fashion. What formal testing tries to avoid by not conducting research and testing in a haphazard manner is the risk of introducing potentially costly design changes that may not lead to any benefits to the end user.

Yet, the danger of guerilla testing comes from poorly planned and executed tests that are not reliable, consistent or meaningful. In this post, we’ll explore some pitfalls of guerilla testing “in the wild,” a few tactics to avoid or minimize the methodology’s weaknesses and tips to improve planning for all research and testing.

Plan B – catering for the uncontrollable

The first and most important difference between guerilla testing and other standard user testing processes is the lack of a controlled environment. You may have carefully picked out a time, location and worked out the groups of people that you want to survey, but when you do turn up, things may not go according to plan. What will you do if some unexpected event happens that week, or if the location becomes unavailable or too crowded/noisy/distracting, and what if you don’t end up running into the type of people you want to speak to? So the first thing to remember when carrying out guerilla testing is always to have at least one backup plan, or even better, to have a plan B and C.

Consistency – stay true to your Q’s

Another common problem with guerilla testing is the temptation to go with the flow when querying the user. Often a particular question or comment triggers interesting insights or unexpected findings, and you become fixated with getting to the bottom of it. This can cause a few different issues, such as not being able to compare results because you haven’t asked the same type of questions, or the questions were asked in ways that produced varying results, or you introduce new variables and behaviour triggers that were not present for other users. In the end you find yourself unable to reconcile all the findings and draw a neat conclusion. So the takeaway message here is to have a focus in mind, stick to the main questions and resist the temptation to chase loose ends. If you must, circle back at the end to dig deeper.

Beyond paper – rich data capture

One of the details that often gets overlooked in guerilla testing is the capturing of information accurately and reliably. Guerilla testing doesn’t mean you are only restricted to pencil and paper.  Although you can get a lot out of paper wireframes, it is not easy for the testers to capture all the feedback on the paper itself or on sticky notes. There’s no shame in taking a PowerPoint presentation or even a semi-interactive prototype into the field. You might even consider a tablet running a usability testing web application if you have the luxury to do so. This way you will be able to have all your data and results captured and stored neatly in one place for you to review later. It also means no more deciphering handwritten notes scribbled down while your mind is focusing on what the user is telling you.

Guerilla testing – what’s it to you?

Last but not least, you should consider what the guerilla test means for you in the grand scheme of things. Not every organization is going to need a research and testing framework document to formalize and standardize the process, but chance are, if you find this to be a useful research tool then you’ll want to know that you can extract reliable and consistent results from your test subjects. So do spend a bit more time thinking about how to eliminate the environmental variables that might affect your users (e.g. testing a weather app on a hot versus cold day might affect people’s mood), consider when and how you approach people to conduct the test, and try to keep the test items simple with a clear focus on the answers you want to get.

With all of these planning details in mind, you should have better luck finding the right balance between flexibility and consistency for your guerilla tests.