Associating “false insights” with research seems ironic, considering that research is frequently the result of hours of fact-finding and correct solution discovery. In essence, research should provide true insights and facts.
However, false insights can result from improper benchmarking. Benchmarks should be chosen carefully to ensure that they represent the system under prototype testing. Also, since humans still perform the cognitive processes for the research, it’s not surprising the resulting study can contain a few misleading insights and incorrect solutions.
The good news is, you can avoid these false insights. Here’s how to ensure your research and analysis are free of errors and, therefore, credible.
1. Identify Your Own Unconscious Biases
Unconscious bias is a term used to describe our associations that can significantly influence our attitudes and behavior.
As humans, it’s expected for us to have unconscious biases toward certain things. Some of these biases are more obvious than others. These biases could be related to our previous experiences, thought patterns, subjective feelings, false positives, or personal beliefs. They could also stem from the many prejudiced points of view we have come to accept and adopt.
It’s up to you as the researcher to identify these biases to avoid false insights in your generative research.
Recognizing that you are biased in some way is the first step. Doing so will force you to rely less on emotional responses and more on logic. You may also use this information to analyze the potential impact of your bias on your research and solution type.
An example of unconscious bias is thinking that you could get more relevant information about human resources from females than males just because, in general, it’s a female-dominated division. A similar thing could be said about the tech industry, where there are generally more males than females.
Consider the following types of unconscious bias when defining them:
However, if identifying your own bias is difficult, you can seek the help of others. Since these biases are hidden, it’s sometimes easier for an outsider to spot them for you.
If you discover a high bias tendency, review your research and analytic solutions more times.
2. Counter Clustering Illusion
Another way to avoid false insights in your research is to counter the clustering illusion. Clustering illusions involve drawing classical insights from a cluster of data you erroneously think is a pattern when they are just random-effects factors. It’s a solution attempt that involves making meaning out of something that doesn’t have meaning at all.
For example, if your traffic increased by 1% in the first month, 2% in the second month, and 3% in the third month after you changed your web hosting for SEO, you might conclude your traffic will rise by 4% in the fourth month. This is, however, classic clustering illusion. Although there seems to be a pattern there, the reality is, the percentages are just random figures.
You must watch out for clustering illusion in your information architecture research. Rely more on fixed-effects factors than on your intuitive thoughts and strong feelings. In addition, you should consider more than short-term streaks when drawing up sudden insights. View the data holistically. Take your time to study the available data to notice the right trends, if any.
3. Shun Sunk Cost Fallacy
This occurs when you aren’t willing to change or abandon your line of action in your moment of solution because you have invested a lot of resources and effort into it.
The sunk cost fallacy is derived from the economic term “sunk cost,” which means money or resources invested that can never be regained.
In a stock market scenario, sunk cost fallacy sets in when you are reluctant to stop investing in a stock that keeps going down because you have spent a lot on it. The market suggests more lucrative alternatives, but you prefer to wait, albeit in futility, for the stock to rise again. In our example above, a company that invested a huge amount of money in hair loss drugs hesitates to stop production of the same even if they have been proven to yield side-effects.
In research, then, the sunk cost fallacy sets in if you have a hard time letting go of a thesis that you have been working on for months. Even though the environmental situation means it is no longer relevant, you decide to hold on because of the resources and effort you invested in it.
Simply put, the level of resources you have put into the research makes you reluctant to entertain a new strategy, even though it has been proven that your current strategy will fail.
To avoid the sunk cost fallacy, you must always look at the bigger picture. This means your mind should be fixed firmly on your research objectives. By doing so, you’ll be more focused on succeeding, even if it means abandoning a failed tactic after investing time and energy.
4. Survey the Right People
It’s essential to survey a diverse group of people to get a more accurate picture of what is happening. Also, you must find people who represent your target audience, not just those who are the easiest to reach.
If your survey is about running a business, your ideal target audience could be CEOs. Or, if your survey concerns the best tools to design logos throughout a complex design process, you can engage graphic designers to get correct, first-hand insights.
Also, you shouldn’t just survey people who are likely to agree with you just because you don’t want anyone to counter your arguments. If you only survey your supporters, you may miss out on valuable feedback from people who have a different position than yours.
To ensure that the right people respond to your survey, you can also formulate questions only your target audience would know how to answer.
For example, if you want to reach content creators, ask about their difficulties while trying to keep up with the latest content trends. Or if you want to reach web designers, ask about the steps you need to take to build a website or website design trends. Their answers will help you determine if they’re actually in the field they say they are.
Once you have the right survey respondents, you can then proceed to ask the questions you need in your questionnaire. Always go back to your research questions and objectives so you can pose the right questions to respondents.
You should also consider the medium through which you want to administer the survey. Use an online survey to capture wider demography across different locations. If your target audience is localized, a face-to-face survey will suffice.
5. Use Mixed Methods Research
Mixed method research involves collecting and analyzing quantitative and qualitative data within the same study of insight problem. Quantitative data comprises numeric data which can be counted and measured. Qualitative data seeks to ask why those numbers are what they are. It studies the traits and characteristics of those numbers.
You may do more comprehensive mixed method research using techniques like user testing, website findability, mobile testing, and A/B testing. When asking research questions, combining both quantitative and qualitative research is essential. The same applies when writing research summaries.
Why? Well, mixed methods research helps you gather true insights, answering both qualitative and quantitative questions. It also ensures you cover as many areas as possible in your research work.
In Closing
By nature, research is meant to be credible and reliable. Research is supposed to give accurate and true insights so it can provide the best multi-step solutions.
But because researchers are also human, false insights are still possible. The good news is, you can avoid these in your research.
Just be conscious of your own biases when formulating your research findings. Also, beware of cluster illusions, which make you confuse random occurrences for facts.
You should also be wary of the sunk cost fallacy, which makes you want to stick to your strategy because of the resources you’ve already invested. Survey the right people, too. Finally, adopt a mixed methods approach.
Follow these tips and you’ll avoid false insights in your research. You’ll ensure only the data speaks for itself.
- How to Avoid False Insights in Your UX Research - January 22, 2023
Give feedback about this article
Were sorry to hear about that, give us a chance to improve.
