The DIY research trend is gaining traction as it allows marketers, product managers, innovators, designers for established businesses and start-ups to add consumer insight to their decision making in faster, easier and cheaper ways.
With impressive advancements in DIY tools, particularly in user experience, they have become a tantalizing option for any marketer who wants to think creatively to maximize time, money or just get more hands-on. Further, there are many marketers with research-prohibitive budgets who now have access to consumer insight. This is a major development in marketing, especially if you believe the mantra, “Some insight is better than no insight.” This idea, however, operates under the assumption that the DIY research will always provide you with accurate insight.
The truth is, DIY research can provide good consumer insight to give businesses guideposts and indicators of a project’s potential success. The risk, however, is knowing whether the insight is actually true.
HBR published a thought-provoking piece this month titled “Defend Your Research – The Internet Makes You Think You’re Smarter Than You Are” by Scott Berinato. The article sites a study conducted by Yale doctoral candidate, Matthew Fisher and colleagues Mariel Goddu and Frank Kiel. In the study, respondents answered a series of questions such as, “How is glass made?” and “Why are there phases of the moon?” Some respondents could search the Internet for answers. Others could not. Then, they were all asked other questions and were asked to assess their confidence in knowing the answers. The finding? Those who could do online searches in the first set of questions vastly over-estimated their own ability to answer new questions correctly.
The study’s author is talking about this finding in a more general sense, but as a market researcher, I can’t help but apply this phenomenon of “cognitive prosthesis” to our new exciting access to consumer insight in a variety of ways via the Internet.
When marketers have prolific access to information via DIY tools like Survey Monkey and others, or even just Internet search, could that access similarly lead to increased confidence that they actually possess more insight about their customers than they actually do?
Could it lead to a false sense of intimacy, ultimately resulting in a misguided gut?
This phenomenon, to me, feels somewhat like a new form of bias that marketers should be aware of and vigilantly protect against.
It also calls to mind a heightened responsibility for professional market researchers to demonstrate their value in the midst of DIY. For example:
- Using experience and expertise in problem framing to help clients ensure the right questions is being asked in the first place. Often, simply asking the wrong questions is the culprit of unactionable and impotent insight.
- Leveraging basic principles of human behavior to guide methods for eliciting insight. It is well known in our field that simply asking people how they feel, how well they like something or whether they will buy it won’t cut it. It takes years of study and/or experience to master the art of inquiry. Without this, you may risk gathering answers, but not necessarily truth.
- Searching for the big idea. Most of the time, the ah-ha moment doesn’t just reveal itself in a data set, or in interactions with consumers. Usually, you have to dig for it. Looking at data in different ways, using statistical techniques, and using proven interviewing techniques that turn questions around are essential to revealing an insight that is new and interesting.
The study authors suggest that “cognitive prosthesis” can be a problem because “in some professions, we want people to be truly knowledgeable, and not have a false sense of knowledge.” (Their example is doctors). But in the market research world, this idea of “metacognitive awareness” (people’s ability to assess how well they can explain things around them) can be considered another form of bias. An emotional investment in something, leading one to think they know more than they really do.
As marketers consider how DIY fits in their overall insight program, it will be important to make an honest assessment of the biases that may be at play and seek outside sources to help frame problems, ask questions and hunt for revelations that can actually impact business.