Getting It Right:
A Checklist for Avoiding Market Research Mistakes
Copyright 2008 by Greenhouse Associates, Inc.
This document may be distributed free of charge provided that the document is
distributed in its entirety, including copyright notices and attribution to Greenhouse
Associates, Inc.
For any questions, please contact us:
Greenhouse Associates, Inc.
2305 North Commonwealth Avenue
Chicago, IL 60614
(773) 404-0780
info@greenhousegrows.com
A Checklist for Avoiding Market Research Mistakes
We conduct market research extensively in our consulting practice. Over the years,
we’ve evolved our approaches and established some guiding principles which we think
are worth sharing:
Use market research as a tool, not the end result: We typically conduct market
research as part of a larger consulting engagement to gain insights required to advise our
clients. For that reason, our market research goes beyond basic fact finding to draw
implications and recommendations. It’s deceptively easy with market research to gather
lots of information, but far harder to make sense of the results. Effective market research
should always ask, “So what?” regarding every fact or observation it gets from its basic
fact finding.
Work backwards: Since market research is a tool for informing decision-making, it
makes sense that all market research projects should start by isolating the critical business
decisions under consideration and should then break them into specific issues. Only after
completing these two steps should one formulate specific questions to be asked of
respondents.
Focus on “must know” information: In our experience, during the process of
organizing a market research project, there is a tendency to expand it by piling on
questions that are nice to know, but not critical to the original inquiry. In some
organizations, market research can be like a college party. Once word gets out, everyone
wants to participate, resulting in research that gets burdened and diluted by questions that
are not critical to the fundamental business decisions at hand.
Get to “Why?” Market research gets more mileage when it reveals underlying needs and
attitudes, rather than just outward behavior. Understanding such motivators can help in
projecting potential market behavior as opposed to asking customers to project their own
behavior. Corollary: Let market research trump common wisdom, not common sense.
Market research can confirm or refute commonly-held views. But when market research
defies common sense, one should challenge it and keep asking “why?” until the results
can be explained logically.
Avoid speculative questions: Market research gets increasingly unreliable as it asks
respondents to project their own behavior. “Would you want a dog that flies?” is a
question that would be difficult for anyone to answer with certainty because of their lack
of experience with such a creature. History is littered with famous stories about falsely
positive or negative research on new products or services. Hewlett-Packard’s early
market research said there was no market for a handheld calculator. Fortunately, CEO
Bill Hewlett discarded the research and pushed ahead with the product.
A Checklist for Avoiding Market Research Mistakes
1
Combine research techniques: While every situation demands its own research plan, we
like approaches that combine qualitative with quantitative research. Online surveys
appear deceptively easy to conduct, but they are surprising difficult to do well, given the
complexity of question development required to actually elicit the information required.
In our experience, online surveys can be far more effective when preceded by at least a
small set of qualitative research to inform the inquiry process. In our experience,
however, online surveys can be far more effective when preceded by at least a small set
of qualitative research to inform them. Furthermore, after completing an online survey,
we favor conducting in-depth qualitative interviews to further probe issues that were
revealed, but not necessarily fully explained, by the online research. It’s an informative
research sandwich that is well worth the extra time and expense.
Apply an appropriate segmentation approach: Whether conducting research among
consumers or businesses, it is critical to divide research subjects into groups with distinct
characteristics so that their responses can be compared. Segmentation schemas most
often use outward characteristics (e.g., size, location, or spending levels). But we’ve
sometimes found that more subtle, behavioral characteristics are more appropriate. For
example, in conducting research among law firms, we have found that size, the traditional
way to segment firms, is less useful than their type of practice. Similarly, among
institutional investment managers, the traditional segmentation approach, by assets under
management, is less informative than investment strategy style, although investment
strategy is not a traditional segmentation criterion. In business research, it is also critical
to distinguish end users from buyers and decision-makers as their needs, attitudes, and
behaviors are often different.
Beware of distortion: Research subjects can respond in ways that intentionally or
unintentionally mask their true feelings. For example, respondents may answer questions
about pricing that they think serves their economic interests, especially if they sense that
research is being conducted by a vendor. Instead of asking such difficult questions
directly, we look for proxies for likely behavior. Rather than asking how much someone
would be willing to pay for a new product or service, we try asking how much they
currently spend (in time as well as money) and how satisfied they are with their current
ways to accomplish the same result.
Enable unexpected results: Effective market research starts with hypotheses, but must
be sufficiently open to wind up with dramatically different conclusions – sometimes far
outside the realm of expected possible answers. Expecting unexpected answers should be
axiomatic in any research, but it’s even more important to design the research so that
truly unexpected results can bubble up. In one consulting engagement, our client, the
CEO of a financial information company, was convinced that his CTO was underdelivering on product technology. Our research reached a far different conclusion: the
technology was meeting customer needs, but the company had failed to market its
product effectively.
A Checklist for Avoiding Market Research Mistakes
2
Make no small plans: Despite the seeming ease with which organizations often
undertake it (especially online surveys), all market research deserves a high level of
diligence. The risk is that bad market research can have as much adverse impact as good
market research can inform and assist.
Market research isn’t just for expansionary times. It is an essential tool when resources
are short and companies need to make difficult decisions about their allocation of
resources. By providing objective evidence, market research can be an effective tool for
clarifying organizational priorities and aligning differing views. Market research is just
as valuable when it validates a company’s strategy as when it causes a change in
direction. One of our client CEOs told us recently, “Your research mostly told us what
we already thought we knew. That was very valuable.”
We are interested in your thoughts on this topic. Please feel free to contact us at
editor@greenhousegrows.com or 773-404-0780.
Greenhouse Associates is a strategic consulting firm serving clients in publishing,
media, online information services, and technology. Our mission is to help clients
develop and grow their digital information businesses through realistic strategies and
practical guidance during execution. Our advisory services span a broad range of
needs, including market research, business planning, competitive analysis, business
development, due diligence, and process improvement.
For more information, please visit www.greenhousegrows.com
A Checklist for Avoiding Market Research Mistakes
3