Research Design Basics
MAR 6648: Marketing Research
January 11, 2011
Overview
• What are the basic types of research?
– What are their goals? What are their limitations?
– How are they done?
• What are some important considerations in
research design?
– How do we know when we’re asking the right
questions?
The Marketing Research Process
Goal formulation
• Define research purpose and objectives
Research design
• Exploratory, descriptive, causal
Data collection
• Databases, survey design, sampling, target respondents
Data analysis and interpretation
Final report
• Communicate findings and recommendations
Research Design
• …Is the detailed blueprint used to guide the
implementation of the research study to
realize its objectives
• The research design is an integral part of the
research proposal
Three Types of Research Designs
• 1. Exploratory
– Used to gain ideas
• 2. Descriptive
– Used to provide numerical information
• 3. Causal
– Used to establish causality
Exploratory Research
• Uses:
– Get ideas
– Develop hypotheses
– Establish priorities for future research
– Provide inputs to questionnaires
• A lot of exploratory research is qualitative in
nature
Exploratory Research
• Typically not used to support major decisions
(without further research)
• High degree of flexibility
– Less structured than other types of research
– Ad hoc in nature
• Number of respondents is small and not
representative of the population
Exploratory Research
• Often uses secondary data
• Sources:
– Census data and government reports
– Trade magazines and reports
– Published research reports
• A good resource is marketresearch.com
• In some cases this can provide all the
information a project needs
• Always check sources for potential bias
Exploratory Research
• Primary data
Observations
Depth interviews
Focus groups
Projective techniques
Descriptive Research
• Give us the answers to…
– Who?
– What?
– Where?
– When?
Descriptive Research
• Typically concerned with:
– Characterizing customers’ demographics,
behavior, and attitudes
– Characterizing competitors’ behavior
• Often guided by initial hypotheses
• Descriptive research designs are much more
rigid than exploratory designs
Example: Segmentation
• MyBestSegments
Causal Research
• Concerned with determining cause-and-effect
relationships
• Usually takes the form of experiments
• Highly structured designs
– Emphasis is on “control”
Really?
A True Experiment
Then:
Vs.
Test mice:
they get the
Alzheimer’s
drug
Control mice:
they don’t get
the Alzheimer’s
drug
Vs.
Causal Research
• Aims to conduct experiments in marketing
– Field experiments
– Test markets
– Lab experiments
• Key question: “When have we identified a true
causal effect?”
– I.e., causal research tells us “why”
Key Points
• Research Design is the blueprint of your
research project
• There are three types of research design:
exploratory, descriptive, and causal
• Projects usually start with exploratory
research, followed by either descriptive or
causal research
Two types of research
Qualitative
• Seeks an in-depth
understanding of behaviors
and attitudes using a small
sample of consumers
• Interested in “qualities”
Quantitative
• Seeks to measure behaviors
and attitudes using
statistically valid samples
• Interested in “quantities”
Qualitative versus Quantitative
I
believe
in the
cause
I feel
less
guilty
Why do
you
donate?
29%
Tax
benefit
s
I don’t
Qualitative: What are the reasons?
5%
Why do
you
donate?
12%
54%
Quantitative: The magnitude of the reasons
Three kinds of research
• Descriptive
– Basic purpose: To observe and record behavior
– How? Case studies, surveys, or observation
– Possible problems
• What if you observe a weird sample?
• What if you don’t watch the right things, or worse,
watch only what you want (confirmation bias)?
– Ask the right questions, or the answers you get will be
uninformative
Three kinds of research
• Correlational
– Basic purpose: To detect naturally occurring
relationships; to see how well one variable predicts
another
– How? Computing statistical relationships, in surveys,
etc.
– Ex. Smoking and cancer, etc.
– Possible problems
• Doesn’t specify cause and effect!
– Kids with bigger shoes are smarter?
• Also, doesn’t specify cause and effect!
The third variable
• We know that watching violence on TV and
aggression in kids is correlated
• But what causes what?
Watching violent TV
Aggressive behavior
Aggressive personality
Neglectful parents
Watching violent TV Aggressive behavior Watching violent TV
Three kinds of research
• Experimental
– The crème de la crème of consumer behavior
research
– Basic purpose: To explore cause and effect
– How? Manipulate one or more factors; use random
assignment; always have a control group
• Independent vs. dependent variables—know which is
which!
– Can be done large- or small-scale, in lab or in field
– Possible problems
• Sometimes not possible ethically
• …or practically
What kind of research is this?
• Researchers poll people in a bar and find that
ratings of the attractiveness of the other sex
increase as it nears closing time
• What kind of research?
• Why might the ratings increase this way?
• Can you improve this study, and show their
finding in a different way?
What kind of research is this?
• You spend an afternoon watching people in a
park and find that more people like to sit on
the grass than on the benches that are
provided
• What kind of research?
• Why should you be careful in interpreting this
result?
What kind of research is this?
• Researchers have some people watch a TV
show with commercials, and others watch the
same show without commercials, to see if
people enjoy the show more or less with the
commercials
• What kind of research is this?
• Any concerns with this set-up?
Key Points
• There are many ways to look at and think
about research designs
– Keep in mind what you want to get out of your
research—a broad picture, or a detailed picture?
– Are you interested in ideas, or numbers and
predictions?
– What are potential limitations on how you can use
your results?
Coke versus Pepsi
• Coke’s market share had
slipped from 60% in the
1940s to less than 24%
in 1983
• What was Coke’s
response?
• Make it sweeter!
Introducing New Coke
• Coke’s Research Strategy
– 2000 interviews in 10 major markets
• Storyboards and mock advertisements: Would you try
it? Would you switch? Would you be upset?
– Focus groups: Mixed results; loyal drinkers were
unfavorable toward the idea
– Blind taste tests: n = 30,000-40,000
• New versus old Coke: 55% vs. 45%
• New versus Pepsi: 54% vs. 46%
– Blind taste tests of all together: n = 191,000
• New Coke won!
But…
• After introducing New Coke on April 23, 1985,
they were forced by consumers and bottlers to
bring the old formula back just 77 days later
• What did Coke do wrong?
Good Brands and Good Taste
•
What’s in a Brand Name?
–
Brand Name vs. No Name (Blind Taste Tests)
75.3
Wise
21.5
Ruffles
75.5
27.5
Pringles
30.7
4.9
Lays
46.4
21.8
0
20
Branded
Blind
40
60
80
• The brand made the chips “crispier,” “less
greasy,” “fresher,” and “more flavorful”!
Vodka: Taste vs. Brands
• The New York Times conducted a blind taste test (Jan. 26,
2005)
– Experts predicted premium would be best (e.g., Grey Goose Vodka)
– In a blind taste test, the least expensive was the best: Smirnoff
McClure et al. (2004)
Implication = Brands change our
fundamental consumption
experience. Branded Coke actually
tastes better.
• People who like “Coke” with
a brand, but not without.
• Without brand, brain
responds as it would to any
tasty substance
• With brand, brain responds
in regions associated with
emotions, self-control, and
well-being
Three Goals
• Validity, Reliability, Representativeness
– Validity: Asking the right question
• Problem definition and research objectives
– Reliability: Receiving the right response
• Reducing response error: Methods/questions
– Representativeness: Generalizing results
• Reducing non-response error: Sampling
Two types of validity
• Internal validity
– Does your independent variable actually influence
your dependent variable?
– In other words, are you measuring what you think
you’re measuring?
Two types of validity
• External validity
– The extent to which your research results can be
generalized beyond the specific settings,
participants, and materials used in the study to
other people, places, and things
– In other words, are you measuring something real
or broad, or something specific?
A third kind
• Face validity
– Is something valid on its face?
• Shouldn’t be the basis for deciding on your
materials, but can help narrow them down
Reliability
• You want to make sure that your data is as
free from random error as possible
• Test-retest reliability:
– Does your test give you the same responses if the
same people take it more than once?
• Equivalent forms reliability:
– Does your test give you similar results as other
similar measures?
Sampling
• Who is going to help you answer your
questions?
• You can’t ask everybody
– Well , sometimes you can: a census
• But usually you can’t, so you need to pick a
sample
Sample
Population
Sampling
• What is the population you’re interested in?
• You’ll want to be specific , but not so specific
that your sample can’t be found or
reproduced
– Look to your research objectives
– If you’ve thought them through, the target
population should be clear
Sampling Frame
• Once you know the population you’re
sampling from, how do you pick the sample?
– How many will you choose?
Sample
Sample Frame
Population
Sampling
Probabilistic Sampling
• Random Sampling
• Systematic Sampling
• Cluster Sampling
• Systematic Sampling
Nonprobabilistic Sampling
• Convenience Sampling
• Judgmental Sampling
• Snowball Sampling
• Quota Sampling
What if your sample doesn’t cooperate?
• Non-response
– What if people say no? What if they don’t answer
the phone?
– How do you make sure you get enough people?
• Self-selection
– The people who agree to do your survey may be
different from those who don’t
Key Points
• Remember what you’re asking people to do
– Does it match consumers’ experience with the
product?
– Does the question make sense? Does it ask what
you thinking you’re asking?
– Will you get the same answer if you ask them
again, or in a different way?
• Remember who you’re asking it of
– Does your sample reflect some of your
consumers, or all of them?
Summary
• There are three types of research we’ll talk about
in this course: exploratory, descriptive, and causal
– These are similar categories as those used in basic
research
• There are three issues to be aware of in
conducting research: validity, reliability, and
representativeness
– Keep them in mind as you create your materials
– Are you asking what you think you’re asking, are you
asking only that, and who are you asking it of?