MARKETING RESEARCH
ESSENTIALS
WITH DATA ANALYSIS IN EXCEL AND SPAA
McDaniel │ Gates │ Sivaramakrishnan │ Main
Chapter Ten:
Primary Data Collection: Experimentation and Test Marketing
LEARNING OBJECTIVES
Chapter Ten: Primary Data Collection: Experimentation and Test Marketing
•
•
•
•
Understand the nature of experiments
Explain the requirements for proving causation
Describe the experimental setting
Explain experimental validity and discuss how
researchers control for extraneous variables that might
affect validity
• Compare types of experimental designs
• Understand test marketing
• Explain the limitations of experimentation in marketing
research
What Is an Experiment?
•
Independent Variable:
–
•
Dependent Variable:
–
•
Sales, Customer Satisfaction
Treatment:
–
•
Price, Packaging, Distribution, Product, Advertising
Independent variable manipulated to measure its effect on
the dependent variable
Extraneous Factors:
–
Things you do not control, such as the weather
Demonstrating Causation
• Determine whether a change in one variable likely
caused an observed change in another
• Causal relationships must show three things:
1. Concomitant variation
2. Appropriate time order of occurrence
3. Elimination of other possible causal factors
Demonstrating Causation
• Concomitant Variation:
– A statistical relationship between variables
• Appropriate Time Order of Occurrence:
• Change in an independent variable occurred before an observed
change in the dependent variable
• Elimination of Other Possible Causal Factors:
– No other independent variables are causing the change
Experimental Setting
• Laboratory:
– Experiments conducted in a controlled setting
• Field:
– Tests conducted outside the laboratory in an actual environment,
such as a marketplace
Experimental Validity
• Internal Validity:
– The extent to which competing explanations for the experimental
results observed can be ruled out.
• External Validity:
– The extent to which causal relationships measured in an
experiment can be generalized to outside persons, settings, and
times.
Extraneous Variables
• History:
– Intervention, between beginning and end of experiment, of outside
variables that might change the dependent variable
• Maturation:
– Changes in subjects occurring during the experiment not related to
the experiment but which might affect subjects’ response to the
treatment factor
• Instrument Variation:
– Changes in measurement instruments (e.g., interviews or
observers) that might affect measurements
Extraneous Variables
• Selection Bias:
– Systematic differences between the test group and the control
group due to a biased selection process
• Mortality:
– Loss of test units / subjects during the course of an experiment
• Testing Effect:
– An effect that is a by-product of the research process
• Regression to the Mean:
– Tendency of subjects with extreme behaviour to move toward the
average for that behaviour during the course of the experiment
Controlling Extraneous Variables
• Randomization
– The random assignment of subjects to treatment conditions to
ensure equal representation of subject characteristics
• Physical Control
– Holding constant the value or level of extraneous variables
throughout the course of an experiment
• Design Control
– Use of experimental design to control extraneous causal factors
• Statistical Control
– Adjusting for effects of extraneous variables by adjusting the value
or the dependant variable for each treatment condition
Experimental Treatment and Effects
• Experimental Design
– A test in which the researcher has control over and manipulates
one or more independent variables
• Treatment Variable
– The independent variable that is manipulated in an experiment
• Experimental Effect
– The independent variable that is manipulated in an experiment
Experimental Terms
• Test Group
– Group is exposed to manipulation (change) of independent
variable
• Control Group
– Group in which the independent variable is not changed
– Group is used for comparison
• Experimental Group
– Effect of the treatment variable on the dependent variable
Experimental Notation
• “X” = Independent Variable:
– Exposure of an individual or group to experimental treatment
– Variable is something the researcher can change
– Goal is to test if the change in the independent variable will cause
a change in the dependent variable
• “O” = Dependent Variable:
– A variable the researcher cannot change directly
– Test to see if changing the independent variable will result in
changes to the dependent variable.
– The dependent variable is “dependent” upon what the researcher
does with the independent variable
Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
One-Shot Case Study Design
• X
O1
Change the independent variable, then measure the change
in the dependent variable to see if there was, in fact, a
change in the dependent variable that the researcher might
conclude resulted from the change in the independent
variable
Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
One-Group Pre-Test–Post-test Design
• O1 X
O2
Same as one-shot design except measure the dependent
variable before the change in the independent variables. The
researcher is establishing a benchmark from which to gauge
the change.
Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Before and After with Control Group Design
Experimental Group:
Control Group:
O1
O3
X
O4
O2
A true experimental design that involves random assignment
of subjects or test units to experimental and control groups
and pre- and post-measurements of both groups.
Selected Experimental Designs
O = The Measurement of the Dependent Variable
X = The Manipulation / Change of Independent Variable
E = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
After-Only with Control Group Design
Experimental Group:
Control Group:
X
O1
O2
Subjects in the experiment are randomly assigned to
experiment and control groups. No pre-measurements of the
dependent variable are taken.
Quasi-Experiments
• Interrupted Time-Series
– Research in which repeated measurement of an effect “interrupts”
previous data patterns
• Multiple Time-Series:
– Interrupted time-series design with a control group
Test Markets: Types
•
•
•
•
Standard
Scanner
Controlled
Simulated (STM)
Test Markets: Costs
• Direct Costs:
–
–
–
–
Advertising expenses
Syndicated research
Coupons, sampling, POP materials
Trade allowances
• Indirect Costs:
–
–
–
–
Diversion of activity from existing products
Possible negative impact of test failure
Possible negative trade reactions
Competition becomes aware of new product idea
Six Steps in a Test Market Study
1. Define the Objective:
– What do you hope to learn?
– What are the characteristics of the people / products of interest?
2. Select a Basic Approach:
– Simulated, controlled, scanner, or standard test?
3. Develop Detailed Test Procedures:
– Generate a full marketing plan
– Determine positioning approach
Six Steps in a Test Market Study
4. Select the Test Market:
–
–
–
–
–
Market should not be over tested
Should have little media spillover
Demographics should be similar to your target population
Market should be large enough to provide useful results
Distribution and other patterns should be similar to the nation
5. Execute the Plan:
– Determine length of time of test
– Finalize the parties involved
6. Analyze the Results:
– Use qualitative and quantitative techniques when possible
Limitations of Experimental Research
• High Cost:
– Is the research affordable?
– Will the research be beneficial and help solve problems?
– Has a cost and benefit analysis been done?
• Security Issues:
– Particularly critical with field experiments
– The competition might be “tipped-off”
– Are the data and findings secure?
• Implementation Problems:
– People who unwittingly get caught in the experiment
– Outside factors unnaturally affecting the experiment
– Participants who intentionally try to skew the results
Copyright
Copyright © 2014 John Wiley & Sons Canada, Ltd. All rights
reserved. Reproduction or translation of this work beyond
that permitted by Access Copyright (the Canadian copyright
licensing agency) is unlawful. Requests for further
information should be addressed to the Permissions
Department, John Wiley & Sons Canada, Ltd. The purchaser
may make back-up copies for his or her own use only and
not for distribution or resale. The author and the publisher
assume no responsibility for errors, omissions, or damages
caused by the use of these files or programs or from the use
of the information contained herein.