12
Experimental
Research
©2013 Cengage Learning. All Rights Reserved. May
not be scanned, copied or duplicated, or posted to a
publicly accessible
website,
in whole
or in part.
©2013
Cengage
Learning.
All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
LEARNING
OUTCOMES
1. Identify the independent variable,
dependent variable, and construct a
valid simple experiment to assess a
cause and effect relationship
2. Understand and minimize experimental
error
3. Know ways of minimizing experimental
demand characteristics
4. Avoid unethical experimental practices
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12-2
LEARNING
OUTCOMES
5. Understand the advantages of a
between-subjects experimental design
6. Weigh the trade-off between internal
and external validity
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12-3
Testing Web Protocols for
Financial Markets
• Technology has drastically
changed financial services.
• How it all occurs is vitally
important to IT directors.
• Researchers designed a
laboratory experiment to
compare performance of
two electronic protocols
(FIX or Financial
Information eXchange and
SOAP or Simple Object
Access Protocol).
• FIX performed better.
• Answers the question "How
does the performance of
these two approaches
compare?"
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12-4
Creating an Experiment
• Subjects
• The sampling units for an experiment, are
usually human respondents who provide
measures based on the experimental
manipulation.
• Independent Variables
• Experimental conditions
• Blocking variables
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–5
Creating an Experiment
(cont’d)
• Main Effect
• The experimental difference in
dependent variable means between the
different levels of any single
experimental variable.
• Interaction Effect
• Differences in dependent variable
means due to a specific combination of
independent variables.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–6
EXHIBIT 12.3
Experimental Graph Showing Results
Within Each Condition
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–7
Designing an Experiment to
Minimize Experimental Error
• Manipulation of the Independent
Variable
• Experimental treatment: the way an
experimental variable is manipulated.
• Experimental Group
• Control Group
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–8
Designing an Experiment to
Minimize Experimental Error
• More than One Independent Variable
• Repeated Measures
• Selection and Measurement of the
Dependent Variable
• Selecting dependent variables that are
relevant and truly represent an outcome of
interest is crucial.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–9
Selection and Assignment of Test
Units
• Sample Selection And Random
Sampling Errors
• Systematic or nonsampling error
• Overcoming sampling errors
◗ Randomization
◗ Matching
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–10
Sample Selection And Random
Sampling Errors
• Experimental Confound
◗ When there is an alternative explanation
beyond the experimental variables for any
observed differences in the dependent variable.
◗ Once a potential confound is identified, the
validity of the experiment is severely
questioned.
• Extraneous variables
◗ Variables that naturally exist in the environment
that may have some systematic effect on the
dependent variable.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–11
Demand Characteristics
• Demand Characteristic
• An experimental design element or procedure
that unintentionally provides subjects with
hints about the research hypothesis.
• Demand Effect
• Occurs when demand characteristics actually
affect the dependent variable.
• Hawthorne Effect
• People will perform differently from normal
when they know they are experimental
subjects.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–12
Reducing Demand
Characteristics
• Use an experimental disguise
• Isolate experimental subjects
• Use a “blind” experimental administrator
• Administer only one experimental
treatment level to each subject
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–13
Establishing Control
• Constancy of Conditions
• Subjects in all experimental groups are
exposed to identical conditions except for
the differing experimental treatments.
• Counterbalancing
• Attempts to eliminate the confounding
effects of order of presentation by varying
the order of presentation (exposure) of
treatments to subject groups.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–14
Ethical Issues in
Experimentation
• Debriefing experimental subjects
• Communicating the purpose of the
experiment
• Explaining the researcher’s hypotheses
• Attempts to interfere with a competitor’s
test-marketing efforts
• Such acts as changing prices or increasing
advertising to influence (confound)
competitors’ test-marketing results are
ethically questionable.
12–15
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Elbow Partner Chat
Review the question and have a chat with a
partner or group of three. Be prepared to
share with the class.
• Suppose researchers were experimenting
with how much more satisfied consumers are
with a "new and improved" version of some
existing product. How might the researcher
design a placebo within an experiment
testing this research question? Is using such a
placebo ethical or not?
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
16
Practical Experimental Design Issues
• Basic versus Factorial Experimental Designs
• Basic experimental designs – a single
independent variable and a single dependent
variable.
• Factorial experimental design – allows for an
investigation of the interaction to two or more
independent variables.
• Laboratory Experiment
• Field Experiments
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–17
The Hidden in Hidden Valley
Ranch
• The company tested
three new flavours,
and all of them sold
out within three days
• Not so fast…a
competitor bought
them all!
• Illustrates the risk
associated with field
tests
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12-18
Within-Subjects and BetweenSubjects Designs
• Within-Subjects Design
• Involves repeated measures because with
each treatment the same subject is measured.
• Between-Subjects Design
• Each subject receives only one treatment
combination.
• Usually advantageous although they are
usually more costly.
• Validity is usually higher.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–19
Within- and Between-Subjects
Designs
EXHIBIT 12.6
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–20
Issues of Experimental Validity
• Internal Validity
• The extent an experimental variable is truly
responsible for any variance in the
dependent variable.
• Manipulation Checks
• A validity test of an experimental
manipulation to make sure that the
manipulation does produce differences in
the independent variable.
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–21
Effects of Extraneous Variables on
Validity
• History Effect
• Cohort Effect
• Maturation Effects
• Testing effects
• Instrumentation Effect
• Selection
• Mortality Effect (Sample Attrition)
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–22
External Validity
• External Validity – the accuracy with which
experimental results can be generalized
beyond the experimental subjects.
• Increased when the subjects in the sample
represent the population.
• The higher the external validity the more
researchers can count on the fact that any
results observed in an experiment will also
be seen in the “real world.”
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
12–23