Experimental Design
Chapter 1
Research Strategies
and the Control of
Nuisance Variables
Introduction

Experimental design
–
–
A plan for assigning subjects to experimental
conditions and the statistical analysis associated
with the plan
Identifies the independent, dependent, and
nuisance variables and indicates the way in which
the randomization and statistical aspects of an
experiment are to be carried out
Formulation of Plans for Collection
and Analysis of Data



Acceptable research hypotheses
Distinction between independent and
dependent variables
Selection of independent variable
–
–

Selection of the dependent variable
–

Quantitative independent variables
Qualitative independent variables
Multivariate designs
Nuisance variables
–
Bias
Research Strategies





Experiments
Quasi-experiments
Surveys
Case studies
Naturalistic observation
Other Research Strategies

Ex post facto studies
–



Retrospective and prospective studies
Longitudinal and cross-sectional studies
Longitudinal-overlapping and time-lag studies
Time-series and single-subject studies
Threats to Valid Inference Making

Four categories of threats to valid inference
making
1.
2.
3.
4.
Statistical conclusion validity
Internal validity
Construct validity of causes and effects
External validity
Threats to Statistical Conclusion
Validity
1.
2.
3.
4.
5.
6.
7.
Low statistical power
Violated assumptions of statistical tests
Fishing for significant results and the error
rate problem
Reliability of measures
Reliability of treatment implementation
Random irrelevancies in the experimental
setting
Random heterogeneity of respondents
Threats to Internal Validity
1.
History
•
2.
Maturation

3.
Processes not related to treatment that occur
within the subject
Testing

4.
Events other than the treatment that occur
between the time the treatment level is assigned
and the DV is measured
Repeated testing may result in familiarity
Instrumentation

Changes in calibration of a measuring instrument
Threats to Internal Validity
5.
Statistical regression

6.
Selection

7.
Prior differences in subjects
Mortality

8.
Occurs when the measurement of the DV is not
perfectly reliable
Loss of subjects
Interactions with selection

Selection-history effects or Selection-maturation
effects
Threats to Internal Validity
9.
10.
Ambiguity about the direction of causal
influence
Diffusion or imitation of treatments

11.
12.
If the subjects in different levels can communicate,
differences may be compromised
Compensatory rivalry by respondents
receiving less desirable treatments
Resentful demoralization of respondents
receiving less desirable treatments
Threats to External Validity
1.
2.
3.
4.
5.
6.
Interaction of testing and treatment
Interaction of selection and treatment
Interaction of setting and treatment
Interaction of history and treatment
Reactive arrangements
Multiple-treatment interference
Other Threats to Valid Inference
Making



Experimenter-expectancy effect
Demand characteristics
Subject-predisposition effects
–
–
–
–

Cooperative-subject effect
Screw you effect
Evaluation apprehension
Faithful subjects
Placebo effect
Controlling Nuisance Variables

General Approaches to Control
–
–

Experimental control
Statistical control
Some Specific Approaches to Control
–
–
–
–
Blind procedures
Deception
Unobtrusive experimentation
Multiple researchers
Controlling Nuisance Variables

Some Specific Approaches to Control
–
–
–
–
–
Debriefing
Experimenter-expectancy control groups
Unrelated-experiment technique
Quasi-control group
Yoked control procedure
Ethical Treatment of Subjects
1.
2.
3.
4.
5.
6.
7.
8.
Responsibility of researcher
Informed consent
Protection from harm
Protection of rights
Research deception
Confidentiality
Debriefing
Accurate and reporting