Chapter 3
The Research Design
Research Design
• A research design is a plan of action for executing a research
project, specifying
•
•
•
•
•
The theory to be tested
The unit of analysis
The necessary observable data
Data-collection procedures
Analytical procedures
Research Design
• The choice of research design is affected by
• The purpose of the research
• Resources like time, money, and skill or ethical concerns
• Every research design has positive and negative attributes.
• You should match a design with your requirements.
Research Design
• All research designs attempt to
•
•
•
•
Establish a relationship between two or more variables
Demonstrate that the results are generally true
Establish whether one phenomenon precedes another in time
Eliminate as many alternative explanations as possible
Research Design
• The goal of scientific research is to find causal relationships.
• Causal relationships have three characteristics:
• Covariation: the alleged cause varies with the supposed effect
• Time order: the cause precedes the effect in time
• Elimination of alternative explanations to isolate causation to one
factor
• Need to avoid spurious relationships (hospital example)
Research Design
• When choosing a research design, it is also important to
consider
• Internal validity:
• Refers to a causal relationship that was not created by a spurious
relationship (a relationship in which a second independent variable
influenced the dependent variable)
• Effects: history, maturation, testing, selection biases, experimental
mortality, instrument decay, demand characteristics
• External validity:
• Refers to the extent to which the results of an experiment can be
generalized across populations, time, and settings
Experimental Design
• Experimental research designs are especially good for isolating
causal factors.
• Experimentation allows a researcher to make causal inferences
with great confidence in the design through control over
exposure to an experimental treatment.
• But, although experiments have great internal validity, they
suffer from weaker external validity.
Experimental Design
 The classical randomized experiment has five basic
characteristics:
1.
2.
3.
4.
5.
At least one experimental group that will have exposure to the
treatment and one control group that will not
Randomly assigned individuals to each group, avoiding selfselection
Controlled administration of the treatment, including the
circumstances under which the experimental group is exposed
Measurement of a dependent variable before and after the
treatment with a pre-test and a post-test; any difference
between the tests can be attributed to the experimental effect of
exposure to the treatment
Controlled environment of the experiment (time, location, and
other physical aspects)
Experimental Design
• Post-test design:
• Shares the characteristics of the classical randomized experiment,
except that no pre-test is used because the sample is truly random
and sufficiently large that one can assume that the control and
experimental group(s) are equivalent
Experimental Design
• Repeated-measurement design:
• Adds to the classic example additional pre-tests, post-tests, or
both in an effort to measure longer-term effects of experimental
treatments
Experimental Design
• Multigroup design:
• A modification of the classic example in which more than one
experimental group is created to compare the effects of different
treatments
Experimental Design
• Field experiment:
• An experiment in a natural setting in which the investigator does
not have control over group membership but does have control
over one or more independent variables
• Causal inferences made using this design are not as strong—but
may be more practical for some situations
Nonexperimental Design
• Nonexperimental designs are characterized by at least one of
the following:
–
–
–
–
Presence of a single group
Lack of control over the assignment of subjects to groups
Lack of control over the application of the independent variable
Inability to measure the dependent variable before and after
exposure to the independent variable occurs
Nonexperimental Design
• Small-N designs:
• Also called case studies or comparative cases studies
• Involve rich, deep understanding of a small number of cases
• May be used for exploratory, descriptive, or explanatory purposes
Nonexperimental Design
• Focus groups:
• Can be used to create hypotheses for testing through other
research designs
• Generally not used to establish causal relationships
Nonexperimental Design
• Cross-sectional designs (survey, aggregate analysis):
• Characterized by measurements of the independent and dependent
variables at approximately the same time
• Data analysis, rather than a treatment, is necessary for making
causal inferences
Nonexperimental Design
• Longitudinal designs:
• Allow for the measurement of variables at different points in time
• Can model change across time; examine the time order of a causal
relationship; and estimate age, cohort, and period effects
Nonexperimental Design
• Trend analysis:
• Analysis of variables measured across periods of generally 20
years or more with a focus on explaining change over time
Nonexperimental Design
• Panel studies:
• Cross-sectional designs that include a time element
• Rely on measurement of the same units of analysis at different
points in time—creating waves of data for analysis over time
• Panel mortality
Nonexperimental Design
• Intervention analysis:
• Measurements of a dependent variable before and after the
introduction of an independent variable that is observed but not
controlled by the researcher
Nonexperimental Design
• Nonexperimental designs are generally characterized as having
less internal reliability but better external validity than
experimental designs.
• There is always a tradeoff when moving from one design to
another.