Modes of Observations
(Research Designs)
– Experiments
– Survey Research
– Field Research
– Unobtrusive Research
– Evaluation Research
• Each of these methods have different
strengths and weaknesses
– And different areas where they are more or
less useful
Criteria for establishing causation
in the social sciences
– Establish a relationship
– Establish time ordering
– Rule out spuriousness
• How do experiments do these things?
Critical Components of
Experiments
• Independent and Dependent Variables
– The “cause” is the experimental manipulation
• Pre-test and Post-test
• Experimental and Control Groups
– RANDOM ASSIGNMENT
• NOTE: This is different from random sampling
– Generalization (random sampling)
– Rule out spuriousness (random assignment)
The “Classic” Experimental Design
RANDOM ASSIGNMENT
SUBJECT
POOL
Treatment
Group
Control
Group
TIME
Pretest Treatment
Posttest
Pretest (Placebo)
Posttest
The Subjects
• Selection of a subject pool
– Effects the extent to which experimenter can
generalize
• Difficult to do “random sample” (must “recruit”
subjects).
• BUT  less of an issue for explanatory research
• Still, “Mice versus Men” (Or Men vs. Women)
• Assignment of subjects to groups
– Randomized
– Matching
Close, but no cigar…
Experimental designs that fall short
• The One-Shot case study
• The One-Group pretest-posttest design
• The Static-Group comparison
• NOTE
– Sometimes, to capitalize on “natural,”
phenomena, research will do these
“quasi-experiments” and try to rule out threats
to internal validity
Other Important Considerations
• Placebo
– The “Hawthorne effect”
• Double Blind Experiments
– Neither subjects (this is typically the case) nor
experimenters are aware of who is in the
control or experimental group
Threats to Internal Validity
•
•
•
•
•
•
•
•
History
Maturation
Testing
Instrumentation
Statistical regression
Selection bias
Experimental mortality
Demoralization
External Validity
• External validity = generalizability
– “Laboratory experiments” versus “the wilds of society”
– Example: the effect of “pre-testing” on subjects
• Subjects (both control and experimental) become sensitized
to the independent variable
– Results may not “work” in real world because people are
sensitized in the same way
– Solution: “Solomon Four-Group Design” (or better yet, no pretesting at all).
• Experiments in general are “artificial” and are
suspect with regard to generalizability
Natural Experiments
• Sometimes called “quasi-experiments”
– They lack one or more of the hallmarks of the
classic experimental design
•
•
•
•
Death penalty research
Low birth-weight study
Study of nuclear power plants
Prejudice and Roots
• Evaluation research often uses quasiexperimental designs
Experiments
• Review and Summary
– Classic experimental design (not used as
posttest only)
• Random assignment as key
– Advantages of Experimental Design?
– Disadvantages?
How does an experimental
design…
– Establish a relationship
– Establish time ordering
– Rule out spuriousness