Quasi Experiments
Non-Experimental Research
Research Methods & Statistics
Fall 2011
Quasi Experiments
• “Research procedure in which the scientist must select
subjects for different conditions from preexisting groups”
– Research Methods, McBurney & White
• “An empirical study used to estimate the causal impact of
an intervention on its target population. Quasiexperimental research designs share many similarities with
the traditional experimental design or randomized
controlled trial, but they specifically lack the element of
random assignment to treatment or control. Instead, quasiexperimental designs typically allow the researcher to
control the assignment to the treatment condition, but
using some criterion other than random assignment.
– Wikipedia, Nov. 17, 2011
It’s a matter of control
True Experiment
• Random assignment of
subjects to condition
• Manipulate the IV
Quasi Experiment
• Selection of subjects for the
conditions
• Observe categories of
subjects
– If the subject variable is the
IV, it’s a quasi experiment
• Control allows ruling out of
alternative hypotheses
• Don’t know whether
differences are caused by
the IV or differences in the
subjects
Other features
• In some instances cannot completely control
the what, when, where, and how
– Need to collect data at a certain time or not at all
– Practical limitations to data collection,
experimental protocol
Validity
• Internal validity is reduced due to the
presence of controlled/confounded variables
– But not necessarily invalid
• It’s important for the researcher to evaluate
the likelihood that there are alternative
hypotheses for observed differences
– Need to convince self and audience of the validity
External validity
• If the experimental setting more closely
replicates the setting of interest, external
validity can be higher than a true experiment
run in a controlled lab setting
• Often comes down to what is most important
for the research question
– Control or ecological validity?
Nonequivalent Control Group Designs
• 2 groups, non-random allocation of subjects
and groups, pre-test, treatment (Y/N), post-test
• Desired pattern:
– Dependent variables have equal pre-test values,
difference seen between experimental and control
groups on post-test (Pre: LL, Post: HL)
– Want to show that any differences that exist did
not impact the value of the variable of interest
Exercise
• Interpret the following graphs (experimental,
control):
– Pre: (L, H) Post: (M, H)
– Pre: (L, M) Post: (H, ML)
– Pre: (L, M) Post: (H, M)
Mixed Factorial Design with One Nonmanipulated Variable
• Example: experiment on pain perception (effect of
caffeine, expected differences between men and
women)
• Protocol:
– 25 men/25 women, each takes part in two sessions, one
week apart
– One session: drink coffee (decaf) and put hand in ice-water
until feel pain
– Other session: drink coffee (caffeinated) and put hand in
ice-water until feel pain
• Between subjects variable (male/female)
• Within subjects variable (caffeine intake)
Non-Experimental Research
Read and understand:
• Gabriella Belli - Nonexperimental Quantitative
Resarch
• “Any quantitative study without manipulation
of treatments or random assignment is a nonexperimental study”
• Experimental research shows cause and effect
• Non-experimental research studies variables
as they exist
Purpose
• Descriptive:
– primary focus for the research is to describe some
phenomenon or to document its characteristics.
• Predictive:
– primary focus for the research is to predict some
variable of interest (criterion) using information from
other variables (predictors).
• Explanatory :
– the primary focus for the research is to explain how
some phenomenon works or why it operates.
Time Frame
• Cross-sectional:
– data are collected at one point in time, often in order
to make comparisons across different types of
respondents or participants.
• Prospective (longitudinal):
– data are collected on multiple occasions starting with
the present and going into the future for comparisons
across time.
• Retrospective:
– look back in time using existing or available data to
explain or explore an existing occurrence.
Techniques
• Surveys
– http://www.socialresearchmethods.net/kb/survey.php
• Interviews
– http://www.socialresearchmethods.net/kb/intrview.php
• Observations
– http://interactionarchitect.com/knowledge/article19991212shd.htm
– http://www.edu.plymouth.ac.uk/resined/observation/obshome.htm
• Unobtrusive methods:
– http://www.socialresearchmethods.net/kb/unobtrus.php