Research Designs
REVIEW
Review -- research
•
General types of research
– Descriptive (“what”)
– Exploratory (find out enough
to ask “why”)
– Explanatory (“why”)
•
Unit of analysis: “object, entity
or process” under study
– Contains the variables
being measured
– Case: A single instance
of a unit of analysis
Review - variable
•
•
•
Any characteristic that is (a) measurable (b) can take on different values in the
population or over time
– Incarceration rate (no. of persons sent to prison/100,000 population)
– Mean number of Boston youth shot dead each month
– Period when Boston youth were shot dead (pre- or post-Ceasefire)
– Height, weight, gender
Types of variables
– Categorical
• Nominal: mutually exclusive categories (e.g., M/F)
• Ordinal: implied ranking (low/medium/high)
– Continuous (e.g., height, weight, scale 1-20)
– Can “transform” ordinal into continuous, and continuous into ordinal
• Low/medium/high  1-3 scale
• 1-20 scale  low (1-5)/medium (6-10)/high (11+)
Coding
– Process of assigning a measurement to a variable
– To use certain statistical techniques, nominal variables are sometimes recoded
as “dummy” variables
• M/F recoded 0 for male, 1 for female
Review - distributions
•
An arrangement of cases in a sample or population according to their values or
scores on one or more variables
•
Statistics – mean, median, mode, range, standard deviation – summarize distributions
Review - association and causation
•
Association means that the
values of two or more variables
change together
– In Boston, the number of
youth shot dead appears
to be associated with the
study period
– After invoking Ceasefire
the mean number of
youths slain by gunfire
drops
•
Causation means that changes
in one variable cause corresponding changes in another variable.
– The causal variable is called the “independent” variable (here it’s the
time period)
– The effect variable is called the “dependent” variable (here it’s the
mean number of monthly deaths)
– So, did Ceasefire cause the reduction?
RESEARCH DESIGNS –
NON-EXPERIMENTAL
Non-experimental designs
Principles of non-experimental designs
• Begin with a hypothesis
– Changes in independent variables(s)  changes in dependent
variable(s)
– Lower income  more crime
• Assess the hypothesis by collecting data on variables of interest.
– Data usually reflects the values of variables at one point in time
– Data can also be collected in “waves,” meaning at succeeding
points in time
– In non-experimental designs investigators only collect data they do nothing that might affect the values of the variables
• Data sources
– Field observations
– Surveys
– Official sources (public records, census, etc.)
Data source: field observations
Non-experimental designs
•
•
•
•
•
•
•
Research question: do police officers take harsher legal measures if youths display a
bad attitude?
Hypothesis: worse demeanor  harsher disposition
Researchers rode along with cops to observe their interactions with youths
Researchers did NOT intervene -- they let things be
Researchers coded...
– Independent variable:
youth’s demeanor (2 values)
– Dependent variable:
officer disposition (4 values)
At a later time they used statistical
techniques to assess whether youth’s
demeanor was associated with
officer disposition in the hypothesized
direction (the worse the demeanor,
the harsher the disposition)
Depending on the strength of this
association they might conclude:
– There is a cause-and-effect
relationship between the variables: hypothesis confirmed
– The association does not go beyond what could be obtained by chance:
hypothesis rejected
Non-experimental designs
Data source: official sources
Panel
4
Panel
6
Non-experimental designs
Data source: surveys
Panel
1
Panel
3
Non-experimental designs
Data sources: surveys + official sources
Panel
2
Panel
5
Issues in non-experimental designs
Non-experimental designs
•
Causal order: Did the change in the independent variable precede (come
before) the change in the dependent variable?
Poverty  crime
•
OR
Crime  poverty
Intervening variables: Could lack of education or living in a violent area
be the more proximate (closer) cause of crime?
Poverty  poor education  crime
Here poverty is still the cause, but it affects crime through intervening
variable education, which is the more proximate cause
•
Spurious relationship: What seems to be a relationship isn’t - it’s bogus!
― Often caused by a strong association between the independent
variable of interest (e.g., poverty) and another independent variable
(e.g., poor social controls) which turn out to be the real cause
Poor social controls  crime
Poverty
RESEARCH DESIGNS –
EXPERIMENTAL
Experimental designs
Principles of experimental designs
•
Purposes
– Eliminate other possible “causes” (e.g., that it’s education, not poverty)
– Set the causal order (e.g., know you are testing crime  poverty)
•
Method
1. Randomly assign cases to two or more groups. Designate one or more
groups as “experimental” and one as “control”
2. Measure the dependent variable (time 1) for each group. Random
assignment insures that the mean values of the independent variable(s)
should be about the same for each group.
3. Intervene in the experimental group by adjusting the level of the
independent variable of interest
4. Post-measure dependent variable (time 2) for each group. If the
differences between experimental and control groups are “statistically
significant” they can be attributed to the intervention.
•
Simple experiment
intervention (adjust level of independent variable)
Experimental group ( X ) DVt1……….IV……….DVt2
Control group ( C ) DVt1…..………………DVt2 (no intervention)
Experimental designs
Hypothesis: officers who complete a special
training program will be less cynical
population:
200 patrol officers
150 males (75%)
50 females (25%)
CONTROL
GROUP
EXPERIMENTAL
GROUP
EXPERIMENTAL
GROUP
CONTROL
GROUP
Randomly Assign
25 Officers
Randomly Assign
25 Officers
Randomly Assign
25 Officers
Randomly Assign
25 Officers
For each group, pre-measure dependent variable officer cynicism
Apply the intervention (apply the value of the independent variable – the program.)
NO
YES
YES
NO
For each group, post-measure dependent variable officer cynicism
Also compare within-group changes – what do they tell us?
Experimental designs
1.
2.
3.
4.
5.
6.
Hypothesis: SOCP reduces recidivism
Independent (causal) variable: SOCP
(yes/no) (categorical/nominal)
Dependent (effect) variable: recidivism
(rearrest rate, continuous)
Randomly assign youths being released
to either X or C
–
Random assignment makes them
equal overall for background factors
such as age, criminal record, etc.
X (experimental group) gets intensive
supervision (SOCP yes)
C (control group) remains with regular
supervision (SOCP no)
Wait two years, compare recidivism
–
Does the X group have a significantly
lower rearrest rate?
–
Does the X group have significantly
lower rates of drug & alcohol use?
Population: youths 12-18 cited
during an 18-month period
EXPERIMENTAL
GROUP
CONTROL
GROUP
Randomly Assign
264 youths
Randomly Assign
265 youths
pre-measure arrest record, drug and
alcohol use, etc.
Apply the intervention
YES
NO
For each group, post-measure
dependent variable measures: arrest
record, drug and alcohol use, etc.
1973 Kansas City Patrol Experiment
Experimental designs
•
•
Research question: Does routine patrol deter crime?
Hypothesis: Routine patrol reduces crime
1. Independent (causal) variable: Patrol
(categorical/ordinal - three levels)
2. Dependent (effect) variable: crime rate
(continuous)
3. Randomly divide an area into 15 beats
4. Measure crime in each beat
5. Randomly assign each a different value of
the independent variable
– Five C (control) beats: same patrol
as usual
– Five X1 (experimental) beats: no patrol
(“R” - reactive - only answer calls for
service)
– Five X2 experimental beats: more patrol than usual
(“P” - proactive - more cars cruising, looking for trouble)
6. After one year compare crime rates
Some issues with experimental designs
Experimental designs
•
•
•
According to the Kansas City experimenters, there was no significant
difference in crime rates between the experimental and control groups.
– Since neither increasing nor decreasing patrol made a difference, the
hypothesis that random patrol can reduce crime was rejected.
However, the experiment was later criticized:
– Level of the independent variable (amount of patrol) was not sufficiently
increased in the proactive beats to be able to demonstrate a statistically
significant effect
– Due to contamination by other units, level of patrol was not sufficiently
reduced in the reactive beats to be able to demonstrate a statistically
significant effect
Other constraints
– Practicality
• Could we experimentally test poverty  crime?
– Ethics
• Should we experimentally test poverty  crime? Can we make
some people poor, then see what happens!
RESEARCH DESIGNS –
QUASI-EXPERIMENTAL
Quasi-experimental designs
Quasi-experimental designs & issues
•
Experiment that lacks random assignment to groups
– Groups might differ along a key independent variable (“matching” often used
to try to make up for this)
•
Experiment without a control group
– An extraneous event might be the true cause of the change in the dependent
variable
•
A non-experimental design that
mimics an experiment
– A known intervention did take
place (e.g., it’s known that the
level of the independent variable
did change at a certain time)
– Measures of the dependent
variable are available for the
periods before and after the
intervention
Quasi-experimental designs
Data source: Vignette
•
•
•
“Vignettes” are brief descriptions of actual or realistic events that are
administered to elicit responses by test subjects to key issues of interest to
researchers
In this example a vignette is used to test the hypothesis that police officers
with military experience are more likely, in domestic violence situations, to
show leniency to other veterans
– Officer’s veteran status  Officer’s disposition
Since everyone was administered the vignette (there was no equivalent
“control group”) the possibility exists that independent variables other than
those tested could explain why officers acted as they did