Choosing Research Designs II
Nonexperimental Methods
The Purpose of Control Variables
• We use control variables to account for
possible alternative explanations we can
think of.
• For example, when I examined whether
democracies are generally more peaceful
than autocracies I included several
control variables.
Explaining Pacifistic Democracy
• Peace (Y) = Democracy (X1) + State
Power (X2) + Development (X3) + # of
Bordering States (X4)
• In the model above, I have more
confidence that Democracy is related to
peace considering I control for the other
variables that may skew my test.
• We need to take care that our theory is not
missing other factors that may undermine
the validity of our theory and tests.
• Our inferences will be flawed if we are
actually capturing other processes through
our variables.
• This means that the validity of our
measures would be undermined.
Several possible problems arise that are
related to model misspecification and
spurious relationships.
•
Thus, we need to control for confounding
factors and alternative explanations!!!
Model Misspecification and
Spuriousness
• Antecedent variable: A variable that
indirectly affects the relationship
between two other variables.
• For example, Ivy league education
increases income.
• However, parental wealth and legacy
admissions affect Ivy league education.
Thus, income of graduates from Ivy
League schools may not be random.
Here Ivy League Parents is an
antecedent variable
Ivy League Parents
Kids
kids
Ivy League
high income
Hence, admission to Ivy schools clearly
not random or pure merit-based, and
thus the income earned by these
people.
Model Misspecification and Spuriousness
• Intervening Variable: These may be
spuriously related to another relationship.
• How can states fight each other if they are not
contiguous with each other? Only the
strongest, with large navies, bases, etc., could
do so.
• Hence, geographic contiguity or distance
is an intervening variable. States may or
may not be more peaceful, but it is hard
to avoid conflict when it is on your
borders.
Model Misspecification and Spuriousness
• Alternative Variables: We also want
to control for variables that would
bias our results if omitted.
• In this case, the X variables in a
model would produce biased
estimates, undermining their validity
and producing error that leads to
inaccurate inferences.
Here is a spurious relationship from my
research
IGOs +
conflicts
+
+
Powerful states
Powerful states both in more IGOs and
conflicts, but these two variables not
directly related but a function of
state power.
Classic Spurious Case
???
Ice Cream Consumption
Crime
+
+
+
Summer Temperatures
Hence we see that despite the fact that ice
cream consumption is correlated with crime,
the real cause is that summer temperatures
increase both ice cream consumption and
crime.
Veronica Says, Beat Marshall!!!
Go Miners!!!
UTEP Fight!
UTEP Win!
I’m going to
Homecoming,
Are you?
Non-Experimental Designs
These studies use data collected or
aggregated from surveys, history, or
government indicators:
• Cross sectional studies
• Panel (cross sectional over a few time
points)
• Longitudinal (time series and pooled
cross-sectional time series)
• Case studies and focus groups
CROSS SECTIONAL Designs
• Statistical or case studies that compare
individuals or subjects across several
variables:
• Surveys comparing peoples’ political views
• Comparison of countries, groups,
organizations along different dimensions,
such as countries with different levels of
development (low, medium, high) relative to
other factors.
Non-Experimental Designs
These studies use data collected or
aggregated from surveys, history, or
government indicators:
• Cross sectional studies
• Panel (somewhat rare)
• Longitudinal (time series and pooled
cross-sectional time series)
• Case studies and focus groups
CROSS SECTIONAL Designs
• Statistical or case studies that compare
individuals or subjects across several
variables:
• Surveys comparing peoples’ political views
• Comparison of countries, groups,
organizations along different dimensions,
such as countries with different levels of
development (low, medium, high) relative to
other factors.
Cross-Sectional Data
ID
State
Abortions/1,000
women
%Bush Conservative score for
04
House delegation
1
Alabama
15
62.5
73
3
Arizona
19.1
54.8
67
4
Arkansas
11.1
54.3
48
5
California
33.4
44.4
41
6
Colorado
18
51.7
67.8
7
Connecticut
23
44
37.6
8
Delaware
34.4
45.8
40
10
Georgia
21.2
58
63.7
12
Idaho
5.8
68.4
90
13
Illinois
25.6
44.5
48.9
14
Indiana
10.6
59.9
69
15
Iowa
9.8
49.9
64.6
16
Kansas
18.3
62
75
Example of a Panel Study
State
Democracy
Illiteracy
HDI
Islamic
Argentina91
7
4.3
0.81
0
Argentina95
7
3.7
0.832
0
Argentina00
8
3.3
0.854
0
Armenia91
7
2.57
0.751
0
Armenia95
3
2
0.708
0
Armenia00
5
1.69
0.754
0
Australia91
10
0
0.892
0
Australia95
10
0
0.932
0
Australia00
10
0
0.942
0
Azerbaijan91
-3
3
.
1
Azerbaijan95
-6
3
.
1
Azerbaijan00
-7
3
0.746
1
Bangladesh91
6
65
0.417
1
Bangladesh95
6
61.9
0.445
1
Bangladesh00
6
59.2
0.497
1
Time Series
• Observations are made over time, which
can provide descriptive information or
used to test hypotheses.
• If testing hypotheses, we track data for a
dependent variable and at least one
independent variable over time (based on
some measure e.g. days, weeks, months,
or years)
Example of a Time Series: Presidential
Approval