10/18/2010
Political Science Scope
and Methods
Onto the Nuts and Bolts…


This week: Measurement (part 1)
Important Concepts

Observation, Measurement,
and Political Implications
Measurement: An Introduction

Steps in Measurement






Putnam Example (and Jackman
critique)
Reliability and Validity

Operational definition
Agreement?
Levels
L
l off measurementt
Nominal
 Ordinal
 Interval
Reliability: Extent to which
measurement procedure yields
same result on repeated trials


Validity: How well the measure we
use correspondents to the
underlying concept



Face validity
y
Construct validity
Multiple measures – inter-item association
ple: 2000 Presidential election
Examp


Reliability and Validity
Operationalization
Reliability and Validity
Unbiasness and Efficiency


Test/Retest
Inter-coder reliability
Validity: How well the measure we
use correspondents to the
underlying concept
Validity Example: Risk Taking

Need a measure of risk-taking
proclivities


Use Gambling acts
V lidity:
Validit


Inter-item association
Construct validity
1
10/18/2010
Gambling Items
1. Many people take chances in some areas of life...games they play, ways they make money, things
like that. Which of the following activities do you take part in – even if you only do it once in a
while?
A. Taking part in football pools
B. Check pools
C. Playing bingo
D. Playing poker
E. Betting on the horses
F. Playing bid whist G. Shooting dice
H. Buying lottery tickets
I. Speculating on land
J. Playing bridge
K. Playing the numbers
L. Entering magazine contests
M. Playing roulette
N. Playing pinochle
O. Baseball pools
P. Buying sweepstakes tickets
Q. Buying raffle tickets
R. Buying stocks
S. Are there any other games you play for money?
.3
.25
Frac tion
.2
.15
15
.1
.05
0
0
.1
.2
.3
.4
.5
Risk Seeking
.6
.7
.8
.9
2. How much do you usually spend [on the gambling activity] (note: coded zero for non-gamblers)
3. Suppose you were betting on horses and were a big winner in the third or fourth race. W ould you
be more likely to continue playing or take your winnings? (note: asked of all respondents)
Table 2: Life Changes
Make Changes in Life?
Unbiasness and Efficiency
Coefficient (S.E.)
Constant
-0.77 (0.23)**
Risk Scale

1.16 (0.24)**
Male
0.33 (0.06)**
White

-0.09 (0.10)
Education
0.66
66 (0.12)
12)**
Age
1.24 (1.03)
Age2
-2.60 (1.12)**
Income
-0.18 (0.14)
N
Unbiased Measure: estimate
centered on the truth
Efficient Measure: reducing the
bounfd
d of uncertainty
t i t around
d a point
i t
estimate as much as possible
2104
Log-Likelihood
-1249.32
Turnout: Presidential Elections 1960-2000
100
90
80
Percent Voting
70
60
50
40
30
20
10
0
1960
1964
1968
1972
1976
1980
1984
1988
1992
1996
2000
Year
2
10/18/2010
Threats to Unbiasness and
Efficiency

Measurement error


Non-random error
Random error



In DV: increases uncertainty
In IV: attenuates estimate of effect (but
careful!)


-
If your IV of interested is correlated
with another IV that is also correlated
with your DV  Bias
Taking it too far…

+
Dislike Italian
Immigrants
Aid England in
War
+
Omitted Variable Bias


Omitted Variable Bias: WWII
Example
Can’t control for every omitted
variable
Control for important plausible
alt
lternative
ti
hypotheses
th
Tradeoff with efficiency
Bottom line: data is precious, use it
wisely
Education
Putnam Example

12 Indicators in 3 areas




Is Putnam the model?



Policy process
Policy pronouncements
P li iimplantation
l t ti
Policy
Validity?
Reliability?
Jackman critique
3
MIT OpenCourseWare
http://ocw.mit.edu
17.869 Political Science Scope and Methods
Fall 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.