Measurement
17.871
Spring 2003
Topics in Measurement
•
•
•
•
From abstraction to measure
Sources of error
What to do about error
Practical ways to improve measurement
The Mapping
Theory
Observation
X
Y
x
y
ex
ey
Mapping from the Abstract to the
Measurement
• Some abstract things we try to measure
– Alienation
– Moral decay
– Democracy
– Party identification
– Fear of defeat
– Terrorism
– Ideology
Sources of Error in Measurement
(Mosteller)
•
•
•
•
•
•
Conceptual or design error
Bad breaks in random sampling
Survey question wording
Non-random out-selection
Transcription errors
Calculation & mechanization errors
What to Do About Error
• Practice safe data
– Know where your data come from
– Watch for anomalies
– Use multiple measurement techniques
– Collect as much data as possible and
disaggregate
Practical things to do about
measurement
• Distinction between a measure and an
indicator
– Measure: straightforward quantification of a
variable of interest
– Indicator: quantification of a variable that is
believed (or known) to be highly correlated
with the “real” variable of interest
Examples of Measures
•
•
•
•
•
Income in $$
Age in years
Votes
Number of wars (hmmmmmm…..)
Campaign contributions
• Measurement issues tend to focus on the quality
of the data-gathering method, especially
sampling
Examples of Indicators
• Public opinion
– Party identification
– Trust in government
– Ideology
• Economics
– Gross domestic (national) product
• Characterizations of political systems
– Freedom
– Transparency
– Democracy
How do we generate indicators?
• Trust others and hope for the best
– GDP, etc.
– Presidential approval
• Use a single measure and hope for the best (or
convince ourselves it’s OK)
–
–
–
–
7-point ideology scale
7-point party identification scale
Codings of “wars” or “rally events”
Bulletin of the Atomic Scientist “nuclear clock”
• Use multiple measures creatively
Multiple-measure indicators
• “Trust in government” battery
– How much of the time do you think you can trust the government
in Washington to do what is right?
– Would you say the government is pretty much run by a few big
interests looking out for themselves or that it is run for the benefit
of all the people?
– Do you think that people in the government waste a lot of money
we pay in taxes, waste some of it, or don’t waste very much of it?
– Do you think that quite a few of the people running the
government crooked, not very many are, or do you think hardly
any of them are crooked?
• External political efficacy battery
– Sometimes politics and government seem so complicated that a
person like me can’t really understand what’s going on
– People like me don’t have any say about what the government
does
– Public officials don’t care much what people like me think.
Trust and Efficacy over Time
Other Multiple-Variable Indicators
• Americans for Democratic Action Support
Scores
• Freedom House freedom assessment
• MIT Teaching Quality
• Transparency International’s “Corruption
Perceptions Index”
• DNominate Scores
Scaling
Or, How to Create Multiple
Indicators More Generally
Imagine an unobserved factor that
gives rise to observable factors
Democracy
Healthiness
Racism
Conservatism
Religiosity
b1
Unobserved
factor
b2
bn
b1
Polyarchy
b2
b3
b4
Measure 1
e1
Measure 2
e2
...
•
•
•
•
•
Measure n
Fairness of
elections
e3
e1
Meaningfulness
of elections
e2
Freedom of expression
e3
Unbiased reporting of
official pronouncements
e4
Fairness of elections
Freedom of expression
Two-factor (pure) polyarchy
example
A B
Polyarchy
C D E F
G H
Polyarchy
I
J
Fairness of elections
What We Observe in the Pure
Case
Freedom of expression
Fairness of elections
Freedom of expression
Two-factor (impure) polyarchy
example
A B
Polyarchy
C D E F
G H
Polyarchy
I
J
Fairness of elections
What We Observe in the Impure
Case
Polyarchy
3.5
Bhutan
Congo, Lebanon
Hungary
USSR,
Libya, Iran,
China
fairelt
Jamaica,
Greece
Egypt, Sri Lanka,
Mexico
Argentina,
U.S., Finland
.5
Peru, India,
Israel
.5
3.5
frexp
Source: http://www.nd.edu/~mcoppedg/crd/polydat.htm
Simple way to create multipleindicator scale with just 1 factor
Si 
N
T
( X i ,t  X t )
i 1
t 1
st
 