17.871
Spring 2006

• From abstraction to measure
• Sources of error
• What to do about error
• Practical ways to improve measurement

Theory X
Observation x e x
Y y e y

Mapping from the Abstract to the
Measurement
• Some abstract things we try to measure
– Alienation
– Moral decay
– Democracy
– Party identification
– Fear of defeat
– Terrorism
– Ideology

Sources of Error in Measurement
• Conceptual or design error
• Bad breaks in random sampling
• Survey question wording
• Transcription, calculation & mechanization errors

• Practice safe data
– Know where your data come from
– Watch for anomalies
– Use multiple measurement techniques
– Collect as much data as possible and disaggregate

Practical things to do about measurement
• Distinction between a measure and an indicator
– Measure: straightforward quantification of a variable of interest
– Indicator: quantification of a variable that is believed (or known) to be highly correlated with the “real” variable of interest

• Income in $$
• Sales of houses in Massachusetts
• Age in years
• Votes
• Number of wars (hmmmmmm…..)
• Campaign contributions
• Measurement issues tend to focus on the quality of the data-gathering method, especially sampling – e.g. measuring sales of houses in
Mass. via realtor records vs official filings

• Most measures are really indicators
• Public opinion
– Party identification
– Trust in government
– Ideology
• Economics
– Gross domestic (national) product
• Characterizations of political systems
– Freedom
– Transparency
– Democracy
• Quality of colleges

• Trust others and hope for the best
– GDP, etc.
– Presidential approval
• Use a single measure and hope for the best (or convince ourselves it’s OK)
– 7-point ideology scale
– 7-point party identification scale
– Codings of “wars” or “rally events”
– Bulletin of the Atomic Scientist “nuclear clock”
– US News and World Report ranking of colleges
• Use multiple measures creatively

• “Trust in government” battery
– How much of the time do you think you can trust the government in Washington to do what is right?
– Would you say the government is pretty much run by a few big interests looking out for themselves or that it is run for the benefit of all the people?
– Do you think that people in the government waste a lot of money we pay in taxes, waste some of it, or don’t waste very much of it?
– Do you think that quite a few of the people running the government crooked, not very many are, or do you think hardly any of them are crooked?
• External political efficacy battery
– Sometimes politics and government seem so complicated that a person like me can’t really understand what’s going on
– People like me don’t have any say about what the government does
– Public officials don’t care much what people like me think.

Other Multiple-Variable Indicators
• Americans for Democratic Action Support
Scores
• Freedom House freedom assessment
• MIT Teaching Quality
• Transparency International’s “Corruption
Perceptions Index”
• DNominate Scores

Scaling
Or, How to Create Multiple
Indicators More Generally

Imagine an unobserved factor that gives rise to observable factors
• Democracy
• Healthiness
• Racism
• Conservatism
• Religiosity
• Teaching quality
• College quality
Polyarchy
Unobserved factor b
1 b
2 b
3 b
1 b n b
2
Fairness of elections
Measure 1
Measure 2
Measure n
Meaningfulness of elections e
1
Freedom of expression e
2 e
2 e
1 e
3 e
3 b
4 Unbiased reporting of official pronouncements e
4