THE SHORTCOMINGS OF EXPLICIT MEASURES
OF BIAS AGAINST FEMALE POLITICAL
LEADERSHIP
Mark Setzler, High Point University
msetzler@highpoint.edu
How explicitly are we measuring gender bias?
• “… Generally speaking, would a male or a female better
handle that issue better?”
• “If two equally qualified candidates were running for
office, one a man and the other a woman, do you think you
would be more likely to vote for the man or the woman?”
• WVS: “How much do you agree with the following
statement: generally-speaking, males make better political
leaders than women do.”
• How widely used is the WVS measure?
Why might explicit measurements be a
problem?
• Studies on racial and ethnic prejudice find that many
respondents shy away from giving socially undesirable
responses, especially if no “face saving” framing statement
is included.
• Individuals with more power understate their prejudices
against minorities with less power, especially when the
interviewer is a minority (example: polling and voting in
mixed-race elections)
• Group solidarity can lead members to under-report bias
and behaviors (example: Evangelicals self-reported church
attendance vs. self-reported anti-LGBT bias)
• Some people may overstate their biases if they belong to
social groups that are publicly intolerant (example, studies
of nativism)
Hypotheses
• Hypothesis 1: A highly explicit survey question measuring
prejudice against female political leadership will report less
bias than a virtually similar question structured to reduce
social desirability effects.
• Hypothesis 2: Males will be more likely to express prejudice
on the masked indicator than the explicit one.
• Hypothesis 3: Non-whites and Democrats will
disproportionately express prejudice exclusively on the
masked indicator.
• Hypothesis 4: Evangelicals and Republicans will
disproportionately express prejudice exclusively on the
more explicit measure
Data and Measurement
• 2008 survey by the Pew Research Foundation that sampled
2,250 individuals.
• The explicit measure: “Which one of the following
statements comes closest to your opinion about men and
women political leaders?”
“Men (Women) generally make better political leaders than
women (men)” and “In general, women and men make
equally good political leaders.”
Coded: Men = 1; Women/Equal = 0
Data and Measurement
• The masked measure: As you may know, our country has 8
women governors out of 50, and 16 women senators out of 100.
There may be many reasons that there are fewer women than
men in high political offices. Here is a list of some of them.
For each, please tell me whether you think it is a major reason, a
minor reason, or not a reason why there are fewer women in
politics.
A battery of 7 statements randomly was presented, including:
Generally speaking, women don’t make as good leaders as men
Coded: Major or minor reason= 1; Not a reason = 0
• IVs: Male, Dem/Republican, Non-white
• Controls: Educational attainment (1-4), Interviewer’s Gender
(Female), Age (Years), & Relig. Attend (1-5)
The proportion of persons accepting the premise that males are
better leader: Explicit versus masked measurement
Sample results: Predicted probabilities of accepting the premise that
males are better leaders (Logistic R)
Differential effects in the probability of expressing gender bias on
only one measure type (Logistic R)
Conclusions
• Measured more discreetly, prejudice against female is much more
common than is typically reported (43% of Americans vs. 20%)
• We may in inaccurately assuming that some groups (evangelicals and
Republicans) are disproportionately more biased against women leaders
and that others (Democrats and non-whites) are more friendly than is
actually the case.
• There’s little evidence that Americans of any type are overstating their
bias despite that some important political forces continue to praise
traditional gender roles and institutions that model them
• Some key questions for future research:
• Will these findings travel outside of the US?
• Are social desirability effects a problem for survey research focusing
on behaviors towards women politicians, too?