Research in Political Campaigns
The Washington Center – December 7, 2007
Agenda
•
•
•
•
•
Introduction
Qualitative
Quantitative
Challenges
Discussion
Research in Political Campaigns
2
Introduction
The Use and Misuse of Research in
Political Campaigns
Introduction
Campaign Questions
•
•
•
•
•
•
Where do the likely voters live?
How do they identify themselves?
What are the key issues?
Why might they change their minds?
Who is winning?
When can this change?
Research in Political Campaigns
4
Introduction
Theories and Tools
John
Locke
1690
Pierre
Simon
Lapace
1786
Research in Political Campaigns
Alexander
Graham
Bell
1876
George
Gallup
1936
Thomas
Dewey
1948
Tim
BurnersLee
1990
5
Introduction
Misusing Research Tools
• Frugging – asking for your opinion and your money
– Candidates, campaigns, and groups
– Bait-and-switch misuse of polling
– www.mra-net.org/resources/abuse.cfm
• Robocalls – prerecorded automated advocacy calls
– Sometimes request the recipient of the call to flood the opposition’s
campaign office or website with “feedback”.
– Misuse of random digit dialing
– www.mysterypollster.com/main/2006/02/roboscam_not_yo.html
• Push polls – advocacy masquerading as research
– Only testing positive on your candidate and negative on the opposition
– Misuse of respondent trust
– www.aapor.org/aaporstatementonpushpolls
Research in Political Campaigns
6
Introduction
Process We Use
With adjustments for feedback …
Candidate
interview
Research
budgeting
Research in Political Campaigns
Messaging
groups
Survey
baseline
Voter
tracking
7
Qualitative Research
The best method to find out how people
think about your candidate or issue
Qualitative
What is Qualitative?
•
•
•
•
Goal is to find out how people think and feel
about a candidate or issue
Small numbers of people interviewed
Results help guide overall strategy and
messaging
Example: Two-hour discussion with 10-12
voters led by a moderator
Research in Political Campaigns
9
Qualitative
What to Test
•
•
•
•
•
•
•
•
Themes and slogans
Debating points
Pamphlets
Advertising
Website
Fundraising appeals
Press coverage (real or mock)
Surrogates
Research in Political Campaigns
10
Qualitative
Tool: Focus Groups
 More art than science.
 Good for exploring issues and specific language used by
the voters to explain their preferences.
 Good for feedback on candidate appearance and
presentation for training; media content and rebuttals.
 Good for revealing latent biases and/or prejudices that
are not apparent in surveys.
 Good for demonstrating how opinions are formed and
how others are affected by strong-willed opinion leaders.
 Good reality check for those in the campaign bubble.
Research in Political Campaigns
11
Qualitative
Methodology
•
Recruit 10-12 voters based on criteria you select (e.g.
swing voters, usually not your base).
•
The campaign team is behind a soundproofed one-way
mirror; sessions are audio and videotaped.
•
Led by an unbiased moderator who is skilled at keeping
the conversation moving but on task.
•
Always do two groups in one night and try to do sets of
groups in different locations, if possible.
•
Focus groups last about two hours and respondents are
paid for their participation.
Research in Political Campaigns
12
Qualitative
Best Practices
• Do not over-pack the agenda
• Mix discussion techniques:
• Word associations
• Open-ended questions
• Top-of-mind answers
• Pre-/post-feedback
• Use handouts and exercises to keep
them interested
• Consider using dial groups to test gut
reactions to positive or negative video
or audio messaging
Research in Political Campaigns
13
Qualitative
Hiring the Team
•
•
•
•
•
•
Careful recruiting
Experienced moderator
Understanding of the subject matter
Same demographic as respondents
Convenient facility location
Cost: $5,000 - $7,500 per group
Research in Political Campaigns
14
Quantitative Research
The most accurate way to learn how
people view your candidate or issue
Quantitative
What is Quantitative?
•
•
•
•
Goal is to find out what a large group of
people think about an issue or candidate
Large numbers of people interviewed
Results help guide choices and tactics, and
gauge campaign performance
Example: National survey of 1,000 likely
voters in a presidential election
Research in Political Campaigns
16
Quantitative
What to Test
•
•
•
•
•
•
•
•
Screening
Political environment
Party and candidate images
Ballot test
Issues
Messaging
Ballot re-test
Demographics
Research in Political Campaigns
17
Quantitative
Tool: Tracking Surveys
 More science than art.
 Telephone surveys of randomly selected registered
and/or likely voters over two or three days.
 Snapshots of the larger population of voters within a
margin of error based on number of interviews.
 Good for following issues, voter preferences and
effectiveness of campaign messaging.
 Good for releasing to the public through the media or
other outlets (e.g. direct mail).
Research in Political Campaigns
18
Quantitative
Methodology
•
Computer-assisted telephone interviewing (CATI) allows
for standardization, speed, and quality of data.
•
Repeat the same questions on each poll reducing the
number as you go from benchmark to tracking to focus on
the messages you can change.
•
When a specific voting bloc becomes critical to winning,
interview more of them (oversample) in the next poll by
adding 200+ people in that group.
•
Demand quick turns from the pollster and consider rolling
samples if you have the budget for it.
Research in Political Campaigns
19
Quantitative
Best Practices
• Keep the survey as short as possible
• Test audio messages, if available
• Mix question techniques:
• Closed-ended questions
• Open-ended questions
• Avoid yes/no questions
• Use balanced scales
• Randomize choices
Research in Political Campaigns
20
Quantitative
Margin of Error
Confidence interval:
usually reported as
margin of error, this is
the +/- percentage
reported on most
surveys.
Equation for the
sample size of a large
population
Confidence level:
number of times out of
100 we will be right
inside the margin of
error; usually 95%.
Research in Political Campaigns
21
Quantitative
Hiring the Team
•
•
•
•
Careful drafting, training, and monitoring
Consultant will likely have a favorite partner
Interviewers need to have a local accent
Cost varies by length of questionnaire,
number of interviews, and type of respondents
(e.g. likely voters, men 18-24, working
women).
Research in Political Campaigns
22
Challenges
Five trends in public opinion research
we are actively monitoring
Challenges
Five Trends to Watch
1.
2.
3.
4.
5.
Declining participation rates
Cell phone only households
Do not call lists
Online polls
Push polls
Research in Political Campaigns
24
Challenges
Declining Participation Rates
•
•
•
Over the past 20 years, response rates have fallen due to
answering machines, Caller ID, cell phones, etc.
The problem is representativeness: would people who avoid
surveys give us different answers than those who do?
We deal with this problem by increasing calls to nonrespondents and using other methods like in-person, postal
mail, and Internet surveys.
Research in Political Campaigns
25
Challenges
Cell Phone Only Households
•
In the United States, 14% of the population now does not
have a landline – up from about 1% in 2000.
– Tend to be young (32%) and single (27%)
– Hispanic (21%) and African-Americans (17%)
•
•
•
The issue is fairness: how do we interview people who only
have cell phones since respondents get charged for the call?
We are only beginning to deal with this by offering
compensation to respondents as well as other incentives.
For more about this trend, read The Birth of Cellular Nation,
a white paper by Andy Arthur of MRI Market Solutions.
Research in Political Campaigns
26
Challenges
Do Not Call Lists
•
•
•
•
Most reputable research firms now keep such a list to weed
out people known to give “Hard Refusals”.
The United States government’s Do Not Call Registry
(www.donotcall.gov) does not include surveys but many
citizens think it does or should.
The issue is what’s next: legislation allowing citizens to optout of surveys altogether?
We are fighting this through education and
lobbying campaigns.
Research in Political Campaigns
27
Challenges
Online Polls
• Web feedback polls are non-scientific
surveys that do not represent the public.
• The media use these polls in their
reporting with a quick disclaimer.
• The issue is deception: a poll is not a
survey if everyone in the population does
not have an equal chance of being
chosen to participate.
• We are dealing with this carefully since
the state-of-the art is changing and there
are safe ways to use the Internet for
public opinion research.
Research in Political Campaigns
28
Challenges
Push Polls
•
The worst kind of polling; undermines the confidence of
voters in public opinion research.
– Very short surveys which read as negative ads
– Non-scientific samples, usually with very large number of calls
– Results are never released to the public since their intent is to defame
•
•
•
•
The issue is credibility: how can voters distinguish between
fair messaging polls and push polling?
All major research organizations have outlawed push polling
but that has not stopped firms from quietly doing it.
The best defense for a push poll is publicity of the attack.
The test: If you don’t want to see your sponsorship on the
front page of La Nación, then do not ask the question.
Research in Political Campaigns
29
Discussion
Quick review of campaign research and
your questions answered
Discussion
Key Takeaways
•
•
•
•
Political research is based on ideas and tools
developed over three centuries.
Qualitative research is best to help shape
campaign strategy and nuanced messaging.
Quantitative research is best to evaluate campaign
planning, tactics, and reception of messaging.
Research is not perfect, even within its defined
margins of error, and must be considered within
the context of other available information.
Research in Political Campaigns
31
Michael D. Cohen, Ph.D.
mcohen@cohenresearchgroup.com
202-558-6300 P | 202-558-6301 F | 703-785-9094 C
10 G Street, NE, Suite 601
Washington, DC 20002