Bias in Surveys and Polls
Bias occurs when a poll or survey produces results that do
not reflect the true opinions or beliefs of the general
population. This is often a result of the methods used to
conduct the survey or the wording of the questions asked.
Non-Response Bias
1. Voluntary Response

In phone-in surveys and internet polls, people are not
likely to participate in the survey unless they are already
likely to agree with the questions asked or are likely to
give opinions related to the subject of the survey.
2. Lack of Response

In some surveys, members of the survey group may
refuse to answer certain questions or may refuse to
participate in the survey. This can particularly happen
with phone surveys or mail-in surveys.

This also can include when a group of people is excluded
from participating in the poll, intentionally or not. For
example, if a phone survey uses only randomly chosen
land-line phone numbers, those who have only a cell
phone would not have a chance to be included in the
survey.
Response Bias
3. Leading Questions

How questions are asked is very important in surveys. A
survey question can be worded in such a way as to direct
a person to answer in only one way.
4. Social Acceptance

Often, when questions about controversial issues are
asked, survey respondents may give answers contrary to
their true beliefs in order to conform to a societal standard
they believe is acceptable.
Formulating Survey Questions
When conducting a survey or poll, what type of questions
and the wording of questions are important considerations.
Poorly worded questions can invalidate the results of a
survey.
 Questions should be asked in a manner that is
balanced.
Consider the questions:
“Do you feel that the increasing cost of the high speed
rail project is too expensive for California?”
“Do you feel that high speed rail will be important to the
future economy of California?”
“Do you approve or disapprove of building a high speed
rail system in California?”
 Questions should not be vague.
For example, the question
“What’s wrong with the economy?”
is vague. It is unclear what the question is trying to
determine.
 The type of question you ask is important:
o In an open question, respondents are free to
formulate their own response.
o In a closed question, respondents choose from a
list of predetermined responses.
For example, the question:
“What is the most important problem facing California
today?”
is an open question, whereas:
“Which of the following do you feel is the most
important problem facing California today?
a) Violence in schools
b) Use of drugs
c) Gun control laws
d) Balancing the State Budget
is a closed question.
When asking closed questions, the order that the choices
are listed should be rotated. This is because earlier
choices in a list are more likely to be chosen than later
choices.
The option “No opinion” should be omitted, since this does
not allow for meaningful analysis.
Here are some questions from recent polls and surveys
regarding same sex marriage. Discuss the issues of bias
and fairness in these questions:
 Should states continue to discriminate against
couples who want to marry and who are of the same
gender?
 Do you support marriage equality?
 Should states be forced to legalize homosexual
marriage over the wishes of a majority of the people?
 Do you think marriages between same-sex couples
should or should not be recognized by the law as
valid, with the same rights as traditional marriages?