Chapter 5
BIAS IN SAMPLING
When we are able to collect data from every member of a population, we are
dealing with a census. It is highly unlikely in most studies that we would
ever be able to do this.
Census Example: One of the few studies where we can use a census (that is have the
information from every member of a the population ) would be SAT scores. The college
board keeps all of this data on hand. They have every SAT score. This is why you will
see many studies using SAT scores
Reasons that a census is not typically used although it would give the best data:
• Expensive
• Time-consuming
• Complex
That is why we need to take a valid sample of the entire population that is
representative of the entire population.
Bias in Sampling
Before we actually look at appropriate sampling methods, let’s take a look at some of the bias that we may
face if we do not design our experiment appropriately :
Selection Bias: occurs when the way that the sample is selected systematically excludes some part of the
population of interest.
Example 1: I want to decide the average number of Science Olympiad members
that are in each class at any one time. I will sample from all of the
AP classrooms in the school.
Why won’t I get a valid representation of the population here ?
Example 2:
A researcher may wish to generalize results of a study consisting of all
residents of a particular city, but the method of selecting individuals
may exclude the homeless or those without telephones. If those people
differ in some systematic way from those that were sampled (income,
for example), the sample is guaranteed to be unrepresentative of the
entire population.
AP terminology : A sampling frame is the list of possible subjects who could be
selected in a sample. If the sampling frame is not equal to the
population, the sample will be biased in the way that the
sampling frame is biased.
Response Bias: Response bias can occur when the way the response of the sample is
swayed by a factor other than the variable at hand.
Example 1: The wording of the question. “Do you or do you not believe that
the Holocaust did not happen ?” Remember, the people that you
sample do not have any consequences of their answers. Many people
Chapter 5
BIAS IN SAMPLING
will just answer without taking too much time to think about the
question, so if it is poorly worded, you can be sure that bias will be
present in their answer.
Example 2 : Honesty. If I am trying to decide the amount of cheating that goes
on in a particular school, I should conduct an anonymous survey.
What do you think will happen if I ask the question, “ Do you
or have you ever cheated?”
Example 3: The order of choices. It is generally accepted that the option listed first
receives a greater number of selections. For example, surveys dealing
with preference of candidates should have different forms that
incorporate different orders of listings.
Example 4: The demeanor of the interviewer. All interviewers must be following
the same instructions.
Non-response Bias: occurs when responses are not actually obtained from all individuals
selected for inclusion in the sample.
Examples: This is likely to occur in telephone surveys and mail surveys because
although you are trying to survey every member of a well chosen
sample, many members of this sample may not respond.
Do you respond to telephone and mail surveys ?
What members of the population do you think do respond to
telephone and mail surveys ?
Household Bias : Since most professional pollsters only interview one member of each
household, it is possible that people who live in large households are
underrepresented in the sample. One possibility to counteract this is to
grant greater weight to the respondent’s answers if they come from
large household.
Voluntary Response Bias: represents members of the population with strong opinions
Example 1: a town meeting is called to decide whether a new sports arena should
be built in town ( many times only those opposed will show up)
Example 2: Who do you think should win on American Idol ?
What other bias occurs on American Idol ?
Convenience Sampling: Don’t Bother…. This would occur is you wanted to find the average heights
of all students on a college campus. You decide to stand on campus and ask
all of those students who walk by there height…….
Little did you know that you were standing right outside of the basketball
gym before practice…. What is the problem with your data ?
Quota Sampling : Interviewers are assigned to interview a fixed quota of members of the population
(ex: 20 males, 20 females) This causes bias because the interviewer can choose
whomever they want.