Bias in Sampling Homework 1.) A newspaper advertisement for USA today: Should handgun control be tougher? You call the shots in a special call in-­‐ poll tonight. If yes, call 1-­‐900-­‐720-­‐6181. If no, call 1-­‐900-­‐720-­‐6182. Charge is 50 cents for the first minute. Explain why this opinion poll bias. This opinion poll is almost certainly bias because only people with strong opinions on the subject-­‐ strong enough that the are willing to spend the time, and 50 cents-­‐ will respond to this advertisement. This type of sampling bias is known as voluntary response bias. 2.) You are on a staff of a member of Congress who is considering a bill that would provide government-­‐sponsored insurance for nursing home care. You report that 1128 letters have been received on the issue, of which 871 oppose the legislation. “I’m surprised that most of my constituents oppose the bill. I thought it would be quite popular,” says the congresswoman. Are you convinced that a majority of the voters oppose the bill? How would you explain the statistical issue to the congresswoman? Letters to legislators are an example of voluntary response bias. The proportion of letters opposed to the insurance should not be assumed to be a fair representation of the congresswoman’s constituents. Only the people with strong opinions on the subject will spend the time to construct a letter. 3.) The list of individuals from which a sample is actually selected is called the sampling frame. Ideally, the frame should list every individual in the population, but in practice this is often difficult. A frame that leaves out part of the population is a common source of undercoverage. a. Suppose that a sample of households in a community is selected at random from a telephone directory. What households are omitted from this frame? What types of people do you think are likely to live in these households? These people will probably be underrepresented in the sample. Households without telephones, or with unlisted numbers will be omitted from this sampling frame (undercoverage and selection bias). Such households will likely be made up of poor individuals (who cannot afford a phone), those who choose not to have phones, and those who do not wish to have their phone number published. b.
It is more common to use random digit dialing equipment that selects the last four digits of a telephone number at random after being given the exchange (the first three digits). Which of the households that you mentioned in your answer to (a) will be included in the sampling frame by random digit dialing? Those households with unlisted numbers will be included in the sampling frame when a random digit dialer is used. Bias in Sampling Homework 4.) A common form of nonresponse bias in telephone surveys is “ring-­‐no-­‐answer.” That is, a call made to an active number but no one answers. The Italian National Statistics Institute looked at non-­‐response bias to a government survey of households in Italy during the periods January 1 to Easter and July 1 to August 31. All calls were made between 7 and 10 pm, but 21.4% gave “ring-­‐
no-­‐answer” in one period versus 41.5% “ring-­‐no-­‐answer” in the other period. Which period do you think had the higher rate of no answers? Why? Explain why a high rate of non-­‐response makes sample results less reliable. The higher no-­‐answer was probably the second period (July 1 – August 31) because more families were likely to be on vacation. Non-­‐response bias of this type might under represent those who are more affluent (and are able to travel). 5.) During the 2000 presidential campaign, the candidates debated what to do with a large government surplus. The Pew Research Center asked two questions of random samples of adults. Both stated that social security would be “fixed.” Here are the uses suggested for the remaining surplus: Should the money be used for a tax cut, or should it be used to fund new government programs? Should the money be used for a tax cut, or should it be spent on programs for education, the environment, health care, crime-­‐fighting and military defense? One of these questions drew 60% favoring a tax cut; the other, only 22%. Which wording pulls respondents towards a tax cut? Why? Response Bias “Should the money be used for a tax cut, or should it be used to fund new government programs?” This wording is likely to pull respondents toward a tax cut as opposed to “Should the money be used for a tax cut, or should it be spent on programs for education, the environment, health care, crime fighting and military defense” because the second wording mentions several very popular alternative uses for tax money. 6.) A newspaper article about an opinion poll says that “43% of Americans approve of the president’s overall job performance.” Toward the end of the article, you read: “The poll is based on telephone interviews with 1210 adults from around the United States, excluding Hawaii and Alaska.” What variable did this poll measure? What was the population in this context? What was the sample? Variable: Approval of the president’s job performance Population: Adult citizens of the U.S. , excluding those citizens of Alaska and Hawaii Sample: The 1210 U.S. adult citizens interviewed. Possible Sources of Bias: Undercoverage (Selection) bias is likely because only adults with phones were contacted, potentially under representing those who can’t afford phones, choose not to have phones, or choose to be unlisted. Bias in Sampling Homework 7.) Comment on each of the following as a potential sample survey question. Is the question clear? Is it slanted toward a desired response? a. Which of the following best represents your opinion on gun control? i. The government should confiscate our guns. ii. We have the right to keep and bear arms. This question will likely elicit more responses against gun control because 1. The government should confiscate our guns makes it seems like one of our rights will be stripped away for no good reason and 2. We have the right to keep and bear arms makes it seem like we get to keep this right which is owed to us. (RESPONSE BIAS) b.
A freeze in nuclear weapons should be favored because it would begin a much-­‐needed process to stop everyone from building nuclear weapons now and reduce the possibility of nuclear war in the future. Do you agree or disagree? The phrasing of this question will tend to make people respond in favor of a nuclear freeze. Only one side of the issue is presented and favored within the question. (RESPONSE BIAS) c.
In view of escalation degradation and incipient resource depletion, would you favor economic incentives for recycling of resource-­‐intensive consumer goods? The wording of the question could be made clearer by using simpler language. It is slanted in favor of incentives by using the words “escalating environmental degradation and incipient resource depletion” in the question for those people that do understand the vocabulary used. (RESPONSE BIAS) 8.) In order to assess the opinions of students at the University of Minnesota on campus snow removal, a reporter for the student newspaper interviews the first 12 students he meets. Why is this sample bias? This sample is bias because it is a convenience sample.