EXPERIMENTAL
DESIGN
Barron’s AP Statistics Review Flash Cards (2008)
EXPERIMENTAL DESIGN 1
 Each of the 30 major baseball teams carries a 40-person roster. A sample of 60
players (5 percent of all 1,200 players) is to be randomly selected to undergo drug
tests. To do this, each team is instructed to put their 40 names in a hat and
randomly draw two names. Will this method result in a simple random sample of
the 1,200 baseball players?
A) Yes, because each player has the same chance of being selected.
B) Yes, because each team is equally represented
C) Yes, because this is an example of stratified sampling, which is a special case of
simple random sampling.
D) No, because the teams are not chosen randomly
E) No, because not each group of 60 players has the same chance of being selected
EXPERIMENTAL DESIGN 2
 Advantage(s) to using surveys as opposed to experiments is (are) that
I. Surveys are generally cheaper to conduct
II. It is generally easier to conclude cause and effect from surveys.
III. Surveys are generally not subject to bias.
A) I only
B) II only
C) III only
D) I and II
E) II and III
EXPERIMENTAL DESIGN 3
 A company wishes to survey what people think about a new product it
plans to market. They decide to randomly sample from their customer
database as this includes phone numbers and addresses. This procedure
is an example of which type of sampling?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple random
EXPERIMENTAL DESIGN 4
 Two studies are run to compare the experiences of low income families
receiving food stamps to those receiving cash subsidies. The first study
interviews 50 families who have been in each government program for at least
2 years, while the second randomly assigns 50 families to each program and
interviews them after 2 years. Which of the following is a true statement?
A) Both studies are observational studies because of the time period involved.
B) Both studies are observational studies because there are no control groups.
C) The first study is an observational study; the second is an experiment.
D) The first study is an experiment; the second is an observational study.
E) Both studies are experiments, because in each, families are receiving
treatments (food stamps or cash).
EXPERIMENTAL DESIGN 5
 A food judge is given an assignment to choose and sample the food at 52 (out of over
20,000) NYC restaurants. She has an assistant list all restaurants whose name begins
with A, assigns each a number, and uses a random number generator to pick two of
these numbers and thus two restaurants. She proceeds to use the same procedure for
each letter of the alphabet and combines the results into a group of 52. Which of the
following are true statements?
I. Her procedure makes use of chance.
II. Her procedure results in a simple random sample
III. Each restaurant in NYC has an equal probability of being selected.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 6
 In a study of Parkinson’s disease, 100 volunteers had incisions made
through their skulls. The patients were randomly sorted into two groups,
one of which had a new drug inserted into the brain. In the other group, the
skulls were closed with no treatment given. The patients did not know who
received the drug. In the weeks to follow all 100 volunteers showed similar
improvement in physical condition. What is this an example of?
A) The effect of a treatment unit
B) The placebo effect
C) The control group effect
D) Sampling error
E) Voluntary response bias
EXPERIMENTAL DESIGN 7
 Fifty migraine patients are randomly selected from hospital records. Half the
patients are told to drink ice water and sit in the dark when they next
experience a migraine; the remaining patients are told to use neither of
these possible remedies. Participants then report back as to any relief, if any.
Serious faults of this experimental design include which of the following?
I. Lack of randomization
II. Probable confounding variables
III. Lack of blinding
A) I only
B) II only
C) III only
D) I and II
E) II and III
EXPERIMENTAL DESIGN 8
 Which of the following is most useful in establishing cause-and-effect
relationships?
A) Complete census
B) A least squares regression line showing high correlation
C) A simple random sample (SRS)
D) A well-designed, well-conducted survey incorporating chance to ensure a
representative sample
E) A controlled experiment
EXPERIMENTAL DESIGN 9
 A sales representative wishes to survey her client base of 47 companies.
She has 47 business cards, all of the identical size, from her contacts in
the companies, and decides to drop them all in a small box, shake them
up, and reach in to pick 5 cards for her sample. This procedure is an
example of which type of sampling?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple Random
EXPERIMENTAL DESIGN 10
 A newspaper advice columnist asks her readers if they would have married their current
spouse if they had to do it over again. Of the 25,000 responses, 80 percent said no. What does
this show?
A) The survey is meaningless because of voluntary response bias.
B) No meaningful conclusion is possible without knowing something more about the
characteristics of her readers.
C) The survey would have been more meaningful if she had picked a random sample of the 25,000
readers who responded.
D) The survey would have been more meaningful if she had used a control group.
E) This was a legitimate sample, randomly drawn from her readers, and of sufficient size to allow
the conclusion that most of her readers who are married would have second thoughts about
marrying their current spouse.
EXPERIMENTAL DESIGN 11
 A researcher planning a survey of heads of households in New York has census
lists for each of the 62 counties in the state. The procedure will be to obtain a
simple random sample of heads of households from each of the counties rather
than grouping all the census lists together and obtaining a sample from the
entire group. Which of the following is a true statement about the resulting
stratified sample?
I. It is more susceptible to bias than would be a simple random sample.
II. It is easier and more cost effective than a simple random sample.
III. It gives comparative information that a simple random sample wouldn’t give.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 12
 Sampling error occurs
A) When interviewers make mistakes resulting in bias
B) When interviewers use judgment instead of random choice in picking the
sample.
C) When samples are too small.
D) Because a sample statistic is used to estimate a population parameter.
E) In all of the above cases.
EXPERIMENTAL DESIGN 13
 Which of the following is most important in minimizing the placebo
effect?
A) Replication and randomization
B) Replication and blinding
C) Randomization and blinding
D) Randomization and a control
E) Blinding and a control
EXPERIMENTAL DESIGN 14
 In general, for a survey to yield useable results:
I. A sample size of n = 30 is usually sufficient.
II. Researchers must be careful in the way questions are worded.
III. Researchers must carefully choose people who they think are
representative of the population.
A) I only
B) II only
C) III only
D) I and II
E) II and III
EXPERIMENTAL DESIGN 15
 A bank wishes to survey its customers. The decision is made to randomly
pick ten customers who just have checking accounts, ten customers who
just have savings accounts, and ten customers who have both checking
and savings accounts. This procedure is an example of which type of
sampling?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple Random
EXPERIMENTAL DESIGN 16
 Which of the following are true statements?
I. If bias is present in a sampling procedure, it can be overcome by dramatically
increasing the sample size.
II. There is no such thing as a “bad sample.”
III. Sampling techniques that use probability techniques effectively eliminate bias.
A) I only
B) II only
C) III only
D) None of the statements are true.
E) None of the above gives the complete set of true responses
EXPERIMENTAL DESIGN 17
 To find out a town’s average family size, a researcher interviews a
random sample of parents arriving at a pediatrician’s office. The average
family size in the final 100-family sample is 3.48. Is this estimate probably
too low or too high?
A) Too low because of undercoverage bias.
B) Too low because convenience samples underestimate average results
C) Too high because of undercoverage bias
D) Too high because convenience samples underestimate average results
E) Too high because convenience samples overestimate average results
EXPERIMENTAL DESIGN 18
 Which of the following are true statements about blocking?
I. Blocking is to experiment design as stratification is to sampling design.
II. By controlling certain variables, blocking can make conclusions more
specific.
III. The paired comparison design is a special case of blocking.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 19
 Two antidepressants are to be compared in the treatment of elderly patients in a nursing
home. Each patient has his or her own room, some with spectacular views of the ocean. This
experimental design is to create homogeneous blocks with respect to window view. How
should randomization be used for a randomized block design?
A) Within each block, randomly pick half the patients to receive each antidepressant.
B) Randomly pick half of all patients to receive each antidepressant, but then analyze separately
by blocks.
C) Randomly choose which blocks will receive which antidepressant.
D) Randomly choose half the blocks to receive each antidepressant for a given time period;
then for the same time period, switch the medication in each block and compare the results.
E) For ethical reasons, allow patients to choose which medication they prefer taking, but then
randomly assign patients to the blocks.
EXPERIMENTAL DESIGN 20
 What is bias in conducting surveys?
A) An example of sampling error
B) Lack of a control group
C) Confounding variables
D) Difficulty in concluding cause and effect
E) A tendency to favor the selection of certain members of a population
EXPERIMENTAL DESIGN 21
 A human resources department plans to survey 100 of the 3,000
employees in the firm. An alphabetical list of the employees is available,
a random number between 1 and 30 is picked, and the sample consists
of the person that far down the list together with every 30th person after
that. This procedure is an example of which type of sampling?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple Random
EXPERIMENTAL DESIGN 22
Two wordings for a questionnaire on independence for Puerto Rico:
I. Would you vote for independence for Puerto Rico?
II. Would you support an independent Puerto Rico separate from the US?
 One of these questions showed 35 percent support for independence while the other showed
45 percent support. Which produced which result and why?
A) The first question showed the 45 percent because of lack of randomization in choice of
subjects as evidenced by wording of the questions.
B) The first question showed the 35 percent because of a placebo effect.
C) The first question showed the 45 percent due to lack of blocking.
D) The first question showed the 35 percent because of response bias due to the wording of the
question.
E) The first question showed the 45 percent response bias due to the wording of the question.
EXPERIMENTAL DESIGN 23
 To conduct a survey on holiday shopping patterns, a researcher opens a
telephone book to a random page, closes his eyes, puts his finger down on the
page, and then reads off the next 100 names. Which of the following are true
statements. Which of the following are true statements?
I. The survey incorporates chance.
II. The procedure results in a simple random sample.
III. The procedure could easily result in selection bias.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 24
 Before taking an exam students either went to bed at their normal times
or were sleep deprived for 4 to 8 hours. Half of each group were given a
caffeine pill before taking the exam. Determine the number of factors,
levels for each, and number of treatments.
A) One factor with two levels, five treatments
B) Two factors, one with one and one with two levels, three treatments
C) Two factors, one with two and one with three levels, five treatments
D) Two factors, one with two and one with three levels, six treatments
E) Three factors, each with two levels, six treatments
EXPERIMENTAL DESIGN 25
A telephone survey with regard to support of a bond issue resulted in:
Age:
21-30 31-40 41-50 51-60 61-70 71-80 Total
For:
45
32
28
25
15
8
153
Against: 30
43
47
50
60
67
297
Which of the following sampling strategies was most likely used?
A) Cluster sampling
D) Stratified sampling
B) Proportional sampling
E) Systematic sampling
C) Simple random sampling
EXPERIMENTAL DESIGN 26
 A television network conducts a weekly survey to determine the
proportion of viewers who watch various programs. For the coming year,
they decide to double the sample size. The main benefit of this is to
A) Reduce undercoverage bias
B) Reduce nonresponse bias
C) Eliminate sampling error.
D) Decrease population variability.
E) Decrease the standard deviation fo the sampling distribution.
EXPERIMENTAL DESIGN 27
 A botanist is running an experiment on two fertilizers that require different
amounts of watering. She has 40 test plots, half of which are in sunny locations,
and half are in the shade. She randomly selects 10 sunny plots and 10 shady plots
for which to use one fertilizer with its appropriate watering, while the remaining
plots are for the other fertilizer with its appropriate watering. Of the following,
which is the most important observation about this procedure?
A) The variables, fertilizer and water, are confounded.
B) The variables, fertilizer and sun, are confounded.
C) The variables, water and sun, are confounded.
D) No variables are confounded.
E) There is a hidden lurking variable.
EXPERIMENTAL DESIGN 28
 Which of the following are true statements?
I. Voluntary response samples often underrepresent people with strong
opinions.
II. Convenience samples often lead to undercoverage bias.
III. Questionnaires with nonneutral wording are likely to have response bias.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 29
 Which of the following are true statements about sampling?
I. Careful analysis of a given sample will indicate whether or ont it is random.
II.
III. Sampling techniques that use probability techniques effectively eliminate
bias.
A) I only
B) II only
C) III only
D) None of the statements are true.
E) None of the above gives the complete set of true responses
EXPERIMENTAL DESIGN 30
 A consumer product agency tests kilowatts per hour for a sample of
refrigerators, each one of three different sizes. which of the following is
true?
A) There are three explanatory variables and one response variable.
B) There is one explanatory variable with three levels of response.
C) Kilowatts per hour is the only explanatory variable, but there are three
response variables corresponding to the different sizes.
D) There are three levels of a single explanatory variable.
E) Each explanatory variable level has an associated level of response.
EXPERIMENTAL DESIGN 31
 Some medical investigations have indicated that nonsmokers are more likely
to develop Alzheimer's disease than smokers. Assuming these investigations
were carefully carried out, what is the most reasonable conclusion?
A) These were probably observational studies, and so no conclusion about
smoking provided protection again Alzheimer’s is proper.
B) Given that these were carefully carried out, they probably were experiments
and thus they do show that smoking provides protection against Alzheimer’s
(although it may have other undesired outcomes!).
C) Without information about use or nonuse of randomization, no conclusion
about cause and effect is possible.
CONTINUED
D) Without use of blinding, for example having all participants smoke
something (cigarettes or a harmless placebo), no conclusion is proper
E) No matter how carefully the investigations were carried out, no
conclusion is possible without knowing the sample sizes.
EXPERIMENTAL DESIGN 32
 A critical difference between experiments and observational studies is
A) An experiment often suggests a casual relationship, whereas an observational
study only suggests an association.
B) Observational studies make use of randomization, whereas experiments do not.
C) Experiments are generally more cost and time effective than observational
studies.
D) Tests of significance can be used on data collected from experiments but not on
data from observational studies.
E) Experiments are free to choose subjects from an entire population, whereas an
observational study only considers a random sample.
EXPERIMENTAL DESIGN 33
 In a survey of 30 elementary school students, those with larger shoe sizes
appear to have higher reading levels. Of the following, which is the most
important conclusion about this observation?
A) Parents interested in their child’s reading level should have their child wear a
slightly larger shoe size.
B) The variables, shoe size and reading level, are being confounded.
C) The sample size is too small for any reasonable conclusion.
D) There is a lurking variable.
E) As long as the sample was randomly selected, a cause-and-effect conclusion is
valid.
EXPERIMENTAL DESIGN 34
 To survey the opinions of people attending a particular Broadway play, a surveyor plans
to select every 25th theater-goer as he or she exits at the conclusion. Will this method
result in a simple random sample?
A) Yes, because each theater-goer has the same chance of being selected.
B) Yes, but only if everyone leaves by the same exit.
C) Yes, because the 24 out of the 25 theater-goers who are not selectd will form a control
group.
D) Yes, because this is an example of systematic sampling, which is a special case of simple
random sampling.
E) No, because not every sample of the intended size has an equal chance of being selected.
EXPERIMENTAL DESIGN 35
Consider the following three events:
I. Although 75 percent of Cubs fans believe they will go to the World Series this year, in
a random sample of 50 Cubs fans, only 30 “believe.”
II. In a survey about literacy, an embarrassed adult deliberately lies.
III. A surveyor mistakenly records answers to one question in the wrong space.
A) I, sampling error; II, response bias; III, human mistake
B) I, sampling error; II, nonresponse bias; III, hidden error
C) I, hidden bias; II, voluntary sample bias; III, sampling error
D) I, undercoverage error; II, voluntary error; III, unintentional error
E) I, small sample error; II, deliberate error; III, mistaken error
EXPERIMENTAL DESIGN 36
 Which of the following are true statements?
I. In well-designed observational studies, responses are systematically influenced
during the collection of data.
II. In well-designed experiments, the treatments result in responses that are as similar
as possible.
III. A well-designed experiment always has a single treatment but may test that
treatment at different levels.
A) I only
B) II only
C) III only
E) None of the statements are true.
D) II and III
EXPERIMENTAL DESIGN 37
 In which of the following studies are cause-and-effect (rather than simple
association) conclusions probably reasonable?
I. Studies noting heights and ages at death tend to show that taller people live longer
than short people.
II. Studies noting that children born prematurely tend to engage in less risky behavior
as adults.
III. Studies showing that animals fed low-calorie diets tend to live longer than animals
on normal diets.
A) I only
B) II only
C) III only
E) Not reasonable in any of the above.
D) I, II, and III
EXPERIMENTAL DESIGN 38
 You wish to survey people who have brought in their cars for service
during the past month. You decide to pick a random sample of gas
stations in the city and then survey all customers from those stations
who had work done during the past month. This procedure is an example
of which type of sampling?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple Random
EXPERIMENTAL DESIGN 39
 Suppose you wish to compare the average height of math/science
teachers to the average height of English/social studies teachers in your
high school. Which is the most appropriate technique for gathering the
needed data?
A) Census
B) Sample survey
C) Experiment
D) Observational study
E) None of these methods is appropriate
EXPERIMENTAL DESIGN 40
 Which of the following are true statements about sampling error?
I. Sampling error can be eliminated only if a survey is both extremely well
designed and extremely well conducted.
II. Sampling error concerns natural variation between samples, is always present,
and can be described using probability.
III. Sampling error is generally smaller when the sample size is larger.
A) I only
B) II only
C) III only
D) I, II, and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 41
 A researcher plans a study to examine the depth of belief in alternative, nontraditional
health procedures among the adult population. She obtains a simple random sample of
100 adults leaving an acupuncture/herbal medicine health center one morning. All but
one of them agree to participate in the survey, consisting of a series of neutrally worded
questions. Which of the following are true statements?
I. Proper use of chance as evidenced by the simple random sample, and neutral wording
make this a well-designed survey.
II. The high response rate makes this a well-designed survey.
III. Selection bias makes this a poorly designed survey.
A) I only
B) II only
C) III only
E) None of these statements is true.
D) I, II, and III
EXPERIMENTAL DESIGN 42
 A candy manufacturer wants to test consistency of the sugar content of
chocolate-covered cherries produced in one factory. The company
decides to randomly select three boxes of chocolate-covered cherries
from each day’s production run at the factory and analyze every cherry
in each of these boxes as to sugar content. What type of sampling is this?
A) Cluster
B) Convenience
D) Stratified
E) Systematic
C) Simple Random
EXPERIMENTAL DESIGN 43
 A town has one high school, which buses students from urban, suburban, and
rural communities. Which of the following sampling techniques is most
recommended in studying attitudes toward military enlistment after high school
graduation?
A) Cluster sample
B) Simple Random sample
C) Stratified sample
D) Systematic sample
E) Voluntary response sample
EXPERIMENTAL DESIGN 44
 Which of the following are true statements?
I. Based on careful use of control groups, experiments can often indicate causeand-effect relationships.
II. Observational studies may suggest relationships, but it would be very difficult
to conclude cause and effect because of the lack of control over lurking variables.
III. A complete census is the only way to absolutely establish a cause-and-effect
relationship.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 45
 What fault do all of these sampling designs have in common?
I. The New York Times plans to make a prediction for the Republican presidential nominee based
on a survey of its readers.
II. An Internet site asks their viewers to vote on their choice for “Movie of the Year.”
III. A statistics teacher randomly picks a sample of his students and interviews each one
concerning the clarity of the teacher’s explanations of new concepts.
A) All the designs make improper use of stratification.
B) All the designs have errors that can lead to strong bias.
C) All the designs confuse association with cause and effect.
D) None of the designs satisfactorily controls for sampling error.
E) None of the designs makes use of chance in selecting a sample.
EXPERIMENTAL DESIGN 46
 A study is made on whether of not a particular review book helps students achieve
higher scores on the AP Statistics exam. In comparing records of 100, half of whom
who purchased the review book, it is noted that the average AP Stat score is higher for
those 50 students who purchased the book. Which of the following are true
statements?
I. While this study indicates a relation, it does not prove causation.
II. There could well be a lurking variable responsible for the seeming relationship.
III. Self-selection here make drawing a conclusion difficult.
A) I and II
B) I and III
C) II and III
D) I, II and III
E) None of the above gives the complete set of true responses.
EXPERIMENTAL DESIGN 47
 Which of the following best explains why we try to guard against
confounding design experiments?
A) Confounding can lead to bias.
B) Confounding can conflict with randomization
C) Confounding can lead to uncertainty as to which variable is cause and
effect.
D) Confounding can make it more difficult to separate subjects into
treatment and control groups.
E) Confounding can negate the benefits of blinding.
EXPERIMENTAL DESIGN 48
 Sampling error is
A) the mean of a sample statistic.
B) the standard deviation of a sample statistic.
C) the standard error of a sample statistic.
D) the result of bias.
E) the difference between a population parameter and an estimate of that
parameter.