TECHNICAL PAPERS ON
HEALTH AND BEHAVIOR MEASUREMENT
TECHNICAL PAPER 69
Sample Design and Demographic Characteristics of Respondents in the
1999– 2000 National STD and Behavior Measurement Experiment
Maria A. Villarroel, Charles F. Turner Elizabeth Eggleston, Alia Al-Tayyib Susan M.
Rogers, Anthony M. Roman, Phillip C. Cooley, Harper Gordek
Reference Citation
Villarroel MA, Turner CF, Eggleston E, Al-Tayyib A, Rogers SM, Roman AM, Cooley PC,
Gordek H. (2006) Sample design and demographic characteristics of respondents in the
1999–2000 National STD and Behavior Measurement Experiment. Technical Papers on
Health and Behavior Measurement, No. 69, Washington DC: RTI Program in Health and
Behavior Measurement.
Sample Design and Demographic Characteristics of Respondents in the
1999– 2000 National STD and Behavior Measurement Experiment
Maria A. Villarroel, Charles F. Turner Elizabeth Eggleston, Alia Al-Tayyib Susan M. Rogers,
Anthony M. Roman, Phillip C. Cooley, Harper Gordek
The 1999-2000 National STD and Behavioral Measurement Experiment (NSBME) randomly
assigned a probability sample of U.S. adults ages 18 to 45 to have sensitive questions asked either by a
human telephone interviewer (T-IAQ: telephone interviewer-administered questioning) or by Telephone
Computer-Assisted Self Interviewing (T-ACASI) technology. This working paper summarizes the
NSBME’s sample design and presents a comparison of the social and demographic characteristics of
subjects in the two experimental conditions. This information supplements study details provided by
Turner et al. (2005) and Villarroel et al. (2006). (Additional information can be found in Roman [2000]
and Nyman et al. [2004].)
Sample Design.
The NSBM experiment was embedded in a telephone survey of a probability sample of women
and men aged 18 to 45 years residing in U.S. households with working land line telephones. The survey
was conducted between September, 1999 and April, 2000. Two sample strata were recruited for this
survey measurement experiment: (1) a sample of the telephone-accessible U.S. household population
aged 18 to 45 (national stratum), and (2) a parallel sample of the population of the city of Baltimore, MD
(Baltimore stratum). A list-assisted random digit dialed (RDD) sample was drawn for each stratum using
the Genesys Sampling System (2002).1
1
This system creates a sampling universe for telephone households by creating all possible "blocks" of 100
telephone numbers. A block is created by using the first eight digits of a telephone number. For each valid telephone
area code (NPA) and each valid telephone exchange (NXX) combination, 100 such blocks are created (i.e., NPANXX-00xx, NPA-NXX-01xx,...,NPA-NXX-99xx). Once these blocks are created, a search is conducted to
determine if any one of the 100 possible telephone numbers appears in a database that contains all directory-listed
residential telephone numbers in the United States (or Baltimore for that stratum). If at least 1 of the 100 numbers is
found in the residential directory database, the entire block is retained in the sampling universe. Blocks with no
residential numbers are eliminated. A simple random sample from all telephone numbers in this sampling universe
1
Sample Execution. For the national stratum, 14,250 telephone numbers were generated, and
12,322 telephone numbers (86.5%) were successfully screened for eligibility (Roman, 2000).2 2,183 of
these screened telephone numbers were found to be residential numbers with one or more eligible
English-speaking respondents aged 18 to 45. One eligible household member of these households was
randomly selected for participation in the survey (without substitution). 1,452 of the 2,183 target
respondents in the national strata completed3 interviews (66.5%), and 91 respondents (4.2%) completed
partial interviews that included at least one substantive questionnaire section. A maximum of 91 calls per
household were made to screen households and complete an interview in the national stratum. Ten
percent of all completed interviews required 25 or more calls. Of 709 completed interviews assigned to
be conducted in the T-ACASI experimental condition, 24 (3.4%) were administered in T-IAQ mode
because respondents did not have a touchtone telephone. (Data for these 24 cases are excluded from our
analyses.) Interview completion rates were higher in the T-IAQ condition than in T-ACASI (71.5%
versus 61.3%). Data for 10 of the 1,452 completed interviews in the national stratum were lost due to a
data processing error.
The second sample stratum was drawn to represent the adult population of Baltimore,
MD.4 For the Baltimore stratum, 7,498 telephone numbers were generated and 6,326 (84.4%)
is then drawn. The sampling universe is updated every three months to include new area codes, exchanges, and
changes in the status of any blocks. Elimination of “zero blocks” from the sampling universe in this procedure
greatly increases sampling efficiency. About 45% of telephone numbers dialed are residential compared to fewer
than 20% if the zero blocks are retained. Brick, et al (1995) have shown that only about four percent of all telephone
households in national surveys are eliminated from possible selection by the elimination of the zero blocks.
2
629 numbers were contacted but refused screening, 177 were determined to be residences but could not be screened
for eligibility, and the status of 1,122 numbers could not be determined typically because the number generated a
ring with no answer on all call attempts. In studies in which random samples of these numbers have been vigorously
pursued, the survey organization found that only four percent of such numbers were working, residential telephones
(Roman, 2000) and thus would have been eligible for inclusion in the NSBME.
3
Interviews were considered “complete” if the respondent completed the 101st of 123 questions in the male version
of the questionnaire and the 103rd of 125 questions in the female version (excluding the closeout section on reactions
to survey).
4
This sample stratum was included to permit comparison of NSBME results to the in-person Audio-CASI and IAQ
measurements made in a parallel experiment embedded in the 1997-98 Baltimore STD and Behavior Survey (Turner
et al. 2002). BSBS comparisons are not included in the present paper because the BSBS used a Paper SAQ
(compared to audio-CASI) to administer questions on same-gender sexual contact.
2
were successfully screened for eligibility. Screening identified 1,072 households with an
eligible respondent, and 697 of these eligible respondents completed interviews (65.0%). An
additional 47 respondents (4.4%) completed partial interviews. A maximum of 82 calls per
household were made to screen households and complete an interview in the Baltimore stratum.
Ten percent of all completed interviews required 26 or more calls. Of 332 completed Baltimore
interviews assigned to be conducted in the T-ACASI experimental condition, 15 (4.5%) were
completed in T-IAQ mode because respondents did not have a touchtone telephone. (These 15
cases are excluded from our analyses.) As in the national stratum, interview completion rates
were higher in the T-IAQ condition, but the difference was smaller in magnitude (67.7% vs.
62.3%).
AAPOR Response Rates. Roman (2000, pp. 23-29) has calculated the AAPOR
response rates (formula RR3)5 to be 62% for the T-IAQ condition and 53% for the T-ACASI
condition in the national strata. In the Baltimore strata, these response rates were 56% for the
T-IAQ condition and 50% for the T-ACASI condition.
Sample Equivalence.
Table 1 presents tests of the equivalence of the sociodemographic characteristics of
respondents completing T-IAQ versus T-ACASI interviews. These results indicate that the
social and demographic characteristics of respondents in the two experimental conditions were
statistically equivalent.
5
The Center for Survey Research (CSR) has taken random samples of telephone numbers of unknown eligibility
status (i.e., ring but never answer) in household surveys and rigorously pursued all avenues to reconcile their status.
CSR has consistently found that only approximately 4% of such numbers are eligible residential telephones. This
figure was used to estimate the number of eligible households among such cases when calculating the NSBME
response rates.
3
References.
Brick JM, Waksberg J, Kulp D, Starer A. (1995). Bias in list-assisted telephone samples. Public
Opinion Quarterly, 59:218-235.
Genesys Sampling Systems. (2002). Genesys Methodologies, <http://www.genesyssampling.com/refernce/genmeth.htm>, Accessed June 24, 2002.
Nyman AL, Roman AM, Turner CF (2004). Comparison of Computer-Assisted Telephone
Survey Methodologies: CATI vs. T-ACASI. (Paper presented at annual meeting of
American Association for Public Opinion Research, Montreal, Quebec, Canada, May
17-20, 2001.) Technical Papers on Health and Behavior Measurement, No. 57,
Washington DC: RTI Program in Health and Behavior Measurement.
Roman AM. (2000). Survey of AIDS and Other Social Issues: Field Report. Unpublished
manuscript, Boston: Center for Survey Research, University of Massachusetts.
Turner CF, Villarroel MA, Rogers SM, Eggleston E, Ganapathi L, Roman AM, Al-Tayyib A.
(2005). Reducing bias in telephone survey estimates of the prevalence of drug use: A
randomized trial of Telephone Audio-CASI. Addictions, 100:1432-1444.
Villarroel MA, Turner CF, Eggleston EE, Al-Tayyib AA, Rogers SM, Roman AM, Cooley PC,
Gordek H. (2006) Same-Gender sex in the USA: Impact of T-ACASI on prevalence
estimates. Public Opinion Quarterly, in press.
4
TABLE 1. Demographic characteristics of respondents by interview mode. (Unweighted data
from 1999-2000 NSBM Experiment, combined National and Baltimore sample strata). [a]
T-ACASI
T-IAQ
%
n
%
n
P (b)
Gender
Female
Male
58.5
41.5
931
58.8
41.2
1,164
>.5
Age
18-25
26-35
36-45
23.1
37.7
39.2
929
24.9
36.6
38.5
1,160
>.5
Marital Status
Married
Cohabiting
Divorced, separated, widowed
Never married
42.8
10.8
13.8
32.7
930
41.3
9.3
14.0
35.4
1,162
0.477
Education
Less than high school (f)
High school
Some college or trade school
College graduate or more
7.8
23.3
35.9
32.9
930
8.3
24.1
34.2
33.4
1,162
>.5
Children in Household
No
Yes
40.4
59.6
929
40.8
59.2
1,162
>.5
Race - Ethnicity
Black (non-Hispanic)
White (non-Hispanic)
Hispanic - Latino origin
Other (non-Hispanic)
23.6
61.7
7.3
7.4
929
24.7
58.7
8.4
8.2
1,150
>.5
Sample
National
Baltimore
66.3
33.7
1,164
0.286
0.434
Demographic characteristic
931
68.5
31.5
c, d
Level of Urbanization
21 largest MSAs
Counties with 85,000+ households
Counties with 20,000-84,999 households
Counties with <20,000 households
36.0
31.3
17.8
14.8
594
36.0
34.4
17.4
12.2
780
Region c, e
Northeast
North Central
South Atlantic
South Central
Mountain
Pacific
15.6
27.9
18.0
17.0
7.5
14.1
617
19.9
24.7
18.1
15.8
7.2
14.3
797
0.367
(a) Analysis compares participants who reached the first question on same-gender sexual behavior, at least
three-fourths of the way into the questionnaire. Analysis excludes 39 respondent assigned to T-ACASI who
were interviewed in T-IAQ because they did not have a touchtone telephone.
(b) P-value for Pearson chi-square test.
(c) National sample only
(d) 21 largest MSAs refers to counties belonging to the top 21 metro areas (baaed on household counts from
1990 Census): Atlanta, Boston-Worcester-Lawrence-Lowell-Brockton, Chicago-Gary-Kenosha, ClevelandAkron, Dallas-Fort Worth, Detroit-Ann Arbor-Flint, Houston-Galveston-Brazoria, Los Angeles-RiversideOrange, Miami-Fort Lauderdale, Minneapolis-St. Paul, New Haven-Bridgeport-Stanford-Waterbury-Danbury,
New York-Northern New Jersey-Long Island, Philadelphia-Wilmington-Atlantic City, Phoenix-Mesa, Pittsburgh,
San Diego, San Francisco-Oakland-San Jose, Seattle-Tacoma-Bremerton, St Louis, Tampa-St. PetersburgClearwater, Washington-Baltimore. Counties with 85,000+ households are counties with 85,000 or more
households in metropolitan areas not included in the 21 largest MSAs. Counties with 20,000-84,999
households are counties with 20,000 or more households not included in the larger categories. Counties
with <20,000 households are all remaining areas not included in the larger categories described above.
(e) Geographical regions: North East (CT, MA, ME, NH, NJ, NY, PA, RI & VT); North Central (IA, IL, IN, KS,
MI, MN, MO, ND, NE, OH, SD & WI); South Atlantic (DC, DE, FL, GA, MD, NC, SC, VA & WV); South Central
(AL, AR, KY, LA, MS, OK, TN & TX); Mountain (AZ, CO, ID, MT, NM, NV, UT & WY); Pacific (AK, CA, HI, OR &
WA).
(f) The T-ACASI condition found 2.5% of respondents reported having an 8th grade education or less while
only 0.8% of respondents told a human interviewer they had left school in 8th grade or earlier. We suspect
that this variation in reported educational attainment may reflect an impact of survey mode — with
respondents being more willing to report very low levels of educational attainment in the more private TACASI interview condition.
Tech69_Table