HANDOUTS
WHO NEEDS RDD?
COMBINING DIRECTORY LISTINGS WITH CELL PHONE
EXCHANGES
FOR AN ALTERNATIVE TELEPHONE SAMPLING FRAME
Thomas M. Guterbock, Director
TomG@virginia.edu
Abdoulaye Diop, Senior Research Analyst
ADiop@virginia.edu
James M. Ellis, Director of Research
jme2ce@virginia.edu
John Lee P. Holmes, Survey Operations Manager
jlh2r@virginia.edu
Kien T. Le, Research Analyst
tkl7b@virginia.edu
Center for Survey Research, University of Virginia
Presented at the Annual Meeting of
American Association for Public Opinion Research
New Orleans, LA
May 16, 2008
WELDON COOPER
CENTER FOR PUBLIC
SERVICE
University of Virginia
ABSTRACT
WHO NEEDS RDD?
COMBINING DIRECTORY LISTINGS WITH CELL PHONE
EXCHANGES
FOR AN ALTERNATIVE TELEPHONE SAMPLING FRAME
Submitted for the AAPOR Annual Meeting, New Orleans, May 2008
The traditional Random Digit Dialing method (list-assisted RDD using a frame of
landline phone numbers) is clearly under threat. The difficulty and costs of completing
telephone surveys have increased due to rising rates of refusal and non-contact. The
completeness of coverage of list-assisted RDD samples has diminished due to the
proliferation of cell-phone only households. The ability of list-assisted RDD to capture
young, mobile, unmarried and minority households is thus diminishing as well.
Increasingly, survey researchers have been adding a cell phone component to their
sampling frames for telephone surveys, despite the increased costs and other issues
associated with RDD calling of cell phones.
Recent research by Guterbock, Oldendick, and others has explored the extent to
which “electronic white pages” (EWP) samples really differ from RDD samples.
Recently, Zogby has released results asserting that these differences are often small and
ignorable. In contrast, Oldendick et al. and Guterbock, Diop and Holian have
emphasized that minority households are seriously underrepresented in EWP samples.
Nevertheless, EWP samples have distinct advantages whenever a survey is aimed at a
restricted geographic area.
This paper considers the feasibility of combining EWP samples with cell-phone
RDD, eliminating the ordinary RDD component from the sampling frame. We analyze
the components of the telephone population, showing that the proposed method would
fail to cover only one segment of the telephone population: unlisted landline households
that have no cell phone. We analyze data, including results from the 2006 National
Health Interview Study, to estimate the size of this segment, its demographic profile, and
the degree to which its demographic and health behavior characteristics are different from
those in the segments that this sampling strategy would capture. We discuss the
implications for constructing efficient and representative telephone sampling frames that
do not rely on list-assisted RDD.
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 2
Table I: 2006 NHIS Sample Characteristics
Frequency
1 cell phone only
2 cell phone with unlisted landline
3 unlisted landline only
4 cell phone with listed landline
5 listed landline only
Total
Missing (-99)
Total
Percent
Valid Percent
Cumulative %
3758
15.5
16.6
16.6
4009
16.5
17.7
34.3
3225
13.3
14.2
48.5
7766
32.0
34.3
82.8
3888
16.0
17.2
100.0
22646
93.3
100.0
1629
6.7
24275
100.0
Table II: 2006 NHIS - Demographics and Health Status of Respondents by Group
Variables
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
Group1
Cell Phone only
Group2
Cell with ULL
Group3
ULL only
Group4
Cell with LL
Group5
LLL only
Total
13.4%
14.2%
31.1%
15.7%
15.7%
8.3%
10.6%
11.6%
13.6%
11.0%
8.0%
15.0%
32.1%
26.6%
48.0%
23.1%
47.5%
32.9%
54.5%
26.6%
37.2%
13.1%
22.8%
27.4%
25.6%
10.7%
28.8%
7.3%
28.4%
17.5%
27.1%
18.9%
22.1%
17.6%
20.6%
20.5%
4.5%
6.7%
9.4%
7.8%
11.5%
7.9%
16.8%
22.9%
28.6%
28.5%
37.7%
27.2%
11.9%
10.9%
9.5%
11.3%
11.2%
11.1%
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 3
Table III: Listed Status of Households in CSR Surveys
PWC CS
Listed phone (yes)
[Prince William County, Virginia]
DMV
Listed phone (yes)
[Virginia Statewide]
Community Shielding
Listed phone (yes)
[National Capital Region RDD]
Critical Infrastructure
Listed phone (yes)
[Nationwide RDD]
2004
74.4%
2006
81.1%
2007
79.1%
2002
81.2%
2003
79.8%
2006
81.2%
2005
73.3%
2005
79.0%
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 4
Table IV: EWP versus RDD+Cell (2006 NHIS)
U
N
Variables
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
[Unlisted is the uncovered group]
(Yc  Yu)
U
(Yc  Yu )
N
Yc  Y
Relative
bias
Yc
Yu
48.5%
48.5%
48.5%
48.5%
47.9%
9.1%
10.0%
31.2%
16.3%
14.2%
14.4%
20.4%
34.8%
39.2%
21.1%
-5.3%
-10.4%
-3.6%
-22.9%
-6.9%
-2.6%
-5.0%
-1.7%
-11.1%
-3.3%
11.7%
15.0%
32.9%
27.4%
17.5%
-2.6%
-5.0%
-1.7%
-11.1%
-3.3%
-22.2%
-33.3%
-5.2%
-40.5%
-18.9%
48.4%
48.7%
48.5%
48.5%
18.6%
9.0%
31.6%
11.3%
22.6%
6.7%
22.5%
10.8%
-4.0%
2.3%
9.1%
0.5%
-1.9%
1.1%
4.4%
0.2%
20.5%
7.9%
27.2%
11.1%
-1.9%
1.1%
4.4%
0.2%
-9.3%
13.9%
16.2%
1.8%
Y
Table V: RDD versus RDD+Cell (2006 NHIS)
U
N
Variables
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
[Cell only is the uncovered group]
(Yc  Yu)
U
(Yc  Yu )
N
Relative
bias
Yc
Yu
16.6%
16.6%
16.6%
16.6%
16.7%
11.3%
11.8%
33.1%
22.0%
15.8%
13.4%
31.1%
32.1%
54.5%
25.6%
-2.1%
-19.3%
1.0%
-32.5%
-9.8%
-0.3%
-3.2%
0.2%
-5.4%
-1.6%
11.7%
15.0%
32.9%
27.4%
17.5%
-0.4%
-3.2%
0.2%
-5.4%
-1.7%
-3.4%
-21.3%
0.6%
-19.7%
-9.7%
16.5%
16.7%
16.6%
16.6%
19.3%
8.6%
29.2%
10.9%
27.1%
4.5%
16.8%
11.9%
-7.8%
4.1%
12.4%
-1.0%
-1.3%
0.7%
2.1%
-0.2%
20.6%
7.9%
27.2%
11.1%
-1.3%
0.7%
2.0%
-0.2%
-6.3%
8.9%
7.4%
-1.8%
Y
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 5
Yc  Y
Table VI: EWP+Cell versus RDD+Cell (2006 NHIS)
U
N
Variables
Demographics
African American
14.2%
18-25
14.2%
HS or Less
14.2%
Renter
14.2%
Less than $20,000
13.7%
Health Related Questions
Smokers (yes)
14.2%
Diabetes (yes)
14.2%
Hypertension (yes)
14.2%
Asthma (yes)
14.2%
[Unlisted landline only is the uncovered group]
Yu
Yc
(Yc  Yu)
U
(Yc  Yu )
N
Y
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
Relative
bias
11.0%
15.3%
30.5%
25.8%
15.7%
15.7%
13.6%
48.0%
37.2%
28.8%
-4.7%
1.7%
-17.5%
-11.4%
-13.1%
-0.7%
0.2%
-2.5%
-1.6%
-1.8%
11.6%
15.0%
32.9%
27.4%
17.5%
-0.6%
0.3%
-2.4%
-1.6%
-1.8%
-5.2%
2.0%
-7.3%
-5.8%
-10.3%
20.3%
7.7%
26.9%
11.3%
22.1%
9.4%
28.6%
9.5%
-1.8%
-1.7%
-1.7%
1.8%
-0.3%
-0.2%
-0.2%
0.3%
20.5%
7.9%
27.2%
11.1%
-0.2%
-0.2%
-0.3%
0.2%
-1.0%
-2.5%
-1.1%
1.8%
Table VII: Summary of estimates from four designs (2006 NHIS)
Variables
Yc  Y
RDD+Cell (total
telephone HH)
EWP
RDD
EWP+Cell
11.6%
15.0%
32.9%
27.4%
17.5%
9.1%
10.0%
31.2%
16.3%
14.2%
11.3%
11.8%
33.1%
22.0%
15.8%
11.0%
15.3%
30.5%
25.8%
15.7%
20.5%
7.9%
27.2%
11.1%
18.6%
9.0%
31.6%
11.3%
19.3%
8.6%
29.2%
10.9%
20.3%
7.7%
26.9%
11.3%
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 6
Table VIII: Summary of bias for four estimates (2006 NHIS)
Variables
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
RDD+Cell (total
telephone HH)
EWP
RDD
EWP+Cell
------
-2.5%
-5.0%
-1.7%
-11.1%
-3.3%
-0.3%
-3.2%
0.2%
-5.4%
-1.7%
-0.6%
0.3%
-2.4%
-1.6%
-1.8%
-----
-1.9%
1.1%
4.4%
0.2%
-1.2%
0.7%
2.0%
-0.2%
-0.2%
-0.2%
-0.3%
0.2%
Table IX: Summary of relative bias for four estimates (2006 NHIS)
Variables
Demographics
African American
18-25
HS or Less
Renter
Less than $20,000
Health Related Questions
Smokers (yes)
Diabetes (yes)
Hypertension (yes)
Asthma (yes)
RDD+Cell (total
telephone HH)
EWP
RDD
EWP+Cell
------
-21.6%
-33.5%
-5.3%
-40.5%
-18.8%
-2.6%
-21.5%
0.5%
-19.7%
-9.6%
-5.2%
1.8%
-7.4%
-5.9%
-10.2%
-----
-9.3%
13.9%
16.2%
1.8%
-5.9%
8.9%
7.4%
-1.8%
-1.0%
-2.5%
-1.1%
1.8%
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 7
GRAPH 1: NHIS data 2003-2006
18.0
16.0
16.7
16.6
16.5
15.1
14.0
12.6
percent
12.0
10.0
8.0
14.2
9.1
7.1
6.0
4.0
2.0
0.0
2003
2004
2005
2006
year
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 8
GRAPH 2: RDD vs RDD+Cell: 2003-2006
5.0%
0.0%
0.0%
2003
0.3%
-0.9%
-0.3%
2004
2005
-3.3%
-5.0%
0.6%
2006
-3.4%
-5.1%
African American
-7.4%
-7.6%
-8.1%
18-25
-9.7%
-10.0%
-11.2%
HS or Less
Renter
Less than $20,000
-15.0%
-16.5%
-17.7%
-19.7%
-20.0%
-21.3%
-25.0%
Year
GRAPH 3: EWP+Cell vs RDD+Cell: 2003-2005
5.0%
2.0%
0.0%
2003
-5.0%
-10.0%
2004
2005
-4.8%
-9.1%
-11.7%
-13.7%
-15.0%
0.7%
-9.2%
-9.5%
-11.2%
-9.4%
-5.2%
-5.8%
-7.3%
-10.3%
African American
18-25
HS or Less
Renter
-12.7%
-14.1%
2006
Less than $20,000
-20.0%
-25.0%
Year
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 9
GRAPH 4: RDD vs RDD+Cell: 2003-2006
10.0%
8.9%
8.0%
7.4%
6.0%
5.6%
4.0%
3.2%
2.0%
6.6%
5.8%
3.5%
Smokers (yes)
Diabetes (yes)
1.5%
Hypertension (yes)
0.0%
2003-1.0%
0.0%
2004
2005
-1.9%
-2.0%
2006
Asthma (yes)
-1.8%
-2.8%
-3.9%
-4.0%
-5.8%
-6.0%
-6.3%
-8.0%
Year
GRAPH5: EWP+Cell vs RDD+Cell: 2003-2005
10.0%
8.0%
6.0%
4.0%
2.0%
Smokers (yes)
1.6%
1.0%
0.0% 0.0%
2003
-2.0%
-4.0%
-1.9%
1.8%
1.0%
0.8%
Hypertension (yes)
0.0%
2005
-1.2%
2004
-2.4%
Diabetes (yes)
-1.9%
-2.6%
-1.0%
2006
-1.1%
Asthma (yes)
-2.5%
-4.2%
-6.0%
-8.0%
Year
AAPOR 2008 ~ Who Needs RDD ~ Guterbock et al. ~ CSR ~ University of Virginia ~ Page 10