Surveying Cell Phone-Only Canadians:
Looking at the Possibilities
Presentation to the
Marketing Research and Intelligence Association
Ottawa Chapter
February 19, 2009
Why the research was conducted
- presence of cell-only households
- higher incidence among younger people, men and singleperson households
- effect on representativeness of RDD samples
- part of wider study on the use the new technologies in
communications
Who conducted the research:
The sampling frame development: ASDE Survey Sampler
The sampling and interviewing: Ekos Research Associates
The analysis: Arcturus Research Solutions
The client: The Public Opinion Research Directorate of Public
Works and Government Services Canada
The current situation
-
14% of households in the US and 5% in Canada are cell-only
- The trend in Canada and especially the US is up
(see citation below)
- cell-only are disproportionately
- one-person unattached
- found in large urban areas
- 18 to 34 yrs
- below-average income
Source: Statistics Canada’s Residential Telephone Survey; and Ambrose, Don; Gray, Derrick; and Halpenny, Gary; Follow-up
on cell-only households – a growing concern for telephone surveys;. VUE, January 2008; pages 16-18
The current situation (continued)
RDD Surveys that exclude cell-only households could encounter
the following:
- increasingly incomplete sampling frame
- non response bias (young adults, single-person households)
If present trends continue, the problem is likely to get worse…
Comparison of Coverage of the Population:
Telephone and the Internet
Telephone: RTSS Survey, Statistics Canada 2006; Internet: New Technologies and Government of Canada Communications, 2007
100%
1%
5%
21%
80%
60%
94%
40%
79%
20%
0%
Telephone
Internet
no coverage
cell-only
Coverage
How the Cell Phone
Only (CPO) sample was
developed
Randa Bell
Vice President
ASDE Survey Sampler
February 19, 2009
• Discussions with clients at 2007 MRIA Conference in
Collingwood
• Following growth closely in US
• 5% of Canadian Households are CPO
(Stats Can December 2006 Report)
Idea comes to life!
Some numbers
• Cell phones first appeared in Canada around 1983
• Today, upwards of 67% penetration
• 3,000+ exchanges reserved for cell phone only
(Telcordia)
Some numbers (continued)
• Each exchange can contain up to 10,000 numbers …
potential of 30 million
• Estimate each exchange to be populated at 50 % …
which means 15 million cell phones to be reached in
Canada
The process
• Huge capacity of ASDE’s IVR System
• Generated numbers at random in cell phone dedicated
exchanges
• Proportionate to population of Canada by province
The process (continued)
• Programmed and recorded survey with one question
only
• Nature of cell phone we were reaching: CPO household
or in addition to landline
• Simplified wording
The questionnaire
• 20 seconds, National Study
• Only one question, nothing to sell
•
•
•
•
Phone line which is not cell phone = PRESS 1
Cell phone only household = PRESS 2
Reimbursement of call = PRESS 3
Repeat question = PRESS 9
The results
•
Two very useful lists
1.
2.
Any cell phone that connected
CPO households (most valuable list)
•
PWGSC chose sample from each list
How to incorporate cell only
samples into RDD
•
Dual frame sampling method
Portion from cell phones
Portion from RDD
•
Augment traditional sample with cell phone numbers (no
overlap)
How to incorporate cell only samples
into RDD
•
Include proportion of CPO households (7%) in sample
•
Possibly assign quota on completed interviews vs. starting
sample
Research Approach
A text message was sent on January 9, 2008 to 2,391
numbers provided by ASDE Survey sampler
English message was: “The Government has hired
EKOS to conduct a study on internet-based
technologies and their use in Canada. You are invited
to participate.”
Average length of interview: 12.8 minutes
Interviews conducted: January 14 to 28, 2008
The Government of Canada New Technologies in
Communications Study: the three modes
Item
Cell-only
RDD
GoC Online
Sample
203
1,718
2,619
Period
January
2008
Oct/Nov
2007
Oct/Nov
2007
Response/
completion
17%
22%
66%
Early Termination of Interviews
Question: “To begin, we would like to confirm that you are not currently operating a motorized vehicle or machinery of any
kind such as a car, truck, van, all terrain vehicle, snowmobile, etc…?” n=293
100%
97%
80%
60%
40%
20%
3%
0%
Not currently driving,
operating
Currently driving,
operating
Terminations owing to Presence of Landline
70 interviews terminated because they have landline in
the household
Recall of Text Message
Question: “Do you recall receiving a text message on your cell-phone or wireless device from our company in the past five to
seven days notifying you that we would be phoning you to complete an interview?
Please Note: All survey respondents who wish will be sent a gift certificate for $10.00 to either Chapters-Indigo or Tim
Hortons, their choice, as a token of our appreciation.” n=203
100%
80%
60%
59%
33%
40%
20%
7%
1%
0%
Yes,
definitely
Yes,
vaguely
No
DK/NR
Preferred Incentive
Question: “As we mentioned at the beginning of the survey, we would like to send you a $10 gift certificate to either ChaptersIndigo or Tim Hortons for taking part in this surveys, would you like to receive a
$10 gift certificate?” n=203
100%
80%
63%
60%
40%
28%
20%
10%
0%
Yes, ChaptersIndigo
Yes, Tim
Hortons
No
The Call Disposition
Total numbers accepted
2,391
Total out of scope
549
Unresolved
920
In-scope non-responding
611
Language difficulty
10
Unavailable
0
Refusals
Break-offs
598
3
In-scope responding
Completes
311
203
Ineligible
77
Quota filled
31
Response rate
16.9%
The Cell-Phone Interview Situation
Response to question, “While completing this interview, were you alone or with other people?” (n=203)
100%
80%
68%
60%
40%
29%
20%
3%
0%
Alone
Family and
friends
Others
Cell Phone Ownership
Response to question, “Are you the sole owner of this cell phone or is its shared with at least one other person 16 years of
age or older?” (n=203)
100%
95%
80%
60%
40%
20%
5%
0%
Sole owner
Not sole owner
Location of the Cell-Phone Interview
Response to question, “When we contacted you, were at …work or school…at home…outdoors?” (n=203)
75%
80%
60%
40%
20%
10%
10%
0%
Work or school
Home
Outdoors
The Sample: by Gender
(n=203)
100%
80%
31%
41%
41%
51%
60%
40%
Women
Men
69%
59%
59%
Sample,
weight
RTSS
20%
49%
0%
Sample,
unweight
Census
The Sample: by Age
(n=203)
100%
80%
60%
40%
20%
39%
28%
15%
38%
26%
16%
31%
6%
0%
16-24
25-34
35-44
55+
Sample,
unweight
Census
The Sample: by Education
(n=203)
100%
80%
44%
58%
56%
60%
23%
40%
27%
28%
20%
33%
15%
16%
Sample,
unweight
RTSS
0%
Census
Postsecondary
High School
<High School
The Sample: by Household Income (in $000)
(n=203)
100%
6%
7%
7%
8%
33%
31%
16%
80%
60%
51%
40%
20%
54%
54%
33%
0%
Sample,
unweight
RTSS
Census
DK/NA
100K+
40K to 99K
0K to 39K
The Sample: by Number of Persons in Household
(n=203)
100%
80%
60%
24%
13%
38%
28%
29%
34%
40%
20%
46%
59%
28%
0%
Sample,
unweight
Cell-only,
RTSS
Census
(RTSS)
3+
2
1
Internet Use: Cell-only versus Landline Respondents, by
Age
Question: “Do you personally use the Internet? This includes using e-mail.”
(cell-only, n=203; landline, n=1718)
100%
88%93%
90% 90%
80%
70%
74%
60%
Cell-only
Landline
40%
20%
0%
16 to 24
25 to 34
35+
Mean Hours per Week on the Internet: Cell-only versus
Landline Respondents
Question: “ln a typical week, how many hours do you spend on the Internet, not including the time you spend sending or
receiving e-mails? Please include all locations, such as home, work, school, etc…”
(cell-only, n=172; landline, n=1,314)
40
20.9
20
18.1
14.1 14
11.3
Cell-only
Landline
8.9
0
16 to 24
25 to 34
35+
Reading or Contributing to Blogs: Cell-only versus
Landline Respondents
Question: “How frequently, if ever, do you… read or contribute to a blog?”
(cell-only, n=168; landline, n=1203)
100%
15%
7%
80%
36%
42%
daily or more
weekly
never
60%
40%
20%
49%
51%
Cell-only
Landline
0%
Participating in Social Network Sites: Cell-only versus
Landline Respondents
Question: “How frequently, if ever, do you…participate in a social networking site like Facebook or Myspace?”
(cell-only, n=168; landline=1,271)
100%
23%
80%
38%
28%
60%
40%
20%
31%
49%
30%
0%
Cell-only
Landline
daily or more
weekly
never
Using Sites like YouTube: Cell-only versus Landline
Respondents
Question: “How frequently, if ever, do you…view, post or download videos from sites like YouTube?”
(cell-only, n=168; landline, n=1340)
100%
18%
11%
80%
60%
46%
54%
40%
20%
43%
28%
0%
Cell-only
Landline
daily or more
weekly
never
Mean Ratings for Web 2.0 Reliability compared to
Traditional Media
“Thinking about the different Internet-based applications and technologies that we have been talking about, do you think that
the information available on these sources is more or less reliable than information obtained through traditional media, such
as television, radio or print? Please use a 5-point scale where 1 means much less reliable, 5 means much more reliable and
the mid-point 3 means the same level of reliability.”
(cell-only, n=168; landline=1,292; online, n=1,216)
3.5
3
2.73
2.88
3.15
2.5
2
Mean
1.5
1
0.5
0
Online
RDD
Cell-only
Significant differences between the Cell-Phone and
Landline Samples: Demographics
(significant at the a=.05 level)
Significant
Not significant
Gender
Age
Official Language
Education
Income
Household Size
Employment Status
Province
Urban/Rural Residence
Significant differences between the Cell-Phone and
Landline Samples: Attitudes
(significant at the a=.05 level)
Significant
Not significant
“Internet more reliable than
traditional media”
“The Government should
invest in Web 2.0”
“Canadians should be
allowed to post content on
GoC web sites”
“The Government should
post material on popular
web sites”
“If GoC material appears on
a web site, it means that the
Government endorses the
web site”
Significant differences between the Cell-Phone and
Landline Samples: Behaviour
(significant at the a=.05 level)
Significant
Not significant
Hours spent on Internet
Keep informed of
Government news and
information
Contribute to blogs
Messages and chat rooms
Contribute to social
networking sites
Download from YouTube
Internet use
Use of pod/video webcasts
Preference for Survey Mode: Cell-Only Sample
Question: “If you were invited to take part in a Government of Canada public opinion survey, which method would you prefer:
completing an online survey or taking part in cell-phone interview, or you have no preference?”
1%
41%
50%
8%
Online
Cell
No preference
DK/NA
Likelihood of Participating in Another Cell-Phone
Survey
Response to question, “How likely are you to agree in the future to participate in a Government of Canada survey over the cell
phone: very likely, somewhat likely, not very likely or not at all likely?”.
60%
51%
40%
33%
20%
12%
4%
0%
Very
Somewhat
Not very
Not at all
Reasons Given for Not Participating in Future CellPhone Survey (number of respondents)
(Among persons who said “not very likely” or “not at all likely”) Response to question, “Why do you say that?”
Number of responses (n=26)
12
11
10
8
8
6
3
4
2
2
2
0
Too
busy/time
Dislike
surveys
Cost
Prefer
Online
DK/NA
Costing: the Final Analysis
Total cost: $20,803
Cost per completed interview: $102.48
$9,432
$10,000
$7,741
$8,000
$6,000
$4,000
$1,800
$1,830
$2,000
$0
SMS, letter,
postage
Incentives
Interviewing
Sample
Development
Costing: Hypothetical Survey of 1,000 Adults
Cost per completed interview 50 cell-only: $102.48
Cost per completed interview 950 landline: $24.21
$28,124
$30,000
$23,000
$25,000
$20,000
$15,000
$10,000
$5,124
$5,000
$0
50 cell-only
950 landline
1000 total
Conclusions
-Surveys of cell-phone only households feasible in
Canada
- Significant differences between cell-only and landline
respondents:
- subject matter
- demographics
- attitudes
- behavioural
Conclusions (continued)
- Canadians will respond
- ethical/respondent burden
- keep the survey short (12 minutes or less)
- high cost relative to other modes
- keep in mind target population of survey
The complete report is also
available
For a copy of the complete report (English) or
the executive summary (English or French),
see the report A Survey of CellularTelephone-Only Households The New
Technologies “Web 2.0” and Government of
Canada Communications Project at
http://www.porr-rrop.gc.ca/index-e.html