Sub-brand to go here
Want to be an Early Bird? Can
encouraging respondents to contact
interviewers to make appointments
increase co-operation and save
costs?
Matt Brown and Lisa Calderwood
Centre for Longitudinal Studies
GSS Methodology Symposium – 27th June
CLS is an ESRC Resource Centre based
at the Institute of Education
Context
•
Survey costs increasing (Stoop, 2005)
•
Increased focus on cost-effectiveness
•
Main element of survey costs is fieldwork
•
Experiment conducted to attempt to reduce fieldwork costs by
encouraging respondents to initiate contact with interviewers to
arrange appointments
Background
•
•
•
•
•
“Early Bird” innovation pioneered by National Longitudinal Studies –
1979 cohort
Respondents sent letter 2 weeks prior to fieldwork inviting them to
call free telephone number to arrange appointment for interview.
W22 (2004): $60/$80 incentive paid if telephoned within 4 weeks of
receiving letter (+ standard incentive ($40) for completing interview).
• 49% took up offer
Some impact on response rates:
• 80% overall
• 83% amongst those offered Early Bird
Big impact on fieldwork effort:
• 3 hours to complete interviewing for Early Birds
• 5 hours to complete interviewing for ‘non’ Early Birds
Research questions
• Can this approach be successful on longitudinal studies in the UK
context?
– Incentives typical in household panel surveys but usually much lower
value than US.
• Can sample members be motivated to be ‘early-birds’ without a
financial incentive?
– Incentives unusual in cohort studies in the UK
– Appeal to ‘helping’ tendencies (Groves, Cialdini and Couper, 1992)
– Increasingly consumer-drive, time-poor society
UKHLS
• Understanding Society: UK Household Longitudinal Study (UKHLS)
• 40,000 households
• Experiment conducted on the ‘Innovation Panel’ – 1500 households
• Develop and evaluate methodologies for longitudinal data collection
• Open call for proposals to carry out experimental designs in a longitudinal
context
• A unique resource for methodological research.
Experiment design
Group
EB offer with
incentive
Treatment
Additional £5 incentive paid to all household members who
completed an interview (standard incentive £5 or £10)
N
347
EB offer – no
incentive
Appeal for help
366
Control – no
EB offer
No request to contact interviewer in advance
375
Implementation
• Two treatment groups sent letter three weeks before fieldwork
– Next wave of study about to begin
– “Opportunity to request an Early Bird Appointment by contacting
interviewer on their mobile phone to arrange your interview at a time that
suits you”.
• Also sent a leaflet “Want to be an Early Bird?” which explained the offer
(identical other than mention of incentive or appeal)
• Given two weeks to contact interviewer to book an appointment for any date
within first 4 weeks of data collection. (Interviewers all issued with mobile
phones).
• Control group just sent letter
Results
• Take-up of offer
• Impact on response rate
• Impact on fieldwork efficiency
Results – take-up of offer
Contacted
interviewer
Made and kept
appointment
N
%
N
%
Base
EB Offer – Incentive
40
11.5
36
10.4
347
EB Offer – No incentive
24
6.6
21
5.7
366
Any Early Bird Offer
64
9.0
57
8.0
713
Results – Response Rates
Productive
interviews
N
%
Base
EB Offer – Incentive
266
76.7
347
EB Offer – No incentive
285
77.9
366
Any Early Bird Offer
551
77.2
713
Control group – No EB
274
73.1
375
EB Offer
Results –
Impact on fieldwork effort
Number of interviewer visits to complete all interviewing
Mean
Max
St. Dev
N
EB Offer – incentive
1.37
4
0.73
36
EB Offer – no incentive
1.45
5
1.0
21
All taking up offer
1.40
5
0.83
57
EB offered –incentive
3.49
14
2.29
231
EB offered – no incentive
3.41
21
2.04
265
EB not offered
3.44
13
2.04
274
All not taking up offer
3.45
21
2.12
770
All
3.31
21
2.12
825
EB Offer taken up
EB Offer not taken up
Results –
Impact on fieldwork effort
Total Number of Interviewer Visits – All issued households
Mean
St. Dev
N
EB Offer – incentive
3.10
2.29
347
EB Offer – no incentive
3.21
2.15
366
Any EB offer
3.16
2.22
713
Control group – no EB
offer
3.37
2.22
375
EB Offer
Early Bird Characteristics
Early Birds
Non-Early
Birds
Sig.
Average age
58.0
48.8
***
Sex (% Female)
74.1
52.0
***
Economic activity status (% Retired)
41.4
25.0
**
Average size
2.2
2.5
-
Children in home (<15) (%)
18.2
23.6
-
Individual
Household
13
Summary and Conclusions
• Low take-up rates:
– Small incentive – in absolute terms?
– Small increase in incentive relative to standard incentive?
– Poor marketing? Emphasis on the term ‘Early-Bird’?
– Materials not read
– Mode effects – Face to face vs telephone?
– Panel loyalty?
• Take-up rate significantly higher if incentive offered
– Appeal to ‘helping tendencies’ unsuccessful?
– More emphasis on how beneficial to the respondent?
14
Summary and Conclusions
• When taken up EB leads to big reduction in interviewer
visits needed to fully complete a case (as per NLSY)
• Low take-up means little impact on overall fieldwork
effort
• Need to boost take-up rates
• Higher incentive rates?
• Better marketing of the EB offer?
15
References
Groves, R.M., Cialdini, R.B. and Couper, M.P. (1992). Understanding the
decision to participate in a survey. Public Opinion Quarterly, 56, 475-495
Stoop, I. A. L. (2005). The Hunt for the Last Respondent: Nonresponse in
sample surveys. The Hague: Social and Cultural Planning Office.
16