Mixed-Mode Approaches in the
Generations and Gender Survey
Past Experience and Future Expectations
Aat Liefbroer
Netherlands Interdisciplinary Demographic
Institute
Overview
Overview of the presentation
 GGP: what is it?
 Past experience
 3rd wave of Dutch GGP 2010-2011
 Future expectations
 Pilot 4th wave in Slovenia 2011
 Blueprint for GGP 2015
 Conclusions
Overview
Aim of the GGP
 To advance our knowledge on intergenerational and
gender relationships, with a focus on understanding
gender inequalities and generational differences
 Internationally comparable data on relationships between
partners and between parents and adult children across
Western societies
 Main vehicles: Generations and Gender Survey (GGS) and
Contextual Database
Overview
Characteristics of the GGS
 Broad age-range (18-79), because understanding demographic
challenges such as population ageing asks for a focus on both young
and old people
 Large-scale surveys on population challenges (N≈10,000) to allow indepth analysis
 Panel design to allow for better causal analysis and studying processes
of adaptation to change
 Cross-national to allow for examining the influence of the social
context (including the policy context)
 Theory-driven questionnaire design
 Contextual macro-level database to allow for multi-level analyses
Overview
GGP Countries
 Wave 1
 18 countries (data for 12 countries currently available)
 Wave 2
 Bulgaria, Netherlands, Australia, Germany, France, Hungary, Italy,
Russian Federation, Georgia (Bulgaria and Germany to be released this
year)
Overview
Challenges for the future of the GGP
 Increasing the number of participating countries
 Extending the data-collection beyond three waves
 Reducing fieldwork costs
 Application of mixed mode design?
3rd wave Dutch GGP
Dutch GGP
 1st wave 2004 ≈ 8,200 respondents, CAPI
 2nd wave 2007 ≈ 6,200 respondents, CAPI
 3rd wave 2010 ≈ 4,300 respondents, mixed mode
 Response rate in 3rd wave: 72% of wave 2 participants
 Costs per respondent in 2nd wave: ≈ €195
 Costs per respondent in 3rd wave: ≈ €115
3rd wave Dutch GGP
Distribution by mode
18%
55%
CAPI
CATI
WEB
27%
3rd wave Dutch GGP
Mode distribution by age
100
80
WEB
CATI
CAPI
60
40
20
0
25-44
45-64
65-89
3rd wave Dutch GGP
Mode distribution by educational
attainment
100
80
WEB
CATI
CAPI
60
40
20
0
low
medium
high
3rd wave Dutch GGP
Missing income info by mode
 Percentage that does not mention earnings from labour
10
8
6
4
2
0
CAPI
CATI
WEB
3rd wave Dutch GGP
Disclosure
 Mean score on loneliness scale by mode (with and without
controls)
1.5
1.4
1.3
without
with
1.2
1.1
1
CAPI
CATI
WEB
3rd wave Dutch GGP
Attrition information (I)
 Percentage that gives permission to contact their partner
to fill out a questionnaire
90
80
70
60
50
CAPI
CATI
WEB
3rd wave Dutch GGP
Attrition information (II)
 Percentage that gives permission to link data to register
data from Statistics Netherlands
100
90
80
70
60
50
CAPI
CATI
WEB
3rd wave Dutch GGP
Attrition information (III)
 Percentage that gives permission to be recontacted for the
fourth wave
100
80
60
40
20
0
no
yes, decide later
yes, participate
CAPI
CATI
WEB
3rd wave Dutch GGP
Conclusions
 WEB more popular than expected
 Missing values on WEB limited
 Small, but systematic mode effects in disclosure
 WEB respondents are less willing to give permission for
additional or future involvement
Pilot Slovenia
Pilot 4th wave Slovenia 2011
 Participating institutions
 University of Ljubljana (Lozar-Manfreda, Petrič)
 University Utrecht (Hox, De Leeuw)
 NIDI (Kveder, Liefbroer)
 Study 1: comparison of mode effects
 Study 2: comparison of mixed mode systems
Pilot Slovenia
Comparison of mode effects
 Aim: testing new pilot and examination of potential mode
effects
 Sample: regular participants in commercial web panel,
randomly assigned to different modes
 Examination of





Item non-response
Scalability of item sets
Distributional characteristics
Length of interview
Evaluation of interview
Pilot Slovenia
Comparison of mode effects
 Item non-response only slightly higher in web, with one
clear exception….
Pilot Slovenia
Income
 Percentage of refusals and ‘do not know’ on question on
household income
50
40
30
refusal
20
don't know
10
0
CAPI
CATI
WEB
Pilot Slovenia
Comparison of mode effects
 Item non-response only slightly higher in web, with one
clear exception….
 Scalability of item sets still has to be analysed
 First analyses of distributional characteristics suggest
 More disclosure on social desirable and difficult questions
 Slightly higher variance in responses
Pilot Slovenia
Example
 Percentage answering ‘yes’ to the question ‘Over the past
12 months, have you thought about breaking up your
relationship?’
40
30
20
10
0
CAPI
CATI
WEB
Pilot Slovenia
Comparison of mode effects
 Item non-response only slightly higher in web, with one
clear exception….
 Scalability of item sets still has to be analysed
 First analyses of distributional characteristics suggest
 More disclosure on social desirable and difficult questions
 Slightly higher variance in responses
 Interview took somewhat longer in CATI mode
 Evaluation of interview most negative in CATI mode
Pilot Slovenia
Duration of survey
65
60
55
50
45
CAPI
CATI
WEB
Pilot Slovenia
Subjective length of survey
 Percentage stating ‘yes’ or ‘definitely yes’ to question
whether the interview was judged too long
40
30
20
10
0
CAPI
CATI
WEB
Pilot Slovenia
Comparison of mixed-mode systems (I)
 Aim: testing of alternative mixed-mode systems
 Sample: two-stage random sample from the population
register
 Examination of
 Response rate of different mixed-mode systems
 Costs of different mixed mode systems
 Evaluation of modes by a random sample
Pilot Slovenia
Comparison of mixed-mode systems (II)
 Comparison of four ‘systems’




CAPI → WEB → CATI (no incentive offered)
CATI → WEB → CAPI (no incentive offered)
WEB → CATI → CAPI (no incentive offered)
WEB → CATI → CAPI (incentive of € 5 offered)
 Difference in response rate across systems
 Difference in costs across systems
Blueprint GGP 2015
Blueprint GGP 2015
 Blueprint for GGP 2015 should be ready by the end of 2012
 Questionnaire GGP 2015
 Sampling design
 Set of fieldwork rules
 Whether to opt for mixed-modes, and if so when?
 Is it an option in the first wave (or in the first wave with a
refreshment sample), or in later waves only?
 What kind of mixed-mode system to prescribe?
Conclusion
Conclusions
 Overall, WEB seems to perform quite well in terms of
answering patterns (item nonresponse, scalability and
distribution of responses)
 WEB leads to very considerable cost reductions
 The big issue that is not yet clear is whether attrition is
negatively affected
 Use of WEB in a first wave seems questionable (but pilot will give
partial answers)
 Use in later waves could lead to higher dropout in future waves as
a result of the legal requirement to ask for permission for recontacting
Thank you for your attention!
Overview
Topics included
 Provision of care to older adults
 Successful ageing
 Economic participation of women and pre-pensioners
 Realisation of fertility intentions
 Balancing work and family life
 Gender equality in household and childcare labour
Members of Consortium Board












Netherlands Interdisciplinary Demographic Institute (NL)
Institut National d’Études Démographiques (FR)
Bocconi University (IT)
Statistics Norway (NO)
Demographic Research Institute (HU)
Norwegian Social Research (NO)
University of Ljubljana (SL)
Utrecht University (NL)
University of York (GB)
Max-Planck Institute for Demographic Research (DE)
Erasmus University Rotterdam (NL)
Population Unit of the UN Economic Commission for Europe