European Commission
DG ECFIN
Directorate General Economic and Financial Affairs
Effect of the changeover to the
mandatory reply on the response rate
for the French surveys
José Bardaji, Olivier Biau,
Vincent Bonnefoy, Jean-Baptiste Pinault
Fourth Joint EU OECD Workshop on
Business and Consumer Surveys
Brussels, 12 -13 October 2009
ECFIN– 12/10/2009
Slide 1
Introduction
DG ECFIN
• In January 2004, two surveys became mandatory:
– Monthly « Activity »
– Quarterly « Investment »
• Biannual « Cash » survey remained voluntary
• Did the response rate improve ?
• Did the new respondents have specific behaviour?
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 2
Agenda
DG ECFIN
• Short presentation of the 3 surveys
• Descriptive analyse of the impact of the
obligation
– On the response rates (short and medium term)
– On the behaviour of new respondents
• Confirmation based on microeconometric study
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 3
3 French Industry Surveys
DG ECFIN
• Monthly: « Activity » survey (11 per year)
• Quarterly: « Investment » (3 per year)
Both included in the JH EU Programme of BCS
• Biannual: « Cash » survey (2 per year)
Not included in the JH EU Programme of BCS
• Same survey sample (n=4.000 firms), stratified
sampling with exhaustive stratum
• Qualitative questions: ↑ = ↓
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 4
DG ECFIN
Common methodology
to analyse the response rate
• To eliminate seasonality in response rate, analyse on
several dates before and after 2004
• Short term (2003-2004):
6 « Activity », 2 « Investment » and 2 «Cash »
• Medium term and econometric analysis (2001-2006):
66 « Activity », 18 « Investment » and 12 « Cash »
• To eliminate the periodically updates of the sample, study
based on balanced samples
• Response: at least one in the questionnaire
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 5
DG ECFIN
Analyse of the response rates
short term: 2003-2004
Activity
Investment
Cash
Before obligation
78,9 %
71,4 %
76,4 %
After obligation
81,2 %
74,3 %
75,3 %
Difference
+2,3 pp
+2,9 pp
-1,1 pp
Rejected
(p-value = 0,0002)
Rejected
(p-value = 0,0002)
Rejected
(p-value = 0,1834)
t-Test (difference=0)
• Significant increase of response rate for « Activity »
(+2 points) and « Investment » (+3 points)
• Not for « Cash »
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 6
DG ECFIN
Analyse of the response rates
medium term: 2001-2006
Activity
Investment
Cash
Before obligation
84,4%
76,0%
82,2%
After obligation
89,7%
84,9%
83,7%
Difference
+5,3 pp
+8,8 pp
+1,5 pp
Rejected
(p-value < 0,0001)
Rejected
(p-value < 0,0001)
Rejected
(p-value = 0,0075)
t-Test (difference=0)
• Strong significant increase of response rate for
« Activity » (+5 points) and « Investment » (+9
points)
• But also for « Cash » (+1.5 point)
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 7
DG ECFIN
Analyse of the response rates
medium term: 2001-2006
Cumulative fonction: P(number of responses ≤ n)
1.0
Before
0.9
After
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
n max =33
0.0
0
3
6
9 12 15 18 21 24 27 30 33
• For « Activity », 60% of firms respond at least 30
times before obligation and 70% after.
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 8
Behaviour of new respondents
DG ECFIN
• Focus on « Activity » - 3 months before and 3
months after the changeover to mandatory reply
• Difference in difference estimator D
D = (Y1 –Y0)Treatment group – (Y1-Y0)Control group
Y1: output AFTER obligation (e.g. balance, %stable)
Y0: output BEFORE obligation
• Same fixed effects in both groups: same trend in
economic outlook
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 9
Behaviour of new respondents
DG ECFIN
Before
After
none
none
one
two
three
Impossible
Nothing after
Nothing after
Nothing after
one
Nothing before
Same
Worst
Worst
two
Nothing before
Better
Same
Worst
three
Nothing before
Better
Better
Control
• Control group: firms that respond to the 6
surveys (no effect of the change over)
• Treatment groups: firms whose number of
responses changes (eg. Better)
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 10
Behaviour of new respondents
DG ECFIN
%stable Q5
• Y = %stable in question Q5 (expectation)
Control Group
Traitment Groups
Control
(1 731 firms)
Nothing before
(106)
Nothing after
(188)
Worst
(235)
Same
(124)
Better
(238)
Before
(1)
After
(2)
Difference
(2)-(1)
55
56
57
2
nd
nd
0
2
2
38
55
56
53
55
58
55
Double difference
nd
nd
-2
0
0
• No specific behavior of the new
respondents.
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 11
Microeconometric study
DG ECFIN
• Statistical unit: firm
• Balanced samples 2001-2006
• Merge with BRN’s database (fiscal declaration and annual survey)
• Size of samples : 1 500 to 2 000
• Modelling the probability of non response: yit=1 ou 0
• Obligation: 1 if t ≥ 2004
• 3 models:
– Without fixed effects (Logit)
– With fixed effects (Random effects Probit and Conditional Logit)
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 12
3 models
DG ECFIN
Logit:
yit* = Xit.β + Xi.γ + εit
Random Effects Probit:
yit* = Xit.β + Xi.γ + ci + uit
Conditional Logit:
yit* = Xit.β + ci + vit
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 13
Random Effects Probit
DG ECFIN
yit* = Xit.β + Xi.γ + ci + uit
Enquête
Activité
Enquête
Investissement
(N = 95 491)
β
σβ
Constante
-1,41
(0,08)
Délais moyens de règlement client
0,35
(0,09)
Salaire moyen
-0,23
(0,13)
Part des salaires dans la production -0,41
(0,14)
Taux d’endettement
0,08
(0,05)
Taux de marge
-0,03
(0,01)
Nomenclature économique de synthèse
Industrie agroalimentaire
0,14
(0,10)
Industrie de biens de consommation 0,12
(0,07)
Industrie automobile
0,24
(0,15)
Industrie de biens d’équipement
-0,03
(0,07)
Référence
Industrie de biens intermédiaires
Énergie
0,33
(0,53)
Chiffre d’affaires
moins de 15 millions d’€
-0,11
(0,04)
Référence
entre 15M€ et 150M€
150 M€ et plus
-0,02
(0,05)
Effectif
moins de 100 salariés
-0,01
(0,05)
Référence
entre 100 et 500 salariés
500 salariés et plus
0,36
(0,05)
Obligation
-0,34
(0,01)
σc
Olivier Biau
European Commission, Economic and Financial Affairs
0,99
(0,03)
Enquête
Trésorerie(1)
(N = 40 919)
(N = 17 600)
Signif.
β
σβ
Signif.
β
σβ
Signif.
***
***
*
***
*
***
-0,95
0,05
-0,22
-0,27
0,08
-0,04
(0,08)
(0,10)
(0,15)
(0,16)
(0,06)
(0,01)
***
-1,47
0,38
0,00
-0,51
0,34
-0,04
(0,13)
(0,19)
(0,20)
(0,28)
(0,13)
(0,01)
***
**
0,11
0,19
0,24
-0,01
(0,10)
(0,07)
(0,15)
(0,06)
0,15
0,24
0,03
0,05
(0,14)
(0,10)
(0,25)
(0,10)
*
*
***
***
référence
0,45
***
-0,12
-0,10
*
1,37
(0,05)
**
-0,23
-0,04
*
-0,25
référence
***
***
***
0,39
-0,43
0,99
(0,06)
(0,02)
(0,02)
(0,51)
***
(0,09)
***
référence
(0,07)
(0,05)
**
référence
(0,27)
référence
-0,07
*
***
***
(0,10)
(0,09)
***
référence
***
***
***
0,49
-0,09
1,19
(0,09)
(0,03)
(0,04)
***
***
***
ECFIN – 12/10/2009
Slide 14
Conclusions
DG ECFIN
• Significant improvement of the response rate for
« Activity » and « Investment »
• Stronger effect at medium term
• Whatever the model, the obligation increased
the probability of response
• Response rate for optional « Cash » survey also
improved, albeit to a smaller extent
Olivier Biau
European Commission, Economic and Financial Affairs
ECFIN – 12/10/2009
Slide 15