Motivation
The CV principle
Results
Discussion & Conclusions
Statistical estimates of the costs of disability:
how robust can they be?
Steve Pudney1
joint work with
Ruth Hancock1,2
1 Institute
Marcello Morciano1,2
for Social and Economic Research, University of Essex
2 Health
Economics Group, University of East Anglia
UCL seminar, June 2014
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Outline
1
Motivation
2
The CV principle
Parametric modelling
Nonparametric estimation
3
Results
Data
Estimates
4
Discussion & Conclusions
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Disability and income distribution
The 2010 State of the Nation report (Cabinet Office 2010, p. 38) stated:
“over one in five of DLA claimants are in the top two income quintiles”
This gives a misleading impression of the targeting of the disability
benefit DLA, because:
post-benefit income was used to construct quintiles
no allowance was made for costs associated with disability
But how to measure disability costs?
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Quantify the costs associated with disability
Pragmatic approach: e.g. scales based on benefit system
design (Hancock & Pudney 2014)
Direct Approach: ask disabled people/“experts” to identify
costs faced as a direct consequence of their disability (Martin
and White, 1988; Thompson et al., 1990)
Indirect Approaches: the compensating variation (CV)
Intensive margin (Engel curve) approach (Jones & O’Donnell
1995)
Extensive margin (Deprivation) approach (Berthoud et al 1993;
Zaidi and Burchardt 2005; Morciano et al 2014)
Subjective meeasures (Groot & Maassen van den Brink 2004,
2006; Stewart 2009)
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
The Compensating Variation
S
D=0
D= 1
s
Y
Y
Y+∆
S = welfare concept; D = disability; Y = income
(X = observed characteristics; U = unobserved heterogeneity)
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
Parametric modelling - e.g. linear regression
Survey measures: S = wellbeing; Y = income; D = disability
Model:
S = αY − β D + U
The cost of disability is a ratio CV = β × D/α
Standard assumptions:
U = random unobserved influences on wellbeing
Possible examples of U:
location (e.g. transport costs)
budgeting ability
Income Y is exogenous: cov (Y ,U) = 0
Disability D is exogenous: cov (D,U)
Assumptions ⇒ unbiased estimates of α and β ⇒ unbiased CV
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
Problems, e.g. endogeneity
Low transport costs (high U) reduces incentive to claim benefit
⇒ cov (U,Y ) < 0
Budgeting ability (high U) is impaired by disability (e.g.
cognitive impairment)
⇒ cov (U,D) < 0
Both problems cause bias:
α is pushed towards 0
CV = β D/α is pushed upwards
if α gets close to 0, the estimated CV ‘explodes’
Standard ‘solutions’ (e.g. instrumental variables) aren’t usually
credible
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
Misspecification bias: endogenenous income
S
D=0
D=
D= 11
CV
Y
Bias
cov (Y ,U) < 0 ⇒ slope under-estimated ⇒ CV over-estimated
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
Misspecification bias: endogenenous disability
S
D=0
D=
D= 11
CV
Y
Bias
cov (D,U) < 0 ⇒ welfare loss over-estimated ⇒ CV over-estimated
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
An alternative direct matching method
Procedure:
for each sampled disabled person with disability D, income Y and living
standards S:
find a matched non-disabled person with same living standards (and other
personal characteristics)
estimate the average disability cost (CV) as the mean difference in
income between disabled and matched non-disabled comparator
Method is nonparametric since we don’t specify an explicit wellbeing
model
The matching approach also has endogeneity bias since we can’t include
the unobservable U in the match variables
But is it more robust?
⇒
How do the biases in the regression method and the matching method
behave as we vary the seriousness of the endogeneity problem?
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Parametric modelling
Nonparametric estimation
Example: linear welfare model, true CV = £90 p.w.
Analytical asymptotic bias of parametric & nonparametric methods:
(i) Parametric
(ii) Nonparametric
mean bias over grid £222
RMSE £394
mean bias over grid £24.7
RMSE £44.7
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Data
Estimates
Family Resources Survey (FRS)
Annual income survey of 25,000 households
Produced by the Department for Work and Pensions in UK;
Sample selection: All 1 or 2-person households with members
over state pension age in Great Britain
2004/5 - 2007/8 Financial Years (approx 34,000 individuals).
Collects detailed information on:
income components;
difficulties with ordinary activities of life;
Household deprivation (“don’t have & can’t afford” or “don’t
have” a set of ‘necessities’).
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Data
Estimates
Health, disability & cost
Health
Biomarkers:
BP, cortisol,
etc
Disorders
Limitations
Cost
Diagnosed
conditions
Difficulties
with activities
Living
standards
External
support
2004/5 - 2007/8 Family Resources Survey
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Data
Estimates
Disability & living standards indicators
Difficulty with... mobility (moving about)
...lifting, carrying etc
...manual dexterity
...continence
...communication
...memory/concentration etc
...recognising physical danger
...physical co-ordination
...other
% giving 1 or more indications
% giving 2 or more indications
Enough money ...to keep home in a decent state?
...to have household insurance?
...to save £10 a month?
...to replace worn out furniture?
...to replace electrical goods?
...to spend yourself?
% households with 1 or more
% households with 2 or more
Pudney et al.
Lone
female
38.3
36.5
15.1
7.2
9.4
6.0
1.5
12.9
12.3
52.4
37.9
88.5
93.3
75.3
80.5
86.1
91.0
36.6
22.5
Lone
male
35.3
30.7
11.2
8.7
9.8
6.8
1.2
11.7
14.0
52.2
34.5
89.9
90.9
78.9
84.8
87.6
94.6
30.8
18.6
Estimating Disability Costs
Couple
female
male
25.3
28.6
23.4
26.6
10.2
9.7
5.0
7.4
5.6
9.4
3.8
6.2
1.1
1.4
7.4
9.0
9.0
10.7
37.6
43.8
24.2
28.9
94.2
96.8
80.6
88.3
92.9
92.6
26.6
14.0
Motivation
The CV principle
Results
Discussion & Conclusions
Data
Estimates
Estimated disability-cost profile
Figure 6
Estimated CV by disability level: weighted disability index
(90% bootstrap confidence intervals: 500 replications)
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Data
Estimates
Overall mean costs
Table 2
Estimated overall mean disability costs
Mahalanobis1
Propensity score1
κw = 4
κw = ∞
κw = 0.005
κw = 0.01
κw = 0.02
CV
£55.36
£54.88
£57.25
£57.31
£57.52
(3.47)
(3.50)
(3.10)
(3.10)
(3.10)
Std err
Support % 2
95.5
100.0
97.8
99.0
99.4
Parametric estimates (standard error)
Linear regression (continuous living standards index) 3
£515.29 (44.65)
Log regression (continuous living standards index) 4
£311.97 (26.32)
Linear ordered probit (count index of living standards) 3
£167.41 (8.45)
Log ordered probit (count index of living standards) 4
£228.87 (15.31)
κw = 2
£55.99
(3.07)
92.2
1
Nearest-neighbour matching, with replacement. 2 Proportion of “treated” sample retained after applying
caliper and common support. 3 Constructed as −b̂d D̄/b̂y , where by , bd are coefficients in a linear
ordered probit model of S on Y , D, X and age 2 , and D̄ is the mean among those reporting disability.
4
Constructed as the mean of Y [exp(−bd D/by ) − 1], where by , bd are coefficients in the logarithmic model.
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Next steps
There is a credibility crisis for CV-type estimates of disability costs
New project (joint with NatCen):
Use parametric & non-parametric methods to predict disability
costs for individuals in UKHLS sample
Select individuls for in-depth interview
Use quali-style interviewing to establish a plausible range for
individual costs
(NB challenge is to design interviews to match the CV
definition which allows for adaptation to disability)
3 approaches = genuine triangulation!
Pudney et al.
Estimating Disability Costs
Motivation
The CV principle
Results
Discussion & Conclusions
Conclusions
The CV is an important contribution to design of health/disability policy
Parametric models are dangerously sensitive to quite small deviations
from ideal assumptions
Nonparametric matching methods may be more flexible and rest on
exogeneity assumptions which are (arguably) no more objectionable
CV estimates are large: mean of £55-60 p.w. (nonparametric) or
£160-520 (parametric)
Nonparametric CV estimates are likely to be conservative
Disability costs are much larger than public support
Next step? In-depth interview follow-ups to investigate predicted
disability costs for individual cases
Pudney et al.
Estimating Disability Costs