Cost-effectiveness analysis of
capitation vs. fee-for-service for
Medicaid patients with severe
mental illness
Richard Grieve
Visiting scholar UC Berkeley
London School of Hygiene and Tropical Medicine
Joint work-Acknowledgments
• Jasjeet Sekhon
– Dept Political Science, UC Berkeley
• Tei-wei Hu, Joan Bloom
– School of Public Health UC Berkeley
Content of talk
• Applied study
–
–
–
–
Context
Methods
Results
Discussion
• Key Methodological issue
Context to the study
• Worldwide concerns about cost of health care
• How best to deploy scare resources to improve
population health?
• Cost-effectiveness analysis (CEA) technique for
guiding resource use towards efficient allocation
(Gold et al, 1997)
• Some governments routinely use CEA to set priorities
(UK, Australia, Canada)
• Its use is controversial, and its uptake variable
(Sheldon et al 2002)
Context to the study
• Lack of explicit use of CEA in US (Neuman et al
2006)
• Concerns about rationing of resources (Aaron et al
2005)
• Historically concern about methodological standards
(Prosser et al 1996, Gold et al 1997)
– How to value outcomes (QALYs)
– How to represent uncertainty
• Published evaluations tend to report costs and
outcomes separately, rather than present full CEA
Context: Case study
• FFS vs capitation for Medicaid cases with severe
mental illness
• Studies found capitation associated with lower costs
– e.g. Bloom et al 2002; Lurie et al 1992
– Clinical outcomes similar (Cuffel et al 2002)
• Studies report costs and outcomes separately
• Unclear whether capitation is more cost-effective?
• Specific Aim of this study: to demonstrate use of CEA
for assessing relative cost-effectiveness
• Illustrate by conducting CEA of capitation vs FFS for
Medicaid cases with severe mental illness
Methods:
case study overview
•
•
•
•
•
•
see Bloom et al (2002)
Legislation passed in Colorado in 1995
Medicaid services for patients with mental illness
Counties divided into 3 groups
– Group 1:FFS
– Group 2: Direct Capitation (DC)
– Group 3: Managed care Behavioral Organizations
(MBHO)
Observational study focused on severe mental illness
Compared costs and outcomes across 3 groups
Before and after introduction of capitation
Methods
definition of ‘interventions’
• FFS: providers reimbursed retrospectively
• DC: Community mental health services (CMHC)
contract with state to organize/provide services
– Not for profit
• MBHO
– Joint venture between for profit private firm
commissioning services
– CMHCs provided some services
• State regulation/ audit
• Contracts every 2 yrs
Methods
selection of groups
• In population
– CMHCs in each county bid for capitation contract
– State selected those bids perceived to be ready
• In Evaluation
– those counties most comparable
– A stratified random sample of cases
– Data for analysis for 522 cases across 3 groups
Methods
cost and outcome measurement
Group 1: FFS
Group 2: DC
Group 3: MBHO
Baseline costs and health status Baseline costs and health status Baseline costs and health status
Group 1: FFS
Post 1: costs and health status
Group 2: DC
Post 1: costs and health status
Group 3: MBHO
Post 1: costs and health status
Group 1: FFS
Post 2: costs and health status
Group 2: DC
Post 2: costs and health status
Group 3: MBHO
Post 2: costs and health status
Methods
cost and outcome measurement
• Costs and outcomes measured over 3 nine
month periods
• Cost measurement
– Medicaid costs claims data, shadow billing
– Considered substitution of health services
• Outcome measurement
– Short form 36 (SF-36); global functioning (GAF)
• Outcome valuation
– Brazier et al (2002)
• QALY calculation: utility score* life years
Methods
Analytical strategy
• Baseline differences in casemix, cost and health status
• Used non-parametric method to match cases
• Genetic matching algorithm (Diamond and Sekhon
2006, Sekhon 2006, Mebane and Sekhon 1998)
• Applied Multivariate matching across 3 groups using
– Previous costs; baseline costs, baseline outcomes, casemix
Analytical strategy
• Reported incremental QALYs and costs separately
• Reported incremental cost-effectiveness
– Reported incremental net benefits (INB)
• INB (A vs B)=λ(ΔE)-ΔTC
– λ-societal willingness to pay health gain
– ΔEi and ΔTCi mean difference in effects and costs
• if INB>0 then ‘accept’ A in preference to B
• Test whether conclusions vary according to λ
– Plot cost-effectiveness acceptability curves
– Probability ‘intervention’ is cost-effective
Results: baseline imbalance
Mean costs ($) FFS vs MBHO
before and after matching (9 month period)
FFS
MBHO
p value
(KS test)
Before matching
4820
6822
0.04
After matching
4820
4581
0.42
For FFS n=151 before and after matching,
MBHO=195 before and n=151 after matching
KS test: Bootstrap Kolomogorov-Smirnov test
Results:
Mean utility and mean QALYs
FFS
DC
MBHO
Pre
0.63
0.63
0.63
Post 1
0.64
0.62
0.64
Post 2
0.63
0.61
0.65
QALYs (18 months)
0.934
0.919
0.954
QALYS: FFS vs DC p=0.48; FFS vs MBHO p=0.30
Results
utilization % any service over each 9 month period
FFS
DC
MBHO
Pre
89.4
89.4
89.4
Post 1
88.8
83.4
77.6
Post 2
83.6
78.8
71.0
Results
Mean Cost ($) per user over each 9 month period
FFS
DC
MBHO
Pre
5374
5375
5095
Post 1
4856
7116
3837
Post 2
4777
9002
4714
Results
Mean Cost per case ($) over each 9 month period
FFS
DC
MBHO
Pre
4808
4805
4581
Post 1
4313
5938
2979
Post 2
3991
7094
3349
costs: FFS vs DC p=0.06; FFS vs MBHO p=0.32
‘conclusions’ from costconsequence
•
•
•
•
•
DC significant increase in cost vs FFS
MBHO non significant decrease vs FFS
No significant outcomes differences
Can we make use of evidence?
CEA
Cost-effectiveness analysis
(over 18 months)
means (95% CI)*
DC-FFS
MBHO-FFS
Incremental cost
4729(-70 to 9596)
-1976(-5929 to 1808)
Incremental
QALY
Incremental net
benefit
-0.016(-0.065 to 0.027)
0.020(-0.015 to 0.059)
-5497(-10591 to -200)
2959(-1203 to 7250)
λ =$50,000 per QALY gained; * bootstrapped bias corrected version
Role of cost-effectiveness analysis
DC vs FFS
MBHO vs FFS
Cost-effectiveness acceptability curves
1.00
0.90
probability cost-effective
p=0.92
0.80
0.70
MBHO vs DC
MBHO vs FFS
DC vs FFS
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
Preliminary conclusions from CEA
• Cost effectiveness analysis useful in extending
traditional cost-consequence approach
• Allows for potential tradeoffs between costs and
outcomes
• In this case study/context CEA found that:
– MBHO model more cost-effective than FFS
– DC model less cost-effective than FFS (or MBHO)
• Observational study but results based on appropriate
methods adjusting imbalance
• Sensitivity analysis applied 2 part model to adjust for
any outstanding differences: findings unchanged
Why the difference?
• MBHO model- for profit stronger incentive
to cost minimise
• Interviews (Bloom et al 2000) suggested MBHO
emphasised maintaining access but reducing costs
per user
• DC areas had less targeted utilisation review
• Different capitation models targeted different
patient groups
CEACS MBHO vs FFS; by diagnosis
1.00
0.90
probability cost-effective
0.80
0.70
MBHO vs FFS-schizophrenia
0.60
MBHO vs FFS- bipolar
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
CEACS DC vs FFS; by diagnosis
1.00
0.90
probability cost-effective
0.80
0.70
DC vs FFS-bipolar
0.60
DC vs FFS schizophrenia
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
Conclusions in context of other
findings
• Caveats
–
–
–
–
small nos, Short followup
selection centres
Only sub sample severe mental illness
limited to morbidity costs
• Other analyses found capitation associated with
cost reductions without selection (Coffman et al
2006, wallace et al 2006)
• Addition to literature suggesting form of
capitation matters
Methodological extensions
use of appropriate methods correcting for
imbalance
• Lack of use of appropriate matching methods in
health services research
• Deeks et al (2003) highly critical of current methods
of bias adjustment in observational studies
• Seriously limits their use in policy making
• Economic evaluations often dependant on data from
observational studies
• Rely on ‘traditional’ methods of casemix adjustment
• Recent interest in use of propensity score methods in
cost-effectiveness analysis (Mitra et al 2005)
• More work in other areas e.g. labour economics
Methods to adjust for baseline
differences
• Genetic matching
• Improves on alternative methods such as multiple
linear regression and propensity scores
• More efficient even when propensity score is known
• When propensity score is unknown
• Genetic matching minimizes bias even where
– distribution of baseline measures are skewed,
– covariates have non linear relationship with
outcomes
A more general solution
Genetic matching (GM) algorithm
Diamond and Sekhon (2006)
• Uses search algorithm (Mebane and Sekhon 1998)
• On basis of stringent non-parametric tests of balance
searches for ‘best’ match between treatment and
controls across baseline covariates
• Previous work demonstrated that when applied to
observational data can replicate the results of RCTs
• Software now available- Sekhon 2006
Baseline cost imbalance for FFS vs MBHO comparison:
pre and post matching
FFS
MBHO
p value
(KS test)
4820
75000
10111
6822
95000
10552
0.04
4820
75000
10111
4558
95550
10427
0.42
Before matching
Mean costs
Max costs
SD
After matching
Mean costs
Max costs
SD
Comparison of method for
adjusting for baseline imbalances
• Compare cost-effectiveness estimates
– from unmatched data with parametric model
adjusts using mean differences (1)
– from unmatched data with parametric model
adjusts using linear adjustment (2)
– From matched data with parametric model
adjusts using linear adjustment (3)
• Parametric model uses 2 stage and log transform
CEACs with different approaches to adjusting for imbalance DC vs FFS
1.00
0.90
probability cost-effective
0.80
No matching parametric model mean adjustment
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
CEACs with different approaches to adjusting for imbalance DC vs FFS
1.00
probability cost-effective
0.90
0.80
No matching parametric model mean adjustment
0.70
No matching linear parametric model
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
CEACs with different approaches to adjusting for imbalance DC vs FFS
1.00
No matching parametric model mean adjustment
0.90
probability cost-effective
0.80
No matching linear parametric model
0.70
matching; linear parametric model
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
25000
50000
75000
100000
125000
value of ceiling ratio (Rc)
150000
175000
200000
Preliminary conclusion for
methodological section
• unbiased estimates of cost-effectiveness important
• Choice of method can make a difference
• Inappropriate method of adjustment overstated
probability intervention was cost-effective
• Genetic matching preferable does not rely on
assumptions routinely violated
• Skewed cost data, non linear relationships
• Method works well even for this smallish case study
because of baseline data on costs, outcomes and
casemix
• Further applications are required