The Response of Ontario Primary Care Physicians to
Pay-for-Performance Incentives
Jeremiah Hurley1,2,3
Jinhu Li1,2,4
1 Department
2 Centre
Gioia Buckley2,3
of Economics, McMaster University
for Health Economics and Policy Analysis, McMaster University
3 Department
4 Melbourne
Phil DeCicca1,2
of Clinical Epidemiology and Biostatistics, McMaster University
Institute of Applied Economic and Social Research, University of Melbourne
September, 2011
Acknowledgments
This research was undertaken as part of the MOHLTC-CHEPA collaborative research
program. We thank Marsha Barnes for initially sponsoring this project and Mary
Fleming for continued support. We are greatly indebted to Delia D’Amore of the
Ontario Ministry of Health and Long-Term Care for sharing her time and expertise
answering our many questions regarding the data and the Ministry’s administrative
processes related to payment policies and regulations. We also thank the following
individuals from the MOHLTC whose assistance has been instrumental at various stages
of this work: Teo Honrado, Mike Collisson, Terry Stevens, Jiahui Wong, Alex Firby,
Kathryn Doresco, James Gonneau, Carol MacIsaac, Barry Monaghan, Patricia Durnford,
Amy Johnston, Mahroo Afrashteh, Alana Salmon and Paul Kaufman. We thank Charles
Sagoe of Cancer Care Ontario for his assistance in helping us obtain data extracted from
CCOs Ontario Breast Screening Program Database. We thank Dr. Leslie Solomon for
his expertise and helpful advice and Andrea Ruskova, Ji Li, Sumangala Shanmuganathan
and Wendy Ho for excellent research assistance.
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
2 / 23
Pay-for-Performance Incentives
In Ontario Primary Care
• Late 1990s: introduced for Primary Care Network (PCN) practices
• 2002: Incorporated into funding model for Family Health Network
(FHN) Practices
• 2004-2008 Agreement: expanded the incentives and the practices
eligible for them
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
3 / 23
Pay-for-Performance Incentives
In Ontario Primary Care
• Late 1990s: introduced for Primary Care Network (PCN) practices
• 2002: Incorporated into funding model for Family Health Network
(FHN) Practices
• 2004-2008 Agreement: expanded the incentives and the practices
eligible for them
Elsewhere
• Pay-forperformance is perhaps one of the most widely advocated
policies of the last decade.
• UK NHS: Quality and Outcomes Framework
• U.S.: private health plans and Medicare program
• Australia: Practice Incentive Progam
• Canada: British Columbia, Manitoba, Ontario, Nova Scotia
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
3 / 23
Evidence Regarding the Effectiveness of P4P
The Overall Literature
• limited empirical literature; much of it methodologically weak
• available evidence is mixed
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
4 / 23
Evidence Regarding the Effectiveness of P4P
The Overall Literature
• limited empirical literature; much of it methodologically weak
• available evidence is mixed
UK NHS Primary Care
• implemented system-wide, so no control group
• only one part of a broader set of quality initiatives
• Campbell et al. (2009)
• interrupted time-series: 1998, 2003, 2005, 2007
• three conditions: asthma, heart disease and type-2 diabetes
• 2005: increased quality for asthma and diabetes; no change for heart
disease
• 2007: increase had leveled off for all three conditions
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
4 / 23
Evidence Regarding the Effectiveness of P4P
US Health Plans
• Rosenthal et al. (2005), Mullen et al. (2010)
• Large network HMO: compare physician groups exposed to incentives
against physician groups not exposed to incentives.
• Rosenthal et al.: cervical cancer screening, mammography and
hemoglobin A1c testing
• evidence of effect only for Pap smear coverage (3.6 percentage points,
or about 10%)
• Mullen et al.: expanded set of services
• positive effect only for Pap smear coverage; possible negative effect for
asthma medication and antibiotic usage
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
5 / 23
Evidence Regarding the Effectiveness of P4P
Systematic Reviews
Giuffrida et al. (1999), Armour et al. (2001), Town et al. (2005),
Peterson et al. (2006), Christianson et al. (2008). Scott et al. (2011)
•
•
•
•
evidence base is relatively weak
evidence is mixed but overall suggests modest effects
outcomes often process measures not necessarily related to quality
expensive per unit of change achieved
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
6 / 23
Our Primary Research Question
Have the performance-based incentive payments for five preventive care
services and for six defined sets of physician services increased the
provision of these services in the target populations?
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
7 / 23
Our Primary Research Question
Have the performance-based incentive payments for five preventive care
services and for six defined sets of physician services increased the
provision of these services in the target populations?
Preventive care services:
•
•
•
•
•
senior flu shots
toddler immunizations
cervical cancer screening
breast cancer screening
colorectal cancer screening
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
7 / 23
Our Primary Research Question
Have the performance-based incentive payments for five preventive care
services and for six defined sets of physician services increased the
provision of these services in the target populations?
Preventive care services:
•
•
•
•
•
senior flu shots
toddler immunizations
cervical cancer screening
breast cancer screening
colorectal cancer screening
Six sets of physician services:
•
•
•
•
•
•
obstetrical services
hospital services
palliative care services
office procedures
prenatal care
home visits
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
7 / 23
The Ontario P4P Incentives in Our Study
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
8 / 23
Our Study
• Design: before-after study with concurrent comparison group
(difference-in-differences)
• Treatment group: physicians in PCRs exposed to the incentives
• Comparison group: physicians who remained in FFS practice (who were
never exposed to the incentives)
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
9 / 23
Our Study
• Design: before-after study with concurrent comparison group
(difference-in-differences)
• Treatment group: physicians in PCRs exposed to the incentives
• Comparison group: physicians who remained in FFS practice (who were
never exposed to the incentives)
• Unit of analysis: individual physician practice
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
9 / 23
Our Study
• Design: before-after study with concurrent comparison group
(difference-in-differences)
• Treatment group: physicians in PCRs exposed to the incentives
• Comparison group: physicians who remained in FFS practice (who were
never exposed to the incentives)
• Unit of analysis: individual physician practice
• Data: 1996-2008: OHIP claims data, Registered Persons Database,
Corporate Provider Database, Client Agency Program Enrolment,
Breast Cancer Screening Database
• enabled us to define each physician’s practice population as of March 31
of each year.
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
9 / 23
Our Study
• Design: before-after study with concurrent comparison group
(difference-in-differences)
• Treatment group: physicians in PCRs exposed to the incentives
• Comparison group: physicians who remained in FFS practice (who were
never exposed to the incentives)
• Unit of analysis: individual physician practice
• Data: 1996-2008: OHIP claims data, Registered Persons Database,
Corporate Provider Database, Client Agency Program Enrolment,
Breast Cancer Screening Database
• enabled us to define each physician’s practice population as of March 31
of each year.
• Features of our design
• assumes that trend for FFS MDs represents well what would have
happened without the incentives for those in the PCR practices
• control for selected physician/practice characteristics
• exploit variation in incentives across PCR groups
• sub-groups analysis: baseline level of provision, physician age, practice
size
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
9 / 23
Study Sample
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
10 / 23
Variable Definitions
Dependent Variables
Preventive Care Bonuses: proportion of a physician’s target practice
population as of March 31 each year who had received the service in
question during the relevant period prior to that March 31.
Special Payments: dichotomous variable taking a value of 1 if a physician’s
service provision met the criteria for the special payment of interest and 0
otherwise.
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
11 / 23
Variable Definitions
Dependent Variables
Preventive Care Bonuses: proportion of a physician’s target practice
population as of March 31 each year who had received the service in
question during the relevant period prior to that March 31.
Special Payments: dichotomous variable taking a value of 1 if a physician’s
service provision met the criteria for the special payment of interest and 0
otherwise.
Independent Variables
Physician Characteristics: age, sex, years in practice
Practice Characteristics: location (urban/rural), size of practice population,
age-sex distribution of patient population, activity levels (e.g., days worked
per year, number of visits, intensity of visits per day), and practice model
(FFS, FHN, FHG, CCM, FHO)
Treatment Dummies: treatment/control group, pre/post policy, relevant
interactions
Other : year dummies, census division
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
11 / 23
Descriptive Statistics: Distribution Across Practice Types
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
12 / 23
Eligibility for Bonuses
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
13 / 23
Descriptive Statistics: Sample
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
14 / 23
Response to Contact Incentive Payment
Proportion of Eligible Family Physicians Who Submitted at Least One
Claim for the Contact Incentive Payment
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
15 / 23
Response to Preventive Care Bonuses
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
16 / 23
Response to Preventive Care Bonuses
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
17 / 23
Response to Preventive Care Bonuses
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
18 / 23
Response to Special Payments
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
19 / 23
Sub-group Analysis: Preventive Care Bonuses
Physician Age
• Pap Smear, Mammogram, Colorectal Cancer: younger physicians
respond more than older
Size of Practice Population
• Senior Flu Shot, Mammogram, Colorectal Cancer: weak gradient
whereby those with larger practice populations respond more than
those with small practice populations
Baseline Level of Provision
• Senior Flu Shot, Mammogram, Colorectal Cancer: those in the low
and middle quartiles of baseline provision responded more than those
in the top quartile
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
20 / 23
Sensitivity Analysis: Preventive Care Bonuses
The findings are robust to:
• exclusion of shadow-billed tracking Q-codes among PCR physicians
• change in practice definition of PCR physicians to include both
rostered and non-rostered patients
• exclusion of CCM or FHG PCR physicians who did not meet
minimum roster size of 450 or 650
• dropping the first year of eligibility for bonuses
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
21 / 23
Conclusion and Implications
• No evidence of any response to the special payments for obstetrical
services, hospital services, palliative care, office procedures, prenatal
care, or home visits
• Evidence of modest response to 4 of the 5 Preventive Care Bonuses
• largest responses for Colorectal Cancer Screening (FOBT) (increase in
coverage from 15% to 23.5%) and Pap Smear (58.9% to 63%)
• small response for mammogram (64.6% to 66.4%) and senior flu shot
(55.4% to 58.2%)
• no response for toddler immunization
• The pattern of responses across incentives likely reflects:
• differential costs of responding
• strength of evidence regarding the effectiveness and appropriateness of
the services
• Overall, the findings suggest that, for these services at least, the
pay-for-performance incentives do not represent a good financial
investment
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
22 / 23
References
Armour B, Pitts M, Maclean R, et al. “The Effect of Explicit Financial Incentives on
Physician Behavior.” Archives of Internal Medicine 2001;161(10):1261-6.
Campbell SM, Reeves D, Kontopantelis E, et al. “Effects of Pay for Performance on the
Quality of Primary Care in England.” The New England Journal of Medicine
2009;361(4):368-78.
Christianson JB, Leatherman S, Sutherland K. “Lessons From Evaluations of Purchaser
Pay-for-Performance Programs: A Review of the Evidence.” Medical Care Research &
Review 2008;65(6-suppl):5S-35.
Giuffrida A, Gosden T, Forland F, et al. “Target payments in primary care: effects on
professional practice and health care outcomes.” Cochrane Database of Systematic
Reviews 1999(4).
Mullen KJ, Frank RG, Rosenthal MB. “Can you get what you pay for?
Pay-for-performance and the quality of healthcare providers.” The RAND Journal of
Economics 2010;41(1):64-91.
Petersen LA, Woodard LD, Urech T, et al. “Does pay-for-performance improve the
quality of health care?” Annals of Internal Medicine 2006;145 (4):265-72.
Rosenthal M, Frank R, Li Z, et al. “Early Experience with Pay for Performance: From
Concept to Practice.” Journal of the American Medical Association
2005;294(14):1788-93.
Town R, Kane R, Johnson P, et al. “Economic Incentives and Physicians’ Delivery of
Preventive Care.” American Journal of Preventive Medicine 2005;28(2):234-40.
Hurley et al. (Centre for Health Economics and Policy Analysis, McMaster University)
23 / 23