HHS Public Access
Author manuscript
Author Manuscript
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Published in final edited form as:
Policy Polit Nurs Pract. 2021 November ; 22(4): 245–252. doi:10.1177/15271544211053854.
The Association of Hospital Magnet® Status and Pay-forPerformance Penalties
Andrew M. Dierkes, PhD, RN1, Kathryn Riman, PhD, RN2, Marguerite Daus, PhD, RN3,
Hayley D. Germack, PhD, MHS, RN1, Karen B. Lasater, PhD, RN, FAAN4
1Department
of Acute and Tertiary Care, University of Pittsburgh School of Nursing, Pittsburgh,
PA, USA
Author Manuscript
2Department
of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh,
PA, USA
3Denver-Seattle
Center of Innovation for Veteran-Centered and Value-Driven Care (COIN),
Eastern Colorado Health Care System, Aurora, CO, USA
4Center
for Health Outcomes and Policy Research, University of Pennsylvania School of Nursing,
Philadelphia, PA, USA
Abstract
Author Manuscript
Author Manuscript
The Centers for Medicare and Medicaid Services’ Pay-for-Performance (P4P) programs aim to
improve hospital care through financial incentives for care quality and patient outcomes. Magnet®
recognition—a potential pathway for improving nurse work environments—is associated with
better patient outcomes and P4P program scores, but whether these indicators of higher quality
are substantial enough to avoid penalties and thereby impact hospital reimbursements is unknown.
This cross-sectional study used a national sample of 2,860 hospitals to examine the relationship
between hospital Magnet® status and P4P penalties under P4P programs: Hospital Readmission
Reduction Program, Hospital-Acquired Conditions (HAC) Reduction Program, Hospital ValueBased Purchasing (VBP) Program. Magnet® hospitals were matched 1:1 with non-Magnet
hospitals accounting for 13 organizational characteristics including hospital size and location.
Post-match logistic regression models were used to compute a hospital’s odds of penalties. In
a national sample of hospitals, 77% of hospitals experienced P4P penalties. Magnet® hospitals
were less likely to be penalized in the VBP program compared to their matched non-Magnet
counterparts (40% vs. 48%). Magnet® status was associated with 30% lower odds of VBP
penalties relative to non-Magnet hospitals. Lower P4P program penalties is one benefit associated
with achieving Magnet® status or otherwise maintaining high-quality nurse work environments.
Corresponding Author: Andrew M. Dierkes, Department of Acute and Tertiary Care, University of Pittsburgh School of Nursing,
3500 Victoria Street, Pittsburgh, PA 15261, USA. amd323@pitt.edu.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Dierkes et al.
Page 2
Author Manuscript
Keywords
reimbursement; incentive; value-based purchasing; medicare; quality of health care; crosssectional studies; hospitals
Author Manuscript
The Centers for Medicare and Medicaid Services’ (CMS) Pay-for-Performance (P4P)
programs aim to improve health care through financial incentives to hospitals for care
quality and patient outcomes. Three P4P programs, in particular, have hospitals compete on
various performance metrics with up to 6% of total hospital CMS reimbursement at stake:
(a) Hospital Readmission Reduction Program (HRRP), (b) Hospital-Acquired Conditions
(HAC) Reduction Program, and (c) Hospital Value-Based Purchasing (VBP) Program
(Centers for Medicare & Medicaid Services, 2020a, 2020b, 2020c). These programs are
mandated for hospitals paid under the inpatient prospective payment system. Depending on
their performance relative to peer institutions, hospitals may receive financial penalties. The
VBP program differs from the HRRP and HAC Reduction Program in that it also extends
upside potential to hospitals. High performing hospitals may receive more than the amount
withheld under VBP, in effect awarding a financial bonus. Existing research documents that
hospitals with better nursing resources, such as better patient-to-nurse staffing ratios, better
nurse work environments, and Magnet® recognition perform better on P4P program quality
metrics (Lasater et al., 2016). Patients in hospitals with these superior nursing resources
are less likely to experience readmissions (Lasater & McHugh, 2016; McHugh et al., 2013;
McHugh & Ma, 2013) and hospital-acquired conditions (Barnes et al., 2016; Shin et al.,
2019), and are more likely to be satisfied with their care (Martsolf et al., 2016; Stimpfel et
al., 2016).
Author Manuscript
The nurse work environment reflects a healthcare organization’s capacity to support nurses’
professional practice, improve job satisfaction, and champion patient safety (Lake, 2002).
One established pathway to improve a hospital’s nurse work environment is achieving
Magnet® status (Aiken et al., 2008; Ulrich et al., 2007). Although the existing evidence
has linked better nursing resources with superior performance (i.e., higher scores) on P4P
program quality metrics, little is known about whether these better P4P program scores
affect financial outcomes.
P4P Programs
Author Manuscript
CMS is the largest funder of health care and a portion of CMS hospital reimbursement
is contingent upon performance in the areas assessed by P4P programs, including but not
limited to the three programs evaluated in this study (Centers for Medicare & Medicaid
Services, 2020a, 2020b, 2020c). The HRRP program monitors unplanned readmissions
for selected patients and hospitals with excess readmissions can lose up to 3% of CMS
reimbursement (Centers for Medicare & Medicaid Services, 2020a). The HAC program
tracks five health care-associated infections; poor performance could cost hospitals an
additional 1% in forgone reimbursements (Centers for Medicare & Medicaid Services,
2020b). VBP includes four quality domains (Clinical Care, Person and Community
Engagement, Safety, and Efficiency and Cost Reduction) and is comprised, in part, of
the eight dimensions of the Hospital Consumer Assessment of Healthcare Providers
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 3
Author Manuscript
and Systems patient experience survey (communication with nurses, communication with
doctors, the responsiveness of hospital staff, communication about medicines, cleanliness
and quietness of the hospital environment, discharge information, overall rating of the
hospital, and care transition) (Centers for Medicare & Medicaid Services, 2020c). The
VBP program withholds 2% of reimbursements and redistributes those funds based on
performance (Centers for Medicare & Medicaid Services, 2020c).
Magnet® Recognition and P4P Outcomes
Author Manuscript
The American Nurses Credentialing Center (ANCC) established the Magnet® Recognition
Program in 1994 to accredit hospitals that demonstrate nursing excellence and favorable
patient outcomes (American Nurses Credentialing Center, n.d.-a). Magnet® status is not just
a recognition of nursing excellence and quality of care but a process to improve hospital
nurse work environments and patient outcomes (Kutney-Lee et al., 2015). A longitudinal
analysis of 136 hospitals between 1999 and 2007 found that achieving Magnet® status is
associated with improvements in hospital work environments and lower odds of 30-day
mortality and failure-to-rescue among surgical patients (Kutney-Lee et al., 2015). As of July
2021, 561 hospitals were Magnet® recognized—roughly 9% of all hospitals in the United
States (American Nurses Credentialing Center, n.d.-a, n.d.-b).
Author Manuscript
Although many studies have evaluated the association of nursing resources such as Magnet®
status and the work environment on the patient outcomes included in Medicare’s P4P
programs, little is known about associated financial implications in terms of whether these
performance improvements translate to penalties avoided. McHugh et al. (2013) found that
hospitals with better nurse staffing had 25% lower odds of penalties under HRRP compared
to those with poorer staffing levels. Lasater et al. (2016) documented that Magnet® status
predicted higher scores on Total Performance (an indicator of VBP performance) and
two VBP domains ([a] Clinical Processes and [b] Patient Experience). Whether nursing
resources are associated with penalties under the HAC program is unknown. Likewise,
the association of nursing resources with overall P4P performance—the overall financial
effect across the three programs—has not been empirically evaluated. A perceived barrier
to achieving Magnet® status is the cost of meeting and maintaining accreditation standards
(Pinkerton, 2005). Whether Magnet® status is associated with hospitals avoiding penalties
is an important, but missing, piece of information for administrators weighing the value
and feasibility of investing in hospital nursing. This study evaluated whether a hospital’s
Magnet® status is associated with individual P4P program penalties and overall effect
accounting for financial penalties and bonuses across three programs.
Author Manuscript
Methods
Data Sources and Sample
This was a cross-sectional analysis of a national sample of hospitals using data sets from
2015 to 2017, including (a) the American Hospital Association (AHA) annual survey, (b)
CMS data made publicly available by Advisory Board (National P4P Map), (c) CMS Final
Rule Impact File, and (d) data on hospital Magnet® status obtained from the ANCC website.
The AHA Annual Survey provided information on nurse staffing and hospital organizational
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 4
Author Manuscript
characteristics. Data from CMS provided information on penalties and additional hospital
organizational characteristics such as case mix index (CMI), disproportionate share, and
wage index. ANCC data provided information on hospital Magnet® status. The final sample
included 2,860 hospitals (2,529 non-Magnet hospitals and 331 Magnet® hospitals).
Measures
Author Manuscript
Our explanatory variable was Magnet® status. Achieving Magnet® recognition is a
multiyear application and approval process and improvements in nursing quality and
associated outcomes may be seen before the status is formally awarded, so we considered
hospitals to be Magnet® if they received recognition as of 2017. Our outcome of interest
was the hospital P4P program and overall penalties. The financial impact within each of
three CMS P4P programs—VBP, HRRP, HAC Reduction Program—and the overall impact
(sum of the financial penalties and/or bonuses) across all programs is referenced as overall
P4P. We dichotomized overall P4P to indicate a positive or negative financial outcome (0
= ≥$0.00, 1 = <$0.00). Penalities are assessed at the individual program level. In order to
arrive at an overall penalty indicator for overall P4P, we used the value of dollars summed
by hospital across programs. Hospitals that received greater than the 2% originally withheld
under VBP were coded as 0 (not a penalty). However, the dollar value of any VBP bonus
counted against penalties accrued under other programs in the analysis of overall P4P
penalty outcomes. If the VBP bonus meets or exceeds the absolute value of any penalties
under both HAC and HRRP combined, overall P4P is coded 0 (not a penalty).
Author Manuscript
We included hospital characteristics in both our matching approach and regression analyses,
similar to prior work in this area (Lasater et al., 2016; McHugh et al., 2013) and consistent
with guidance on including variables in propensity score models (Stuart, 2010) and as
covariates when modeling treatment effects (Nguyen et al., 2017). Continuous variables
included hospital size (number of beds), Herfindahl–Hirschman Index (a measure of market
competition calculated as the sum of the squares of the market share of each hospital
competing), CMI (a measure of patient acuity represented by the average relative diagnosisrelated group weight of a hospital’s inpatient discharges), disproportionate share (a measure
of payments intended to offset costs incurred by hospitals treating a large or disproportionate
number of indigent patients), percent Medicare patients, percent Medicaid patients, wage
index (a ratio of the area’s average hourly wage to the national average hourly wage), the
ratio of medical residents to beds, and nurse staffing (registered nurse hours per adjusted
patient day). Dichotomous and categorical variables included technology status (capacity to
perform organ transplantation or open-heart surgery), ownership (for-profit versus nonprofit
institution), core-based statistical area (a measure of population concentration), and region
(geographic areas of the United States as defined by the U.S. Census Bureau).
Author Manuscript
Analysis
We conducted analyses using matched and unmatched samples. Magnet® hospitals were
matched 1:1 with non-Magnet® hospitals using nearest neighbor propensity score matching
without replacement to identify non-Magnet® hospitals that otherwise had the observable
characteristics associated with Magnet® institutions. These variables were presented in our
discussion of measures and are listed in Supplementary Table 1. A logistic regression
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 5
Author Manuscript
model produced propensity scores representing the probability of Magnet® status based on
these variables. Each Magnet® hospital was paired with the “nearest” available non-Magnet
hospital, where the distance between hospitals is the difference in propensity scores. This
approach approximates a randomized controlled trial in that the matched pairs of hospitals
are similar on all observable characteristics except for the treatment (in this case, Magnet®
recognition). This allows us to attribute an observed effect of the treatment with greater
confidence than a regression analysis of unmatched cross-sectional data (Silber et al., 2001).
Matching improves the homogeneity of all other observable characteristics across Magnet®
and non-Magnet hospitals and helps account for selection bias—that these characteristics
might be associated with a hospital’s decision to pursue Magnet® recognition and therefore
result in systematic differences mirroring Magnet® status.
Author Manuscript
The balance of covariate distribution between Magnet® and non-Magnet hospitals signals
the quality of the match. Standardized mean differences (SMDs) are the difference in
means between groups (Magnet® vs. all non-Magnet hospitals in the prematch analysis
and Magnet® vs. non-Magnet controls in the postmatch analysis) divided by the standard
deviation (SD) of each covariate for Magnet® hospitals. Incorporating the SD into the
calculation normalizes the scale across covariates to aid in comparison. The closer the
resulting SMDs are to zero, the better the match. Consistent with prior research employing
propensity score matching to pair Magnet® and non-Magnet hospitals, we adopted 0.2 as the
threshold for an adequate match (Lasater et al., 2016). Statistical models employed logistic
regression to estimate the odds of penalties associated with Magnet® status. These models,
when adjusted, included as controls the same variables used in the matching approach for a
doubly robust analysis to account for residual imbalance (Nguyen et al., 2017).
Author Manuscript
Results
Author Manuscript
Table 1 presents the descriptive characteristics of the national sample of hospitals, before
and after propensity score matching. Seventy-seven percent of hospitals received overall P4P
penalties, with rates differing by the individual program. Twenty-three percent of hospitals
received penalties under HAC, 46% under VBP, and 84% under HRRP. As indicated by
the “pre-match” SMDs greater in absolute value than 0.2, the 331 Magnet® hospitals as a
group differed from the 2,529 non-Magnet hospitals across all hospital characteristics. After
narrowing the non-Magnet group to 331 matched controls, these differences were greatly
diminished—represented by “post-match” SMDs that were in all cases <0.2 in absolute
value, and in most cases <0.1. The change in these SMDs and the difference between
using all data versus matched data is visually apparent in Figure 1. The penalty rates
between Magnet® and non-Magnet hospitals in this matched cohort varied by program. The
percent of Magnet® and non-Magnet hospitals penalized under overall P4P and HAC were
similar (overall P4P, 76% vs. 78%; HAC, 30% vs. 28%, respectively). More Magnet® than
non-Magnet hospitals were penalized under HRRP (85% vs. 80%). The opposite was true of
VBP in Magnet® versus non-Magnet hospitals (40% vs. 49%).
Table 2 presents the results of the logistic regression analyses evaluating the odds of
penalties associated with Magnet® status after adjusting for hospital characteristics in both
the unmatched and matched samples. Magnet® status was associated with lower odds of
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 6
Author Manuscript
penalties for VBP across all models. Within the fully adjusted models, overall P4P crossed
our threshold for statistical significance in the unmatched data only (odds ratio [OR]: 0.66,
95% confidence interval [CI]: 0.48–0.91, p < .05). In the matched data, while the direction
of effect for both overall P4P and VBP was unchanged from the unmatched analysis, these
associations were statistically significant for VBP penalties only (OR: 0.66, 95% CI: 0.48–
0.92, p < .05). In the final matched, adjusted models, HRRP is notable for its large effect
size (OR: 1.40) in contrast to HAC (OR: 1.02), although neither result was statistically
significant.
Sensitivity Analysis
Author Manuscript
Magnet® status is a signal for good nurse work environments; however, hospitals can
have excellent nurse work environments without holding Magnet® recognition. Therefore,
we conducted a sensitivity analysis using a subset of hospitals in four states (California,
Florida, Pennsylvania, and New Jersey) for which we had detailed data about nurse
work environments to evaluate the robustness of our main findings. Supplementary Table
1 presents a comparison of the primary and sensitivity analyses. Using data from the
2016 RN4CAST-U.S. survey of nurses, we created hospital-level measures of the nurse
work environments derived from frontline nurse informants’ reports using the Practice
Environment Scale of the Nursing Work Index (PES-NWI). (Lake, 2002). Additional details
of the survey methodology are provided elsewhere (Lasater et al., 2019).
Author Manuscript
Author Manuscript
The propensity score method of matching hospitals on Magnet® status, a binary indicator,
cannot be used to match continuous measures, such as PES-NWI. Nonbipartite matching is
designed for matching on nonbinary measures (Lu et al., 2011). We employed nonbipartite
matching to mirror our primary analysis of Magnet® hospitals by pairing hospitals with
higher- and lower-quality work environments, generating two equal groups of hospitals
that were similar on all other observed characteristics. The approach involved several
steps. First, the “treatment” of interest—hospital-level nurse work environment score—was
categorized by quartile in order to differentiate levels of work environment quality. Next, an
ordinal logit model including all hospital covariates produced maximum likelihood estimates
representing the distance between hospitals, much like the propensity scores in our primary
matching method. Finally, these were used to balance covariate distributions across groups.
This approach accomplished two aims: to pair hospitals across groups such that (a) the
distribution of observed covariates was balanced and (b) the difference in exposure to a
better work environment was maximized across all pairs. Within each pair, the hospital with
the higher quality work environment was labeled “high PES-NWI” and the hospital with
the lower quality work environment was labeled “low PES-NWI.” This sensitivity analysis
examined 482 hospitals: 241 in lower quality work environments and 241 in higher quality
work environments. This process was accomplished using the R package “nbpMatching”
developed by Lu et al. (2011).
Supplementary Table 2 presents the descriptive characteristics of the four-state data set and
the results of nonbipartite matching. Although the 482 hospitals in the four-state data had a
higher overall rate of P4P penalization than the national sample (83% vs. 77%), the penalty
rate by program mirrored that of the national sample presented in our primary analysis,
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 7
Author Manuscript
ranging from HAC (28%) to HRRP (85%) (23%–84% in the national sample). The greatest
difference in penalty rates by program within the four-state data between hospitals with
“low” and “high” work environment scores was observed for VBP (15 percentage points).
Differences were statistically significant at the p <.05 level for VBP and overall P4P only.
Hospitals with better work environments had substantially lower rates of penalties across all
individual programs and for overall P4P. As in the primary analysis, the low SMDs (<0.01 in
all cases) signal adequate matching results.
Author Manuscript
Author Manuscript
Supplementary Table 3 presents the results from logistic regression models that are identical
in structure to those in the primary analysis except for the independent variable, where
this sensitivity analysis substituted work environment quality for Magnet® status. The
unmatched models regressed on an ordinal variable derived from the continuous PES-NWI
score by dividing hospitals into work environment quality groups based on quartile: poor
(bottom quartile), mixed (middle two quartiles), and best (top quartile). A unit increase in
work environment quality represents a change from “poor” to “mixed” (or from “mixed”
to “best”) work environments. This is consistent with existing approaches to modeling
PES-NWI in nursing health services research (Lake et al., 2019). The matched models
represent the change in odds of penalties between hospitals with relatively higher versus
lower PES-NWI scores. The results of the sensitivity analysis paralleled the primary analysis
with two key differences. As with Magnet® in the primary analysis, we found an inverse
relationship between the quality of the work environment and VBP, while the HRRP and
HAC programs did not cross our threshold for statistical significance. However, the direction
of effect for HRRP in the final matched, adjusted model was opposite that of the primary
analysis, making it more consistent with the other programs and our hypothesis that better
nurse work environments are associated with lower odds of penalties. In contrast to the
primary analysis, overall P4P in this sensitivity analysis, as with VBP, was significant across
all models at the p < .01 or p < .001 level.
Discussion
Author Manuscript
Our analysis is the first to document that Magnet® hospitals are less likely to be penalized
under the VBP P4P program. We found no significant differences in odds of HRRP
and HAC program penalties or overall P4P penalties between Magnet® and non-Magnet
hospitals in our fully adjusted models using matched data. In a sensitivity analysis, we
found large effects for VBP as an individual program associated with better hospital nurse
work environments. Together, the primary and sensitivity analyses may reveal the work
environment as an underlying factor by which Magnet® hospitals might avoid P4P program
penalties. Magnet® is just one strategy for improving nurse work environments. Although
Magnet® hospitals are known for the quality of their work environments, not all hospitals
with higher quality work environments are Magnet®-recognized (McHugh et al., 2016). For
hospital administrators, this may be actionable evidence to improve the quality of care and
reduce P4P penalties by investing to improve hospital nurse work environments.
The only study to document an association between nursing organizational factors and
penalties under a P4P program found that hospitals with better nurse staffing were less likely
to receive HRRP penalties (McHugh et al., 2013). Our study provides a more complete
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 8
Author Manuscript
Author Manuscript
picture by documenting the odds of penalties for each program separately and the overall
effect across three P4P programs. This is important because avoiding penalties under one
program does not guarantee avoidance in another program. Furthermore, under VBP, the
amount a hospital receives may be less than (a penalty, in this study), equal to, or greater
than (in effect, a bonus) the amount withheld by CMS. As a result, a hospital may incur
a penalty in one of the individual programs that is counteracted by a VBP bonus. Both
Magnet® recognition and better work environments were associated with lower odds of
VBP program penalties across all models. The effects of Magnet status and better work
environments on HRRP and HAC penalties were statistically insignificant across all adjusted
models. Importantly, HRRP and HAC are distinct from VBP in that they specifically focus
on clinical outcomes and aspects of care. It is possible that Magnet® recognition and work
environments have less of a direct impact on specific causes of readmission and infection
rates and more to do with a summative effect. Although we do not have evidence that
penalties under HRRP and HAC were different between Magnet® and non-Magnet hospitals
or between hospitals with relatively high- and low-quality work environments, it is possible
that hospitals with higher quality work environments, such as Magnet® hospitals, are more
likely to receive a bonus under VBP. The combination of significance in VBP and overall
P4P and insignificance in HRRP and HAC in the sensitivity analysis suggest that the impact
of better work environments on VBP may be great enough to counteract the financial impact
of any penalties in these other programs.
Author Manuscript
P4P penalties represent financial losses for hospitals. They are different from but related
to performance scores. Although Lasater et al. (2016) have shown that Magnet® status is
associated with better scores on VBP, because performance is evaluated relative to other
hospitals in the program, better P4P program scores do not always equate to avoided
penalties. Comparable changes in scores across all hospitals may yield the same cohorts
of hospitals penalized under each program. By examining penalties directly instead of the
underlying change in P4P program scores, this study bridges this gap by providing evidence
for Magnet® status associated with penalties avoided. This bolsters the case for Magnet® by
demonstrating evidence of favorable financial impacts associated with such recognition.
Author Manuscript
The effectiveness of pursuing Magnet® recognition or otherwise improving hospital nurse
work environments to improve patient outcomes is well-documented in the literature. There
is evidence to suggest that this association may be causal, including Magnet® status
(Kutney-Lee et al., 2015) and the work environment (Lake et al., 2020; Sloane et al., 2018).
The association of Magnet® status and the quality of hospital work environments with
P4P programs is an important consideration of interest to administrators seeking to justify
greater investments in nursing. Reductions in financial penalties may offset the expense
of pursuing Magnet® recognition or other improvements in the nurse work environment.
This study contributes to this body of literature by recognizing both Magnet® recognition
and improvements in the work environment as innovative management interventions that
leverage the nursing workforce to improve outcomes and lower costs of care.
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 9
Limitations
Author Manuscript
Author Manuscript
This study is not without limitations. However, we have taken steps to mitigate these
weaknesses. For example, cross-sectional data ordinarily limits the interpretation of
results to associations. Our matching approach, which adjusts for potential selection bias
of Magnet® hospitals and associated systematic differences between Magnet® and nonMagnet hospitals, addresses this limitation by significantly improving the comparability
of the groups regarding observed characteristics such that the effect may be attributed
to the “treatment.” In both our primary and sensitivity analyses, this was achieved by
balancing the probability of receiving the “treatment” (i.e., Magnet® recognition or a
better work environment) across the treatment and control groups. It is important to note
that these matching approaches cannot account for unobserved factors, such as quality
improvement initiatives conducted in an organization. These limitations emphasize the need
for longitudinal and interventional studies evaluating the outcomes advantages of Magnet®
hospitals.
Conclusions
This study examined the impact of Magnet® recognition and, in a sensitivity analysis,
higher quality work environments, on the odds of P4P program penalties. We found that
Magnet® hospitals and those with better nurse work environments were less likely to lose a
portion of their CMS reimbursement as a result of poor performance in P4P programs. The
findings provide evidence that investments to improve hospital nursing through the Magnet®
journey or other interventions to improve the hospital work environment may be offset by
subsequent savings in reduced P4P program penalties.
Author Manuscript
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgments
The findings of this study are solely the responsibility of the authors. The authors would like to thank Duy Do and
Morgan Peele for analytic support, Kyle Campbell for assistance in interpreting Advisory Board data and guidance
in choosing covariates, and Malia Meyer for assistance in organizing our response to peer reviewers.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of
this article: This study was supported by grants from the National Institute of Nursing Research (grant numbers:
T32NR007104, R01NR014855, LH Aiken, PI, T32HL007820, Kahn, PI).
Author Manuscript
Biographies
Andrew M. Dierkes is a registered nurse and a health services and policy researcher
studying the nursing workforce at the intersection of cost, quality, and outcomes. He is an
assistant professor at the University of Pittsburgh School of Nursing, Department of Acute
and Tertiary Care.
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 10
Author Manuscript
Kathryn Riman is a practicing staff nurse and health services and policy researcher
interested in critical care nursing, critical care organization and management, and quality
improvement. She is a postdoctoral scholar in the Department of Critical Care Medicine
at the University of Pittsburgh School of Medicine and Treasurer of the AcademyHealth
Interdisciplinary Research Group on Nursing Issues.
Marguerite Daus research is focused on the optimization of healthcare transitions,
specifically from hospital to home, for high-risk patients with social needs with the goal
of improving patient outcomes. She is an advanced HSR&D fellow at the Denver/Seattle
Center of Innovation VA Eastern Colorado Health Care System.
Author Manuscript
Hayley D. Germack is a nurse and health services and policy researcher with a focus
on improving health outcomes for people with mental health disorders and optimizing the
role of the nursing workforce in the delivery of mental health care. She is an assistant
professor at the University of Pittsburgh School of Nursing and chair of the AcademyHealth
Interdisciplinary Research Group on Nursing Issues.
Karen B. Lasater program of research focuses on the impact of nursing on patient
outcomes and measuring the value—in terms of outcomes and costs—of improving hospital
nurse resources. She is an assistant professor at the University of Pennsylvania School
of Nursing in the Center for Health Outcomes and Policy Research and a fellow of the
American Academy of Nursing.
References
Author Manuscript
Author Manuscript
Aiken LH, Buchan J, Ball J, & Rafferty AM (2008). Transformative impact of magnet
designation: England case study. Journal of Clinical Nursing, 17(24), 3330–3337. 10.1111/
j.1365-2702.2008.02640.x [PubMed: 19146592]
American Nurses Credentialing Center. (n.d.-a). About magnet. Retrieved December 16, 2020, from
https://www.nursingworld.org/organizational-programs/magnet/about-magnet/
American Nurses Credentialing Center. (n.d.-b). Find a magnet organization. American Nurse
Credentialing Center.
Barnes H, Rearden J, & McHugh MD (2016). Magnet® hospital recognition linked to lower central
line-associated bloodstream infection rates. Research in Nursing & Health, 39(2), 96–104. 10.1002/
nur.21709 [PubMed: 26809115]
Centers for Medicare & Medicaid Services. (2020a, January 6). Hospital readmissions reduction
program (HRRP). Centers for Medicare & Medicaid Services. https://www.cms.gov/Medicare/
Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program
Centers for Medicare & Medicaid Services. (2020b, January 6). Hospital-Acquired condition (HAC)
reduction program. Centers for Medicare & Medicaid Services. https://www.cms.gov/Medicare/
Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/HAC-Reduction-Program
Centers for Medicare & Medicaid Services. (2020c, January 6). The hospital
value-based purchasing (VBP) program. Centers for Medicare & Medicaid
Services. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/ValueBased-Programs/HVBP/Hospital-Value-Based-Purchasing
Kutney-Lee A, Stimpfel AW, Sloane DM, Cimiotti JP, Quinn LW, & Aiken LH (2015). Changes in
patient and nurse outcomes associated with magnet hospital recognition. Medical Care, 53(6), 550.
10.1097/MLR.0000000000000355 [PubMed: 25906016]
Lake ET (2002). Development of the practice environment scale of the nursing work index. Research
in Nursing & Health, 25(3), 176–188. 10.1002/nur.10032 [PubMed: 12015780]
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 11
Author Manuscript
Author Manuscript
Author Manuscript
Author Manuscript
Lake ET, Riman KA, & Sloane DM (2020). Improved work environments and staffing lead to less
missed nursing care: A panel study. Journal of Nursing Management, 28(8), 2157–2165. 10.1111/
jonm.12970 [PubMed: 32017302]
Lake ET, Sanders J, Duan R, Riman KA, Schoenauer KM, & Chen Y (2019). A meta-analysis of the
associations between the nurse work environment in hospitals and 4 sets of outcomes. Medical
Care, 57(5), 353–361. 10.1097/MLR.0000000000001109 [PubMed: 30908381]
Lasater KB, Germack HD, Small DS,& McHugh MD(2016). Hospitals known for nursing excellence
perform better on value based purchasing measures. Policy, Politics, & Nursing Practice, 17(4),
177–186. 10.1177/1527154417698144
Lasater KB, Jarrín OF, Aiken LH, McHugh MD, Sloane DM, & Smith HL (2019). A methodology for
studying organizational performance: A multistate survey of front-line providers. Medical Care,
57(9), 742. 10.1097/MLR.0000000000001167 [PubMed: 31274782]
Lasater KB, & Mchugh MD (2016). Nurse staffing and the work environment linked to readmissions
among older adults following elective total hip and knee replacement. International Journal for
Quality in Health Care, 28(2), 253–258. 10.1093/intqhc/mzw007 [PubMed: 26843548]
Lu B, Greevy R, Xu X, & Beck C (2011). Optimal nonbipartite matching and its statistical
applications. The American Statistician, 65(1), 21–30. 10.1198/tast.2011.08294 [PubMed:
23175567]
Martsolf GR, Gibson TB, Benevent R, Jiang HJ, Stocks C, Ehrlich ED, Kandrack R, & Auerbach DI
(2016). An examination of hospital nurse staffing and patient experience with care: Differences
between cross-sectional and longitudinal estimates. Health Services Research, 51(6), 2221–2241.
10.1111/1475-6773.12462 [PubMed: 26898946]
McHugh MD, Aiken LH, Eckenhoff ME, & Burns LR (2016). Achieving kaiser permanente quality.
Health Care Management Review, 41(3), 178. 10.1097/HMR.0000000000000070 [PubMed:
26131607]
McHugh MD, Berez J, & Small DS (2013). Hospitals with higher nurse staffing had lower odds
of readmissions penalties than hospitals with lower staffing. Health Affairs, 32(10), 1740–1747.
10.1377/hlthaff.2013.0613 [PubMed: 24101063]
McHugh MD, & Ma C (2013). Hospital nursing and 30-day readmissions among medicare
patients with heart failure, acute myocardial infarction, and pneumonia. The Journal of Nursing
Administration, 43(10 Suppl), S11. 10.1097/MLR.0b013e3182763284 [PubMed: 24022077]
Nguyen T-L, Collins GS, Spence J, Daurès J-P, Devereaux P, Landais P, & Le Manach Y
(2017). Double-adjustment in propensity score matching analysis: Choosing a threshold for
considering residual imbalance. BMC Medical Research Methodology, 17(1), 1–8. 10.1186/
s12874-016-0277-1 [PubMed: 28056835]
Pinkerton SE (2005). The financial return on magnet recognition. The Journal of Continuing Education
in Nursing, 36(2), 51–52. 10.3928/0022-0124-20050301-01 [PubMed: 15835576]
Shin S, Park J, & Bae S (2019). Nurse staffing and hospital-acquired conditions: A systematic review.
Journal of Clinical Nursing, 28(23–24), 4264–4275. 10.1111/jocn.15046 [PubMed: 31464017]
Silber JH, Rosenbaum PR, Trudeau ME, Even-Shoshan O, Chen W, Zhang X, & Mosher RE (2001).
Multivariate matching and bias reduction in the surgical outcomes study. Medical Care, 39(10),
1048–1064. 10.1097/00005650-200110000-00003 [PubMed: 11567168]
Sloane DM, Smith HL, McHugh MD, & Aiken LH (2018). Effect of changes in hospital nursing
resources on improvements in patient safety and quality of care: A panel study. Medical Care,
56(12), 1001. 10.1097/MLR.0000000000001002 [PubMed: 30363019]
Stimpfel AW, Sloane DM, McHugh MD, & Aiken LH (2016). Hospitals known for nursing excellence
associated with better hospital experience for patients. Health Services Research, 51(3), 1120–
1134. 10.1111/1475-6773.12357 [PubMed: 26369862]
Stuart EA (2010). Matching methods for causal inference: A review and a look forward. Statistical
Science: A Review Journal of the Institute of Mathematical Statistics, 25(1), 1. 10.1214/09STS313 [PubMed: 20871802]
Ulrich BT, Buerhaus PI, Donelan K, Norman L, & Dittus R (2007). Magnet status and registered
nurse views of the work environment and nursing as a career. JONA: The Journal of Nursing
Administration, 37(5), 212–220. 10.1097/01.NNA.0000269745.24889.c6
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Dierkes et al.
Page 12
Author Manuscript
Author Manuscript
Figure 1.
Standardized bias plot summarizing matching performance across all covariates,
highlighting the change in standardized mean differences between using all data and
matched data.
Author Manuscript
Author Manuscript
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Author Manuscript
Author Manuscript
258 (78)
252 (76)
755 (1,482)
249 (75)
HHI, mean (SD)
High-tech, N (%)
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
0.27 (0.12)
1.79 (0.29)
8.33 (2.32)
Disproportionate share, mean (SD)
Case mix index, mean (SD)
Nurse staffing, mean (SD)
100 (30)
99 (30)
49 (15)
Midwest
South
West
52 (16)
104 (31)
100 (30)
75 (23)
8.12 (2.63)
1.76 (0.28)
0.27 (0.14)
1.05 (0.21)
49 (12)
19 (9.8)
0 (0.0)
7(2.1)
324 (98)
240 (73)
726 (1,170)
6 (1.8)
0.16 (0.30)
392 (322)
514 (20)
1,057 (42)
582 (231)
376 (15)
6.79 (2.64)
1.52 (0.28)
0.31 (0.17)
0.98 (0.21)
52 (14)
21 (13)
212 (8.4)
498 (20)
1,819 (72)
923 (37)
874 (1,598)
587 (23)
0.06 (0.17)
205 (191)
1,937 (77)
549 (22)
1,174 (46)
2,132 (84)
0.090
Non-Magnet hospitals
(N = 2,529)
Notes: Nurse staffing is registered nurse hours per patient day. Wage index is for fiscal year 2017.
83 (25)
Northeast
Region, N (%)
1.04 (0.19)
Wage index, mean (SD)
19 (9)
50 (11)
1 (0.3)
Rural
% Medicaid pts., mean (SD)
10 (3.0)
Micro
% Medicare pts., mean (SD)
320 (97)
Metro
Core-based statistical area, N (%)
11 (3.0)
0.19 (0.37)
Residents per bed, mean (SD)
For-profit, N (%)
440 (297)
Total hospital beds, mean (SD)
Overall P4P
92 (28)
98 (30)
HAC
163 (49)
134 (40)
VBP
264 (80)
0.287
280 (85)
0.314
Matched controls
(N = 331)
HRRP
Penalties, N (%)
Propensity score
Magnet® hospitals
(N = 331)
563 (20)
1,156 (40)
682 (24)
459 (16)
6.97 (2.65)
1.55 (0.30)
0.31 (0.17)
0.99 (0.21)
52 (13)
21 (12)
213 (7.5)
508 (18)
2,139 (75)
1,172 (41)
860 (1,585)
598 (21)
0.07 (0.21)
232 (219)
2,189 (77)
647 (23)
1,308 (46)
2,412 (84)
n/a
All hospitals
(N = 2,860)
−0.16
−0.26
0.17
0.24
0.66
0.93
−0.32
0.33
−0.21
−0.21
−1.47
−0.97
1.38
−0.90
−0.08
−1.11
0.36
0.79
1.04
Prematch
n/a
−0.03
−0.03
<0.01
0.06
0.09
0.11
0.05
−0.01
−0.01
0.06
0.06
−0.05
−0.07
−0.07
0.02
0.08
0.09
0.16
0.12
Postmatch
Standardized mean differences
Author Manuscript
Hospital Characteristics and Performance on P4P Programs, Before and After Matching.
Author Manuscript
Table 1.
Dierkes et al.
Page 13
Author Manuscript
Author Manuscript
Author Manuscript
Author Manuscript
Standardized mean differences approach zero as the quality of the match improves.
SD = standard deviation; HHI = Herfindahl-Hirschman Index; P4P = Pay for Performance; HRRP = Hospital Readmissions Reduction Program; VBP = Value- Based Purchasing; HAC = Hospital-Acquired
Conditions Reduction Program; n/a, not applicable; pts., patients.
Dierkes et al.
Page 14
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.
Author Manuscript
Author Manuscript
Author Manuscript
0.70* (0.52–0.95)
1.09 (0.78–1.53)
0.79* (0.62–0.99)
1.52** (1.18–1.96)
0.97 (0.74–1.28)
HAC
Overall P4P
1.02 (0.71–1.46)
0.75 (0.50–1.11)
0.66* (0.48–0.91)
0.66* (0.48–0.92)
1.40 (0.89–2.19)
1:1 Matched (N = 662)
0.81 (0.60–1.10)
0.58*** (0.44–0.76)
1.18 (0.81–1.70)
Unmatched (N = 2,860)
Adjusted
p < .001.
***
p < .01
p < .05
**
*
Notes: Coefficients represent the change in odds of a penalty under each individual program (HRRP, VBP, and HAC) and in aggregate across all three programs (overall P4P) associated with Magnet versus
non-Magnet hospitals. Hospital covariates include technology status, percent Medicare/Medicaid patients, core-based statistical area, hospital ownership, Herfindahl–Hirschman Index, residents per bed,
total hospital beds, case mix index, disproportionate share, wage index, registered nurse hours per adjusted patient day, and U.S. census region. P4P = Pay for Performance; HRRP = Hospital Readmissions
Reduction Program; VBP = Value-Based Purchasing; HAC = Hospital-Acquired Conditions Reduction Program.
0.90 (0.63–1.30)
1.40 (0.93–2.08)
1.02 (0.74–1.40)
VBP
1:1 Matched (N = 662)
HRRP
Unmatched (N = 2,860)
Unadjusted
Unadjusted and Adjusted Logistic Regression Models, Before and After Matching of the Association Between Magnet® Status and P4P Program
Penalties.
Author Manuscript
Table 2.
Dierkes et al.
Page 15
Policy Polit Nurs Pract. Author manuscript; available in PMC 2022 August 22.