Maximizing impact of quality
measurement research on policies
and programs
Helen Burstin, MD, MPH
Senior Vice President, Performance Measures
National Quality Forum
Academy Health Webinar
May 27, 2010
www.qualityforum.org
Quality Enterprise Functions:
Contributions of NQF
Health Services Research
Establish National
Priorities
Endorse Measures,
Practices, and SREs
Align Payment and
Other Incentives
Two dimensional framework:
•National Priorities Partnership
•Top 20 conditions
•Over 600 measures covering
all settings, including Safe
Practices and SREs
•Analysis of measurement
implications of various
payment reform models
Identify Measure Gaps
Build Data Platforms
Improve Performance
•Agenda for Measure
Development and Endorsement
•Health Information Technology
Expert Panel
•Webinars
•Measures database
Measure Development
Publicly Report Results
Evaluate
•Guidance for performance
reporting on safety
•MAPs & Dashboard
•Measure use evaluation
2
www.qualityforum.org
Quality Measurement in Evolution
•
•
•
•
•
Drive toward higher performance
Shift toward composite measures
Measure disparities in all we do
Harmonize measures across sites and providers
Promote shared accountability & measurement
across patient-focused episodes of care:
– Outcome measures
– Appropriateness measures
– Cost/resource use measures coupled with quality
measures, including overuse
www.qualityforum.org
Patient-Focused Episodes of Care Model
End of Episode
Population at Risk
Evaluation &
Initial
Management
Rehabilitation &
Follow-up Care
- Risk-adjusted health
outcomes (i.e. mortality &
functional status)
PHASE 3
- Risk-adjusted total cost
of care
PHASE 2
PHASE 1
Clinical episode begins
Time
Appropriate Times Throughout Episode
- Determination of key patient attributes for risk
adjustment
- Assessment of informed patient preferences and
the degree of alignment of care processes with
these preferences
- Assessment of symptom, functional, and emotional
status
www.qualityforum.org
Episodes Model Measurement Domains
• Patient-level outcomes (better health)
–
–
–
–
–
Morbidity and mortality
Avoidance of complications (e.g., HAIs)
Functional status
Health-related quality of life
Patient experience of care
• Processes of care (better care)
– Technical
– Care coordination and transitions
– Alignment with patients’ preferences
• Cost and resource use (overuse, waste, misuse)
– Total cost of care across the episode
– Indirect costs
www.qualityforum.org
NQF
Consensus
Development
Process
Consensus
Development
Process Steps
•
Call for nominations for steering committee
and technical panels
•
Call for measures
–
Online measure submission form
•
Measure evaluation
•
Member and public comment
•
NQF member voting
•
Consensus Standards Approval Committee
(CSAC) and Board endorsement
•
Appeals
www.qualityforum.org
Conditions for Consideration
1. The measure is in the public domain or an intellectual
property agreement is signed.
2. The measure owner verifies there is an identified
responsible entity and process to maintain and update the
measure.
3. The intended use of the measure includes both public
reporting and quality improvement.
4. The requested measure submission information is
complete.
• Generally, measures should be fully developed and tested so that
all information needed to evaluate the measure is available.
• Measures that have not been tested are only potentially eligible
for a time-limited endorsement
• Measure owners must verify that testing will be completed within
12 months of endorsement.
7
www.qualityforum.org
NQF Evaluation Criteria
• Importance to measure and report
• What is the level of evidence for the measures?
• Is there an opportunity for improvement?
• Relation to a priority area or high impact area of care?
• Scientific acceptability of the measurement
properties
• What is the reliability and validity of the measure?
• Usability
• Can the intended audiences understand and use the results
for decision-making?
• Feasibility
• Can the measure be implemented without undue burden,
capture with electronic data/EHRs?
8
www.qualityforum.org
Importance to Measure and Report
• The specific focus of what is measured should be
considered important enough to expend resources for
measurement and reporting, not only that it is related
to an important broad topic area.
• These concepts are addressed in separate sub-criteria
for
– Relation to an NPP goal or high impact aspect of
healthcare
– Evidence to support the measure focus
– Opportunity for improvement
www.qualityforum.org
Scientific Acceptability of Measure
Properties
• Precise specifications
• Reliability, validity, and discrimination (testing is
expected to demonstrate reliability and validity)
• Demonstration of comparability if more than one data
source/method is allowed
• Specifications should allow for identification of
disparities (e.g., stratification)
• Adequacy of risk-adjustment
• Appropriateness of exclusions
www.qualityforum.org
Usability
• Requires evidence that the measure results are
meaningful and understandable to intended audiences
and useful for both public reporting and informing
quality improvement.
– This is consistent with NQF policy of not endorsing
measures solely for quality improvement.
– Measures should be harmonized and provide a
distinctive or additive value to existing endorsed
measures.
www.qualityforum.org
Feasibility
• Extent to which the required data are readily available,
retrievable without undue burden, and can be
implemented for performance measurement.
– Required data are routinely generated concurrent
with and as a byproduct of care delivery.
– Required data elements are available in electronic
sources OR credible, near-term path to electronic
collection
– Data elements are specified for transition to EHRs
(NQF Quality Data Set)
www.qualityforum.org
Linkage of HIT and Measurement
Data
Data
Sources
Source
s
Performance
Measures
EHRs and
HIT tools
E-Infra
structure
www.qualityforum.org
Disparities Measurement
• Assessment of quality and safety by race, ethnicity,
primary language and SES status needs to become a
routine part of performance measurement
• Explore direct methods for collecting race, ethnicity,
primary language, and SES data in an efficient,
effective, patient-centered manner or consider indirect
methods
• Identify measures that are “disparity-sensitive” that
should be routinely stratified: prevalence, impact of
the condition, impact of the quality process, quality
gap
14
www.qualityforum.org
Health Services Research: Case Study
• Patient safety has been a longstanding topic for
health services research
• Safety hazards and safety improvements may be
difficult to measure.
• Safety measures can be difficult to capture as
valid rates:
–
–
–
–
–
Safety events are uncommon or rare
Few events have standardized definitions
Dependence on self-reporting
Denominators are frequently poorly defined
Handling of “close calls”
15
www.qualityforum.org
Patient Safety Reporting: Issues (1)
• Low-frequency events:
– Some events are perceived as being so serious that a single
event should be reported.
– Usually no adjustment for volume, predisposing, or risk factors.
• Variability in consequences or harm:
– Some events are, by definition, associated with significant harm
– Others may be indisputable patient safety events but may not
lead to any harm or may lead to relatively transient harm
– The same type of event (e.g., patient fall) may cause significant
harm in one patient and no harm in another
– Important flag for needed improvement.
16
www.qualityforum.org
Patient Safety Reporting: Issues (2)
• Problems of event identification:
– Dependent on recognition and documentation of the event
– Events that do not cause harm (e.g., fall without injury) may be
less likely to be recognized.
– Occurrences that do not reach the patient (near misses/close
calls), such as wrong site prepped but surgery not performed,
are less likely to be reported
• Event-free intervals:
– Relevance or usefulness of reporting intervals free of event
occurrence (e.g., absence of bloodstream infections over a
period of years)
17
www.qualityforum.org
Thank You
Helen Burstin, MD, MPH
hburstin@qualityforum.org
18
www.qualityforum.org
www.qualityforum.org
National Priorities
• Population health
- Key preventive services
- Healthy lifestyle behaviors
• Patient/family engagement
– Informed decision-making
– Patient experience of care
– Patient self-management
• Safety
– Hospital-level mortality rates
– Serious adverse events
– Healthcare-Acquired
Infections
• Care Coordination
– Medication reconciliation
– Preventable hospital
readmissions
– Preventable emergency
department visits
• Palliative Care:
– Relief of physical symptoms
– Help with psychological, social
and spiritual needs
– Communication regarding
treatment options, prognosis
– Access to palliative care
services
• Overuse
– 9 major areas
19
www.qualityforum.org