How to evaluate quality in
medicine
Eva Stensland MD, PhD
National Network of Competence for
Medical Quality Registries
08.12.14
Quality in medicine - definition
“The degree to which health services for individuals and
populations increase the likelihood of desired health
outcomes and are consistent with current professional
knowledge.”
Institute of Medicine. (2001). Crossing the
Quality Chasm: A New Health System for the
21st Century. Washington D.C.: National
Academy Press.
Six elements of quality
• Safe – avoiding injuries to patients from the care that is supposed to
help them.
• Effective – providing services based on scientific knowledge to all
who could benefit and refraining from providing services to those not
likely to benefit (avoiding underuse and overuse).
• Patient-centered – providing care that is respectful of and
responsive to individual patient preferences, needs, and values and
ensuring that patient values guide all clinical decisions.
• Timely – reducing waits and sometimes harmful delays for both
those who receive and those who give care.
• Efficient – avoiding waste, in particular waste of equipment,
supplies, ideas, and energy.
• Equitable – providing care that does not vary in quality because of
personal characteristics, such as gender, ethnicity, geographic
location, and socioeconomic status.
Institute of Medicine. (2001). Crossing the
Quality Chasm: A New Health System for the
21st Century. Washington D.C.: National
Academy Press.
Challenges when presenting and evaluating
quality in medicine
• Different groups can have different reasons for measuring quality
and hence different measurement criteria and emphasis.
• Clinicians or those who manage and provide clinical care might be
interested in evaluating quality so that they can monitor and improve
the services they are providing to individual patients.
• Regulators might be interested in ensuring that care provided by a
hospital/health care organization meets a minimal standard and/or is
making credible efforts to improve care quality.
• Patients/population might be most interested to know if treatment
in their local hospital is safe or to get information they can use to
choose hospital.
Monitoring health care (Donabedian)
Structure
Material resources
Human resources
Process
(activity)
Diagnosis
Treatment
Result/outcome
Mortality
Morbidity
Quality of life
Structure
Process
Outcome
QUALITY
If quality-of-care criteria based on structural,
process, or intermediate outcomes are to be
credible, it must be demonstrated that variations in
the attribute they measure lead to differences in
health status outcomes.
What indicators should we choose?
• Of the structural indicators, measures that predict variations in
processes or outcomes of care have the greatest utility
•
Process indicators are especially useful when:
- quality improvement is the goal of the measurement process
- short time frames are necessary
- performance of low volume providers is of interest
- and when tools to adjust or stratify for patient factors are lacking.
Comparisons of process data are easier to interpret and more sensitive to
small differences than comparisons of outcomes data.
•
Outcome data are useful if:
- long time-frames are possible
- performance of whole systems should be studied
- or if a high volume of cases are available.
Outcome data are most useful for tracking care given by high-volume
providers over long periods of time, and for detecting problems in
implementation of processes of care.
Palmer RH: Int J Qual Health Care
1998;10:477-83
Factors determining the outcome
Factors that are frequently included in risk adjustment
models include:
• patient demographic
• psychosocial characteristics (such as age, sex, and functional
status)
• lifestyle factors (smoking, alcohol use)
• severity of the illness that is the focus for measurement
• health status
• co-morbid conditions.
Risk adjustment is important before comparing patient outcomes
across hospitals or providers. Process indicators (eg. adherence to
guidelines) might be more appropriate for comparing quality
between hospitals





Quality indicators
An ideal indicator would have the following key characteristics:
(i) indicator is based on agreed definitions, and described
exhaustively and exclusively
(ii) indicator is highly or optimally specific and sensitive, i.e. it
detects few false positives and false negatives
(iii) indicator is valid and reliable
(iv) indicator discriminates well
(v) indicator relates to clearly identifiable events for the user
(relevant to clinical practice)
(vi) indicator permits useful comparisons
(vii) indicator is evidence-based
Mainz J Int J Qual Health Care 2003;15:523-530
The use of quality indicators in a quality register
RIKS-HIA quality index
Quality indicator
0,5 point
1 point
Reperfusion
80 %
85 %
Reperfusion in recommended time
75 %
90 %
Coronary angiography
75 %
80 %
Heparin
90 %
95 %
ASA
90 %
95 %
Clopidogrel/Ticlopidin
85 %
90 %
Beta blocker
85 %
90 %
Lipid-lowering drugs
90 %
95 %
ACE-inhibitor
85 %
90 %
Swedeheart: Annual report 2013
OECD: Health Care Quality Indicators
Health at a glance 2013:
Case-fatality in adults >45 within 30 days after admission
for AMI
Health at a glance 2013
Patient-reported data
• PROM: patient reported outcome measures
• PREM: patient reported experience measures
• (PRI: patient reported incidents)
• PROMs includes patient reported symptoms, function, healthrelated quality of life, ratings of health care
• Patient-reported measures can correlate poorly with physiologic
measures.
• Patients with the same clinical status or physiologic state may
have different responses to the condition.
Norwegian Registry for Spine Surgery
- Function
(Oswestry Disability Index ODI) range 0-100 where 0= no disability
- Back and leg pain
(numerical pain scale - NRS)
range 0-10 where 0= no pain
- Health-related quality of life
(EQ-5D) 0=death and 1= perfect health
- Global score of outcome of
surgery 7 point scale
Lumbar dics
herniation:
Pain, loss of
function
and quality
of life
Can we define success criteria for lumbar disc surgery?
Estimates for substantial amount of change in core outcome measures.
Solberg T. Acta Orthopaedica 2013; 84 (2):196-201
Aim: estimate cut-off values for success criteria for the Oswestry
disability index, pain scale and HRQL
The cut-off values for success for the mean change scores were 20 (ODI), 2.5 (NRS
back), 3.5 (NRS leg), and 0.30 (EQ-5D). The ODI and leg pain scale showed the best
ability to discriminate success.
Summary
• Part of the complexity in evaluating quality of care is that
different groups can have different reasons for measuring
quality and hence different views on variables to choose and
measurement criteria.
• Elements of quality:
• Clinical measures and patient-reported measures assess
different aspects of quality
Tromsø
Thank you!