Measuring quality processes
and outcomes
Rod O'Connor PhD
www.RodOConnorAssoc.com
Rod O'Connor & Associates P/L
Adjunct Senior Lecturer in Health
Outcomes Measurement, SHSM, UNSW
Email: rodocon@RodOConnorAssoc.com
59 Gipps St. Birchgrove NSW
Ph: +61-2-9555 9916 Mob: 0413 60 70 73
Measuring Quality Processes and Outcomes
Measurement of outcome quality
Origins:



Economic - Demand to assess rival health
programs in a context of increasing pressure on
resources
Medical effects - move to comprehensively
evaluate clinical therapies
Recognition that patients assess their health
differently to their doctors
Measuring Quality Processes and Outcomes
U.S. President’s Advisory Commission on
Consumer Protection and Quality in the
Health Care Industry - ‘Quality First: Better
Health Care for All Americans’
Role of quality measures in internal quality
improvement initiatives ‘provide the tools that allow health care
providers and organizations to undertake focused
improvement programs to improve the quality of
care they furnish, whereby quality is measured both
before and after an intervention’

Measuring Quality Processes and Outcomes
Need to capture multiple dimensions of
quality

the processes associated with the delivery of care
the
outcomes obtained
Measuring Quality Processes and Outcomes
Health Care Processes (treatment)
Admission
Surgery
*
*
*
*
Technical Care
Information
Consideration
Informed consent
Discharge
Outcomes
* Mortality
* Clinical status
* Functional
status
* HQOL
Measuring Quality Processes and Outcomes
Treatment (process) and outcome
indicators are both required:


treatment indicators to reveal the source of
outcome variation, and because case mix
variation can make outcomes hard to interpret
outcome indicators to reveal the state of health
of patients (and ex-patients), and to identify
unexpected effects and effects of most concern
to the patient - mortality, morbidity, functional
status, HQOL, and ‘satisfaction’ all relevant.
Measuring Quality Processes and Outcomes
Desirable to have core data sets, evaluated
against criteria (U.S. report)
Scientific
soundness (i.e., reliable, valid, adjusted)
Importance of the quality concern;
Relevance to various users;
Potential to foster improved health status or well-being
Evidence basis
Interpretability, and Actionability ( the degree to which
steps can be taken to address the concern);
Feasibility; and Ease/ Cost-effectiveness of measurement.
Measuring Quality Processes and Outcomes
But essential to encourage innovation
(U.S. report)
Identification of core sets of quality measures
should not lock in approaches to quality
measurement and reporting.
Innovation can be fostered by … placing limits
on the breadth of standardization efforts (e.g.,
number of measures) so compliance with the core
reporting sets does not consume resources better
devoted to innovation or internal quality
improvement

Measuring Quality Processes and Outcomes
Practical measurement Issues 1:
Measures need to be valid
Important
that instruments measure what
they are intended to
Increasing focus on measuring care from the
patient’s perspective
Measuring Quality Processes and Outcomes
The cost of poor measures
The cost of poor measures is wasted and
potentially misleading information, potentially
leading to:
failure
to improve services
discrimination against good treatments
development of poor quality services
Measuring Quality Processes and Outcomes
Need to specify the measure, its purpose,
and the action that flows from variation in
the measure
What
is the decision to be made?
Which variable needs to be estimated to guide the
decision?
How can the variable be most simply & directly
measured?
What degree of change is needed to provoke a
response, and what should the response be?
Measuring Quality Processes and Outcomes
Practical measurement Issues 2:
Indicators need explicit definition,
quantification, and standardisation
Each
indicator, and its purpose, need to be clearly
defined, as well as the conditions under which it is
to be measured.
quality
measures need standard operational
definitions, eg. caesarean section rates need to
specify the age range of women to be included in
the measure, the timeframe etc.
Measuring Quality Processes and Outcomes
Indicators require:
a. operationalised definitions
eg. 'some reduction in dexterity' , given the
example of 'difficulty smoothly placing key in lock'.
b. explicit quantification of phrases by removing terms
such as 'severely' and 'moderately’
- thus 'seriously disruptive' became 'disrupts work for at least 60
minutes per day'.
c. domains clarified & emphasised
- eg. ‘communication difficulties’ refers to sensory impairment,
not cognitive impairment or language knowledge.
Measuring Quality Processes and Outcomes
Practical measurement Issues 3:
When assessing outcomes, may need to
make adjustments for risk (particularly
with one-off measures, eg. mortality)
Outcome
indicators can employ mortality data,
clinical outcomes, HQOL and functional status
measures (disease-specific or generic). But
outcomes result from a complex mix of factors that
relate to intrinsic patient characteristics as well as
quality of care.
Measuring Quality Processes and Outcomes
The problem of attribution
Adjusting for differences in baseline risk can be
extremely difficult
persons with complex illnesses, multiple coexisting
diseases, or other significant risk factors are more likely
to do poorly, even with the best care, than healthy
individuals (Iezzoni, 1997).
there is often a long delay between care given and an
outcome occurring, leading to uncertainty as to which
episode of care (or which carer) was responsible.
outcomes of interest may be infrequent (the "small n"
problem)
Measuring Quality Processes and Outcomes
Practical measurement Issues 4:
Patient-centred outcomes are important,
as well as clinical outcomes
Most
outcomes research still focuses on mortality
and morbidity rates (did test-group patients live
longer, show significant improvements in
physiological measures of health status, were there
untoward side-effects or complications). Relatively
few research studies have tended to assess
outcomes in terms of HQOL(health-related quality
of life), disability, patient satisfaction.
Measuring Quality Processes and Outcomes
Patient-centred measurement, such as
HQOL, can potentially change treatment
practices by identifying relatively
neglected areas of major concern to
patients
Measuring Quality Processes and Outcomes
Patients
assess their health differently to their
doctors, both according to degree of improvement
and the areas of greatest concern .
 Doctors tend to stress physical much more than
psycho-social aspects of health
The separation between doctor and patient
judgements tends to increase as the doctor becomes
more senior.
Measuring Quality Processes and Outcomes
Patient-focussed outcomes include:
the patient's report of the degree to which their
overall health has improved
 the extent to which the functional and qualityof-life areas of greatest concern to patients have
been remedied
 patient perceptions of the health care
experience (beware of reports on 'satisfaction').

Similarly, ‘appropriateness of care’ is
ultimately assessed according to the extent it
delivers outcomes valued by the patient)
Measuring Quality Processes and Outcomes
Psychometric measures can develop better
services from the patient’s perspective
Illness-specific measures of health-related quality of
life (based on self-report) can be systematically
decomposed to reveal the areas of key importance to
patients, which typically differ from the judgement of
doctors.
Thus HQOL measures can play a diagnostic as well as
an outcome-measurement role.
Measuring Quality Processes and Outcomes
Caution:
When using generic HQOL measures,
recognise that they may be insensitive and
misleading - test them rigorously
Eg. Sickness Impact Profile (SIP)
For
all patients, self-assessed dysfunction & global
SIP, r=0.52; for speech pathology only, r=-0.01
Measuring Quality Processes and Outcomes
The Present
Concerns regarding the validity and application
of existing indicators, concerns regarding risk
adjustment, relatively little attention to
validity/reliability.
HQOL and client satisfaction tests considered
soft and of doubtful merit. Where HQOL
used, usually reliance on omnibus measures.
Measuring Quality Processes and Outcomes
The Future
The disciplines of epidemiology, psychometrics, and
health economics will become more integrated.
Clinical measures and cost/ efficiency, HQOL/
functionality, and client satisfaction measures will
approach equal status. HQOL and other psychometric
tests will play a major diagnostic role, to identify issues
and areas most important to the patient, and target
services/treatments to them.