Pergamon
International Journal for Quality in Health Care. Vol. 8, No. 2, pp. 101-104, 1996
Copyright © 1996 Elsevier Science Ltd. All rights reserved
Printed in Great Briuin.
PD: S1353-^505(96)00018-X
1353-1505/96$ 15.00+0.00
Counterpoint
Occasionally, the Journal will feature "Counterpoint"—one or more guest editorials presenting a
variety of views on the topic of a paper published in the same issue of the Journal. Our goal is to
further discussion of a topic that is currently under debate in academic and professional circles. We
invite our readers to write letters to the Editor adding their opinions on the topic.
Using Outcome Data to Measure Quality in Health Care
101
health care and the complementarity of
structure/process measurement and outcome
measurement.
The two major problems with existing health
outcome measures (including health status)
which limit their utility as accountability tools
are risk adjustment and attribution. The inputs
to health care processes (otherwise known as
patients!) vary widely. Risk adjustment models
which adequately and appropriately account for
all the confounding factors for given health care
interventions are currently unavailable [1,2].
As highlighted by Dr Goldfield in this issue, the
powerful influence on ultimate health status of
hereditary and environmental factors unrelated
to traditional health care delivery can make the
attribution of any given health status change to
a specific health care intervention (or series of
interventions) difficult. When a series of interventions is involved (as in chronic illness treatment regimens), it may be difficult to ascribe
health status changes to any individual intervention. This is a particular problem if it is
proposed to use health status measures as
accountability tools in a health care system such
as Australia's, where hospital and community
care are typically provided by unrelated teams
of care-givers, and hospital-based and community care are purchased by different governmental agencies—although less of a problem in
integrated service delivery models such as
Managed Care. Problems with attribution of
health status changes to given health care interventions and imperfect risk-adjustment models
make comparisons between the health care providers' performance based upon current outcome measures tentative at best. With
continued research and ongoing measure
Downloaded from by guest on March 6, 2016
Debates in health care quality forums have
shifted in recent decades from whether quality
• can be measured to how best to measure quality
in health care. The few stalwarts clinging to a
notion of quality as being intangible have been
left behind. Interest is focused upon the selection of measurement sets which reliably and
credibly inform about health care service quality. The challenges in developing and implementing measurement sets appropriate to the
needs of the various participants in health care
(purchasers, providers and consumers) are considerable and the use of health status measures
within a health outcomes quality monitoring
framework does have difficulties. Proponents of
individual measures or systematic approaches
to measurement occasionally adopt such strident advocacy that it would appear that other
measures or approaches to measurement are
without value or validity. Health status and
other outcome measures may appear to be
offered as alternatives to structure or process
measures (which they are) which are in competition with them (which they are not). The
ultimate goal of health systems is the optimization of the health of treated individuals and
populations, and the "gold standard" for quality
measurement will thus always be health outcome measures. However, all health outcome
measures currently available have a number
of relative weaknesses. The complexities of
health care demand a balanced portfolio of
structure, process and outcome measures in
quality monitoring to fulfil the varied requirements of all participants in health care. Presentation of outcome measures as replacements for
all other quality measures ignores the glaring
deficiencies of most quantification efforts within
102
development, outcome measures, including
health status measures, will improve, and their
utility will increase. In clinical medicine, incomplete solutions to complex diagnostic and therapeutic dilemmas are rife. "Best available"
solutions are routinely applied whilst better
solutions are developed or sought. Quality
monitoring in health care demands the same
pragmatism, with implementation of "best
available" outcome measures (including health
status measurement instruments) whilst developing improved versions (e.g. better able to
address chronic illness or the needs of particular
socioeconomic groups) for future application. If
existing measures are well characterized, well
understood and all health care participants are
cognisant of the limitations of measurement,
linkages between outcome measures and provider sanctions or rewards will be made
acknowledging these limitations. In this way,
purchaser/provider conflict and consumer misinformation will be avoided.
tus and measures of patient satisfaction [3].
Others exclude economic analyses from health
outcome measures, preferring to address the
value of health care separately [4,5]. Our concepts of "health" certainly embrace characteristics reflected in both clinical and health status
measures. Doctor Goldfield proposes that
health care providers contribute to health status
assessment. Providers already do contribute to
health status measures by virtue of their participation in instrument design. Beyond this contribution, I would prefer to see health status
assessment remain as a "customer focused/
patient centred" activity. Providers participate
in health outcome formulations by designing
and applying relevant clinical outcome
measures which quantify their perspectives on
the health impact of any given intervention.
When health outcomes are being used to monitor the quality of a particular health care intervention, it is logical to address satisfaction with
care as one component of outcome assessment,
as the acceptability of care is interwoven with
observed clinical and health status outcomes.
Should health outcomes be used to monitor
system-wide quality or community health, the
feasibility and utility of satisfaction measures
may be more questionable. Economic analysis
of health care performance is inevitable—be it
as an integral component of health outcome
assessment or a separate value assessment
based on an outcome/cost review. The definition for health outcomes recently adopted for
use within Australian health care (a health outcome is a change in the health of an individual or
group of people or population which is attributable to an intervention or series of interventions) [6] does not accommodate economic
measures.
The question of what level of health system
performance to tackle with outcomes measurement (e.g. individual provider, provider
groups, individual facility, network of facilities
or community-based assessment) will only ever
be answered when it is agreed as to why
measurement is undertaken. If the objective of
measurement is to ascertain the community
impact of any given health care intervention or
health care delivery model, there is little point
in focusing on facilities, networks or individual
providers. Doctor Goldfield argues for health
status measures as integrators of global influ-
Downloaded from by guest on March 6, 2016
Health outcome measures can be used for
several distinct purposes:
• Individual patient or population health
monitoring
• Effectiveness research
• Quality improvement endeavours
• Provider accountability
All of these are worthwhile applications of outcome measures. Doctor Goldfield suggests that
current limitations of health status measures
render them of more value as monitors of population health/community health, integrating all
influences on health, than as the basis of conventional health outcome applications. I would
argue that health status measures are of value in
each of the above circumstances. Each of these
contexts for measurement demands different
attributes of health status measures, with an
increased stringency demanded when health
status measures are used in particular settings,
e.g. for provider accountability. Traditional approaches to quality monitoring may see outcomes measured in clinical, health status,
customer satisfaction and economic performance domains. There is a trend to convergence
of these aspects of outcome into a global assessment of health outcome. Expert opinion on how
all-encompassing health outcome measures
should be varies. Donabedian, for example,
separates outcome measures such as health sta-
Counterpoint
Counterpoint
successful outcome analysis, and various State
Health Departments are actively progressing
pilots of outcome-focused health care quality
monitoring. These projects include widespread
application of health status measures, in particular disease states, within health care facilities and in community health profiling (the most
common instrument in use being the Australian
version of SF36). Commonwealth and States
have contributed to the establishment and
maintenance of a National Health Outcomes
Clearing House to help track and inform on
outcome projects underway within Australia
and to link with related international initiatives.
Outcomes measures including health status
assessment are vital tools for health care quality
monitoring. As well as difficulties with riskadjustment and attribution (see above), current
outcomes measures have additional weaknesses:
1. Some outcomes are sufficiently rare to provide little statistical power for comparative
analyses.
2. Outcome analysis has little ability to inform
clinical practice and contribute to clinical
Quality Improvement (i.e. it says how well it
worked, not why it works).
3. Outcomes may be long after any intervention and there may be difficulty in agreeing on the optimal time for outcome
assessment.
4. Consumers care about structure and process
quality, not just outcomes. They wish to
know at the time of service delivery that
services are delivered in a way that maximizes their chances of an ultimately favourable outcome.
Measures of structure and process redress
these deficiencies. Although frequently there is
no proven link between the measured structural
or process attribute and eventual outcomes, if
such linkages are known to exist, structure and
process measures may act as "outcome proxies". Given the current "state of the art" in
quality measurement (be that structure, process
or outcome measures) in health care, it is essential that we combine measurement approaches
to generate a useful profile of service quality.
I believe health status measures will remain
a valuable component of health outcome
measurement for health care quality monitoring. This does not preclude their use for indi-
Downloaded from by guest on March 6, 2016
ences on health, acknowledging and attributing
impacts on health status of factors other than
health care delivery. This is a valid use of health
status measures which differs from their use in
health outcome measures, where the goal is to
identify and monitor only that component of
health status change attributable to health care
interventions. Both are important uses of these
valuable monitoring tools (i.e. health status
measurement instruments). If the objective of
health status measurement is the comparison of
performance of health care facilities within a
geographical region, then only facility-scale
measures of health outcomes are of value. Unfortunately, the objectives of health care quality
monitoring are all too frequently multiple, conflicting and vague [7]. The absence of clearly
stated monitoring objectives will see measurement sets judged by divergent criteria, as those
adjudicating assume they have divined the
"true" objective of those applying the particular
measurement instruments.
Outcome measurement, like all quality monitoring initiatives, should be accompanied by
explicit objectives which determine the framework for measurement and direct the criteria
for measure adequacy. Information systems
capturing patient-level data may allow reformatting of data to generate measures for health
care quality for an individual provider, facility
or network of facilities and community health
status. If data exist to link health care provision
and other non-health care influences on health
to community health status measures, it may be
feasible to fractionate the attribution of any
observed community benefit.
The Australian health scene is currently moving towards an output/outcome focus which will
consider health outcomes embracing clinical,
health status and patient satisfaction and see
economic outcomes addressed simultaneously
[8]. As such, we have as yet no "traditional" use
of either outcome measures generally or health
status measures specifically. It is acknowledged
within health management in Australia that
considerable preparatory development work is
required to make an outcome focus in health
care quality assessment feasible [9]. The Commonwealth Department of Human Services and
Health is currently supporting initiatives such as
the National Hospitals' Outcomes Program to
define appropriate issues and methodologies for
103
104
vidual patient monitoring, community health
monitoring, effectiveness research or quality
improvement initiatives. Detailed knowledge
of health status measures defines the caveats
drawn around conclusions based upon the use
of these measures in any given circumstance.
Rather than restricting health status measures
to any particular circumstance (e.g. community
health monitoring), I would advocate their
application to contribute to the achievement of
any appropriate objective in health assessment,
provided there is a clear enunciation of that
objective and a full understanding of any limitations to their use in any given circumstance.
Counterpoint
4.
5.
6.
7.
8.
REFERENCES
9.
Professor Neil Boyce
Quality Improvement Clinician
The Alfred Healthcare Group
Prahran, Victoria
Australia
Downloaded from by guest on March 6, 2016
1. Iezzoni L I, Ash A, Schwartz Z M, Daley J,
Hughes J S and Mackiernan Y D, Predicting who
dies depends on how severity is measured: implications for evaluating patient outcomes. Ann
Intern Med 123: 763-770, 1995.
2. Localio A R and Hamory B H, A report card for
report cards. Ann Intern Med 123: 802-803,1995.
3. Donabedian A, Explorations in quality assessment
and monitoring, Vol. 1. The definition of quality
and approaches to its assessment. Health Administration Press, Ann Arbor, MI, 1980.
Greenfield S, The challenges and opportunities
that quality assurance raises for technology assessment. In: Quality of care and technology assessment (Lohr K and Rettig R, Eds). National
Academy Press, Washington DC, 1992.
Banta D, Developing outcome standards for quality assurance activities. Qual Assur Hlth Care 4:
25-32, 1992.
Australian Health Ministers' Advisory Council
Proceedings, Canberra, 1993.
Lansky D, Butler J B V and Waller F T, Using
health status measures in the hospital setting:
from acute care to outcomes management. Med
Care 30: 57-73,1992.
Nienaber I and Wildavsky A, The budgeting and
evaluation of Federal Programs or Money doesn't
grow on trees. Basic Books, New York, 1973.
Duckett S J, Health policy issues in the 1990s.
Money, Measurement and Movement. The Quarterly 28: 6-11,1995.