CHILDREN AND FAMILIES
EDUCATION AND THE ARTS
The RAND Corporation is a nonprofit institution that helps improve policy and
decisionmaking through research and analysis.
ENERGY AND ENVIRONMENT
HEALTH AND HEALTH CARE
INFRASTRUCTURE AND
TRANSPORTATION
This electronic document was made available from www.rand.org as a public
service of the RAND Corporation.
INTERNATIONAL AFFAIRS
LAW AND BUSINESS
NATIONAL SECURITY
Skip all front matter: Jump to Page 16
POPULATION AND AGING
PUBLIC SAFETY
SCIENCE AND TECHNOLOGY
TERRORISM AND
HOMELAND SECURITY
Support RAND
Browse Reports & Bookstore
Make a charitable contribution
For More Information
Visit RAND at www.rand.org
Explore RAND Europe
View document details
Limited Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law as indicated in a notice appearing later in this
work. This electronic representation of RAND intellectual property is provided for non-commercial use only. Unauthorized
posting of RAND electronic documents to a non-RAND Web site is prohibited. RAND electronic documents are protected
under copyright law. Permission is required from RAND to reproduce, or reuse in another form, any of our research
documents for commercial use. For information on reprint and linking permissions, please see RAND Permissions.
This product is part of the RAND Corporation technical report series. Reports may
include research findings on a specific topic that is limited in scope; present discussions
of the methodology employed in research; provide literature reviews, survey instruments, modeling exercises, guidelines for practitioners and research professionals, and
supporting documentation; or deliver preliminary findings. All RAND reports undergo rigorous peer review to ensure that they meet high standards for research quality
and objectivity.
How health systems make
available information on
service providers
Experience in seven countries
Mirella Cacace, Stefanie Ettelt, Laura Brereton,
Janice Pedersen, Ellen Nolte
Prepared for the Department of Health within the PRP project
“An ‘On-call’ Facility for International Healthcare Comparisons”
EUROPE
The research described in this report was prepared for the Department of Health within the
PRP project "An 'On-call' Facility for International Healthcare Comparisons".
The RAND Corporation is a nonprofit research organization providing objective analysis
and effective solutions that address the challenges facing the public and private sectors
around the world. RAND’s publications do not necessarily reflect the opinions of its
research clients and sponsors.
R® is a registered trademark.
© Copyright 2011 by RAND and London School of Hygiene and Tropical Medicine
All rights reserved. No part of this book may be reproduced in any form by any electronic
or mechanical means (including photocopying, recording, or information storage and
retrieval) without permission in writing from the copyright holders.
Published 2011 by the RAND Corporation
1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1200 South Hayes Street, Arlington, VA 22202-5050
4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213-2665
Westbrook Centre, Milton Road, Cambridge CB4 1YG, United Kingdom
RAND URL: http://www.rand.org
RAND Europe URL: http://www.rand.org/randeurope
To order RAND documents or to obtain additional information, contact
Distribution Services: Telephone: (310) 451-7002;
Fax: (310) 451-6915; Email: order@rand.org
Executive summary
There is growing interest in the public release of information on the performance of
healthcare providers as a means to improve the quality of care and promote transparency
and accountability. Countries have made considerable investments in creating systems to
make available such information. However, there is lack of systematic comparative
assessment of such systems. Thus, there is a need to better understand the drivers behind
and leaders of initiatives and their aims and objectives; the nature, frequency and source of
information provided; their availability to and usability by the public, and, finally, whether
and how these systems might have an impact on the quality of care.
This report aims to contribute to filling these gaps by reviewing information systems
reporting on the quality or performance of providers of healthcare (‘quality information
systems’) in seven countries: Denmark, England, Germany, Italy, the Netherlands, Sweden
and the United States. Data collection involved a review of the published and grey
literature, complemented by information provided by key informants in the selected
countries using a detailed questionnaire.
The review highlights that as the policy context for quality reporting in countries varies, so
also does the nature and scope of quality information systems within and between
countries. Systems often pursue multiple aims and objectives, which typically are:
•
to support patient choice
•
to influence provider behaviour to enhance the quality of care
•
to strengthen transparency of the provider–commissioner relationship and the
healthcare system as a whole
•
to hold healthcare providers and commissioners to account for the quality of care
they provide and the purchasing decisions they make.
However, system objectives are not always well defined and documented, which makes it
difficult to assess whether, and how well, systems achieve their intended aims.
Similarly, quality information systems typically address a number of audiences, including
patients (or respectively the general public before receiving services and becoming
patients), commissioners, purchasers and regulators. Ultimately, however, the main users
of information systems are the providers themselves as the publication of information
provides an incentive for improving the quality of care. Most systems have the stated aim
to make information on provider performance available to patients thus supporting choice
of provider. Yet there are examples of systems that, in their present format, seem to be ill-
xiii
International Healthcare Comparisons
RAND Europe
suited to meet patients’ information needs as the nature of information collected and their
presentation might require specialist knowledge in order to be usable and useful. This
reflects, to a certain extent, the origins of several of the information systems reviewed here,
which have developed gradually, often through a combination of some form of ‘bottom
up’ initiatives and ‘top down’ regulation. In Sweden, for example, the national quality
registries were initially developed by clinicians and professional networks, while
requirements for quality reporting for hospitals in Germany were defined by the legislator.
The Weisse Liste in Germany is developed by a private foundation in collaboration with
associations of patient and consumer organisations.
As quality information systems vary in their objectives and target audiences, so does the
nature and scope of data provided. In most countries, several systems co-exist, frequently,
although not always, serving slightly different purposes. The scope ranges from those
whose sole purpose it is to inform about one aspect of performance, typically waiting times
(e.g. Ventetider in Denmark and Väntetider i Vården in Sweden), to systems that give users
access to detailed information about the quality of care delivered by individual providers.
Where detailed information on provider performance is available, it tends to focus on
hospitals with only a minority of systems also providing information on the quality of
primary and/or preventive care (e.g. NHS Choices in England, Zichtbare Zorg in the
Netherlands, and Öppna jämförelser in Sweden). Furthermore, only a few systems provide
patient ratings based on systematic surveys, including patients’ experience (e.g. NHS
Choices, Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)
in the United States (US), and KiesBeter in the Netherlands), patient satisfaction
(Qualitätskliniken in Germany) or provide an opportunity for direct feedback from
patients (e.g. the ongoing pilot project by AOK-Gesundheitsnavigator in Germany).
Governments and health authorities play a key role in operating and funding quality
information systems. Where healthcare governance has been devolved to regions or
countries, quality reporting typically also involves regional governments. Furthermore,
professional groups are involved, for example by operating the databases on clinical quality
in Denmark or the national quality registries in Sweden. Dr Foster Intelligence in England
has originated in the private for profit sector and is operated as a public–private
partnership.
Although countries reviewed here have developed a variety of quality information systems
for a range of different audiences, the evidence about the impact of these systems is fairly
limited. Systematic evaluative work is mainly available for the US, suggesting that public
reporting can improve the quality of care in some areas. However, the effects of
information systems on quality of care are difficult to isolate as these frequently are part of
broader quality initiatives. Also, the requirements for increased reporting necessitate
improved documentation, which may explain some of the (initially) observed effects of
quality reporting on improvements.
Quality improvement can result from selection and/or as a consequence of provider
behaviour change. These mechanisms are however difficult to disentangle. Findings from
six of the seven countries reviewed here suggest that patients rarely search out information
about healthcare providers. Low uptake of published information suggests that the
available data does not sufficiently meet patients’ information needs, also highlighting lack
xiv
RAND Europe
Executive summary
of systematic involvement of patients in the development of quality information systems
(with NHS Choices in England being a notable exception). Growing evidence suggests
that other user groups, such as managers and providers, indeed use comparative
information to improve care where public reporting occurred. It is important to note that
information systems can encourage changes in provider behaviour even if the public makes
limited use of them. This supports the notion of an association between public reporting
and quality improvement, which operates largely through provider behaviour change.
More systematic research is needed, however, to understand the underlying mechanisms.
Based on the evidence reviewed here, we identify a number of considerations for the design
of quality information systems. Thus, the purpose of publishing information on the quality
of service providers needs to be well defined, as does the primary target groups that are
supposed to be using the system. Where patients or the public are the main intended users,
information systems must ensure access and usability so as to minimise the possibility of
unequal access to web-based resources. Stakeholder involvement can improve acceptance of
information systems and therefore potentially increase its use. It will therefore be necessary
to decide early on which stakeholders to include and at what stage of the process.
Several findings highlighted the necessity to improve and ensure the high quality of data
provided. Mechanisms for ensuring validity, reliability and consistency of reported data
should go hand in hand with safeguarding the completeness of data. Mandatory
approaches can be effective, but also incentive-driven schemes, for example ‘pay-forreporting’ (Hospital Compare, US) can be successful in achieving high participation of
healthcare providers. As there might be an incentive for some providers to manipulate data,
control mechanisms are indicated. Finally, and perhaps most challenging, there is a need
for systematic evaluation of information systems to assess the cost–benefit relation of
information systems as well as their overall value for money. This includes rigorous
measurement of potential impacts on provider behaviour and detailed understanding of
the mechanisms at work. Further learning from international experience can be helpful in
achieving these aims.
xv