DRAFT for WDC WS on Nov. 12, 2010
This project is funded by the European Union within the framework of the Pilot
Project on Transatlantic Methods for Handling Global Challenges in the European
Union and the United States. The general objective of the Pilot Project, created
through a European Parliament initiative, is to promote mutual understanding
and learning among EU and US policy researchers and policymakers on a number
of challenges with a global dimension.
ARGOS PB Benefits, 19 October 2010
1 of 27
DRAFT for WDC WS on Nov. 12, 2010
“Transatlantic Methods for Handling Global Challenges in the
European Union and the United States”
Transatlantic Observatory for Meeting Global Health
Policy Challenges through ICT-Enabled Solutions
1
ARGOS eHealth
Draft Policy Brief
on
Policy needs and options
for a common approach
towards measuring
Adoption, usage and benefits of eHealth
Gesellschaft für Kommunikations- und Technologieforschung mbH,
Bonn, Germany
American Medical Informatics Association (AMIA),
Washington, DC, USA
Partners Healthcare System, Harvard Medical School
Boston, MA, USA
Bonn/Washington, DC/Boston, MA
ARGOS is funded by the European Union. This is one of the
seven projects under the Pilot Project ‘Transatlantic Methods for Handling Global Challenges in the European Union
and the United States’.
In Greek mythology, Argus Panoptes (Ἄργος Πανόπτης) or Argos was guardian of the heifer-nymph Io and son of Arestor. He
was a primordial giant whose epithet "Panoptes" (i.e. "all-seeing") led to him being described as having multiple (often a hundred)
eyes. The name therefore fits with such kind of transatlantic eHealth observatory.
1
ARGOS PB Benefits, 19 October 2010
2 of 27
DRAFT for WDC WS on Nov. 12, 2010
Preface:
What is ARGOS about: ...
Purpose of this paper:
A key output of ARGOS will be three Policy Briefs. They will concisely analyse and summarise
project results on the three topics of the project on policy needs and options regarding
• Interoperability in eHealth and Certification of Electronic Health Record systems (EHRs);
• Measuring adoption, usage and benefits of eHealth solutions;
• Modelling and simulation of human physiology and diseases - Virtual Physiological Human
(VPH)
and provide recommendations for developing together and aligning trans-Atlantic eHealth policy
strategies and cooperation in these three topical fields, including setting concrete goals, proposing adoption measures and processes to be followed.
On measurement and benefits, the ARGOS Proposal formulated these objectives:
“Better understanding the benefits (and costs), i.e. their overall clinical and socio-economic impact, identifying challenges and success factors, as well as measuring and globally benchmarking the concrete usage of eHealth solutions are key policy priorities not only of national governments, but also of institutions like WHO or OECD. ...[The objectives will be] to analyse current
policy thinking, compare challenges and outcomes and draft a roadmap towards developing
advanced global approaches for these issues.”
Overall length: 15 - 25 pages, A 5 format or slightly larger
Genesis:
This draft reflects the initial issues as discussed in Barcelona during the WoHIT Conference in
March, the OECD workshop in Barcelona - with contributions from ARGOS - on measuring
eHealth, results of the eHealth Strategies workshop in Brussels in September, 2010, and discussions with stakeholder communities (professionals, citizen groups, media, ...), at scientific
conferences like the Annual Global Health Technology Assessment international (HTAi) Conference, June 2010, Dublin, Ireland, and with professional and scientific experts in the context of
various other meetings and eHealth activities.
KAS, EMP, V03, 16 October 2010
ARGOS PB Benefits, 19 October 2010
3 of 27
DRAFT for WDC WS on Nov. 12, 2010
Key (concise) Policy Messages (policy issues and context; policy measures)
to be done later
Executive Summary
to be done later
ARGOS PB Benefits, 19 October 2010
4 of 27
DRAFT for WDC WS on Nov. 12, 2010
Policy Brief
Contents
1
2
Analysis framework .......................................................................................... 6
What are the issues? ........................................................................................ 6
2.1
2.2
2.3
3
Key issues - a European policy perspective
Key issues - a USA perspective
European Union evidence
4.1.1
4.1.2
4.1.3
4.2
14
EU evidence at the macro-level: diffusion and usage ...........................................14
EU evidence at the micro-level: benefits assessment ..........................................18
Activities of EU Member States ............................................................................19
United States of America evidence
4.2.1
4.2.2
20
USA evidence at the macro-level: diffusion and usage ........................................20
USA evidence at the micro-level: benefits assessment ........................................20
Research challenges and methodological issues encountered ................. 21
5.1
5.2
6
7
9
11
Past and present realisation activities and results ...................................... 14
4.1
5
6
7
8
Policies, strategies, approaches presently pursued ...................................... 9
3.1
3.2
4
eHealth and the quest for a new model of healthcare
Evidence needed at the macro-level: diffusion and usage
Evidence needed at the micro-level: benefits assessment
Gathering evidence at the macro-level: diffusion and usage
Assessing benefits and costs at the micro-level
21
22
Experiences, lessons learned, needs for future actions ............................. 25
Recommendations for transatlantic exchange, policy harmonisation,
collaborative initiatives .................................................................................. 26
ARGOS PB Benefits, 19 October 2010
5 of 27
DRAFT for WDC WS on Nov. 12, 2010
1 Analysis framework
When analysing the respective national health policy and related eHealth strategy perspectives
regarding measuring of adoption, usage and benefits of eHealth solutions, the following analysis
framework will be applied. Table 1 depicts a linear policy development and implementation process. Of course, in reality such processes are complex, recursive, disruptive and multi-level in
character. Nevertheless, for analytical purposes and as sketched in Figure 1 below, it is useful
to regard them as a linear process involving usually these basic steps:
TABLE 1: BASIC STEPS OF THE POLICY LIFE CYCLE:
1) Identification of the issues guiding high level policy goals and objectives (Problem definition)
2) Mustering support for the policy (Agenda setting)
3) Drafting and agreeing on a policy document including the identification of a strategy to
realise it, the implementation process, measures and resources needed (Adoption)
4) Initiating new or using established organisational structures to implement it and executing the measures foreseen (Implementation)
5) Controlling and evaluating the timely policy realisation, outcomes and performance
(Evaluation)
6) Feedback of results into adjusted or new policy development (Feedback).
FIGURE 1: THE POLICY LIFE CYCLE
Problem
Definition
Agenda
Setting
Policy
adoption
Implementation
Evaluation
Feedback
Source: © empirica 2008, based on Brewer and Jones2
In the framework of this work, it will not be possible to cover in detail all the steps involved in the
policy life-cycle. Especially the problem definition and agenda setting part are often hidden from
the public and would require knowledge of internal policy processes in ministries and the parliament.3 The most visible parts of this cycle are the policy-adoption and implementation stages.
Measurement of outcomes and evaluation of the impact of a policy in the field of Healthcare and
eHealth, which is still a rather rare occurrence in most countries, are key concerns of this policy
brief.
2 What are the issues?
2.1
eHealth and the quest for a new model of healthcare
Health policy-makers are challenged by insufficient human and capital capacity to meet demand
for services, and to prioritise finite budgets. Demographic change, rising incidence of chronic
2
Brewer, G. D. and P. DeLeon (1983). The foundations of policy analysis, Dorsey Press Homewood, Illinois. See also Jones, C.
O. (1984). An introduction to the study of public policy, Houghton Mifflin Harcourt P.
3
Jann, W. and K. Wegrich (2003). "Phasenmodelle und Politikprozesse: der policy cycle." Lehrbuch der Politikfeldanalyse.
München: 71-103.
ARGOS PB Benefits, 19 October 2010
6 of 27
DRAFT for WDC WS on Nov. 12, 2010
disease among young and old4, and unmet needs for more personalised care are driving the
pressure to introduce a new model of healthcare, a redesigned smart health system. This quest
for integrated care close to the home, better involvement of patients and improved efficiency are
not new.5
But the context has changed: for the first time the enabling capacities of advanced information
and communications technology (ICT) solutions can now enable our health systems to focus
indeed on realising this smart health system paradigm.6 ICT systems for clinical decision support7, chronic disease and population management8, shared access to patient data and coordination of clinical pathways9, for public health risk and epidemiological surveillance, for
knowledge generation from structured data and better training, and its translation into health
service routines are within reach.10
Fostering eHealth systems has been among the earliest research topics supported by the EC’s
Framework Programmes.11 eHealth denotes all applications of ICT supporting and interconnecting health service processes and health system actors, both at the local level and remotely.
But in spite of a 40 year struggle to mainstream eHealth solutions, their diffusion into many potential application fields is still meagre, and evidence on the various types of benefits forecast
and heralded as a new panacea for health policy woes is limited.
Despite notable successes in a few settings12, regrettably for all actors involved, be they policy
makers and administrators at the European or national/regional level, be they health service
providers, taxpaying citizens or other stakeholders, are left wondering whether this expensive
and long-term commitment is paying off.
2.2
Evidence needed at the macro-level: diffusion and usage
For eHealth to deliver the expected benefits, it requires on the one hand policies to foster actual
deployment (including investments, improvement of interoperability, reducing legal barriers and
others) and on the other hand measures of deployment and the effectiveness of current and
future take-up. To monitor and control the development, implementation and success/impact of
eHealth strategies and implementation measures, it is necessary to measure the diffusion and
usage of such applications across health service actors and organisations. At the macro-level of
the overall health system, this necessitates a clear understanding of the role of measurements /
indicators and benchmarking in policy and their use for policy making.
A general approach to the use of quantitative indicators and benchmarking for policy is outlined
in Figure 2. An indicator system requires as a starting point the definition of the area of activity
in broad terms in form of policy challenges. As such, it will be too broad to be directly measured and therefore requires a translation of these challenges into more concrete concerns and
policy objectives, which can be used as a basis for the specification of policy actions. Measurable results of policy action, i.e. the specification of policy targets in terms of outputs, outcomes, and impacts are the next step to be conducted. From these, indicators can be derived
(through “operationalisation”), which can then be used for a policy evaluation. This typically
means comparison of achievements against set targets. These can be referred back to policy
objectives and policy challenges.
4
Neal Halfon; Paul Newacheck (2010). Evolving Notions of Childhood Chronic Illness. JAMA 303(7), pp. 665-666.
Actually, they have been public health policy desiderata for more than 50 years: Leroy E. Burney (1954). Community OrganizationAn Effective Tool. American Journal of Public Health, Vol. 44 (1954), pp. 1-6
6
Karl Stroetmann et al. eHealth is Worth it - The economic benefits of implemented eHealth solutions at ten European sites. Luxembourg: Office for Official Publications of the European Communities, 2006
7
Bates NEJM
8
DM ref
9
Grant R DM
10
Alexander Dobrev et al. . Interoperable eHealth is Worth it - Securing Benefits from Electronic Health Records and ePrescribing.
Luxembourg: Office for Official Publications of the European Communities, 2010
11
Ilias Iakovidis, Octavian Purcarea (2008). eHealth in Europe: from Vision to Reality. In: B. Blobel et al. (eds.). eHealth: Combining
Health Telematics, Telemedicine, Biomedical Engineering and Bioinformatics to the Edge. Studies in Health Technology and Informatics Vol.134, pp.:163-8.
12
Chaudry
5
ARGOS PB Benefits, 19 October 2010
7 of 27
DRAFT for WDC WS on Nov. 12, 2010
The resulting indicators can then be defined on different levels:13
• policy outputs, which are comparatively easy to specify using operational indicators;
• policy outcomes, which are more difficult to operationalise as they relate to the underlying
policy objectives; and
• policy impacts, which are most difficult to specify because measurable effects typically only
occur in the long-term, and because the relationship between individual effects and causes
are often very difficult to disentangle from each other.
•
FIGURE 2: THE USE OF INDICATORS AND BENCHMARKING FOR POLICY-MAKING
Policy
challenge
• Broad area for
activity and
investigation
• The "mission
statement"
• Too broad to be
directly measured
Policy
Policy
Policy
targets
targets
targets
Policy
objectives
• Translate
challenges into
more concrete
concerns
• Basis for
specification of
policy actions
Implementation
Policy
evaluation
Indicators
Indicators
Indicators
• The measurable
results of policy
actions
• Outputs  outcomes
 impacts
• Benchmarking:
What is the current
status?
• Benchmarking:
Has there been
progress towards
meeting objectives?
• In case of nonachievement,
assessment of
reasons
• Benchlearning:
What can be
learned from
others’ experience?
Source: © empirica 2007
Benchmarking diffusion and usage will play a central role in monitoring progress in exploiting
opportunities for economic growth and jobs by promoting an open and competitive digital economy. A mix of indicators is needed to measure the different aspects of the objectives that are to
be achieved. Policy emphasis now focuses more on complex issues of impact and usage of
technologies in the wider economy and benchmarking must become more sophisticated. It is
necessary to build on existing work and continue to track some indicators consistently but monitoring of progress now requires indicators that are flexible and timely.
2.3
Evidence needed at the micro-level: effective use and benefits assessment
However, availability of such macro-level data covers only one, albeit key, aspect of overall policy monitoring and assessment. The other side is to assess, at the micro-level of individual
stakeholders and local/regional health systems, the concrete benefits (and also the costs) associated with investing in and implementing eHealth solutions and systems. These benefits arise
directly from effective use of ICT (or as often described in the US policy framework “meaningful
use”).
Such micro-level socio-economic evaluations of eHealth systems and solutions can offer significant benefits to health policy decision-makers. These include providing evidence-based information to help decision-makers in eHealth to identify the:
13
cite Shortelle S. Program Evaluation in Healthcare
ARGOS PB Benefits, 19 October 2010
8 of 27
DRAFT for WDC WS on Nov. 12, 2010
• type and scope of benefits for patients, carers, healthcare professionals, healthcare provider entities, other stakeholders, and any public good to consider when making decisions to invest
in e-health applications
• de facto beneficiaries and cost bearers of eHealth
• number and type of users, their levels of utilisation, and so required and future capacity
• impact on meeting healthcare demand
• enabling changes to healthcare models and regional networks that become possible
• benefits for clinical audit and governance
• scale of the critical investments needed in training and change management
• potential changes to the costs of providing healthcare and the potential to generate additional income
• impact on the future ICT infrastructure needed to support eHealth
• impact on third party payers.
A major reason for the relatively slow progress in eHealth deployment is the lack of awareness
of and empirical evidence on benefits. There is a need to disseminate existing best practices
and the associated benefits as well as to examine existing methodologies of assessment and to
propose common approaches to proving benefits of interoperable solutions using coherent and
quantitative (scientific) methods.”14 Secondarily, healthcare payment models may impede an
efficient market for healthcare goods and services supported by ICT. Under certain reimbursement models, the stakeholder investing in ICT may not be the beneficiary if benefits accrue to a
different stakeholder.15
A rigorous evaluation framework for identifying and measuring the benefits and costs from
eHealth investments and deployment is needed for two reasons:
1) in order to demonstrate the potential, including the points of high-level impact, and
2) in order to analyse incentives structures and thus identify fields of required policy initiatives
and action.
Success factors and lessons for future initiatives are a valuable by-product of micro-level benefit
assessment activities.
Rigorous evaluation findings that show the extent to which aspects of eHealth applications have
not succeeded are also very valuable to decision-makers, because they will reveal the risks that
they are facing. Taken together, these two main types of evaluation findings will help to remove
inhibitors, show best practice, support future investment decisions and create enablers for
change in eHealth.
3 Policies, strategies, approaches presently pursued
3.1
Key issues - a European policy perspective
Measurement of adoption, usage and benefits of eHealth solutions has always been high on the
EU agenda. This was clearly stated in the European Commission eHealth Action Plan of 2004.
Under Issue 3: Working together and monitoring practice, topic Disseminating best practices,
the European Commission recommended that “eHealth should be supported by the widespread
dissemination of best practices. These should include ... assessments of cost benefits ...” and
asked “to develop a strong evidence basis for the case for e-Health”. Furthermore, with respect
to the topic of Benchmarking, it was voiced that “progress also needs to be measured. ...This
means assessing and quantifying the added value that eHealth is expected to deliver. ...These
measures should be accompanied by proper monitoring of e-Health’s impact on health and
14
Tender specifications: Study on Economic Impact of Interoperable Electronic Health Records and ePrescription in Europe,
SMART N°2007/0048 – OJ 2007/S 100-122426, p.5
15
Value of ACPOE, CITL
ARGOS PB Benefits, 19 October 2010
9 of 27
DRAFT for WDC WS on Nov. 12, 2010
health care in the Community. All stakeholders should have a role in this process which should
feed in to further improvements in e-Health systems and services.”
“During the period 2004-2010, every two years, the European Commission will publish a study
on the state of the art in deployment, examples of best practices, and the associated benefits of
e-Health. By the start of 2005, Member States, in collaboration with the European Commission,
should agree on an overall approach to benchmarking in order to assess the quantitative, including economic, and qualitative impacts of e-Health.”16
These recommendations were also recognised recently in the European Council Conclusions
on Safe and efficient healthcare through eHealth17 where EU Member states committed to cooperation on eHealth in view of maximising the benefits for their own patients. It also welcomes
the collaboration between a number of Member States in the epSOS large scale pilot project18,
the €22m initiative jointly funded by the Commission and 12 Member States, which seeks to
develop cross-border interoperability of summaries of electronic health records and ePrescriptions, where impact assessment will play a key role. It calls upon Member States to create a
high level governance group to address the issue of interoperability. It also calls upon the
Commission to “organise an evaluation, at the appropriate intervals, of the health benefits and
cost-effectiveness of the use of different eHealth services, building on knowledge accumulated
at EU and national levels.”
eHealth monitoring and benchmarking need to be seen in the context of the objectives of the
i2010 strategic framework19. i2010 was initiated by the European Commission in 2005 to create
a European Information Society for growth and employment and formed part of the wider partnership for growth and jobs set out by the European Council in the spring of 2005.
In the Extended Impact Assessment of the i2010 Initiative it was already mentioned that “progress with connectivity has been rapid and the policy priority has switched to usage of ICT and
their impact on businesses, governments and citizens. In addition, it is important for benchmarking to show market developments ... A core policy aim would be to promote advanced services,
and benchmarking should then include a mechanism to identify and monitor market developments.”20
Later on it reads on the i2010 website: “Comparing the development of the information society
in the Member States on the basis of certain indicators as well as best practice is an important
part of i2010 assessment. Such benchmarking is carried out on the basis of statistical surveys
and studies. The results are available in the benchmarking section and every year they are reviewed in the i2010 Annual Report.”21
In the i2010 Benchmarking Framework22, eHealth is only referred to very briefly in the following
statement taken directly from the Framework document: “Finally, public services beyond eGovernment (i.e. services of public interest not necessarily provided by public bodies, such as
health or education) should also be covered possibly on a case study approach. In the case of
eHealth monitoring should be done with indicators developed in consultation with health spe-
16
Commission of the European Communities - COM (2004) 356: Communication from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions: e-Health - making health care
better for European citizens: An action plan for a European e-Health Area, Brussels, 2004-04-30.
17
Council conclusions of 1 December 2009 on a safe and efficient healthcare through eHealth. Official Journal of the European
Union (2009/C 302/06), http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2009:302:0012:0014:EN:PDF (2980th Employment, Social Policy, Health and Consumer Affairs Council meeting, Brussels, 1 December 2009)
18
European Patients Smart Open Services – epSOS, www.epSOS.eu
19
COMMISSION OF THE EUROPEAN COMMUNITIES. COMMUNICATION FROM THE COMMISSION TO THE COUNCIL,
THE EUROPEAN PARLIAMENT, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE
REGIONS: “i2010 – A European Information Society for growth and employment” {SEC(2005) 717}. Brussels, 1.6.2005, COM(2005)
229 final
20
COMMISSION STAFF WORKING PAPER - COMMUNICATION FROM THE COMMISSION “i2010 – A European Information
Society for growth and employment”. EXTENDED IMPACT ASSESSMENT {COM(2005) 229 final - Brussels, 01.06.2005,
SEC(2005) 717/2, p. 14.
21
http://ec.europa.eu/information_society/eeurope/i2010/measuring_progress/index_en.htm
22
i2010
High
Level
Group:
i2010
Benchmarking
Framework,
April
2006:
http://ec.europa.eu/information_society/eeurope/i2010/benchmarking/index_en.htm
ARGOS PB Benefits, 19 October 2010
10 of 27
DRAFT for WDC WS on Nov. 12, 2010
cialists, as agreed at the first workshop. Monitoring of wider public services will contribute to
benchmarking of quality of life”.
As concerns policy implementation of the above recommendations in practical EU research policy and promotion, measuring and monitoring usage, diffusion, and benefits of eHealth solutions
has increasingly become a prerequisite for research proposals of the 7th Framework Programme for EU research (FP7). For generating evidence of scientific, technical, commercial,
social, or environmental impacts, recently, assessment and evaluation approaches must be
convincingly self-administered to projects. In order to ensure the maximisation of tangible clinical as well health system/societal benefits, quantitative indicators of added value and potential
impact are meant to demonstrate and also support the use of foreground, that is, exploitation of
project outputs, beyond a project’s life-cycle. Assessment indicators ought to be formatively
employed to validate and evaluate eHealth solutions and related research in monetary terms,
with respect to individual patients, the overall healthcare organisation, and at the public health
level.
3.2
Key issues - a USA perspective





Current policy framework overview
Assessment activities: Meaningful Use
Related concerns: HIE
Moving toward Phase 2 and 3 of MU
Sustaining HIT (ICT) adoption process and effective use
Coordinated US health Information technology policy formulation began with the creation of the
Office of the National Coordinator for Health Information Technology (HIT) by an Executive Order from President Bush in 2004.23 Prior to this act, HIT related policy issues arose in a largely
uncoordinated process across public and private sectors. In 1998 a milestone occurred when
the National Committee on Vital and Health Statistics (NCVHS -- a federal advisory committee
composed of private sector experts), reported that the nation’s information infrastructure could
be an essential tool for promoting the health of US citizens in its seminal concept paper, “Assuring a Health Dimension for the National Information Infrastructure.”24 Since the time of the
NCVHS report, other initiatives have helped to further define the best approach to apply information and communication technologies to the health sector. In 2002, the Markle Foundation (a
private US foundation) organized a public-private collaborative, Connecting for Health, which
brought together leaders from government, industry, and health care, and consumer advocates
to improve patient care by promoting standards for electronic medical information. A year later,
the collaboration of more than 100 public and private stakeholders in Connecting for Health
achieved consensus on an initial set of health care data standards, and commitment for their
adoption from a wide variety of national health care leaders25.
Similar activities were underway in the US federal sector. In March 2003, the Consolidated
Health Informatics (CHI) initiative involving the US Department of Health and Human Services,
the Departments of Defense (DoD), and Veterans Affairs (VA), announced together uniform
standards for the electronic exchange of clinical health information to be adopted across the
federal health care enterprise. These standards facilitate information exchange, with privacy
and security protections, to make it easier for health care providers to share relevant patient
information and for public health professionals to identify emerging public health threats.26
23
Executive Order number, date
get cite
25
get cite
26
get cite
24
ARGOS PB Benefits, 19 October 2010
11 of 27
DRAFT for WDC WS on Nov. 12, 2010
A very significant related HIT policy initiative resulted from efforts to improve drug safety, coverage, and utilization in the US. At the end of 2003, President Bush signed into law the Medicare
Prescription Drug Improvement and Modernization Act (MMA) of 2003. Among other new initiatives, the law includes important provisions for HIT. MMA requires the Centers for Medicare and
Medicaid Services (CMS) to develop standards for electronic prescribing, which will be a first
step toward the widespread use of electronic health records (EHR). In addition, the MMA required the establishment of a Commission on Systemic Interoperability to provide a road map
for interoperability standards.27
In April 2004, President Bush issued Executive Order 13335 calling for widespread adoption of
interoperable EHRs within 10 years, and established the position of National Coordinator for
Health Information Technology. EO13335 charged the National Coordinator with developing,
maintaining, and directing:
“ … the implementation of a strategic plan to guide the nationwide implementation of interoperable health information technology in both the public and private health care sectors
that will reduce medical errors, improve quality, and produce greater value for health care
expenditures.”
The first National Coordinator for HIT, Dr. David Brailer, in conjunction with the Secretary of
Health and Human Services Mr. Tommy Thopson produced the first strategic framework for
HIT: The Decade of Health Information Technology: Delivering Consumer-centric and Information-rich Health Care.28 This framework outlined 4 overarching gals and 3 associated strategies for each (Table XXX). These fundamental goals and strategic objectives serve as a foundation for many of the policy development efforts since that time.
Table XXX: The Brailer-Thomson Framework for Strategic Action. (2004)
Informing clinical practice
Providing incentives for EHR adoption
Reducing the risk of EHR investment
Promoting EHR diffusion in rural and underserved areas
Interconnecting clinicians
Enhancing regional collaborations
Developing a national health information network
Coordinating federal health information systems
Personalizing care
Encouraging the use of personal health records
Enhancing informed consumer choice
Promoting the use of telehealth systems
Unifying public health surveillance architectures
Streamlining quality and health status monitoring
Accelerating research and dissemination of
evidence
Improving population health
27
get cite
Brailer DJ, Thompson TG. The Decade of Health Information Technology: Delivering Consumer-centric and Information-rich
Health Care. Framework for Strategic Action. July 21, 2004.
28
ARGOS PB Benefits, 19 October 2010
12 of 27
DRAFT for WDC WS on Nov. 12, 2010
With the leadership provided by the Office of the National Coordinator for Health Information
Technology, efforts began to coordinate both public sector and private sector policy development activities for HIT. The President’s Information Technology Advisory Committee (PITAC) in
June 2004 issued a draft report, “Revolutionizing Health Care Through Information Technology,”
which stated that the overall quality and cost-effectiveness of U.S. health care delivery bear
directly on three top national priorities of national, homeland, and economic security29.
In July 2004, the Markle Foundation Connecting for Health group released a timely report that
details specific actions the public and private sectors can take to accelerate the adoption of information technology in health care. Connecting for Health's “Preliminary Roadmap for Achieving Electronic Connectivity in Healthcare” contains recommendations in three categories: creating a technical framework for connectivity, developing incentives to promote improvements in
health care quality, and engaging the American public by providing information to promote the
benefits of electronic connectivity and to encourage patients and consumers to access their own
health information.30 This consensus statement represents a significant alignment of recommended policy and initiatives across a broad coalition of stakeholders.
IN June 2008, the Office of the National Coordinator released a significant update to the HIT
Strategic Plan.31 The 2008 ONC Strategic Plan has two goals, Patient-focused Health Care and
Population Health, and four objectives under each goal (See Table XXX). The themes of privacy
and security, interoperability, IT adoption, and collaborative governance recur across the goals,
but they apply in very different ways to health care and population health. Achievement of the
eight objectives is tied to measurable outcomes. The Plan articulates 43 strategies that describe
the work needed to achieve each objective. Each strategy is associated with a milestone
against which progress can be assessed, and a set of illustrative actions to implement each
strategy.
Table XXX. The US ONC Coordinated Strategic Plan for Health IT (2008)
Goal 1) Patient-focused health care: Enable Objective 1.1 – Privacy and Security: Facilitate
the transformation to higher quality, more cost- electronic exchange, access, and use of elecefficient, patient-focused health care through tronic health information while protecting the
electronic health information access and use privacy and security of patients’ health inforby care providers, and by patients and their mation
designees.
Objective 1.2 – Interoperability: Enable the
movement of electronic health information to
where and when it is needed to support individual health and care needs
Objective 1.3 – Adoption: Promote nationwide
deployment of electronic health records and
personal health records that put information to
use in support of health and care
Objective 1.4 – Collaborative Governance:
Establish mechanisms for multi-stakeholder
29
30
31
Get cite
get cite
The ONC-Coordinated Federal Health IT Strategic Plan: 2008-2012. June 3, 2008
ARGOS PB Benefits, 19 October 2010
13 of 27
DRAFT for WDC WS on Nov. 12, 2010
Goal 2) Population health: Enable the appropriate, authorized, and timely access and use
of electronic health information to benefit public health, biomedical research, quality improvement, and emergency preparedness.
priority-setting and decision-making to guide
development of the nation’s health IT infrastructure
Objective 2.1 – Privacy and Security: Advance
privacy and security policies, principles, procedures, and protections for information access and use in population health
Objective 2.2 – Interoperability: Enable the
mobility of health information to support population-oriented uses
Objective 2.3 – Adoption: Promote nationwide
adoption of technologies and technical functions that will improve population and individual health
Objective 2.4 – Collaborative Governance:
Establish coordinated organizational processes supporting information use for population
health
The goals, objectives, and strategies of the 2008 Plan portray what must be done, in a coordinated manner distributed across the US public and private sectors, to achieve an interoperable
health IT infrastructure in the US in support of patient-focused health care and population
health. This Plan is primarily federally focused with many of the strategies proposed in the Plan
designed to harmonize activities in the public and private sectors. This approach is designed to
ensure that federal resources allocated to health IT, while supporting the individual and distinct
missions of the Departments, are also positioned to realize maximum benefit for the nation as a
whole.
***Meaningful Use
the most significant HIT related policy event in the US, however, is the passage of the ARRA
legislation by the US Congress, date (American Recovery and Reinvestment Bill), which contains the HITECH Act (Health IT Economic and Clinical Health Act). This legislation includes a
wide array of major policy initiatives directed at stimulating the adoption and “meaningful use” of
health IT.
***get my summary
4 Past and present realisation activities and results
4.1
4.1.1
European Union evidence
EU evidence at the macro-level: diffusion and usage
4.1.1.1 The policy context and measurement framework
The European Union (EU) i2010 Benchmarking Framework of 2006 for measuring the Information Society monitored progress in the 3 pillars of the i2010 Initiative:
– Completing the Single European Information Space
– Strengthening R&D in ICT and the take up of eBusiness
ARGOS PB Benefits, 19 October 2010
14 of 27
DRAFT for WDC WS on Nov. 12, 2010
– Achieving an inclusive European information society
Its broad scope concerned both ICT research and various application domains, and it put particular emphasis on the impact of ICT use. eGovernment, including eHealth issues, were covered by the third pillar.
As of 2011, it will be followed by the Benchmarking Digital Europe 2011 – 2015 framework
which was endorsed by the i2010 High Level Group on 09 November 2009 in Visby, Sweden 32.
It proposes a conceptual framework for the collection of statistics on the information society as
well as a list of core indicators [EU speak for measures] to be used for benchmarking. It provides a context for measurement and suggests new areas for investigation. For the forthcoming
period, a summary list of indicators to be collected routinely has been proposed for the following
fields33:
A) The ICT sector
B) Broadband and connectivity
C) ICT usage by households and individuals
D) ICT usage by enterprises
E) ePublic services
eHealth is to be covered partially in the context of these indicators as well as through ad-hoc
studies on issues for which official statistics are not available. As concerns the former, consumer data on “seeking health information (on injuries, diseases, nutrition)”, “making an appointment with a practitioner” as well as “consulting a practitioner online” will be collected biannually.
The European Commission has been reporting annually on the list of benchmarking indicators
through its i2010 annual report that contains detailed country profiles for the EU, Norway, Iceland, and more recently also Croatia.34 With the advent of 2010, the i2010 Strategy has come
to an end,35 and is followed by a new initiative – the Digital Agenda36 -, which is one of seven
so-called Flagships of the Europe 2020 Strategy37 – the Commission’s proposal on a new economic strategy for the EU.
The Europe 2020 Strategy focuses on “three mutually reinforcing priorities:
• Smart growth: developing an economy based on knowledge and innovation.
• Sustainable growth: promoting a more resource efficient, greener and more competitive
economy.
• Inclusive growth: fostering a high-employment economy delivering social and territorial cohesion.”38
eHealth does not only feature in the mentioned Digital Agenda Flagship, but will also become a
key aspect in the Smart Growth Flagship called “Innovation Union” with various “European Innovation Partnerships” (EIP), where the initial EIP will be on “Active and healthy ageing.”39.
32
i2010 High Level Group: Benchmarking Digital Europe 2011-2015: a conceptual framework, 27 October 2009:
http://ec.europa.eu/information_society/eeurope/i2010/docs/benchmarking/benchmarking_digital_europe_2011-2015.pdf
33
Ibidem; for the full list of indicators see pp. 18-21
34
COMMISSION OF THE EUROPEAN COMMUNITIES. Europe’s Digital Competitiveness Report - Main achievements of the
i2010 strategy 2005-2009. Brussels, 04.08.2009, COM(2009) 390. Fore more details, see the accompanying COMMISSION
STAFF WORKING DOCUMENT. Brussels, 04.08.2009, SEC(2009) 1103 - An overview of all documents can be found at
http://ec.europa.eu/information_society/eeurope/i2010/key_documents/index_en.htm
35
For 2010, see the final report: European Commission. Europe’s Digital Competitiveness Report 2010. Commission staff
working document, Volume 1, Brussels, 17.5.2010, SEC(2010) 627
36
For details, see http://ec.europa.eu/information_society/digital-agenda/index_en.htm
37
EUROPEAN COMMISSION. EUROPE 2020 - A strategy for smart, sustainable and inclusive growth. Brussels, 3.3.2010,
COM(2010) 2020
38
Ibidem, p. 3
39
See European Commission. Europe 2020 Flagship Initiative Innovation Union. Brussels, 6.10.2010, COM(2010) 546 final,
ANNEX III European Innovation Partnerships - Aims and scope of a pilot European Innovation Partnership in the field of active and
healthy ageing, pp. 40-42
ARGOS PB Benefits, 19 October 2010
15 of 27
DRAFT for WDC WS on Nov. 12, 2010
4.1.1.2 Data sources and surveys
Statistical information will continue to be mainly collected through the following sources:
• Eurostat surveys on ICT use by households/individuals and enterprises, plus other official
statistics;
• data on connectivity provided by National Regulatory Authorities;
• ad-hoc studies on different issues for which official statistics are not available;
• regular annual surveys on the availability of online public services.
In the 2011-2015 benchmarking framework, Eurostat - the Statistical Office of the European
Communities40 - surveys represent the main source of statistical information, with additional
data to be provided by the Communication Committee (COCOM).41 The ICT use surveys will
keep the current structure, including core indicators (to be maintained for tracking development
over time) and special modules focusing on different topics each year. When specific policy
needs cannot be covered by official statistics, ad-hoc surveys/studies will be conducted, with
particular attention to the issue of quality and reliability. Member States will be involved in the
process of validation of these statistics in the framework of a specific group on measurement.
4.1.1.3 Specific eHealth issues
There is a range of relevant policy domains so far only partially covered by Eurostat surveys.
These include healthcare and eHealth. Monitoring the use of ICT to support a more efficient
and safer delivery of health services is increasingly regarded as highly important. The Eurostat
survey on ICT use can provide some evidence on the use of the internet for health related purposes, but for a more exhaustive picture, the analysis has to cover also the use of ICT by various health sector actors, where presently only ad-hoc surveys deliver data.42
Recently, the European Commission has launched a series of studies on measuring usage and
diffusion of eHealth. The “eHealth Indicators” (2007-2008)43 study involved a representative
European general practitioners survey in all 27 EU Member States as well as Norway and Switzerland, also including 29 Country Briefs reporting on the situation in the respective country
compared to the rest of Europe. Almost 7,000 primary care physicians where surveyed on their
use of ICT and internet for communication with patients, and between primary and secondary
care and other health system actors.
A Commission study on eHealth benchmarking (2008-2009) provided additional input in terms
of a comprehensive measurement framework and on potentially useful, related measures44. The
study identified and collected quantitative and qualitative evidence of eHealth deployment and
use in the European Union, Norway, Iceland, Canada and the United States - with a particular
focus on measurements specified in the European Union's 2004 eHealth Action Plan.45 Sources
covered include healthcare associations and bodies, international organisations such as OECD
and WHO, IT industry, national statistical institutes and authorities at regional and national level.
The outputs of the study include an online knowledge base with almost 100 eHealth Benchmarking data sources in 31 countries with a total number of around 4,500 eHealth indicators46.
Late in 2009, the EC has launched a third study in this series, this time on the use of ICT and
eHealth solutions by medical staff in hospitals, both for administrative and clinical processes.
The overriding aim is to obtain a better understanding of the current level of digitalisation, the
actual usage (compared to availability as surveyed in other studies), and the perceived benefits
40
http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/
Through the COCOM - Communications Committee - National Regulatory Authorities collect data to supplement Eurostat
activities, e.g. in the field of broadband penetration .
42
http://ec.europa.eu/information_society/eeurope/i2010/docs/benchmarking/gp_survey_final_report.pdf
43
http://ehealth-indicators.eu/
44
http://ec.europa.eu/information_society/eeurope/i2010/docs/benchmarking/ehealth_ii_bench_final_report.pdf
45
Commission of the European Communities: e-Health - making health care better for European citizens: An action plan for a
European e-Health Area. COM (2004) 356, Brussels, 2004-04-30
46
see http://kb.ehealth-benchmarking.eu/search.do#
41
ARGOS PB Benefits, 19 October 2010
16 of 27
DRAFT for WDC WS on Nov. 12, 2010
from and effectiveness of eHealth solutions.47 Results should become available at the beginning of 2011.
4.1.1.4 OECD activities48
On July 6-7, 2009, at their 5th Session, OECD Health Committee delegates had already expressed their support for further work on the application of ICT solutions in two areas:
• Option 1: development of a standardized survey for international comparison of adoption
and use of ICTs in the Health Sector
• Option 2: review the use of EHRs for data collection for population-based quality indicators
These projects had not been anticipated in the 2009/2010 OECD Programme of Work and
Budget, but they were being put forward because of strong demand from experts in two subgroups (the National Experts on ICTs in Health Systems, and the OECD Health Care Quality
Indicator (HCQI) group).
To avoid double work and, on the other hand, take advantage of potential synergies, OECD and
the ARGOS Initiative discussed their respective briefs and plans, and agreed to cooperate on
option 1 mentioned above, with the lead to be allocated to OECD. Involvement of WHO was
highly welcomed.
An initial common workshop was the OECD special session towards international cooperation
in measuring the adoption and usage of IT in the health sector. It took place on March 16, 2010,
at the World of Health IT Conference in Barcelona, Spain. The meeting was devoted to further
scoping a proposal for international work to measure adoption and use of ICTs in the health
sector (OECD paper DELSA/HEA(2009)7) and, specifically, to discuss, also in light of the ARGOS brief,
• shared needs;
• key lessons from work to date;
• a strategy to employ common measures while preserving continuity of each nation’s measurements; and an action plan to develop shared measures and approaches to meet highpriority policy needs.
Delegates from Member States confirmed their countries’ interest in an international benchmarking initiative and welcomed the attempts to develop a set of common indicators and definitions. It was noted that the impacts of ICT implementation in the health sector are still very
much an open question and reliable and credible measurements of adoption and use would be
a useful starting point.
On this issue, the following observations were made:
• Comparable reliable measurements can be a powerful motivating tool (what you do not
measure will not get done) and are necessary to understand the value of programmes and
initiatives.
• Benchmarking provides a useful means to identify countries which may be facing similar
implementation challenges, compare and understand possible success factors, and understand where improvements are most significant (we need to measure not just what we are
doing but how we are doing it).
• A joint international effort can help countries identify what can be measured, how to develop
meaningful comparators and the ways to collect data (there is already value added in agreeing on what to compare).
• Measurement can help address resource allocation and impacts questions. (Have we well
invested our money? Are our healthcare professionals using ICT tools? For what purpose?
Are we improving the quality and efficiency of our health systems? )
47
http://ec.europa.eu/information_society/eeurope/i2010/studies/
The following is based on the OECD DRAFT SUMMARY OF MAIN POINTS paper about the OECD SPECIAL SESSION
TOWARDS INTERNATIONAL COOPERATION IN MEASURING ADOPTION AND USE OF INFORMATION TECHNOLOGIES IN
THE HEALTH SECTOR, on MARCH 16, 2010, at the WORLD OF HEALTH IT Conference in BARCELONA, Spain, compiled by
Elettra Ronchi and Niek Klazinga.
48
ARGOS PB Benefits, 19 October 2010
17 of 27
DRAFT for WDC WS on Nov. 12, 2010
•
International comparisons should not be used to rank countries but should be seen as a
means to enhance collaboration across countries.
4.1.2 EU evidence at the micro-level: benefits assessment
Measuring the benefits and costs and assessing the socio-economic impact of eHealth solutions has been a key policy concern at the EU level since the start of this century. Commissioned by the EC, the eHealth IMPACT study (2005-2006)49 developed a generic methodology
for the socio-economic evaluation of eHealth applications. It is a context adaptive model, so it
can be applied to a wide diversity of applications, from clinical settings to supply chain solutions,
in the respective health system context. The model is based on the concept of benefit-cost
analysis.
Benefits concern a great variety of detailed measures related to better quality of care, greater
efficiency and/or improved access to health services. Benefits for all potential stakeholders can
be analysed, be they individual persons, organisations, or the health system as a whole/society.
Special attention has been paid to identifying the benefits to, and impact on, citizens. The concept of cost-avoidance is important in identifying benefits. These are the costs estimated for
achieving the ICT-based performance without ICT, which may prove prohibitive.
Cost measures include the initial and continuous eHealth investments, such as those in ICT and
change management, as well as the running costs of eHealth systems and solutions.
Whereas typical publications on benefits (and sometimes also costs) of eHealth are either
based on very limited, subsidised experiments and pilot experience, or concern potential benefits heroically extrapolated from limited, often atypical experience, the eHealth IMPACT study for
the first time looked at a diversity of longer term, routine eHealth (systems) solutions integrated
into standard health service provision. They were, albeit, also atypical in the sense that they
were purposefully selected on the assumption to provide exemplar case studies of how to do it
successfully. For the first time, the empirical ex-post results of the study showed that for a wider
health system context that, given the right approach, context and implementation process, benefits from effective eHealth investments are indeed better quality and improved productivity,
which in turn liberate capacity and enable greater access. But the results also showed that
eHealth system investments are usually longer-term endeavours not delivering net cash savings.
The EHR IMPACT50 study (2008-2009), building on the foundations laid in the eHealth IMPACT
study, investigated the socio-economic impact of interoperable electronic health record (EHR)
and ePrescribing systems in Europe and beyond. Core to the project was a detailed qualitative
analysis of eleven good practice cases in Europe, USA and Israel. Nine of these also underwent a quantitative assessment of their socio-economic impacts. Each case studied represents
a sustained solution in routine operation.
The goals of the EHR IMPACT study required an inductive and adaptive empirical approach.
Two perspectives were applied, the socio-economic, and a narrower, financial one within the
socio-economic. This dual perspective in the EHR IMPACT methodology provides a rigorous
evaluation of the long-term impacts of interoperable EHR and ePrescribing systems. The case
studies provide empirical insights that underpin findings on the socio-economic impact of interoperable EHR and ePrescribing systems and the factors that need to be in place to accelerate their successful deployment.
The cases show that there is no single, ‘right’ strategy for implementing interoperable EHR and
ePrescribing systems. Decisions to invest in such solutions must devise and adopt strategies
that fit their local or regional setting, and be designed to succeed by meeting clearly identified,
measurable needs. Transferability of some technology and tools to other contexts is more viable
49
eHealth is Worth it - The economic benefits of implemented eHealth solutions at ten European sites. Luxembourg: Office for
Official Publications of the European Communities, 2006 (56 pp. - ISBN 92-79-02762-X). Electronic file:
http://europa.eu.int/information_society/activities/health/docs/publications/ehealthimpactsept2006.pdf
50
www.ehr-impact.eu; publication upcoming
ARGOS PB Benefits, 19 October 2010
18 of 27
DRAFT for WDC WS on Nov. 12, 2010
than transferring specific functionalities and organisational features. The specific roles and priorities of healthcare professionals and HPOs differ between jurisdictions and healthcare systems,
limiting transferability of success stories mainly to principles, tools and techniques rather than
specific EHR and ePrescribing systems. The most transferable features are the experiences
and capabilities gained, and requirements for success identified.
By taking the socio-economic perspective, the cases illustrate that initiatives can achieve returns of close to 200% on their total investment, and an average of about 80% over some nine
years. These represent excellent returns from a wide range of benefits, but must be seen as
longer-term investment to support a longer-term strategy for improving clinical performance.
The EHR IMPACT study provides evidence on concrete implementations at the local and regional level. The gains from EHR and ePrescribing systems rely on access to information regardless of place and time, and from re-using information for multiple purposes. Without meaningful sharing and exchange of information, the gains would be marginal and not justify the cost
of investments. The other is to ensure continuous engagement and a productive dialogue between clinical and administrative users on the one hand, and ICT experts on the other.
Healthcare professionals, who are essential users of eHealth systems, are too often not sufficiently involved.
It follows upon earlier research and adds well-founded empirical evidence to the pool of
knowledge about the potential of “eHealth in the context of healthcare reform programmes”51.
Further micro-level evidence has been collected in the context of the Good eHealth project
(2006-2008)52, now followed up by the ePractice portal53. Good eHealth project sought to promote examples of good practices in eHealth which have not yet been so widely publicised. The
goal was to make them available for use as good models and experiences. The objectives of
this study were to identify good practices and their associated benefits, develop and implement
proven approaches to wider dissemination and transfer real-life experiences, and stimulate and
foster accelerated take-up of eHealth by addressing the common challenges of eHealth and
lessons learned.
Another currently running study is developing an evaluation methodology for telemedicine applications.54
4.1.3 Activities of EU Member States
The topic of ex-ante impact assessment as well as of formative and ex-post summative evaluation has gained considerable momentum across Europe. The scope and procedures used are
very diverse however, and a systematic comparison of approaches, techniques/tools applied
and specific applications or processes evaluated is not possible.
Countries with well advanced eHealth infrastructures and services have often carried out assessments of the benefits of their eHealth investments. A recent study55 on eHealth strategies in
Member States has generated more knowledge on the topic at this point in time.
Around one-half of the surveyed 33 countries (including the four home countries of the UK)
mention a specific body of one form or another as being responsible for evaluation activities.
These include: national eHealth-platform (Belgium), Estonia State Audit Office, Centro Nazionale per Informatica nella Pubblica Amministrazione (CNIPA, Italy), Department of IT of Ministry of Health (Lithuania), Centre for Health Economics (Latvia), National Institute for Health
and Welfare - THL/Ministry of Health (Finland). Some mention a diversity of entities/agencies,
two the involvement of research institutes. In decentralised healthcare systems such as Italy
and Spain, regional evaluations prevail over systematic national level assessments.
Some of these bodies have only recently taken up responsibility for the evaluation role and –
from the evidence offered – have not yet finalised any particular evaluation.
51
Council Conclusions 1 December 2009, p. 2
http://www.good-ehealth.org/about/introducing.php
53
http://www.epractice.eu/
54
MethoTelemed: http://www.telemed.no/methotelemed.4567567-51256.html
55
http://www.ehealth-strategies.eu/
52
ARGOS PB Benefits, 19 October 2010
19 of 27
DRAFT for WDC WS on Nov. 12, 2010
Past Member State activities include:
• A detailed German report focusing on the (additional) costs of the planned advanced electronic health insurance card and some of its planned applications, largely neglecting benefit
aspects.
• Selected Spanish activities like an evaluation undertaken within the framework of cooperation provided by the Quality Plan for the National Health System, within the context of an collaboration agreement signed by the Carlos III Health Institute, an autonomous agency of the
Ministry of Health and Consumer Affairs, and the Department of Health of the Basque Country Government (OSTEBA).
• During its recent EU presidency, the Swedish government commissioned a study on the
benefits from eHealth56, which focused on rough estimates of the potential for benefits from
eHealth implementations based on singular experiences being extrapolated to several countries and neglecting costs.
Seven countries report on actual (Ireland, England, Switzerlan) or planned (France, Slovenia,
Slovakia, Bulgaria) assessments of the impact of investments in the eHealth domain. As such
analyses are expected to lead to an optimisation of resource allocations not only with respect to
planned investments, but also for already running activities, one can expect more attention to be
paid to such socio-economic and change management aspects in future.57
So far only a single country, Switzerland, has officially subscribed to undertaking an initial
‘rough’ and later a detailed Regulatory Impact Analysis (RIA) of specific eHealth legislation under discussion at the parliamentary level right now. RIA is a systemic approach to critically assessing the positive and negative effects of proposed and existing regulations and nonregulatory alternatives. As employed in OECD countries since 1974, it encompasses a range of
methods. At its core it is an important element of an evidence-based approach to policy making.
The United Kingdom is another exam-ple of almost continuous evaluations of the National Programme for IT (NPfIT) of the Na-tional Health Service in England by a wide variety of actors.
A few Member States also mention EC co-financed studies which have provided them with
some insights into the socio-economic impacts of eHealth solutions (Denmark, Malta, Sweden).
The two studies cited were the Institute for Prospective Technological Studies (IPTS) commissioned study of state-of-play vis-à-vis eGovernment and eHealth in ten New Member States
(2005-2008) and the EC-commissioned eHealth Impact study analysing 10 routine applications
across Europe (Denmark, Sweden) (2005-2006).
4.2
United States of America evidence
4.2.1 USA evidence at the macro-level: diffusion and usage
AMIA/Blackford please add a short summary
4.2.2
USA evidence at the micro-level: benefits assessment
AMIA/Blackford please add a short summary
56
Cf. Ministry of Health and Social Affairs in Sweden (2009): eHealth for a Healthier Europe. Opportunities for a better use of
healthcare resources.
57
Stand 2007: http://www.ehealth-era.org/documents/2007ehealth-era-countries.pdf
ARGOS PB Benefits, 19 October 2010
20 of 27
DRAFT for WDC WS on Nov. 12, 2010
5 Research challenges and methodological issues encountered
5.1
Introduction
Frame the issues
5.2
Gathering evidence at the macro-level: diffusion and usage
From the OECD paper:
At the OECD/ARGOS Barcelona meeting in March, the following concerns were raised over
possible country-specific interpretations of the various indicators discussed:
Possible Measures to Benchmark Adoption and Use of ICTs in the Ambulatory Sector
Notes:
The Table is adapted from the OECD‟s compilation of most common indicators to measure adoption and use of
Health ICTs (from the report: “Achieving Efficiency Improvement in the Health Sector through ICTs”). The table does
not list all of the variations in the data collections or the measurement units for the available indicators. The OECD „s
study included the following countries: Australia, Canada, the Czech Republic, Finland, France, Norway, New Zealand, Spain, Sweden, the United States. It also included surveys from the European Commission and the Commonwealth Fund
Participants are invited to consider:
•
•
what are the potential "core" measures for all nations to track
which of these measures may add value in surveys to measure ICT adoption
ARGOS PB Benefits, 19 October 2010
21 of 27
DRAFT for WDC WS on Nov. 12, 2010
•
whether a joint effort to standardize the definitions of these concepts is required
The three individual countries that are listed were selected to provide a cross-national comparison. Canada, Australia, and the United States are three non-EU nations where ICT adoption has been measured at both the local and
national levels through various surveys.
Source: OECD Paper on Barcelona meeting, ANNEX III.
However, it was noted that this would be resolved if the group could first agree on the meaning
of the terms used. Participants agreed that a most promising approach today is to characterize
ICT applications by the features and functions they offer. This approach provides a framework
on which to base international agreement on definitions.
Countries may have been surveying ICT adoption by health service providers for several years
by now. This raises the question what such countries would gain from participating in an international exercise if benchmarking would primarily focus only on a first cluster of indicators
(adoption). It would be more interesting to compare the adoption curve/rate of adoption of specific kinds of ICT applications with countries where there is scope to learn from other countries.
The use of a “modular survey” approach may therefore be to be recommended.
On next steps, the following was contemplated:
• A feasible way forward would be to establish three/four small expert sub-groups tasked to
create spanning definitions for the fifteen indicators.
• The sub-groups should work primarily on-line and through an electronic discussion group.
• Provided the groups could make some progress, the OECD could organise a follow-up
meeting at OECD headquarters in Paris.
5.3
Assessing benefits and costs at the micro-level
There are several limitations of presently undertaken individual eHealth evaluations, such as:
• Benefits are not based on or measured with data from high quality (random) surveys, validated tools and survey instruments, or (in the extreme, randomised controlled) clinical trials
(RCTs).
• Data and attributes of cause and effect are not precisely aligned, and rely on assumptions
and estimates
• Only one, or too few, dimensions of eHealth are included in the evaluation
• Limited account is taken of prerequisites and enabling investments, such as ubiquitous
(broadband) communication networks
• Economic evaluations do not deal with financing implications for healthcare entities like
credit rating, affordability in the public sector
• The characteristics of the healthcare setting and the idiosyncrasies of national health service systems are not properly dealt with
• The impact on other entities, both within and outside healthcare, are excluded from evaluation, but are often very critical and may impact results considerably
• Findings and conclusions cannot be validly transferred to other settings.
For evaluations to be useful in advancing viable eHealth investment, it is essential that some of
these factors are addressed. This will also allow a look "behind the scene", to attempt to identify
and understand the reasons why at one site an eHealth investment indeed triggered a positive
increase in efficiency, and why this did not materialise in another, perhaps very similar context.
Citation from the report of the USA CBO58:
58
Congressional Budget Office (CBO): Evidence on the Costs and Benefits of Health Information Technology. The Congress of
the United States - Congressional Budget Office, Washington,. DC, May 2008
ARGOS PB Benefits, 19 October 2010
22 of 27
DRAFT for WDC WS on Nov. 12, 2010
"The RAND researchers attempted to measure the potential impact of widespread adoption of
health IT—assuming the occurrence of “appropriate changes in health care”—rather than the
likely impact, which would take account of factors that might impede its effective use. For example, health care financing and delivery are now organized in such a way that the payment
methods of many private and public health insurers do not reward providers for reducing
costs— and may even penalize them for doing so. B The RAND study was based solely on empirical studies from the literature that found positive effects for the implementation of health IT
systems; it excluded the studies of health IT, even those published in peer reviewed journals,
that failed to find favorable results. The decision to ignore evidence of zero or negative net savings clearly biases any estimate of the actual impact of health IT on spending. B The RAND
study was not intended to be an estimate of savings measured against the rates of adoption
that would occur under current law, but rather against the level of adoption in 2004. That is, the
researchers did not allow for growth in adoption rates that would occur without any changes in
policy, as CBO would do in a cost estimate for a legislative proposal." "The potential of health IT
to reduce spending for health care depends in large part on its ability to make care more efficient by cutting the cost of delivering services, avoiding redundant services, and improving providers’ productivity. Evidence from the literature on health IT, however, does not uniformly support the possibility of such savings. The potential for savings appears to depend heavily on their
source and whether that source is in a hospital or in an ambulatory care setting (such as a clinic
or a physician’s office). In addition, savings are difficult to assess because the trimming of costs
in one area of a physician’s practice, for example, may be offset by increased costs or reduced
efficiency in another area. Estimating the impact of some potential sources of savings, especially those arising from greater exchange of information among providers, insurers, and patients, is
especially difficult because health IT networks are in an early stage of development. Furthermore, health care providers and hospitals that were early adopters of health IT may have been
motivated by particular characteristics of their organizations or operations that made them more
likely than nonadopters to achieve benefits from health IT—in which case the outcomes they
have seen might not be generalizable. Evidence of savings in the health care sector as a whole
from adopting health IT is also limited." "As discussed earlier with regard to the RAND study,
reductions in the average length of hospital stays are unlikely to result in cost savings of a similar proportion to the reduction in average length of stay, such as that found by the Mekhjian research team (that is, of 5 percent or more). In particular, reductions in stays that stem from performing various hospital functions more quickly are not likely to cut costs as much as will reductions that result from improving care—for example, by diminishing the number of adverse drug
reactions. Reducing the length of time required to process a lab test or diagnostic image from
the time it is ordered to the moment the results are delivered only speeds up the delivery of
care; it does not necessarily reduce the amount of care provided or its associated cost."
Note on Gartner study for Sweden Presidency 200959:
Study results are largely based on speculation ("could be avoided", "could be freed", "an opportunity for"), not on convincing evidence. For each "benefit" (p. 17) they mention only one source
(p. 72 ff) - probably one can find many other sources with quite different or even negative values
as well. Also
- such studies are usually pilots etc., i.e. their extrapolation to the overall system is more than
questionable (for 40 years we have now seen probably more than 10,000 of such reports on
pilots, experiments etc. etc. (and, in small, shorter term studies you will usually find positive reactions - the Hawthorne effect is well known in methodology). Another aspect is that the good,
intelligent, curious doctors engage in such studies, not the "average" ones. Cf. "The benefits
presented in this chapter are projections based on evidence of benefits reported on individual [!]
case studies." p. 19 [!!! - how can you generalise???]
Gartner (prepared for the Swedish Presidency of the European Union): eHealth for a Healthier Europe! – opportunities for a
better use of healthcare resources. 2009
59
ARGOS PB Benefits, 19 October 2010
23 of 27
DRAFT for WDC WS on Nov. 12, 2010
- you simply cannot directly compare the Dutch with the English, the French with the Swedish
health system and apply data from one to the other (seems many extrapolations are based on
NHS England data). Often even hospitals in neighbouring cities are very different and will, with
the same technology, generate different outcomes.
Note on present status and applicability of conventional HTA approaches to eHealth issues:
Health Technology Assessments (HTA) are usually carried out in order to provide a particular
competent health authority or stakeholder with information about accurate judgments on technologies, but limited by the respective idiosyncrasies of their health system. Even though some
of the principles that inspire HTA should remain valid when assessing benefits and costs,
eHealth solutions put ICT technology in a whole different context, also in terms of assessment.
Virtually all of the detailed frameworks for HTA have been developed with more conventional
technologies in mind: the term “technology” in HTA commonly refers to drug testing or medical
devices, but not to health information technology based systems and solutions. HTA programmes, moreover, predominantly use integrative (meta-level) approaches, with particular
attention to formulating findings that are based on distinguishing between stronger and weaker
evidence drawn from already available multiple, primary data studies. Yet the very research
challenge behind benefits assessment of health ICT solutions is to develop frameworks that are
at all capable of producing primary data, as the lack of robust data and evidence, and the tools
to arrive at, is the major deficiency that prohibits deployment, awareness and diffusion.
5.4
Assessing the anticipated clinical and socio-economic benefits from RTD
initiatives
More and more, even in the case of basic research initiatives and projects, like the European
VPH initiative, RTD endeavours are being marked from the very beginning by a strong interest
in early applications of their expected final results and outcomes to clinical practice, and a demand for measurable evidence that such complex technology is actually worth to society the
cost and the complexity it involves.
E.g., as the first wave of VPH projects is unravelling, and as the first batch of VPH technologies
is getting closer to the time when they will enter first clinical trials, there is a growing attention
within the VPH community to the issue of how to quantitatively assess these technologies in
term of safety, efficacy, clinical and socio-economic impact. A first, apparently trivial answer to
this question is “Health Technology Assessment” (HTA). In many contexts, it has become the
preferred approach for policymakers to base decisions on health technology investments and
reimbursement policies on factual evidence, gained as independently as possible from the
many biases that tend to revolve around the introduction of a new health technology.
HTA embraces a diverse group of methods that can be grouped into two broad categories. Primary data methods involve collection of original data, ranging from more scientifically rigorous
approaches such as randomized controlled trials to less rigorous ones such as case studies.
Integrative methods (also known as "secondary" or "synthesis" methods) involve combining
data or information from existing sources, including from primary data studies. These can range
from quantitative, structured approaches such as meta-analyses or systematic literature reviews
to informal, unstructured literature reviews (Goodmann, 2004).
However, VPH technology is so radically different from anything else used in hospitals that a
systematic assessment is required already in the phase where specific tentative clinical indications are being searched. Due to its very innovative nature, medical professionals have no general understanding of the potential of VPH technology, and thus have difficulties in driving the
process that associates the output of such basic technological research to the solution of a concrete clinical problem. Whereas 99% of diagnostic medical technology measures one specific
variable, VPH technology does something radically different: it predicts. It generates new
knowledge from a set of individual patient and cohort data. Different to clinical decision-support
systems, this knowledge is not already available somewhere, it is new and original, and not in
ARGOS PB Benefits, 19 October 2010
24 of 27
DRAFT for WDC WS on Nov. 12, 2010
possession of the medical professional (although to be clinical useful, this knowledge needs to
be integrated and combined with that of the medical professional). In addition, the integrative
nature of most VPH technology implies that the knowledge that emerges cannot be linearly derived from the available data, but it results form complex non-linear interaction between various
sets of information. This means also that no clinician can assess the validity of such a prediction
just by looking at it.
All of this implies totally new challenges for benefits assessment and requires new approaches
and methods to be defined and identified in some detail. There is need for a more dynamic approach to impact assessment of medical technologies, and in particular of VPH-based health
technology. The purpose of many impact assessments involving clinical assessment of medical
technologies is to measure the change of cost-benefit ratio during the development and during
the early phases of innovation, often for each stage of development and prototype, and dynamically per each year. Therefore we need an approach that also takes into account the technology dynamics by emphasizing socio-dynamic processes. Similar to the so called constructive
technology assessment (CTA) (Douma, Karsenberg, Hummel, Bueno-de-Mesquita, & van
Harten, 2007), first described in the 1980s, a comprehensive assessment can be combined with
an intentional influence in a favourable direction to improve quality (related to so-called formative evaluation). Early assessments (including feedback into ongoing developments -- “action‐oriented assessments”) occur, but choices will be primarily about promises: the role of
expectations and promises (and concerns), and the need to understand their role in ongoing
developments.
Using VPH here only as an example, we expect that future eHealth technologies will need a
whole new assessment framework, significantly different from those currently available, and
deeply entangled with the design and development phases of these eHealth solutions.
6 Experiences, lessons learned, needs for future actions
to be done later
Question: Use an integrated approach mingling EU and USA perspectives and lessons learned,
or report separately on this here?
6.1
Macro-measures:
OECD suggests to use its model survey framework which takes a staged approach in moving
international measurement work forward. To be useful in all contexts, a „model survey‟ is composed of separate, self-contained modules that ensure flexibility and adaptability to a rapidly
changing environment. The use of core modules (as an add-on to existing country surveys or as
a stand alone survey) allows measurement on an internationally comparable basis. Additional
modules and new indicators can be added to respond to evolving or country-specific policy
needs in this area.
At the OECD/ARGOS Barcelona meeting, it was suggested to organise macro measures according to the following four categories or steps:
1. Adoption
2. Modes of Use/Purpose of Use
3. Critical Success Factors
4. Outcomes/Impacts
6.2
Micro measures
ARGOS PB Benefits, 19 October 2010
25 of 27
DRAFT for WDC WS on Nov. 12, 2010
6.3
Measures related to RTD activities in eHealth
7 Recommendations for transatlantic exchange, policy
harmonisation, collaborative initiatives
to be done later
ARGOS PB Benefits, 19 October 2010
26 of 27
DRAFT for WDC WS on Nov. 12, 2010
References
ARGOS PB Benefits, 19 October 2010
27 of 27