This project is funded by the European Union within the framework of the
Pilot Project on Transatlantic Methods for Handling Global Challenges in the
European Union and the United States. The general objective of the Pilot
Project, created through a European Parliament initiative, is to promote
mutual understanding and learning among EU and US policy researchers and
policymakers on a number of challenges with a global dimension.
2010
ARGOS Discussion Paper
EuroRec
The European Institute for Health Records
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
9 November 2010
2
Contents
Introduction............................................................................................................................................. 6
EuroRec Corporate Board........................................................................................................................ 7
Rationale for quality labelling and certification ...................................................................................... 8
Introduction......................................................................................................................................... 8
Benefits to healthcare organisations as purchasers of EHR systems .................................................. 9
Benefits to the supplier industry ....................................................................................................... 10
A valuable resource for purchasers and suppliers ............................................................................ 11
State of the art of quality labelling and certification ............................................................................ 12
Introduction....................................................................................................................................... 12
Methods used.................................................................................................................................... 12
Situation before 2007 – information collected through the Q-REC project ................................. 12
Situation in 2010 ........................................................................................................................... 13
Belgium .............................................................................................................................................. 13
Situation before 2007 .................................................................................................................... 13
Situation in 2010 ........................................................................................................................... 19
Denmark ............................................................................................................................................ 21
Situation before 2007 .................................................................................................................... 21
Situation in 2010 ........................................................................................................................... 29
Greece ............................................................................................................................................... 29
Situation in 2010 ........................................................................................................................... 29
Ireland ............................................................................................................................................... 30
Situation before 2007 .................................................................................................................... 30
Situation in 2010 ........................................................................................................................... 38
Serbia ................................................................................................................................................. 39
Situation in 2010 ........................................................................................................................... 39
Slovenia ............................................................................................................................................. 40
Situation in 2010 ........................................................................................................................... 40
United Kingdom................................................................................................................................. 41
Scope of the labelling procedure .................................................................................................. 41
Rationale........................................................................................................................................ 42
Description of the procedure ........................................................................................................ 42
Economic Model ............................................................................................................................ 44
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
3
Criteria ........................................................................................................................................... 44
Testing Scenarios ........................................................................................................................... 49
Lessons learned ............................................................................................................................. 49
USA .................................................................................................................................................... 50
Situation before 2007 .................................................................................................................... 50
Situation in 2010 ........................................................................................................................... 56
Canada – eHealth Collaboratory ....................................................................................................... 63
Scope of the Labelling Procedure .................................................................................................. 63
Rationale........................................................................................................................................ 63
Description of the Procedure ........................................................................................................ 64
Economic Model ............................................................................................................................ 64
Criteria ........................................................................................................................................... 64
Testing Scenarios ........................................................................................................................... 65
Lessons Learned ............................................................................................................................ 65
France ................................................................................................................................................ 65
Scope of the labelling procedure .................................................................................................. 65
Rationale........................................................................................................................................ 66
Description of the Procedure ........................................................................................................ 66
Economic Model ............................................................................................................................ 66
Criteria ........................................................................................................................................... 67
Testing Scenarios ........................................................................................................................... 67
Lessons learned - Comments......................................................................................................... 68
Germany ............................................................................................................................................ 68
Situation in 2010 ........................................................................................................................... 68
Norway .............................................................................................................................................. 69
Situation in 2010 ........................................................................................................................... 69
EuroRec Products & Services ................................................................................................................. 70
Repository ......................................................................................................................................... 70
Introduction ................................................................................................................................... 70
Workflow process that applies to EHR quality and conformance criteria .................................... 70
Statistics......................................................................................................................................... 72
EuroRec Tools .................................................................................................................................... 74
Definition of the Tools ................................................................................................................... 74
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
4
EuroRec Seal ...................................................................................................................................... 75
Seal Level 1 .................................................................................................................................... 75
Seal Level 2 .................................................................................................................................... 77
Release of the EuroRec Profile for EHR Compliant to Clinical Trial Requirements ....................... 80
European research projects .................................................................................................................. 81
Past projects ...................................................................................................................................... 81
Current projects ................................................................................................................................ 81
Future projects .................................................................................................................................. 81
Projects’ fact sheets .......................................................................................................................... 82
Q-REC ............................................................................................................................................. 82
RIDE ............................................................................................................................................... 83
EHR-Implement ............................................................................................................................. 84
EHR-QTN.......................................................................................................................................... 85
ARGOS ........................................................................................................................................... 86
HITCH ............................................................................................................................................. 87
EHR4CR .......................................................................................................................................... 88
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
5
Introduction
The EUROREC Institute (EuroRec) is an independent not-for-profit organisation, whose main purpose
is promoting the use of high quality Electronic Health Record systems (EHRs) in Europe.
It comprises a permanent network of National ProRec centres and provides services to industry
(developers and vendors), healthcare systems and providers (buyers), policy makers and patients.
National ProRec centres in Europe
The current membership position of the Institute is set out in the following
table
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
6
The Primary objects of the Institute are as follows :

To Federate and support Members (ProRec Centres) and Associates of the Institute;

To promote, advance and participate in research on development, implementation, use and
transfer of health records and related health informatics undertakings in a European context;

To promote, advance and engage in assuring the efficiency, effectiveness and quality of
health records and related health informatics undertakings in a European context;

To advance and secure acceptance of the importance of comprehensive patient / citizen
focused health records as a central feature of effective health systems in a European
context;

To promote and advance investment in the development and promulgation of health records
and related health informatics undertakings in a European context;

To promote European and Global cooperation in health informatics undertakings designed
to improve quality and interoperability of health records, especially through quality labelling
and certification.
Development, promulgation and systematic application of EHR quality labeling and Certification
Schemes constitute a central area of activity and focus for the Institute. This paper outlines the
prevailing status of EHR certification globally, compares and contrasts established Schemes and
poses opportunities for convergence of activity in the domain designed to advance certification
endeavors generally.
EuroRec Corporate Board
Under reconstituted Articles of Association, a new Corporate Board was established in November
2009 comprising the following membership:

President: Georges De Moor

Secretary General: Gerard Hurl

Vice-Chairman: John O’Brien

Treasurer: Jos Devlies

Vice-Presidents
o
Organisation Portfolio: Leo Ciglenecki
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
7
o
Services Portfolio: Gerard Freriks
o
Projects Portfolio: Dipak Kalra
o
Stakeholder Relationships Portfolio: Christopher Nolan
Figure: New EuroRec Executive Board
Rationale for quality labelling and certification
Introduction
Certification, Licencing and Accreditation endeavours are now well established and accepted in
Healthcare. They are recognised as essential to quality assurance and improvement advancement.
With the increasing investment in and exponentially expanding central role of and dependence on
ICT in Health, particularly in the core areas of patient diagnosis, treatment and care there is a
growing recognition of the need to quality assure the content, capacities, capabilities and
performance of related Systems as they are applied in the domain. This has led to the emergence in
recent years of a variety of quality labelling/certification Schemes for EHR’s of varying levels of
completeness and sophistication with differing degrees of application success in various jurisdictions.
The rationale for developing and applying these Schemes is multifold.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
8
Firstly, potential for harm may lie in the design of some EHR systems (such as failure in design logic,
poor or confusing presentation of clinically relevant information) or even failure in logic generally
etc... Many of these system deficiencies are inbuilt, and may not be apparent to the user (healthcare
professional). EHR Certification Schemes are designed, inter alia, to identify and assure remedy of
such failings.
Secondly, Healthcare information, in particular clinical information, is often distributed over a
number of information Systems (primary care, secondary care…). As patient medical data is
increasingly shared across these Systems with the aim of supporting high quality integrated care, the
interoperability of these Systems becomes paramount. Providing assurance in this area is a prime
requirement and value proposition of high quality EHR Certification Schemes.
Thirdly, certification of EHRs is essential to assure that EHR systems are sufficiently developed,
comprehensive and robust to facilitate realisation of anticipated benefits. EHR procurers and end
users (e.g. hospital directors and physicians) frequently claim that EHR systems and related product
quality, data portability and interoperability are difficult to judge. Certification eliminates this
concern for procures/users.
Finally, EHR Certification hugely risk ameliorates the payer/investor position and significantly
advances the robustness of the Business Case for considerably increasing investment in ICT in Health.
Benefits to healthcare organisations as purchasers of EHR systems
One of the main reasons for quality labelling and certification of EHRs across Europe is to provide a
more secure purchasing environment for the customers of the EHR supplier industry. Quality
labelling and certification offers many advantages to the purchasers of EHR systems of which the
following are prime examples:

Choosing a certified EHR product will give greater confidence that it will perform all of the
basic functions required and has the potential to significantly improve the success of
implementation;

A key requirement to support quality outcomes is that an EHR functions as specified and the
availability of quality assured products improves prospects in this area;

There is growing evidence that suggests significant benefits can be obtained from investment
in quality assured EHRs, arising from improvements in the quality and safety of healthcare
services and also the possibility of cost savings. Purchasing a certified EHR product has the
potential to improve patient care and service;

The quality and suitability of EHR products may be difficult for a purchaser to assess that the
product is robust enough to deliver what is expected. The EuroRec inventory of resources,
which underpins the certification process, contains defined standards for functional criteria,
data and communications architecture, interoperability and security which will assist in
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
9
specifying EHR requirements, what an EHR product is expected to provide and how it will
meet the needs as specified;

Quality labelling/certification of EHRs has potential to significantly reduce investment and
associated risks by:
o
o
o
o
o
o

Companies supplying systems who have undergone the certification process will have
demonstrated that they have:
o
o
o
o
o
o


Specifying expectations/content of EHRs;
Specifying interoperability requirements of EHR systems;
Specifying deployment standards;
Educating vendors/users on requirements of successful EHRs;
Driving performance standards convergence;
Certifying compliance with expectations, content, interoperability, deployment and
standards requirements for specific EHRs.
A robust and proven product and solution portfolio;
Ensured their EHR systems are of demonstrably good quality;
Commitment/responsibility –towards providing system support;
Skills in technology delivery support and innovation;
Understanding of healthcare service delivery issues;
Experience and reputation in the healthcare sector.
Quality labelling and certification offered by EuroRec provides an independent, unbiased,
professional and trustworthy quality assurance mechanism, it supports risk assessment and
analysis, and provides a benchmark to support and evaluate value for money (VFM)
initiatives;
Certification of EHR products can lead to an improved market confidence with larger volumes
and lower marketing and development costs to vendors thus leading to the potential for
lower ownership costs over the lifetime of the EHR. The tendering process can be both
simplified and the costs reduced for the purchaser and supplier communities. It will also be
more transparent and improved by:
o
o
o
Having best practice validated for systems assessment methodology;
Potential time reduction through speedier short listing and evaluation;
Utilising standard criteria and certified product suppliers.
Benefits to the supplier industry
With the introduction of Europe -wide quality labelling and certification standards and criteria for
EHRs by EuroRec, subsequently adapted for each country’s needs, the supplier industry can expect
numerous benefits to emerge that are fundamental to securing the wider adoption of known quality
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
10
labelling and certification standards and criteria. It is important that both purchasers and providers of
EHR systems see real and significant benefits in this process and for the suppliers these include:

More efficient product development and extended product life cycles against known market
requirements;

Certification of EHR products can lead to improved market confidence with larger volumes
and lower marketing costs for suppliers. The tendering process can be simplified and become
more transparent. The time and costs involved can be reduced for the purchasers and
suppliers;

A liberated market in which previously locally developed and locally marketed products have
the potential to appeal to a Europe -wide audience;

Industry co-operation can be greatly improved by convergence towards known standards
and criteria to the betterment of those healthcare organisations served. The quality and
professionalism of the delivered product and service will be enhanced over to-day’s level and
thus the industry enjoys an enhanced reputation;

Suppliers have more opportunity to ‘mix and match’ their solutions stimulating a market in
which both purchasers and suppliers have wider choice;

Quality labelling will greatly support the development of national based programmes for EHR
with choice of certified solutions rather than central governments/ executive agencies
becoming too prescriptive and thus reducing supplier choice and options.
A valuable resource for purchasers and suppliers

For both purchasers and suppliers, participation in and utilisation of the EuroRec
certification process will also provide access to:
o
o
o
o
o
Trustworthy resources that contribute to producing or validating high quality and
more interoperable EHRs;
Criteria on which quality labelling and certification is based, and which also indicate
the fine-grained requirements that must be met by “good” systems;
Guidance materials that explain the way in which accreditation works, and how tests
may be performed;
Knowledge artefacts such as archetypes, as a trusted (quality-assured) source of
specifications that may be downloaded and applied to EHR systems (e.g. in the
design of templates and forms);
Guidance materials on how good knowledge artefacts can be designed;
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
11
o
o
References and pointers to published work that underpins the criteria (such as
standards to which conformance is required) or to externally-produced knowledge
resources (such as published terminologies and code sets);
References and pointers to “trustworthy” software meeting suitable quality criteria
(e.g. open source components, implementable technical specifications such as XML
schemas and other EHR-related documentation).
State of the art of quality labelling and certification
Introduction
The present overview of the state of the art of quality labelling and certification has been presented
in deliverable 2.1 ‘State of the Art Report’ of the Q-REC project. The state of the art described in the
following sections covers the situation up till 2007. If more recent information for a country was
available, this has been indicated by a section “Situation in 2010”.
Methods used
Situation before 2007 – information collected through the Q-REC project
Four European countries and the American initiative of CCHIT were identified as having implemented
EHR system quality labelling/certification procedures.
The participating PROREC centres of these countries were requested to collect the information about
the current EHRs labelling and certification procedures used in their countries. Complementary
interviews during on site visits were conducted to complete the picture of the organisation
(stakeholders, economic model, buyers and sellers of EHRs…).
The on-site visits provided the necessary detail to fully understand the implemented procedures and
processes, which differ considerably between countries (scope, implementation, criteria, modus
operandi etc…).
The analysis material provided by the PROREC centres completed with the results of the “on site
visits” were discussed and structured by the project team in the following topics:
Scope of the labelling procedure: different certification objectives are aimed at within the different
countries
Description of the procedure: a summary description of the procedures is presented, including
periodicity of the labelling cycle, declarative or benchmarking type of procedure, pricing
mechanisms, outcome and market impact of the labelling procedure, user involvement and user
perception, juridical and economical aspects and results obtained.
Economic model: special attention is given to the economic model behind the labelling procedure.
This model will drive the impact of the labelling activity on the current market. If no demonstrable
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
12
advantage is perceptible to the software vendor (seller) and or the user (buyer), one should question
the use (advantage) of such labelling procedures. Advantages for the patient are obvious and will be
commented where appropriate in the description of the procedures. The economic model will
become a determinant factor when developing the business plan and ditto framework for European
labelling and certification of EHRs in WP4.
Criteria used: quality labelling and/or certification require strictly defined quality criteria, which can
easily be tested during the benchmarking activities. Wherever formal criteria are available, these will
be analysed and inventoried.
Testing scenarios will be analysed including the benchmarking procedures when existing.
Benefits and advantages of labelling EHRs. This part is included in the section “Lessons Learned” of
the different country analysis reports. Regarding the CCHIT part within this report the analysis is
based on the information found on their WEB site. During the HIMMS congress in San Diego (2006)
initial contacts and collaboration intentions where established by the European delegates of the
QREC project. This allowed us to clarify the procedure where appropriate with the responsible
persons within CCHIT.
Situation in 2010
Information was collected by means of a webbased questionnaire (HITCH project), or by directly
contacting the several ProRec representatives (e.g. through the EHR-Q-TN project).
Belgium
Situation before 2007
Scope of the labelling procedure
Quality labelling of Electronic Healthcare Records in Belgium started in 2000. The quality labelling of
software packages for EHRs was initially developed for ambulatory care and for software packages
used by General Practitioners in their private practices. Progressively the quality labelling of EHRs
was extended to other software packages i.e. Physiotherapy, Dentistry and recently nursing and
pharmacy. The implementation & operation of the quality labelling/certification procedure falls
under the responsibility of the Ministry of Health1.
Quality Labelling never intended to standardise the data structures, implementation aspects or
internal working of EHR software implementation. Quality labelling in Belgium aimed at:

Verifying the compliance with legal and ethical aspects of data protection when storing
patient information (administrative and clinical).

Verifying the respect of patient’s privacy when archiving confidential data.

Providing access to reference documents, healthcare databases (drug information), validated
means to support clinical decision making
1
Information on quality labelling in Belgium can be found on http://www.health-telematics.be/ and
http://www.health.fgov.be/EMDMI/
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
13

Verifying the implementation of the interoperability criteria allowing the exchange of patient
records between different software package implementations, without the loss of clinical
information, in particular the import from and export to the summary patient record, called
“Sumehr”.
In 1999 Prorec-BE was asked to develop evaluation criteria to support the “quality label” of GP
software systems. Initially 333 criteria were identified. It was agreed that conformance testing of
these 333 criteria at once would not be feasible taking into account the reality of the market. It
would have resulted in a failure of all software packages present on the market and therefore it was
decided not to proceed to define the criteria upfront for the coming years, but to setup a working
group with the software vendors to define year after year the criteria to be tested. This working
group was called “Transfer” group.
Rationale
The labelling and certification procedure is embedded in a legal framework consisting of 4 main laws:
1. Royal Decree of the 4th of April 2002 legitimising the labelling and verification of medical
software for Electronic Healthcare Records management.
2. Royal Decree of the 15th of April 2002 defining a multidisciplinary working group responsible
for the software of the EHRs in ambulatory care settings. This working group will be
responsible to further elaborate the criteria, the testing scenarios and to provide assistance
to the other working groups of experts as appointed by the ministry of Health.
3. Royal Decree of 4th of September 2002 nominating the representatives to the ad hoc expert
group.
4. Royal Decree of the 6th of February 2003 defining the terms and conditions under which a
user of an EHR software can be subsidised to partially cover the maintenance costs of its
EHR software. The amount of the subsidy is defined within this document at 743 €/year.
In addition to the legal framework, several scientific associations (scientific) and the order of
physicians were consulted. Al the comments and reports are available on the ministerial WEB site.
The patient rights regarding the storage and distribution of personal medical information within
EHRs are registered in the law of 22nd of August 2002. This law also defines the modalities of care
delivery from the patient’s viewpoint (freedom of choice, right of information about its own health
status, right of privacy protection, right of complaining and arbitration and the right to have a
sufficient level of quality delivery of care).
Description of the procedure
The “Transfer” working group consisted of representatives of the EHR software industry, end-users,
independent experts appointed by the Ministry of Health, representatives of several scientific
associations. This group became later the working group “LABELLING” of the Ministerial Committee
of Health Telematics Standardisation who defines the criteria that should be tested during next
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
14
years’ benchmarking sessions. The criteria need to be regularly adapted; others need to be made
more explicit. The criteria defined 5 years ago have therefore to be reviewed every year depending
on the current evolution of technology, standardisation, coding systems, legislation etc.
A permanent dialogue between this working group and the ministerial department representatives
guarantees an appropriate yearly quality labelling procedure for EHR software.
Once the criteria agreed in the above working group, subgroups start finalising the testing scenarios.
These testing scenarios are developed by experts appointed by the ministry of health. The scenarios
represent a consistent case (as close as possible to a real situation), and are developed to a sufficient
level of detail so that unambiguous benchmarking can be organised. Every detailed step of a scenario
comprises a reference to the criterion or sub-criterion that is tested. The example test scenario in
annex B has been used for the benchmarking session in the quality labelling exercise of 2005.
The scenarios and preparative instructions have to be available on the WEB site of the Federal
Ministry of Health minimum three months before the benchmarking sessions. Software vendors have
to subscribe to the benchmarking session days. The benchmarking sessions of 2006 will be organised
Friday the 9th of June and Saturday the 10th of June.
End users assisted by the software vendor personnel are conducting the tests. The software vendor’s
presence is allowed to provide technical assistance to the user and to provide more technical
explanation whenever requested by the examinators.
The experts can only evaluate the functionality of a software application. They are not allowed to
evaluate the presentation of the functionality in so far that the presentation is not part of the
function (e.g. an error message in a green colour can be commented). The technical design (the way
the function was implemented) cannot be subject of a certification or labelling exercise. The latter
one remains the full responsibility and the exclusive authority of the software vendor.
After the benchmarking sessions a deliberation session draws up the final list of software
applications that succeeded the tests. This deliberation session is organised between the Ministry of
Health representatives and the examination experts. Until now, no software vendors are allowed
during the deliberation session in spite of many requests of the software industry. The final results of
the deliberation session are presented within 15 days after the tests on the WEB site of the Federal
Ministry of Health.
Once the list is published, users owning one of the accredited software packages can apply for
subsidies at the Ministry of Health (see Economic Model).
Economic Model
Since 2000 the Belgian Health Insurance selected the “sponsoring” model to improve the quality of
the Electronic Healthcare Record software. This economic model was legally defined by the Royal
Decree of the 6th of February 2003.
The following terms and conditions legally apply to obtain the financial support:

The software has to pass the test

No advertising is allowed in the EHR software

The user has to apply for the financial support
o
The user declares to use this year the software to manage patient records
o
The software vendor approves the client status of the user for the current year
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
15

The application can be performed electronically via the WEB site of the Federal Ministry of
Health

The financial support amounts to 743 € per year/per user since 2002.
No costs are involved for the software vendors. The procedure, invitation of experts (examination,
scenario development) is entirely financed by the Federal Ministry of Health.
Criteria
Every year a subset of the 333 original criteria is refined and submitted to the labelling procedure of
that year. These criteria are then referred to by the test scenarios. The criteria are structured in the
following categories:
1. General criteria: (e.g. the software should be conceived for the management of patient
records in primary care)
1.1. Security aspects linked to the application access
1.2. Data Availability
2. Administrative data management
2.1. Patient Identification
2.2. File’s access
2.3. File’s characteristics
2.4. Patient’s personal characteristics
2.5. Insurability data
2.6. Administrative period management (e.g. hospitalisation, …)
3. Medical data
3.1. Nature and content
3.2. Overview
3.3. Codes and internal dictionaries
3.4. Specific interfaces
3.4.1.Drug prescription
3.4.2.Vaccination
3.4.3.Prevention
3.4.4.Ergonomic aspect
4. Data structure
4.1. Health Care element
4.2. Health Approach
4.3. Contact
4.4. Sub-contact
4.5. Service
5. Decision support
6. File management support
6.1. Schedule
6.2. Production of document and reports
7. Coordination, connectivity, communication
7.1. Import of documents
7.2. Export of standardised messages documents
7.3. Received messages management
7.4. Encryption
8. Requests: data extraction
8.1. Data extraction
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
16
8.2. Export and de-identification of data
9. Medical and legal aspects
10.Electronic signature
11.Service and continuity
11.1. User documentation
11.2. Technical documentation
11.3. Contractual after sales services.
Testing Scenarios
Detailed test scenarios have been used during benchmarking sessions to certify the different
software applications presented for labelling. An example scenario can be found in Annex B. That
particular scenario was used for the labelling exercise of the year 2005. The analysis of all testing
scenarios used until now defined a common structure. This structure is constantly discussed and
refined to obtain a better performance of the benchmarking sessions.
Structure of the Testing scenario document
1. Preparative activities
1.1. Equipment
1.2. Software (EHR application)
1.3. Documents and software documentation
1.4. Patient setup
1.5. Lab data
1.6. Referrals
2. Test session
2.1. Start
2.2. Enter patient data
2.3. Simple case
2.4. Structured case
2.5. Vaccination section
2.6. Care planning
3. Help, structure of the screens (display)
4. Docs and reports (mainly official reporting to Health insurance)
5. Selection of data
6. Closing of the session and saving patient data.
Lessons learned
In order to fully understand the impact of labelling/certification of EHRs in Belgium we need to
describe the situation on the EHR software market before Labelling started (1998-1999):
1. The market of EHR softwares consisted of +/- 80 software vendors providing EHR software to
approximately 7000 users (primary care physicians working in private practices). These users
represented +/- 55% of the primary care physicians in private practices.
a.
7 of these software vendors provided 50% of the installed base
b. Many very small vendors (IT, Healthcare professionals…) with a typical installed base
varying from 50 up to 300 users.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
17
2. Interoperability did not exist and no international nor national standards to exchange patient
information were available
3. The vendor liability (financial & technical) was questionable at that time. It was the time that
many very small software vendors became insolvent and had to stop their EHR software
activities. Many reasons were identified for this user hostile situation:
a. Not enough resources to cope with evolving software technology (Windows 3.1, 95,
2000, XP…) and hardware technology like USB 1.0/2.0, Firewire IEEE1394
b. New functionality requirements (more/better decision support, prescription support,
reporting facilities like Adverse Drug Events, ever changing legal requirements for reimbursement etc… )
c. What if bankruptcy?
i. In many cases the users had to re-edit the patient records or
ii. They had to order a custom made - expensive conversion programme from
the new software vendor if the file format and datastructures of the existing
software were documented, which often was not the case.
d. Missing facilities and resources for user training & education
e. Inappropriate or non-existing helpdesk or user support
After five years of labelling the advantages became visible:
•
Improved supplier reliability. There are less EHR software vendors on the market (max. 20)
and all of them have a respectable number of customers installed. Smallest installed base =
250 users, largest installed base 2100. Market size 12.000
•
All software vendors, even the ones with the most advanced applications, recognise that
quality labelling/certification had a positive impact on the development of functions/features
within their software.
•
A remarkable interoperability improvement as a result of a two stage process
implementation within the labelling cycles:

Labelling of import & export features of patient information (clinical and
administrative) between two users of the same software package (labelling
2004)

Import/export between different software packages without any information
loss, using standardised data structures (labelling 2005)

Improved integration of messaging (lab results, admission & discharge,
examination reports, medical images etc…)

In depth implementation of standards (CEN TC251 standards)

Extended encoding of information (ICD-9 and ICPC 2bis)
More involvement of the software industry at the level of benchmarking is highly requested by
the industry. A proposal to change the procedure was submitted at the end of 2004. This
proposal mainly requested more timing flexibility i.e. to apply for the labelling cycle during a
period of 4-5 months/year instead of two days, as it is organised today. This flexibility will
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
18
introduce an additional cost (experts have to be more available), which the industry is ready to
take up.
Situation in 2010
Quality Labelling for EHRs
One of the tasks of the Belgian eHealth platform (founded in 2008) is to verify "whether the software
for the management of computerised patient files complies with the functional and technical ICT
norms, standards and specifications" and to register such software. Quality labelling of EHRs already
started back in 2000, and covered software packages used by general practitioners, physiotherapists,
dentists, nurses and pharmacists. A new labelling session for EHRs for general practitioners will take
place during (the second half of) 2010.
Description of the procedure
The new labelling session for EHRs for GPs will start on October 1st 2010. The tests will take place at
the vendors’ premises. Detailed scenarios will be used during the tests. The vendors have a demo
scenario at their disposal. A scenario consists of several testing criteria. For the present labelling
session, the criteria are listed below (see “Testing Criteria”). A test session is limited to 4 hours. The
eHealth platform will give their decision on passing the test no later than 21 days after the test has
taken place. Candidates that have not successfully passed the test, can request one (or more) new
test sessions. In that case, the applicants have to pay again for the tests, and the new test sessions
will again evaluate all criteria (not only the failed ones). Software that has successfully passed the
evaluation tests, will be “registered”. Users of “registered” (quality labelled) software will then
receive financial incentives.
Testing criteria
The criteria that will be used for the 2010 quality labelling are provided below:
1. Different users can access the application simultaneously. The software guarantees a unique
and persistent identification of current and past users.
2. Each access to a patient file is persistently traceable, human accesses as well as any other
kind of accesses.
3. The system manages the access rights to the EHR considering the following parameters: - the
quality and/or role of the user (e.g. kind of medical function, administrative function) - the
nature of the accessed patient data (e.g. history data, personal notes of the healthcare
professional, administrative date, allergies, etc..) - the intended process: reading, writing,
modifying...
4. The system enables identification of the patient based on the Social Security Number.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
19
5. The system is able to read the eID card as well as the Social Security Card to identify the
patient.
6. The system identifies, at creation, possible double patient records based on name, first
name, gender and date of birth or based on the Social Security Number. The system prevents
the creation of more than one patient record with the same Social Security Number.
7. The system displays presence or absence of a Global Medical Record agreement, the date of
signature and the ID (Professional ID) of the responsible healthcare professional.
8. The system offers a mapping to ICPC-2 and to ICD-10 coding schemes in case of using a
proprietary terminology.
9. The system uses the ATC and the CNK (national package code) when prescribing a brand
medicinal product, being compatible with the authentic source of medicinal products
available through the eHealth-platform.
10. The system displays, when selecting a medicinal product, the "cheap" products as defined in
the Royal Decree of 15 September 2005, modifying article 73§2 of the law related to the
mandatory social security, coordinated on 14 July 1994.
11. The systems enables definition, data entry and management of "care elements" or "health
issues".
12. The system exports standardised electronic messages, type "Sumehr" (Summarised
Electronic
Health
Record).
https://www.ehealth.fgov.be/standards/kmehr/en/transactions/home/transactions/index.x
ml
13. The system offers a viewer on the Summary Electronic Healthcare Record to validate its
content.
14. The system enables partial or complete export of all the available data out of one or more
patient records, after applying a query. The exported data are accessible through external
tools for database management or statistical tools, that are readily available on the market
at the time of testing.
15. The system exports, in order to validate the technical documentation, the complete dataset
of at least ten chronic patients, as defined in criterion 14. The supplier deposits a complete
description of the exported files. ProRec-BE will store the files and the documentation and
keep the content confidential. The files and the documentation will be available to any
requesting supplier in case of "incapacity" of the original supplier. The export files will be
validated by two randomly designated competing system suppliers, up to the integration of
the patient data into their own application. Each supplier will validate the data out of two
competing systems. Suppliers with more than one application on the market will validate
three different competing systems, also designated at random. The supplier accepts that a
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
20
recuperation of patient data from an "incapable" system will never cost more than one
annual funding provided by the Social Security system for using a labeled EHR system.
16. The system is able to identify the yearly contact group of patients (i.e. patients that have
been in contact with the user during the past year) having their Social Security Number
entered in their file.
17. The system integrates the for free available plug-in "Evidence Linker", enabling the users to
access for diagnostic statements good practice requirements and recommendations offered
on the web site of CEBAM.
18. The systems accesses and uses the encryption tool offered by the eHealth platform.
19. The system supports the data entry of preventive services related to cervical cancer as well
as managing, based on approved good practice requirements, patient information and recalls
or reminders.
20. The system supports the data entry of preventive services related to colorectal cancer as well
as managing, based on approved good practice requirements, patient information and recalls
or reminders.
21. The system supports the data entry of preventive services related to breast cancer as well as
managing, based on approved good practice requirements, patient information and recall
Denmark
Situation before 2007
Scope of the labelling procedure
Currently, the EHR conformance testing and certification processes in Denmark address both the
EHRs in primary care and the EHRs in hospitals.
Regarding certification of EHRs in secondary care (hospitals) this development is closely linked with
the National BEHR (Basic Structure for Electronic Health Records) project, initiated by the Danish
National Board of Health 2000. Based on standards elaborated by the National Board of Health, this
project creates the foundation harmonised concepts for the EHRs in hospitals. The certification of
EHRs in hospitals is currently in a pilot phase.
An important aspect of the EHR implementation is to ensure that vendors of EHR systems are
involved in testing and the further development of BEHR to stimulate the use of the concept model
in their solutions. One of the tools to ensure a high degree of consistency when implementing BEHR
has been addressed in the IT strategy by specific action. The Danish National Board of Health
launched in June 2003 a National BEHR validation project, with the objective to establish a number of
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
21
reference implementation of the BEHR model in the Danish Hospitals. An important outcome of the
project was to assess the clinical impact of using the BEHR model.
The Danish EHR Observatory (see below) was given the job to assess the project within 3 areas:
1. Certification: validate to what extend the EHR prototypes are based on the national BEHR
model
2. Exchange of information between EHR systems
3. Assessment of the clinical impact by using BEHR.
The Danish Electronic Health Record Observatory corresponds to the PROREC centres found in other
European countries. It was launched in 1998 by the Danish Ministry of Health. The purpose of the
EHR Observatory is to support the realisation of the National Health IT strategy by monitoring and
assessing the development, implementation and EHR application in the Hospitals. The EHR
Observatory is disseminating national and international experience and best practice to the EHR
actors. The Danish EHR Observatory is funded by the Ministry of Interior and Health, the Association
of Danish Regions and the Copenhagen Hospital Corporation.
Regarding the certification of EHRs in primary care, the scope is limited to the message standards
within primary care and between primary care and secondary care. This certification was initiated in
2000 and has been in regular use since 2002. All communication is based on a Danish version of UNEDIFACT and six different EDI standards for (discharge letters, referrals, lab requests and results,
prescriptions and reimbursements). There are several sub-types for each EDI standard, so the total
different messages is 36.
To ensure unambiguous description of the standards and to avoid misinterpretation or erroneous
display of the communicated data, the original Message Implementation Guides (MIGs), were
customised and made more precise. Consensus was achieved by involving Health Professionals
Advisory Groups which included representatives from the relevant Medical Societies. The work from
the healthcare professionals were validated and adjusted by standardisation experts from the
software vendors.
Agreement to implement the standard were made between the various parties resulting ion
contracts between the relevant software vendors offering systems to the health sector, all 15
hospital owners and regional reimbursement organisations (counties), all laboratories, etc.
The new MIGs consisted of 36 standards in separate documents each covering a specific interface,
i.e. a biochemistry lab result sent from laboratories to GP systems. The new MIGs were distributed to
the vendors and made assessable via the web. Training courses were organised for programmers
from all participating software vendors.
After implementation of the new MIGs, the vendors applied for a certification by the National Health
network, MedCom. MedCom has developed a software product: EDI-CHECK, where each type of
message can be tested according to the new MIGs. The results of the tests and certifications are
published on the MedCom website.
A “sender” makes a series of documents covering all information stated in a specified test set. The
message is sent it to MedCom, which runs it through the EDI-CHECK programme. A check report
without any errors is needed before a certificate can be issued.
For a “receiver”, MedCom sends several test documents by EDI the receiver must show different
types of documents are received and presented according to the MIGs. If the sender or receiver
passes the check without any errors, a certificate can be issued.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
22
In the following sections, the certification process regarding EHR-systems in hospitals is elaborated.
Rationale
In 2003 the Ministry of Interior and Health approved a new national IT strategy for the Danish Health
Care Service for the period 2003 to 2007. The purpose of the National IT Strategy for the Danish
Health Care Service is to establish a common framework for the full digitalisation of the health care
service during the period 2003–2007.
The fiscal agreement between the government and the counties (i.e. the hospital owners) states as a
common goal that electronic health records based on common standards is to be implemented in all
Danish hospitals by the end of 2008. The common standards in question are mainly the EHR model
(The Basic Structure for Electronic Health Records, BEHR) and a national terminology system.
Currently preparations are made for SNOMED CT to substitute the national classification system.
The aim of the common standards and common terminology for health records and IT systems is that
data can be shared and that they can efficiently support interdisciplinary high quality care. The
purpose is also to enable communication of data between EHRs and other IT systems without forcing
the involved parties in the health care service to utilize systems from the same vendor.
The dissemination of EHR is monitored and surveyed by the EHR Observatory where the number of
beds covered by EHR is used as an indicator for the national coverage. In June 2005 the national
average coverage was approximately 30%.
Description of the procedure
The EHR model
The procedure of certification of EHRs in hospitals is based on requirements from the BEHR, and the
principles of the model are therefore described briefly below.
Diagnostic
Evaluation
consideration
Diagnosis
Goal
Planning
Outcome
Executing
Intervention
Process
Information
Figure: Basic Structure for Electronic Health Records
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
23
The “Basic Structure for Electronic Health Records” is a generic information model for clinical
information systems published by the National Board of Health. This model sets the national
standard for EHRs. It is characterized by using the clinical problem solving process as the structural
backbone for all clinical information and it is designed to directly support continuity of multiprofessional health care across all environments of the entire health care service.
The Clinical Process is a specialization of the general method for problem solving and in the health
care domain the BEHR model includes at a generic level: ‘Diagnostic Consideration’ leading to a set of
‘Diagnosis’, ‘Planning’ leading to a set of ‘Interventions’, ‘Execution’ of ‘Interventions’ leading to
‘Outcome’ and ‘Evaluation’ of ‘Outcome’ validated by ‘Goal’.
1. Diagnostic Consideration
Diagnostic Consideration is a process of collecting and analysing facts in order to understand the
patient condition and determine the problems faced. This process implies that the health
professionals, describes the problems that are in focus. The documentation of the problems is
expressed primarily as structured diagnoses by all kind of health professionals (doctors, nurses,
physiotherapists). ‘Diagnosis’ is a clinical judgment where an actual or potential patient condition is
in focus. Within the context of the conceptual model this professional judgment is defined in a much
broader sense than a solely medical view.
2. Planning
Planning is a process during which interventions to be performed are outlined according to expected
or desired outcomes. This process implies that health professionals add a number of concrete
interventions for diagnostics, treatment, prevention, care, rehabilitation etc. The BEHR model
requires that one or more diagnosis should be indicated for each intervention.
3. Outcome
Outcome is a documentation of the actual results from the performed interventions. In this context,
outcome is seen broadly as information about the patient's condition, i.e. results of examinations as
well as different kinds of treatments and preventive actions e.g. medication, nursing, surgery,
rehabilitation programmes etc.
4. Goal and evaluation
A goal in BEHR has to be operational and is the documentation of what is expected or desired
outcome of intervention. If the expected outcome does not meet the expected goal, the health care
professionals have to continue a new round in the BEHR model.
The requirements
The requirements from the national Board of Health can be summarised as follows:
1. The EHR system has implemented the BEHR model, including the central use cases, the
Reference Information Model (RIM) and the business rules.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
24
2. The EHR system is designed as a multi user system.
3. The EHR system is user friendly and has an acceptable performance
4. The EHR system is stable and includes all aspects of security
The BEHR model is specified in a number of documents and includes use-cases, business rules, RIM
model, data descriptions and XML-schemas. The specifications include also a data set and data
scenarios to validate if an EHR system is compliant with BEHR specifications. The data for validation is
extensive and includes approximately 500 individual tests.
Only the first requirement point is included in the formal test set. The business rules also include
aspects of data security and data consistency. The other points are judged informally by the
evaluators. The security aspects mentioned in point 4 is related technical issues like to secure logon
and transaction logging, and is not a part of this certification.
B-EHR documents
Use cases
B-EHR test documents
Rules
Specification
Test-set
Scenes
Test-data
Logical model Other docs.
Test plan
EHR prototype
Test protocol
Test log
Usability
Delivery report
• Approved
• Not Approved
Figure: Methodology for EHR certification
The certification methodology is used to validate to what extend an EHR system is compliant with the
specifications as shown on the figure above – i.e. the BEHR specifications and the BEHR test
documents.
The test of an EHR system is done in two phases. In the first phase the test is done internally by the
vendor of the system (self assessment). When the vendor finds, that the system is mature to pass the
test, a new test is performed together with the evaluators.
The purpose of the self-assessment phase is to allow the vendor to try-out the methodology
internally and ensure that the expected result of the assessment can be fulfilled. When the start for
the self-evaluation is agreed with the evaluators, the vendor has maximum 2 weeks to perform the
self-assessment. During the self-assessment, the vendor documents the result of each individual test
in a test protocol and a test log. In case the test protocol documents a result below an agreed
threshold, the vendor requests the evaluators to perform the final assessment.
The purpose of the assessment phase is to examine to what extend the EHR system is compliant with
the BEHR specifications. The assessment process is fully documented and allows that parts of the
test, which fails can be assessed later on. The time for the assessment is scheduled to maximum 2
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
25
days and the results of the individual tests are recorded in a test protocol and test by external
evaluators.
By the end of the assessment a delivery report, which concludes the overall result of the test is
prepared. The delivery report is starting by concluding if the EHR system passed the assessment (Yes
or No). If the EHR system did not passed the test it can be dependent approved which means that the
vendor has to fix some minor errors. In case that the result documents many deviation from the
BEHR specifications or that important elements is not implemented accurate a new assessment is
needed. The assessment phase is finalised by the EHR Observatory and the vendor is signing the
delivery report.
A prerequisite for running the assessment phase is that the vendor documents that the EHR
installation and environment correspond to a real operation. This implies that the installation
includes all necessary components and resources to avoid that the tests is not influences by
irrelevant causes.
Economic Model
Since the certification process is not yet formally established, the economic model and organisation
is not decided.
I the current situation a co-funded model have been used. The National Board of Health has
supported a part of the adjustment to the BEHR model and funded the external evaluators. The
vendors have not paid for the validation, but have provided resources for the self assessment and
the external evaluation.
Criteria
The criteria are related to the use cases, which is a part of the BEHR specification.
There are criteria in the following areas:
Administrative data management
 Admit patient
 Handle information about relatives
 Handle central reporting
General functionality
 Provide patient overview
 Find record
 Read record
 Write record
Specific clinical functionality
 Handle indications
 Handle diagnose hierarchies
 Handle interventions hierarchies
 Refine hierarchy objects
 Document assessment
 Document planning
 Document activities
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
26


Document evaluation
Display all above information including
 Notes on diagnoses, planning and activities
 Complications
 Goals
 Intervention results
 Patient basic information
Decision support
 Monitor documentation
 Display guidelines
Data management
 Correct information
 Invalidate information
 Display corrected information and corrected status
Testing Scenarios
For each of the 500 tests, the following it is described:

The pre-condition, i.e. the information the system contain and state of the system before the
test is run

The steps to be performed during the test

The post-condition, i.e. the expected state of the system should be in after the test

A discriminator indicating if the test is passed
This information was provided as supporting documents to the test protocol itself. These documents
are described below.
The pre-condition information is presented as power points showing the example data and status of
various objects. The same diagram is often covering several tests.
The scenarios of selected post-conditions, presented as simplified user interfaces.
It is not mandatory to use the example data in the pre- and post-condition documents. In practice
they are not used as such, but they provide a better understanding of what the test is aimed to
check. It is therefore used as reference material during the procedure.
The test plan describes each step of the test. The plan includes descriptions of

The background material

The self assessment

The external assessment

The time schedule for the test process

Requirements for the testing environment

Criteria for acceptance of the test
 The documentation of the test
The test protocol contains is the actual lists of the tests. This is made in a spreadsheet with a
registration facility to be used during the test.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
27
Both the self assessment and the test performed by the evaluators are using the same test protocol,
based on the BEHR test-set. An example of a filled protocol is shown below.
Technical validation of GEPKA projects
Testprotocol for XXX County pilot
Validation date
Test
nr
Sub
UseCase
101
1
Indskriv patient
102
103
1
1
Indskriv patient
Indskriv patient
104
105
1
1
Indskriv patient
Indskriv patient
100
105
106
106
106
107
107
107
108
108
109
Topic
01 Admit patient
Der er mulighed for at angive en tid som vedrører
en hel arbejdsgang.
Det er muligt at håndtere SOO'er.
Det er muligt at manipulere diagnoser.
13-14 April 2004
Approved
NOT approved
Blank
Approved of total
tested
303 = 61,3%
183 = 37,0%
5 = 1,0%
62,9%
x
x
x
x
Det er muligt at manipulere interventioner.
x
Det er muligt at håndtere, at et forløb skifter
x
forløbsansvar.
2
(regel)
Ændringer i forløbsansvar skal ske konsekutivt.
1
Indskriv patient Det er muligt at håndtere, at en tilstedeværelse
x
skifter pladsressource.
2
(regel)
Til en Tilstedevaerelse kan der kun være én åben
Pladsressource af en given art.
3
(regel)
Pladsressourcer af given art skal afløse hinanden
konsekutivt.
1
Indskriv patient Det er muligt at håndtere modtagen henvisning
x
(diagnose findes ikke i forvejen).
2
Indskriv patient Systemet sørger for, at der til en tilstedeværelse
x
altid er oprettet interventioner til indhentning af
basisoplysninger.
3
(regel)
Pladsressourcer kan ikke tilknyttes
Tilstedevaerelse, som ikke eksisterer.
1
Indskriv patient Det er muligt at håndtere modtagen henvisning
x
(diagnose findes i forvejen).
MIE 2005
2
Indskriv patient Systemet opretter ikke yderligere interventioner af
x
typen 'Indhent CAVE-oplysninger' og 'Indhent
basisoplysninger', hvis der allerede findes åbne
interventioner af denne type.
1 Figure:
Indskriv patient
Det er muligt
håndtere, at patient
ankommer
An example
of at
a fragment
of filled
test protocolxfor
som konsekvens af planlagt tilstedeværelse.
x
x
NOT approved of
total tested
38,0%
kan ikke se tiden ved opret + visning. Tid vises først når SOO e
x
x
x
x
x
x
x
x
x
systemet går ned
x
x
x
x
x
x
MEDIQ
er testen korrekt?
x patient"
er altid med anvar (egen afd. er default)
the use case "admit
Lessons learned
In the original test-set, there was no differentiation between mandatory (must be fulfilled) and semioptional (should be fulfilled) and optional (recommended) requirements. It is suggested that this is
implemented in the future. Furthermore, the test set could probably be reduced, since similar
functionality is tested in various ways.
A prerequisite for the assessment was that the EHR installation and execution environment
correspond to a real operational environment and that the installation had all necessary components
and resources. However it might not be possible to fulfil this requirement, due to security reasons
and practical reasons.
Experience has shown that the evaluators found a lot of errors in the EHR systems, which was not
identified during self-assessment. Thus, the external assessment methodology can also be used by
the vendors to improve the quality of the system.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
28
Situation in 2010
MedCom is responsible for development and maintenance of standards for electronic
communication in the health sector. This includes definition of content and practical testing and
certification of the vendors’ implementation. About 15 different electronic patient record systems
are interoperable in the primary care sector and four different homecare records are used in the
municipalities. Through a consensus process with vendors of almost 100 systems, all patient
management systems for hospitals, GPs, homecare and pharmacies have incorporated a common
messaging system and have been tested and certified by MedCom. In 2004-2005 a new standard for
EHR in Hospitals (B-EHR) was developed by the National Board of Health, including approx. 500
functional statements for testing EHR applications. The functional statements were used as Danish
requirements in the QREC project. However it was decided to terminate B-EHR in 2008 and today
Denmark does not have any certification for EHRs in Hospitals.
MedCom has established a test centre and developed a set of test tools to support eHealth
implementation and provides test and certification to the suppliers. All certification is done by
MedCom. In order to get certified, the vendor has to pay a fee.
The certification is for standards for "messaging" with focus on cross-sector communication for
frequent messages in large volumes (eq. prescriptions, discharge letters, referrals, consultations
notes, laboratory results and reimbursement). All Danish vendors for EHR systems (primary and
secondary care), community care systems, PAS-systems, laboratory, dentists, physiotherapy,
RIS/PACS etc. have implemented the standards and have been tested and certified by MedCom.
For each standard MedCom has developed a national profile for the implementation and certification
(in Danish).
Greece
Situation in 2010
The information in the following sections has been collected in August 2010.
The certification and quality labeling procedures experience in Greece has been part of the Regional
Health Information Networks development process and have been performed per project for each
one of the health regions. Currently, there is no unified approach to certification.
Certification concerns both functional and interoperability aspects of the information system. A third
party, the technical adviser of the regional health authority, prepares a set of scenarios of use against
which the functionality of the regional health information network software components are tested.
Scenarios are based on work flows within health organizations (Hospitals, health centers). The
scenarios are reviewed and updated with input from key users. Software components of the RHIN
are then tested and certified for satisfying the scenarios and user requirements. The second level of
certification is performed concerning integration both within and across healthcare organizations.
Interoperability among software components of the RHIN (i.e. end user applications) is tested in
order to provide certification of the system at the integration level.
All health care domains are targeted including ERPs, patient administration software, medical
records, booking, LIS, etc.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
29
The type of certification is a third party certification. There is one technical adviser per regional
health network project for the duration of the implementation of the regional health systems. In
collaboration with key users from the region, scenarios and requirements are established.
The economic model behind the certification activities in Greece is that certification occurred as part
of the implementation of Regional Health Information Networks. A specific budget was allocated for
each of the projects and concerned the overall activities of the technical adviser of each project.
Ireland
Situation before 2007
Scope of the labelling procedure
In order to understand the history, current status and future prospects for labelling/certification of
Electronic Health Records and other related systems in Ireland, it is necessary to have some
background knowledge of the Irish health system. In particular, it is important to appreciate the
cultural and organisational factors that have shaped and continue to determine the approach to
developing Information Technology in Irish healthcare.
Traditionally, the Irish health system has been very fragmented. For example, the population of 4
million is served by approximately 2,500 General Practitioners. About 40% of those GPs work in
single handed practices. While the concept of the integrated Primary Care Team is promoted by the
Department of Health and Children (www.dohc.ie), real progress has been slow to date due to
funding and cultural issues. Similarly, there is a lack of appropriate integration between the Primary
and Secondary Care sectors despite the success of electronic communications projects like Healthlink
(www.healthlink.ie).
In terms of service provision, a medical card scheme has been in operation since the early 1970s
providing free GP services and prescription drugs for eligible persons: that is all individuals aged 70
and over and any other person that meets a low income means’ test threshold. As the economy has
thrived, the level of eligible persons has fallen now to just under 30%. There is essentially universal
entitlement for in-patient hospital care but despite this almost 50% of the population have taken out
private insurance to cover their secondary care needs.
The institutional structure was also characterised by widespread fragmentation and unnecessary
duplication until the creation of the Health Service Executive (HSE) (www.hse.ie ) in January 2005.
That national body replaced over thirty previously independent regional and national healthcare
entities. It will be the HSE rather than the Department which will now drive initiatives such as the
Electronic Health Record –and related matters such as software certification- forward. Another new
body, the Health Information and Quality Authority (www.hiqa.ie) has been given formal
responsibility for standard setting and accreditation of healthcare information technologies and
information systems. However, HIQA is not yet operational and, therefore, it remains to be seen how
the relationship between it and the HSE will operate in terms of which body leads the way in
determining standards in a practical as opposed to formal manner.
Quite separate from the aforementioned healthcare bodies, there is the National Standards
Authority of Ireland (www.nsai.ie). NSAI formulates industry and product standards through
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
30
consultation with all interested parties bearing in mind market conditions and developing
technologies and taking into account demands for quality, design, performance, safety and
envirnonmental requirements. The primary means of achieving consensus is through a Standards
Consultative Committee. One of the Standards Consultative Committees is the Health Informatics
Standards Committee (HISC). There is also a wider ICT Standards Consultative Committee. The HISC
Committee has been very active in trying to establish working understanding with both the HSE and
HIQA.
Given its longstanding expertise in standards setting, especially its involvement with
international standards bodies, it is desirable that HIQA and the HSE draw on and work with NSAI in
developing an Electronic Health Record.
The creation of the Electronic Health Record has been widely stated –in official strategy papers and
reports including the National Health Strategy (Quality and Fairness, 2001) and the National Health
Information Strategy (Health Information, 2004) - as a major goal of the Irish health system.
However, the (i) fragmented structure described above together with (ii) the unique and sometimes
competing mix of the private and the public combined with (iii) a longstanding lack of trust between
service providers and the State have meant that while most stakeholders are positive in principle to
the idea of an EHR there are significant real obstacles towards its implementation.
Even simple and universally accepted measures that must be undertaken to progress EHRs in Irish
healthcare are sidestepped or pushed down the priority agenda. For example, there is no universal
personal health identifier in Ireland. However, there is a State issued Personal Public Social Number
(which originated as a tax and social security identifier) which individuals use in their dealings with
the State. A doctor may use the PPSN as a unique identifier for his or her medical card patients but
commits a criminal offence if he or she asks private patients for their number. The problems
associated with this scenario have been fully debated and the need for action publicly recognised but
nothing has been done and there are no active legislative proposals to address the matter.
To date, in Ireland, the only systematic and successful attempt to introduce certification for
healthcare management information systems was by the National General Practice Information
Technology Group (www.GPIT.ie). The Group was established as a broadly based representative
body by the Department of Health in the late 1990s with the aim of promoting information
technology in general practice.
Rationale
General practice is at the core of the Irish health service. However, the State has no direct way of
controlling or measuring how GPs use their computer systems. Levels of computerisation in Irish
general practice have risen sharply in recent years following a prolonged period of apathy. For
example, a survey conducted by GPIT in 2003 found that 83% of Irish GPs have a computer in their
practice and 88% of that figure have a computer in the consulting room. The survey also found that
most GPs see their investment in computerisation as having positive practice and patient benefits.
(See GPIT website for details from surveys in 2001, 2003 and 2005.)
The rationale for introducing certification system for GP Practice Management Systems was simply
that it made obvious sense to so do.2 It was hoped that the benefits that were inherent in such
2
The first software certification arrangements for general practice management systems in Ireland was
introduced in 1994 by the Department of Health & Children in conjunction with the Irish College of General
Practitioners. That system was administered by the Centre for Software Engineering in Dublin City University. It
was intended from the outset that the process would be reviewed in the light of experience and changing
times. In 1998, the then newly established National GPIT Group began examining the area. After a long review
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
31
process would spur others in the health system to adopt similar and compatible measures in their
own sectors. As the GP software certification scheme was entirely voluntary (see Economic Model
below), it had to appeal to all the stakeholders (individual GPs, their representative bodies, software
suppliers and a rage of health agencies and hospitals).
The GPIT Group identified the importance of Practice Management Systems not just to the GPs who
use them but to the wider health system as a key element of an integrated IT approach that would
support the development, over time, of the EHR. Promoting confidence in software systems among
users was also seen as central to the process. GPs want their Management Systems to deliver and
they tend, understandably, to judge the value of IT in terms of how these systems perform. The
certification system was seen as needing to deliver an objective and tested level of quality assurance
for any doctor purchasing a new software system guarantees a practical re-assurance of quality.
Accordingly, the bottom line for the certification process was to define baseline standards which
suppliers needed to meet if they wanted to be certified. These were known as the Requirements for
Certification (RFC).
Description of the procedure
The declared objectives of the certification testing process were to ensure that suppliers’ systems
met the mandatory requirements for practice management systems, according to the Requirements
for Certification specification 2002 (RFC02). Certification testing was never intended as a substitute
for comprehensive system testing by the suppliers. They were advised not to submit their software
for testing until they have completed their own internal testing programme. This latter point was
clearly more relevant in the case of new products entering the market rather than to the products
that were already in the market at the time of the introduction of the new certification programme.
The certification testing process set out to measure each system against the specification to ensure
that the functionality met the mandatory requirements in full and that the expected results are
obtained. Requirements were classified as mandatory, highly desirable and desirable. It was always
planned that those that were identified as highly desirable in any particular testing period would be
mandatory next time around.
The criteria for certification were based on each system meeting the mandatory requirements in
relation to:
•
•
•
•
•
data structure and coding;
data input,
validation and processing;
the content specified for display and outputs; security
control and audit requirements.
process dominated by a very open consultative process, the Group felt it was time to radically restructure the
certification programme to make it more flexible and workable from the perspectives of all the stakeholders.
The new scheme was implemented in 2003.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
32
The General Functionality and System Architecture requirements together with other background
material on certification can be found at www.gpit.ie/software.html. This provides details of test
conditions which were intended as examples to give suppliers an indication of what would apply in
the testing process.
The results of the certification process were published on the National GPIT Group website and
advised to health agencies. In order to promote a positive image of certification, it was decided that
vendors whose products did not meet the required standards would be given the option to withdraw
their products rather than have them failed.
Where the product was successfully tested, certification was issued to the company concerned in
respect of the product. Such product certification status was valid until the next round of
certification with the proviso that GPIT Group reserved the right to suspend certification status
should there be any degradation of the system or supplier performance that would result in the
system no longer meeting the RFC standard.
While all the stakeholders supported the certification process they tended to have their own
particular ambition for it. For example, individual clinicians wanted certification to act as a guide to
purchasing a practice management system while their professional education and training body (Irish
College of General Practitioners) were mainly concerned that the RFC should be used for setting and
enforcing guidelines for evidence based medicine and best practice. Software suppliers wanted
certainty. Health agencies wanted to ensure a greater degree of strategic control.
Economic Model
Ireland’s public expenditure on health has been buoyed by the very strong growth in recent years.
Expressed as a percentage of GDP, it remains below the EU average, but as a percentage of GNP it is
now the highest in the EU and in per capita terms it is the third highest. When it comes to IT
development, the amount spent is small. The amount that is actually available for capital spending
on IT in 2006 (as per the Official Estimates) is set at €70m –that is no increase on 2005 (which itself
was no material increase on 2004). In a recent European study, it emerged that Ireland spends less
than 0.5% of its health budget on IT, compared to more than 4% in the Netherlands. There are three
main reasons for this scenario in Ireland. First, IT spending has never enjoyed a high profile and has
traditionally been a poor relation in healthcare funding allocations. Second, there were a series of
well publicised failed ICT healthcare projects of 2005 which has made it even less attractive
politically. Thirdly, clinicians and patients have not promoted it as a priority.
As a proportion of total health expenditure, ICT currently only consumes in the order of 1%. This low
level of investment in ICT has left the health system in a very weak position with respect to its ability
to manage itself and obtain value from the remaining 99% of health spending.
A major study on eHealth in Ireland was published by the Information Society Commission in
December 2004 ~An e-Healthy State? (www.isc.ie ). The report found that Ireland currently spends
less on ICT in healthcare than (i) investment levels internationally and (ii) accepted ICT investment
levels in other economic sectors. It recommended a significant increase in Government spending on
ICT in the healthcare sector, to deliver benefits and savings to all stakeholders.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
33
In the general practice area, a grant scheme for investment in computerisation existed during the
1990s but was phased out. There is, however, a scheme to encourage doctors to prescribe drugs
rationally (called the Indicative Drugs Target Saving Scheme). This allows GPs to keep any monies
saved on a notional prescribing budget for use in practice development. The final decision on
whether a doctor’s proposed expenditure is within the IDTSS is made by local health management.
Given that the available evidence demonstrated that savings under the IDTSS were the principal
source of GPs’ expenditure on computerisation, there was agreement among everyone involved in
the certification programme that only certified software systems should qualify for support under
the drugs saving scheme. It was felt that this arrangement would act as an appropriate and
compatible incentive to a voluntary certification programme: that is, no software supplier was forced
to submit is product for certification and no doctor was forced to buy only a certified system but
both were incentivised to do so.
Criteria
This RFC is detailed and structured and consists of the following parts:
Part A General Functionality
This part details requirements in relation to practice and patient administration, clinical and
prescribing activities and reporting facilities. It details each required function, followed by the
data which should be stored and the testing criteria that may be used in the certification
process. The data to be stored is held in Schedules numbered in accordance with the
required function which will provide for easy modification of these data requirements as they
change over time. There are specific functions detailed that do not have related data items
and will be followed by the testing criteria only.
Part B System Architecture Requirements
This part covers general aspects of services and requirements that impact on other parts of
RFC including, confidentiality and security, Year 2000 conformance, coding and data
portability. As there is no data requirement relating to these functions, each function is only
followed by the testing criteria as required.
Part C Supplier Services
This part includes requirements on support, documentation and training and identifies the
need for practices to work in partnership with suppliers to benefit from the training and
support offered.
Part D Messaging and Information Exchange
This parts sets out, in the absence of any national strategic guidelines, the broad
requirements for the exchange of EDI and e-mail messages between GPs and the full range of
relevant agencies. This section, in particular, may come under very regular review as this
technological area and the associated standards develop and embed.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
34
Part E Future Strategy
The purpose of this section is to provide suppliers with an indication of other IT initiatives and
developments that may be included in future versions of RFC. This section does not form part
of the actual certification process and therefore, has no requisite testing criteria.
Testing Scenarios
The Test Pack (available on the GPIT website) is comprehensive and covers tests for everything in the
certification specification. It includes the Mandatory as well as the Highly Desirable and Desirable
items. Most tests in the test pack follow the order of the Requirements for Certification
specification, but in some instances, this is not possible for practical reasons. The elements of the
testing arrangements are set out below. The website also contains a Priority Document which was
prepared during the consultation period and was intended to help parties understand what would be
in the RFC. It should be read in conjunction with both the RFC and Test Pack (which it mirrors).
Certification Testing
Conformance testing is a pre-requisite for certification. Testing will be carried out by the national
GPIT Group. The tester will have control of the testing process and if products are deemed
incomplete or ill prepared testing will be suspended. The General Functionality and System
Architecture requirements parts provide details of test conditions in the boxes below the required
functions and are intended as examples to give suppliers an indication of what will apply in the
testing process. However, it would be inadvisable for suppliers to base their systems developments
solely on the examples provided. Testing will check for both the presence of a requirement and
where appropriate, its ease of use. The results of the certification process will be published on the
National GP IT Group website upon completion.
Timing and Location of the Test
Certification testing is normally conducted at the suppliers’ premises using their hardware. During
the testing, the systems will be operated entirely by the suppliers. Suppliers need to provide an
operator who is familiar with all aspects of the system.
Observations
Each observation is noted and discussed during testing, and a summary report is provided on
completion of testing for reference or action, as appropriate. Observations are reviewed and given
an appropriate priority as follows:


Critical: - Any observation which relates to major data corruption, major omissions in
functionality or any other serious condition, thereby making it impractical to proceed with
the certification test. In this case, arrangements will be made to recommence testing after a
suitable interval, according to the availability of the tester;
Major: -Any observation where the system is prevented from working correctly, gives
incorrect results or omissions in functionality, thereby preventing or making it impractical for
large areas of the system to be tested;
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
35

Medium: -Any observation where the system is prevented from operating correctly, gives
incorrect results or omissions in functionality, that can be allowed for in certification testing
and enable testing of that part of the system to continue;
Low: -Any observation where the system gives the correct results but requires a cosmetic
change to conform to the requirements.

Regression Testing
The resolution of the observations raised during testing will require re-testing of the affected areas
once the necessary corrective action has been taken. The extent of the re-testing will depend on the
nature of the observation and the consequent impact of the necessary corrective action. If a
significant number of substantial software amendments have to be made as a result of observations
raised during testing, it may be necessary to repeat earlier testing to prove the integrity of the
software amendments.
PART A
1.
1.1
1.2
1.3
2.
2.1
2.6
2.2
2.3
2.4
2.5
3.
3.1
3.2
3.3
3.4
4.
4.1
4.2
4.3
4.4
4.5
5.
5.1
5.2
5.3
5.4
5.5
GENERAL FUNCTIONALITY
PRACTICE INFORMATION
The Practice
Practice Staff
Related Organisations
PATIENT REGISTRATION
Patient Details
Patient Import File
Minimum Registration
Ease of Registration
Patient Uniqueness/Duplicates.
Data Take On
PATIENT HISTORY OR MEDICAL SUMMARY
Summary Display
Summary of Relevant Clinical History
Family and Social History
Drug Allergies and Sensitivities
CONSULTATIONS
Consultation History
Record Consultation Details
Structured Clinical Details
Clinical Protocols
Ease of Consultation Data Entry
INTERVENTIONS/INVESTIGATIONS
Practice Based Interactions/Investigations
Referrals
Requests
Vaccination/Immunisations
Ante-Natal and Post-Natal Visits
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
36
6.
RECALL FACILITY
7.
GENERATE FEES
8.
PRESCRIBING
8.1
Drug Database
8.2
Generic Prescribing
8.3
Practice Formularies
8.4
Generate and Print a Script
8.5
Script Management Facilities
9.
DISPENSING (optional)
9.1
Dispensing System
10.
REPORTING FACILITIES
10.1 Standard Reports
10.2 Ad Hoc Reports
10.3 Batch Reports
10.4 Letter Writer
11. APPOINTMENTS
12. ACCOUNTS/BILLING
13. DATA MERGE
PART B SYSTEM ARCHITECTURE
1.
IDENTIFICATION AND AUTHENTICATION
2.
ACCESS
3.
AUDIT.
4.
INTEGRITY
5.
SECURITY
6.
YEAR 2000
7.
DATA PORTABILITY
8.
SYSTEM CONFIGUATION
PART C SUPPLIER SERVCIES
1.
Support Contracts
2.
Help Desk and Fault Logging
3.
Documentation
4.
Training
5.
User Groups
Lessons learned
It is difficult to be certain about what lessons, if any, have been learned. There are several reasons
for this situation. The significant institutional reform process now underway in the Irish health
system has created some uncertainty about roles which has affected promotion of a standard setting
process in healthcare ICT. There is some evidence that certain parties hold the view that buying in a
systems-wide ready made commercial package obviates the need for a major standards setting
process. On the other side, there is increasing recognition of the need for standards in areas of
electronic communications, namely HL7.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
37
At a more concrete level, in December 2004, the GPIT Group commissioned a formal independent
evaluation of the Certification process. The Report (published in 2005) reached the following
conclusions which it is hoped will serve, at least, as a basis for future thoughts on certification:

The Certification Programme introduced by the National GPIT Group in 2003 was an
important innovation in healthcare ICT but it now needs to move forward in line with
changes in the requirements and structures of the Irish healthcare system.

A voluntary system of certification of GP Practice Management Systems (PMS) is the
appropriate mechanism for improving the quality of Irish GP computer systems. It needs to
be part of a coherent strategy to ensure GPs are financially driven to only use certified
systems.

Free choice of GP PMS systems with appropriate certification to ensure value for money,
adequate functionality and the ability to communicate with the wider health service is the
most appropriate way forwards for Ireland.

The ownership of a Certification Programme should rest with those in the Irish healthcare
system charged with setting standards. On the basis of the reform programme, this means
the Health Information and Quality Authority when operational. However, at a policy level,
the Department of Health & Children and, at an implementation level, the Health Service
Executive have very important roles to play.

The person or other agency carrying out the actual product assessment on behalf of the
standard setting body should have both technical and healthcare experience to be able to
evaluate the system properly.

The Specification should describe standardised ways in which the data should be recorded
and exported. National standards are needed in several areas. The way in which these data
are handled within the individual GP systems should be left to the designers of the systems.

The Certification Programme should be extended to include messaging, coding and data
standards. The ICPC clinical coding system should be mandated of Ireland at the present
time.

To ensure necessary buy-in, there should be consultation with all the stakeholders in moving
the present Certification Programme forward.

The re-designed RFC should be published in an unambiguous manner with all the information
necessary to design systems and test them made available.
Situation in 2010
In 2006 the Health Service Executive and the Irish College of General Practitioners have restructured
the National General Practice Information Technology (GPIT) Group. One of the tasks of the GPIT is to
promote interoperability and health informatics standards in the health services, and to certify GP
Practice Software Management Systems. Between November 2008 and February 2010, five software
products have achieved a quality label. The procedure has been described in detail in the “General
Practice Software Management Systems – Requirements for Certification 2007” document, which
can be downloaded via the GPIT’s website. The conformance tests concern both core – and
interoperability functions. The testing takes place at the vendors’ premises. A representative of GPIT
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
38
is present. Vendors have to provide a specific amount of networked computers, printers and a
backup system. The software must be preloaded with at least two hundred test patients. The
certification process consists of seven steps:
1. The GPIT group informs the vendors of the certification period and request the
necessary documentation
2. The vendors provide the requested pre certification documentation and request the
test date
3. GPIT agrees on the test date and the test environment
4. The GPIT representative performs the testing
5. The GPIT representative recommends whether the vendor has passed the test, has
to completely retest, has to partially retest. The vendor can also withdrawn from the
testing
6. GPIT Group takes a decision whether the software has passed certification and
publishes the list of certified software
7. Vendor has right of appeal (to an independent committee)
Products that have achieved certification will obtain a certification logo issued by the GPIT group.
Software companies can use this logo for dissemination purposes (website, brochures, ...).
According to the GPIT group, certification is considered important because of the following reasons:

It ensures that the software supports the requirements of general practice;

It helps in coordinating the development of general practice systems with developments in
health service information systems;

It provides a roadmap for further development of EHRs.
Serbia
Situation in 2010
The first activities related to certification and / or quality labelling procedures in Serbia began
through the Serbia Health Project, funded by the World Bank, in 2006. One of the tasks of this project
was the development and implementation of hospital information system (HIS) and software for
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
39
primary health care (PHC SW), and the objective was to build a national standard for software in the
health sector, that is at this stage, ensuring interoperability framework.
In this regard, the Ministry of Health, supported establishment of ProRec Serbia, which has become
full member of the EuroRec Institute in 2008. The founders of ProRec Serbia are key persons of
Ministry of Health, Health institutions, Health Insurance Fund, Software Houses and Belgrade
University.
In 2009, Serbian Government adopted the Rulebook on more detailed contents of technological and
functional requirements for the establishing the integrated health information system (IT Rulebook).
This legal document contains the technical requirements for all software products and functionalities
for HIS and PHC SW, for now. Also, all requirements of EuroRec Seal Level 1 (2008) are implemented.
The new version of the IT Rulebook is planned for 2011. Currently, several major IT projects being
implemented in Serbia and all the software that will take part in them must be in accordance with
the national IT Rulebook. At the same time, the largest software providers from Serbia will apply for
EuroRec Seal Level 2 until the end of 2010.
In the present version, the certification / quality labelling procedures define functional requirements
for hospital information system (HIS) and software for primary health care (PHC SW). Also, the
national IT Rulebook defines the coding system and a set of relevant data. So the interoperability of
the whole system is provided. Within the certification/quality labelling procedure the primary,
secondary as well as tertiary health care is targeted (hospital information systems and software for
primary health care).
The type of certification/quality labelling is a third-party assessment. The procedures are performed
by the IT Commission of MoH. A number of IT Commission members are also members of ProRec
Serbia.
Slovenia
Situation in 2010
Currently (August 2010), the only EHR related certification and/or quality labeling procedure in
Slovenia is the one initiated by the EuroRec Institute. Moreover, the Slovenian SW vendor Audax
d.o.o. was the first in Europe been awarded the EuroRec Seal Level 2 certificate for its EHR products.
Prorec Slovenia has acted as the main driver in the procedure of obtaining the certificates. The
Slovenian Ministry of Health has shown serious interest in these developments. It seems that there
might be room for the development of some sustainable EHR certification/quality labeling
procedures within the currently run e-Health state project. The other quality labeling procedure of
the software systems has been and is being performed by the Health Insurance Institute of Slovenia.
However, the focus of the assessment lies with the reimbursement procedures rather than with the
EHR functionalities.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
40
Prorec Slovenia has taken the approach that combines both self-assessment and third-party
assessment procedures. However, the final assessment is always done by a third-party evaluator.
There is currently no particular economic model behind the so far performed EHR certification
activities. The main driver for the software vendors to participate in the certification / quality labeling
activities is to gain market advantage over their competitors.
United Kingdom
Scope of the labelling procedure
The NHS (England) can be considered as having two distinct phases of quality assuring its healthcare
information systems:

an accreditation approach, with published functional and non-functional specifications and a
formal external conformance testing process;

managed procurement, in which the specifications form part of the procurement process.
The accreditation approach has been used for GP computer systems between 1989 and 2001. The
managed procurement approach has been adopted by the National Programme for IT, from 2002.
Although no formal accreditation standards from before 2001 are still in force, it is valuable to
consider how GP systems have developed, and these accreditation artefacts in particular, since they
still represent good examples of system specification and accreditation testing processes. These GP
systems are generally regarded as amongst the best internationally.
The Hospital systems sector has never undergone a quality labelling or accreditation process.
Although it has been muted on odd occasions since 1978 very little has transpired by way of action.
Under various NHS/DoH programmes such as Körner, HISS and Resource Management these
attempted to restrict the suppliers in some way either by creating an ‘approved solution’ as in the
case of the various HISS procurements, or standards as with Körner that were never fully
implemented by the Regions.
In 2002 the NHS (covering the whole of England, with a population of 54 million persons) launched a
major new IT procurement initiative known as the National Programme for IT (NPfIT). This 6.2 billion
pound programme, running to 2010, is the largest public IT procurement project internationally. It
was managed primarily through a single contractual phase in which England was divided into five
regional “Clusters”. For each, a major global-scale company (Local Service Provider) was sought to
commit to deliver comprehensive IT solutions through a series of private sub-contracts. For these
systems, functional quality criteria form part of the contracting process. Each Local Service Provider
(LSP) has engaged several key sub contractors to actually deliver hospital, general practice or
community systems which comprise the ‘core systems’. Most of the LSP sub contrators are existing
systems suppliers, although some have not previously supplied systems to the UK.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
41
Rationale
General Practice systems in NHS England, prior to 2002
Computers first appeared in general practice in the 1960's as ad hoc systems developed in university
research departments by pioneer enthusiasts, with early commercial systems appearing in the late
1970's sponsored by the Department of Health. In the mid-80's the market grew rapidly through the
initiative of two companies offering GP computing systems at no cost in exchange for a commitment
of access to aggregated prescribing and contact data supported in part by the Pharmaceutical
Industry. Despite widespread concerns at the ethical implications of this, both companies enjoyed
considerable success up to 1991. However these early systems had a strong bias towards drug
prescribing and other clinical requirements were later addressed with some difficulty. The revenue
from the sale of this data proved to be much lower than anticipated, precipitating the collapse of
these free schemes.
In the late 1980's the Department of Health introduced a 50% reimbursement of all computing costs
in general practice. This encouraged the growth of the GP computer industry, and at its peak around
50 companies were actively supplying this market. UK general practice now has one of the highest
levels of computerisation in Europe, although practices vary in the extent to which the computer
system is used to capture routine consultation data.
In order to ensure that reimbursement was indeed being given for good quality systems, a formal
accreditation process was introduced in 1990 and has been progressively incremented until 2001. At
no point has legislation required the use of accredited systems, partly because general practitioners
have historically been self-employed contractors to the NHS and not direct NHS employees.
However, conditional part-reimbursement has acted as a significant driver towards to adoption of
accredited systems.
UK (England) NHS National Programme for IT (NPfIT)
The business driver for quality healthcare information systems within the NPfIT programme is
embodied within the procurement specification, which includes a set of functional requirements for
which each contractor has needed to tender a proposed solution. Although the criteria have now
been made public (see Section 7.3 below) the respose of each contractor has not, and the
governance and quality assurance processes to be used to verify the solutions are also not
transparent.
Description of the procedure
General Practice systems in NHS England, prior to 2002
UK general practice systems have, over the past 12 years, been conformance-tested against
progressively more detailed and rigorous functionality and safety criteria. These standards, known as
Requirements For Accreditation (RFA) defined a formal testing and approval mechanism for all GP
systems. Initial versions, between 1990-5, focused on the provision of GP Links (patient registration
and service delivery claims) and the incorporation of READ codes. RFA Version 3 (1996) required that
a substantial clinical record system be included. RFA Version 4 (1997) contained minor extensions to
version 3. The final published form (RFA 99, actually implemented in 2001) contained additional
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
42
security features & compatibility with NHSnet, required the incorporation of PRODIGY (national
prescribing guidelines for a wide range of common primary care conditions) and MIQUEST (a remote
access audit and population morbidity data extraction and analysis tool). An update to RFA 99 (RFA
99.2) added the ability to support pathology report messages and PKI with encryption.
Items forming part of a pre-test declaration and documentation set for RFA99



















RFA 99 Application Form
RFA 99 Documentation Requirements checklist
Explanatory Notes, if appropriate
RFA 99 Pre-Test Declaration Form
GP-CARE - Support Service Specifications
Escrow-Style Agreement
RFA 99 Training Undertaking (checklist of topics)
Samples Of User Documentation
Audit Trails (specification and examples)
Downloading Patient Data (how to extract the data for one patient from the system, and the
format it will take)
Uploading Patient Medical Data
File Definitions (database schema)
Backup And Restore documentation
Handling Of Dates After Year 2000
User Licence Agreement
Licences For Drug Data And Clinical Terms
NHS IT Standards Questionnaire
RFA 99 Pre-Conformance Declaration Form
RFA 99 Conformance Statement
Pre-testing procedures are defined for those aspects of electronic communication that can be tested
remotely, such as the ability to generate and receive NHS standard messages. In order to verify
responses to these, a test version of the GP system to be validated has to be set up by the supplier
with test (patient and staff) data, and evidence of conformance is in part provided by appropriate
response messages and in part through screen captures showing that relevant information has been
imported.
Physical accreditation testing was performed initially by specialised consultants (such as Hoskyns) but
in recent years by the NHS itself. The process could last 2-4 weeks of full time testing by one
examiner, usually working with 1-2 representatives of the supplier and undertaken at the premises of
the supplier.
The costs of RFA conformance testing have usually been borne by the supplier, but some specific add
on connectivity modules, such as pathology message import, have occasionally been centrally
funded.
The Royal College of General Practitioners has historically been involved in a steering group that
defined the criteria, and draft versions of the specifications have always been sent out to suppliers
for comment some months before being finalised.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
43
UK (England) NHS National Programme for IT (NPfIT)
No details have been published about how nationally procured or LSP systems will be verified, nor
what detailed functions each such system will actually have implemented. However, it is apparent
that a ‘backdoor’ approach to establishing some standards is emerging as part of the implementation
processes across the clusters. As a prerequisit to achieving robust system software models that are
capable of volume roll-out the CfH together with the LSPs and their EPR sub contractors are working
to National code sets and cluster code sets for key data items, intermediate file formats (IFF) for data
loading, standard function sets and defined standards for business logic.
Economic Model
General Practice systems in NHS England, prior to 2002
As described in Section 7.2, a reimbursement approach has been taken to stimulating the adoption of
accredited systems. The reimbursement level has generally been 50% of capital costs, but for some
years when EDI was being encouraged a 100% reimbursement was offered for communications
equipment (a comms server and modem). This extent of reimbursement proved sufficient to
effectively eliminate all non-conformant suppliers from the market-place within several years.
UK (England) NHS National Programme for IT (NPfIT)
The economic model under NPfIT is much more straightforward – only centrally-procured systems
will eventually be deployed. However, at present an agreement has been made to allow GPs to
choose their own systems, without any clear indication of if or how such chosen systems will be
accredited.
Criteria
General Practice systems in NHS England, prior to 2002
As an example, the most comprehensive RFA published specification (RFA99) contains criteria under
the following sections.
Part CR: Core Requirements and Services
Section CR.1 Privacy and Security
Section CR.2 Year 2000
Section CR.3 Read Codes
Section CR.4 NHS Number
Section CR.5 NHS Data Standards
Section CR.6 System Configuration
Annex CR.A.1 New NHS Number Check Digit Calculation
Annex CR.A.2 NHS Data Standards
Part ST: Supplier Services - Support and Training
Section ST.1 Introduction
Section ST.2 Support Contracts
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
44
Section ST.3 Documentation
Section ST.4 Training
Section ST.5 Testing
Annex ST.A.1 Confidentiality Clause
Annex ST.A.2 Training Undertaking
Annex ST.A.3 Training Partnership Agreement
Part GF: General Functionality
Section GF.1 Information about the Practice and Related Organisations
Section GF.2 Information about Patients
Section GF.3 Prescribing and Dispensing
Section GF.4 Reporting Facilities
Annex GF.A.1 National Organisation and Practitioner Codes
Part MI: Messaging and Information Exchange
Requirements Summary
Section M1.1 Introduction
Section M1.2 Overview
Section M1.3 NHSnet
Section M1.4 Electronic Data Interchange (EDI)
Annex M1.A.1 References and Specifications
Annex M1.A.1.1Further references
Annex M1.A.2 Contacts
Annex M1.A.3 Useful Web Sites
Part KR: Knowledge Related Functionality
Section KR.1 MIQUEST
Section KR.2 PRODIGY
Part SS: Strategic Statement
Section SS.1 Introduction
Section SS.2 RFA – Purpose and Scope
Section SS.3 Privacy and Security
Section SS.4 Pathology Results Messages and GPnet
Section SS.5 Other Clinical Messages – Radiology and Discharge
Section SS.6 PRODIGY
These specifications amount to 125 pages of material, much of which comprises formal functional
requirement statements. They were supplemented by detailed EDI message specifications (which are
much more voluminous) and some specific annexes dealing with organ donor registration and a
national data standards project (dealing with Read code audit data sets).
UK (England) NHS National Programme for IT (NPfIT)
The specifications of the systems to be tendered were published as an “Output Based Specification”
(OBS), drafts in early 2003 and a final version in August 2003. This large document set comprised
some high-level scene setting overview material and sets of specification statements for arrange of
functional modules to be delivered. Unlike the previous Requirements for Accreditation of general
practice systems, this specification was expressed at a higher level but with more comprehensive
functional coverage (because the systems are broader in scope than GP systems). For each
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
45
(numbered) specification statement, suppliers have been required to define how their proposed
solutions would meet the specification.
This information is confidential, as is the contracted price for each module. Whilst recognising the
need for commercial confidence, there has been widespread concern that the new hospital and GP
systems are being developed without any mechanism for peer review of the designs, or even the
knowledge of which of these many specifications each supplier has undertaken to meet, or how.
In addition to local hospital and GP systems, the OBS specified several core components that have
been procured as national applications:








Personal Spine Information Services
Personal Demographics Service
Transaction and Messaging Spine
Spine Directory Services
Workflow/Rules Services
Terminology Service
Clinical Spine Applications
Secondary Uses Service
The OBS Part 1 document, 95 pages, specifies these national applications and also defines some
generic criteria for Programme and Project Management, including:
 Design and Build of Solution
 Testing and Acceptance
 Piloting
 Implementation
 Training
 New Systems and Services
 Development
 Operational Support
 Change Control
OBS Part 2, 608 pages, specifies the Local Service Provider (LSP) systems to be delivered for each
Cluster. This specification is divided into functional modules, for each of which there is a general
overview and a high level description of the target achievements for 2004 and 2010. The functional
modules defined in OBS Part 2 are listed below.
 Patient Index
 Prevention, screening and surveillance
 Assessment
 Integrated Care Pathways and Care Planning
 Clinical Documentation, including Clinical Noting and Clinical Correspondence
 Care management
 Scheduling
 eBooking
 Requesting and order communications
 Results reporting
 Decision support
 Prescribing and pharmacy
 Diagnostic and Investigative Services
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
46












Digital Imaging including specification for a Picture Archiving and Communications
System (PACS) Solution
Document Management
Financial Payments to Service Providers
Maternity
Social Care
Dental Services
Maintain Patient Details
Emergency\Unscheduled Care
eHealth and Clinical Development
Information to support secondary analysis and reporting
Surgical Interventions
For each module, the specifications listed are generally at quite a high level, and although the
document is long overall, the details for each module at times appear sparse.
Example: Clinical Noting specification
(re-formatted for this document, numbering removed)
Clinical Noting Clinical noting will need to support both structured (e.g. operation notes for a
specific surgical procedure) and unstructured notes (e.g. by providing the ability to record
notes of a telephone conversation, and to store freehand drawings, annotated diagrams and
digital images).
Structured notes will require the use of template formats, which can be locally tailored for
specific types of clinical notes.
Clinical notes shall be attributable to the author. Nothing may ever be removed from the
notes and any changes to the notes shall be audited.
Standard clinical coding and terms shall be applied at specific, locally-defined points, where
relevant. Clinical notation shall incorporate different formats and media, including body
maps, drawings, and graphical, textual and numerical information, as well as audio and video
records.
Clinical noting may require supporting technology to translate a Clinician’s notes into an
electronic format (e.g. from tablet P.C.s, audio equipment or where Clinician has used
drawings/diagrams etc.).
The service shall support the production of a range of documentation; e.g. discharge
summaries (immediate and final); out-patient clinic letters; and referral letters.
The service shall support the delivery of documents such as, but not limited to: paper; email
and structured Messages (e.g., HL7 V3).
The service shall enable the creation of letters from standard templates to which particular
comments can be added. The service shall pre-populate the templates (e.g., use rule-based
intelligence to select which address is appropriate from the information stored in the
patient's Patient Record or the PDS). Any amendments, corrections and/or changes shall
have Audit Trails; and, once sent out, documentation shall be made read-only.
The service shall support enlarging of print for patients with visual impairments and also
translation into Braille, or, recording onto audio tape. It would be desirable that any patient
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
47
information leaflets (e.g. “Rights Under the Mental Health Act”) or condition- specific
information be printed in a language other than English, if such preference is indicated at the
point of registration.
The service shall support user-selected batching and printing of letters; e.g.: by priority; by
GP; by practice; by document type; and by postcode, (to fit with Post Office discounts).
The systems shall support printing of associated documents; e.g., check lists, maps, leaflets,
instructions, appointment cards, etc.
The service shall support the storage of clinical correspondence, including discharge
summaries.
All documents generated shall be stored against the patient's Patient Record. The service
shall enable viewing of clinical correspondence.
Access to documents shall be controlled by role-based access and limited to authenticated
users. All user access shall be recorded and have Audit Trails. Configuration management
shall apply to the documents. Changes, amendments, corrections and deletions shall be
recorded in the Audit Trails. The service shall support integrated, specialist, multi-disciplinary
(including Social Care and mental health) team records with ICPs shared across teams,
disciplines and organisations.
The service shall support locally- defined, role-based "views" of patient data. The service shall
be able to receive a nationally-defined discharge notification, and other nationally-defined
Messages. The service shall be capable of producing and exporting Patient Record summaries
with nationally-defined content and formatting, including SAP and CPA information. The
service shall enable pointers to be held within the Patient Record giving access to: (a)
correspondence sent or received in respect of the patient; and (b) any other relevant
documents originating from outside the system. This shall include the ability to store or link
to scanned documents, where appropriate. The service shall enable templates for standard
and non-standard letters to be set up and accessed, drawing on data from the system for
pre-formatting and entry of existing field contents but allowing additional free text. Where
appropriate, nationally-defined templates shall be used.
It shall be possible to electronically mail these documents, where appropriate.
The service shall enable appropriate copies of summary reports to be sent to any GP, care
coordinator, Social Services or independent care provider, as appropriate.
The service shall provide the ability to record contemporaneous notes as part of the care
process. This includes operation notes for surgical procedures, medical notes with are
supplementary to assessments, medical history, treatment notes, community and social care
notes and such like.
The service shall enable the creation of referral communications which should incorporate
links with referral protocols and guidance. The referral letter shall be linked to the electronic
booking process.
The service shall enable draft notes to be created and edited prior to committing the note to
the record. Once a note has been committed to the record it can only be altered through
audited change control. Correction or updates shall only be recorded as supplementary to
the original notes.
The service shall enable clinical notes and documentation to be accessible through different
views according to the user’s requirement, this will include as examples: chronological view
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
48
of patient notes within or across care events; by care profession; and as part of another
function, e.g. as part of a care pathway.
Part 3 of the OBS, 214 pages, defines a set of criteria for common service functions:











Service Levels: Availability and Response Times
Business Continuity / Disaster Recovery
Volumetrics, Scalability and Extensibility
Technology Refresh
Helpdesk
Information Governance
Standards
Clinical Communications
Interfaces to/Use of Existing Health Services Infrastructure (links to other National Services)
Information for Secondary Purposes
Data Quality and Data Quality Management
It might be noted that, as an example, the section on standards lists only six (sparsely populated)
pages of criteria that are all expressed at a very generic level, such as those for SNOMED-CT:
 All new systems shall support the SNOMED CT standard.

All existing systems should support SNOMED CT.
Testing Scenarios
The accreditation process for general practice systems pre-2002 has been outlined above.
For NPfIT systems, it is difficult at this stage to be confident about the quality crieria that will actually
be met by each “approved” system (as opposed to what criteria were requested), and how each
system will be evaluated. There has been no publlished information about an external quality
assurance process, so no testing information can be provided.
Lessons learned
General Practice systems in NHS England, prior to 2002
In the early to mid nineties, when GP systems were of very variable quality, the introduction of RFA
served to communicate to the general practice community that their systems were being taken
seriously, and also that in order to be part of a national health service it was reasonable that the
service should have some expectations of the quality of such systems. It also communicated to the
suppliers that intensive marketing to ill-informed GP purchasers was not to be their longer-term
survival strategies. Although it took many years for the smaller suppliers to fade out (with a few
minority suppliers still in the picture in 2006) the majority of practices have migrated to one of the
main three providers. The strength of this has been to eliminate poor quality systems, but the
weakness has been to suppress innovation and competitiveness, since most vendors have some
degree of data lock-in and most geographical areas are now single-supplier communities.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
49
A second problem has been that, whilst initial RFA criteria were well-recognised to be generic and
consensual, RFA 4 and onwards has included an increasing number of centrally-driven agendas which
have dominated the evolution of GP systems in place of user-driven functional requirements.
A major flaw of the reimbursement process has been that a system must be accredited to be
reimbursed, but no specific RFA level was mandated. This means that many practices have continued
to use quite old accredited systems and not replaced them with newer versions. This surfaced as a
major problem when a draft RFA 2001 was circulated, refined and then withdrawn because it was
recognised that too few practices had yet upgraded to an RFA 99 system, and that pushing the
minimum standard to RFA 2001 was nationally unaffordable!
A vacuum has now been created in the future evolution and accreditation of GP systems, and it is
unclear how future quality labelling will now occur.
UK (England) NHS National Programme for IT (NPfIT)
It is too early for lessons on quality labelling to become apparent from the National Programme, but
the absence of information in response to some of the earlier sections of this report for the UK
shows that a lack of transparency is at minimum leaving uncertainly in the market-place as far as
users are concerned. Also, as delivery delays continue to frustrate Trusts so they are increasingly
moving to adopt ‘interim solutions’ which further reduces the likelihood of any accreditation or
quality labelling processes occurring in the near term
USA
Situation before 2007
Scope of the labelling procedure
The Certification Commission for Healthcare Information Technology’s (CCHITSM) mission is to
accelerate the adoption of healthcare information technology throughout the US by creating a
credible mechanism for the certification of healthcare IT (HIT) products.
CCHIT was founded in 2004 with sponsorship from three industry associations in HIT:
 The American Health Information Management Association (AHIMA),
 The Healthcare Information and Management Systems Society (HIMSS) and
 The National Alliance for Healthcare Information Technology (the Alliance).
In September 2005, CCHIT was awarded a $7.5 million, three year contract by the Department of
Health and Human Services (HHS) to develop certification criteria and an inspection process for
Electronic Health Records (EHRs) and the infrastructure components through which they
interoperate.
The scope or the labelling procedure was, at least in 2006, limited to ambulatory care.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
50
In-patient care information systems has been identified as the second target group. CCHIT expects to
start certification of In-patient care information systems during 2007.
Rationale
The rationale of CCHIT Certification has been described by them in their CCHIT Certification
Handbook.
CCHIT’s Certification Program assures the healthcare industry and healthcare consumers that
EHRs with CCHIT Certification can expected to deliver the following benefits:
 Increase the transparency of the EHR marketplace and reduce risk for physicians and
hospitals that select, purchase, and implement HIT.
 Redirect HIT investment toward EHRs that have the necessary functionality to improve the
quality, safety, and efficiency of care.
 Protect the privacy of health information by requiring adequate security standards within
EHR products and network infrastructure.
 Ensure the interoperability of EHRs through standards-based compatibility with the emerging
national health information network (NHIN) architecture.
 Encourage healthcare purchasers and payers to offer incentives to physicians and other
providers to adopt EHRs.
Taken together, these benefits have the potential to accelerate the widespread adoption of
robust, interoperable EHRs.
There are no legal requirements to apply for certification. The endorcement of the CCHIT approach
by the Department of Health and Human Services indicates on the other hand given to the quality
and the interoperability of health information systems by the US Health Authorities.
Description of the procedure
The applicants sign a CCHIT Agreement with the basic rules governing the certification process and
liability matters between the applicant and CCHIT. The agreement stipulates that retesting and
appeals are allowed.
New versions of the application should be re-certified in case of substantial differences with the
previous certified version. The vendors therefore files a “Statement of Substantial Equivalence” in
case of “minor” changes to the application.
Certification is only done for Comprehensive EHR Products yet on the market, meeting all the CCHIT
criteria or collection of applications meeting together all the certification criteria.3 Pfre-market
certification is “Conditional” and starts to be a full CCHIT Certified Product as soon as references are
given to CCHIT.
3
Under responsability of one provider submitting the dossier.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
51
The full procedure has been described in the CCHIT Certification Handbook, available on the web-site
of CCHIT, www.cchit.org. Hereby a short overview of the procedure, more a TOC of the handbook.
Application process
1. The applicant completes the application
2. CCHIT reviews the application
3. Submission of signed certification agreement and payment of the certification fee
4. The applicant submits self-attestation material, no later than 10 days after submitting the
certification application
5. CCHIT conducts a “desktop review” of the self-attestation material, eventually informing the
applicant of deficiencies.
6. Scheduling of the Inspection Dates, one date for Functionality tests and one date for
Interoperability/Security inspection.
Inspection process
The inspection process is a rigorous, structured review of the product by inspection of the selattestation material, a jury-observed demonstration of clinical functionality and a jury-observed
demonstration of security functionality.
Test scripts are used for that purpose. The test scripts are available on the web site of CCHIT.
1. Functional inspection

Is done by three clinical experts, at least of them being a practicing physician.

Test are done by web conferencing

Collation of the votes, based on a majority rule. Non-compliant if two or more jurors marked
a test as “non-compliant”

Same day retest is foreseen, eventually allowing the applicant to include adjacent steps to
adequately demonstrate the compliance to the test

Completion of the functionality inspection by recollection of the votes
2. Time allotment of 8 hours for the functional inspection, $1.000 per supplementary hour if
required.
3. Security inspection is done the same, except that direct access to the system may be required.
4. Time allotment for security inspection: 4 hours, though 2 hours seems generally enough
5. Review of self-attestation material: the most practical and reliable method of assessing
compliancefor certain criteria is through self-attestation and review of related material. The
applicant has to provide convincing material.
6. Retesting with a new jury can be requested by the applicant if he believes that the inspection
process results do not accurately reflect the compliance of the applicant’s EHR product. The new
jury will test only the non-compliant items. The retest will be recorded. Retest is free of charge if
it can be done within 3 hours.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
52
7. Appeal for Commission Review can be applied after retesting still results in being non-compliant
and if the applicant believes, in good faith, that the results doe not accurately reflect the
compliance of the applicant’s EHR product with the Certification Criteria.
Certification outcome
Certification outcome is notified by email within five days of its decision.
1. Products achieving certification receive a Certification Document and the Seal. The
announcement will be done on CCIT’s next Batch Announcement Date, together with the
announcement of other products certified. The applicant can only make use of the certification
once announced by CCHIT.
2. Products failing can reapply when they want by resubmitting an application and after paying a
first-year Certification maintenance fee.
Economic Model
CCHIT is clearly a “private” initiative with the ambition to be self-sustaining and any way increasingly
independent from external funding over the next years.
Vendors needs to pay a Certification Fee of $23.000 to qualify, each year that the chooses to get his
product certified. The Certified Vendor who chooses, during the three years following a previous
certification, not to apply for recertification against updated criteria in subsequent years pays an
annual Certification Maintenance Fee of $4.700.
Vendors are obviously free to qualify for certification.
Private funding seems not to be sufficient, certainly not in the initial phase. The CCHIT raised, after
the initial sponsorship in 2004, funding in 2005 from different professional organisations, more
especially the American Academy of Family Physicians (AAFP), the American Academy of Pediatrics
(AAP), the American College of Physicians (ACP), the California HealthCare Foundation (CHCF),
Hospital Corporation of America, McKeson, Sutter Health, United Health Foundation, and WellPoint
Health Networks, Inc. In September 2005, CCHIT was awarded a $7.5 million, three year contract by
the Department of Health and Human Services (HHS) to develop certification criteria and an
inspection process for Electronic Health Records (EHRs) and the infrastructure components through
which they interoperate.
The total funding granted to CCHIT is therefore4 estimated at some $10 million, at least, for the first
three years. This means that over 500 IT systems needs to be labelled at an average rate of $20.000,
average of the fee for new applicants and annual renewal applicants.
Up to now 33 has been certified. 11 out of 17 applicants were certified during the second application
period, August 1 – 14, 2006.
4
No concrete figures available.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
53
The economic model should be considered as public-private initiative, until proven that CCHIT
succeed in being a self-sustaining organisation.
The fee paid by the applicants should be considered as important and most probably not feasible in
Europe. This fee might have an influence on the market and be a risk for the certification as such,
certainly once the organisation get that self-sustaining or independent status, the system providers
being at that moment the “clients” of the certification “authority”.
Criteria
The CCHIT Test Criteria were developed in working groups and validated by a large number of
“experts”. They are described by CCHIT as “These criteria were developed based upon available,
widely-accepted industry standards.” The source documents have been listed. That list is available to
anyone.
The CCHIT original 2006 criteria for ambulatory care are subdivided in 4 sections: functionality (266
criteria), interoperability (26 criteria), security (32 criteria) and reliability (16 criteria).
Not all the criteria are considered yet as mature enough to be tested, to be certified. Compliance to
134 out of the 266 functionality criteria has not been certified in 2006. 7 other functional criteria are
only tested informally, to get some more information on the implementation of these criteria on the
field. 6 security criteria, 15 reliability criteria have been certified in 2006. Interoperability has not
been certified in 2006.
The functional criteria mostly sufficient “general” to be used in other healthcare environments in the
United States but also in other countries. Some criteria are typical for the US, more especially
regarding administrative and financial issues, but rather limited in number, e.g. statements 211, 229
and 235.
Some criteria should be considered as ‘national’ implementation options, e.g. regarding the content
of an electronic healthcare record summary (53 and 228) or what has to be considered as vital signs
(65) or the required field for a medication item (30).
Another difference with most European specifications seems us to be the “list” based approach for
the description of most functionalities. European specifications are more addressing individual data
elements in an EHR.
The CCHIT criteria are nevertheless very near to the European approach, most probably because they
are addressing the same medical / clinical specialty, the ambulatory care.
The CCHIT criteria for ambulatory care are still evolving very quickly. The 2007 criteria for
interoperability, security and reliability are far more elaborated than the 2006 ones. The test
scenario’s for 2007 also illustrate important changes in the business function descriptions: at least 5
new test criteria have been added in the Test Scenario 2007, content has slightly been modified for
over 10 different criteria.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
54
Too many and too quick changes in the criteria has been experienced as a risk for the labelling
procedure. An additional problem will be that on a given moment IT products will be certified against
different sets of ‘mandatory’ criteria, considering that a vendor is not obliged to re-certificate during
a period of three years .
CCHIT is actually in the process of finalising a set of criteria for In-Patient Care IT systems. It contains
299 business functions and 75 interoperability test criteria. Security and Reliability test criteria for InPatient care are not made available yet, January 2007. Certification of in-patient care products is
expected to start in 2007.
No development has been described for other health professions.
For more information on CCHIT and for the lists of criteria, visit the website: www.cchit.org
Testing Scenarios
The certification process is based on test scripts.
These “test scripts were developed to simulate realistic clinical use scenarios, with each step in the
script mapped against one or more of the criteria. The CCHIT inspection process uses a combination
of documentation review/self-attestation, jury-observed demonstration, and jury-observed technical
testing, in a sequence guided by the test scripts, to confirm compliance of an applicant’s EHR product
with the certification criteria.”
Testing scenarios are available on the web site of CCHIT: www.cchit.org
Lessons learned
Certification seems to be possible in a professional way and based on an industrial initiative, but it
remains, even then an expensive process requiring public funding, even in the United States.
Certification at European level, with an increased level of complexity and cultural as well as
regulatory difference will be even more expensive, if at least the healthcare community wants it
done professionally.
The first reactions of the market (of ambulatory EHR systems) seems positive as yet more than 40
applications applied for that certification.
The reactions are on the other hand not unanimously positive. Most criticism is on the fee, the lack
of legal status (even if this is considerably improved since the grants obtained from the Department
of Health and Human Services (HHS) and on “interpretation” of the tests.
The initiative is too recent to come to final conclusions.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
55
Situation in 2010
In 2009 the American Recovery and Reinvestment Act (ARRA) was enacted, which contains the
Health Information Technology for Economic and Clinical Health Act (HITECH Act). Within this HITECH
Act it is foreseen to invest around $19 billion to health information technology. The Office of the
National Coordinator (ONC) for Health Information Technology as well as the Centers for Medicare
and Medicaid services play an important role within this federal stimulus plan.
Meaningful use of EHRs
Eligible professionals and – hospitals can apply for HITECH incentives if they use a certified EHR in a
“meaningful” matter. “Meaningful use” of EHRs encompasses three aspects:

Use of certified EHR technology in a meaningful manner

Certified EHR technology must be connected in order to make electronic exchange of health
information possible

Clinical quality measures must be submitted
Certified EHR technology
Certified EHR technology can either be a complete EHR or a combination of EHR modules, each of
which meets the requirements included in the definition of a qualified EHR (see definition below),
and which has been tested and certified in accordance with the Certification Program established
with the ONC.
Qualified EHR
A qualified EHR is an electronic health record which contains health-related data on an individual
that includes patient demographic data and clinical health information. Furthermore, the EHR must
have the capacity to provide clinical decision support, to support physician order entry, to capture
and query information relevant to health care quality, and to exchange electronic health information
with, and integrate such information from other sources.
Meaningful user
Eligible professionals and eligible hospitals can be considered as meaningful users if they can
demonstrate meaningful use of certified EHR technology during a specified reporting period.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
56
Stage 1 of Meaningful use
At present in the US, only Stage 1 of Meaningful use is being implemented (year 2011). Stage 1
focuses on aspects of data capturing and – sharing. Five health outcome policy priorities have been
defined, each of which is mapped to care goals:
“Improve the quality, safety, efficiency of health care and reduce health disparities”





Provide access to comprehensive patient health data for a patient’s health care team
Use evidence-based order sets and Computer Provider Order Entry (CPOE)
Apply clinical decision support at the point of care
Generate lists of patients who need care and use them to reach out to patients
Report information for quality improvement and public reporting
“Engage patients and families in their health care”

Provide patients and families with timely access to data, knowledge, and tools to make
informed decisions and to manage their health
“Improve care coordination”

Exchange meaningful clinical information among professional health care teams
“Improve population and public health”

Communicate with public health agencies
“Ensure adequate privacy and security protections for personal health information”


Ensure privacy and security protections for confidential information through operating
policies, procedures, and technologies and compliance with applicable law
Provide transparency of data sharing to patient
For each of the care goals defined, objectives (certification criteria) and measures have been
proposed for eligible hospitals and – providers.
Stage 1 objectives and measures for eligible professionals and eligible hospitals
 Use CPOE
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
57
 Used for at least 30% of unique patients with at least one medication in their medication
list

Implement drug-drug, drug-allergy, drug-formula checks
 Function is enabled

Maintain an up-to-date problem list of current and active diagnoses based on ICD-9-CM or
SNOMED CT®
 At least 80% of all unique patients have at least one entry or indication of none recorded

Maintain active medication list
 At least 80% of all unique patients have at least one entry or an indication of none

Maintain active medication allergy list
 At least 80% of all unique patients have at least one entry or an indication of none

Record demographics
 At least 50% of all unique patients have demographics recorded

Record and chart changes in vital signs
 For at least 50% of all unique patients aged > 2 years, record blood pressure and BMI;
additionally, plot growth chart for children 2 – 20 years

Record smoking status for patients aged > 13 years
 At least 50% of all unique patients aged > 13 y have “smoking status” recorded

Incorporate clinical lab-test results into EHR as structured data
 At least 40% of all clinical lab tests results are incorporated

Generate lists of patients by specific conditions to use for quality improvement, reduction of
disparities, and outreach
 Generate at least one report listing patients with a specific condition

Report ambulatory quality measures to CMS or the States
 For 2011, through attestation

Implement 1 clinical decision support rule relevant to specialty or high clinical priority,
including diagnostic test ordering, along with the ability to track compliance with that rule
 Implement 1 clinical decision support rule

Provide patients with an electronic copy of their health information upon request
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
58
 At least 50% of all patients who request an electronic copy of their health information
are provided it within 3 business days

Capability to electronically exchange key clinical information among providers of care and
patient-authorized entities
 Perform at least one test of certified EHR technology’s capacity to electronically
exchange key clinical information

Perform medication reconciliation at relevant encounters and each transition of care
 Performed for at least 50% of relevant encounters & transitions of care

Provide summary care record for each transition of care and referral
 Provided for at least 50% of transitions of care & referrals

Capability to submit electronic data to immunization registries and actual submission where
required and accepted
 Performed at least one test of certified EHR technology’s capacity

Capability to provide electronic syndromic surveillance data to public health agencies and
actual transmission according to applicable law and practice
 Performed at least one test of certified EHR technology’s capacity

Use certified EHR technology to identify patient-specific education resources and provide
that information to the patient if appropriate
 More than 10% of all unique patients are provided with patient-specific education
resources

Protect electronic health information created or maintained by the certified EHR technology
through the implementation of appropriate technical capabilities
 Conduct or review a security risk analysis and implement security updates as necessary
Stage 1 objectives and measures for eligible professionals only
 Generate and transmit permissible prescriptions electronically
 At least 40% is transmitted using certified EHR technology

Send reminders to patients per patient preference for preventive/follow-up care
 Reminder sent to at least 20% of all unique patients
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
59

Provide patients with timely electronic access to their health information within 96 hours of
information being available to the eligible professional (EP)
 At least 10% of all unique patients seen by the EP are provided timely electronic access
to their health information

Provide clinical summaries for patients for each office visit
 Clinical summaries are provided for at least 50% of all office visits within 3 business days
Stage 1 objectives and measures for eligible hospitals only
 Provide patients with an electronic copy of their discharge instructions and procedures at
time of discharge, upon request
 At least 50% are provided with this upon request

Capability to provide electronic submission of reportable lab results, as required by state or
local law, to public health agencies and actual submission where it can be received
 Performed at least one test of certified EHR technology’s capacity
Clinical Quality Measures
Beside the several certification criteria an EHR must meet (see list above), several Clinical Quality
Measures must be reported. For eligible professionals, a total of 90 Clinical Quality Measures have
been proposed. They must report on two sets, a core set (consisting of clinical quality measures
related to “tobacco use”, “blood pressure measurement” and “drugs to be avoided in the elderly”)
and a specialty set (the professional must choose one medical discipline out of a list of 15 disciplines,
and must report on the Clinical Quality Measures belonging to that discipline). Eligible hospitals must
report on 35 Clinical Quality Measures (more related to the management aspects). In 2011, these
clinical quality measures must be reported by attestation. It is foreseen that from 2012 these quality
measures will be submitted electronically.
Financial Incentives and Penalties
Depending on applying for HITECH incentives through Medicare or Medicaid Centers, the eligible
professional can receive a maximum $44.000 or $63.750 respectively. For eligible hospitals, e.g.
requesting incentives through Medicare, an initial amount of $2 million is foreseen with an extra
amount per discharge ($ 200 for the 1150th – 23000th discharge). Important to mention is that from
2015, penalties will be applied to professionals and hospitals which have not reached Stage 3 of
Meaningful Use!
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
60
Authorized Testing and Certification Bodies
ONC-Authorized Testing and Certification Bodies (ONC-ATCBs) are authorized by the National
Coordinator to test and certify that EHR technology is compliant with the standards, implementation
specifications, and certification criteria.
At the moment, three organisations have been selected as ONC-Authorized Testing and Certification
Bodies:

Certification Commission for Health Information Technology (CCHIT)

Drummond Group, Inc.

InfoGard Laboratories, Inc.
CCHIT
The Certification Commission for Health Information Technology (CCHIT®) is an independent, not-forprofit organization founded in 2004 with sponsorship from The American Health Information
Management Association (AHIMA), The Healthcare Information and Management Systems Society
(HIMMS) and The National Alliance for Healthcare Information Technology (the Alliance).
In 2006, CCHIT started with the certification of ambulatory EHRs. From 2007, also inpatient EHR
systems were also within the scope of CCHIT’s activities. In 2008 the “Ambulatory 2008 EHR
Certification” program was introduced. The “Health Information Exchange Certification” program
was launched in October 2008 (till September 2009). In 2011 two Certification Programs are
introduced : the “Preliminary ARRA 2011 certification: EHR Technology for Eligible Providers” and
the “Preliminary ARRA 2011 certification: EHR Technology for Hospitals”.
On CCHIT’s website a “Certification Handbook” is available with detailed information on the
Certification Program’s terms and conditions, the certification process and the marketing policies.
Before the American Recovery and Reinvestmant Act (2009), CCHIT was the officially recognised US
certification body. CCHIT has been officially authorized as ONC-ATCB on September 3rd 2010.
Drummond Group Inc.
DrummonD Group Inc. (DGI) has been founded in 1999. The company has an interoperability test lab
and offers various testing services (auditing, quality assessment, conformance testing) and
certification services. DGI has become an ONC-ATCB authorized company on September 3rd 2010.
InfoGard Laboratories Inc.
InfoGard Laboratories was founded in 1993. Its main mission is to provide accredited IT security
assurance services. The company provides testing and evaluation services (e.g. security of IT products
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
61
and networks, third party security testing services, etc…). The company was officially recognised as
ONC-ATCB on September 24th 2010.
NIST
As a result of the American Recovery and Reinvestment Act of 2009, NIST is developing, in support of
the Office of the National Coordinator (ONC) for Health IT, tools for testing the conformance with the
“meaningful use” of EHRs criteria and standards (http://healthcare.nist.gov).
For each of the “Meaningful use” Certification Criteria, draft test procedures are available on NIST’s
website for download.
Figure: Overview of NIST’s draft conformance test procedures
These draft test procedures contain detailed information on following aspects:

Certification criteria;

Informative test description;

Referenced standards;

Normative test procedures (which information is required from the vendors, what the
required test procedure is, what inspection needs to be done);

Example test data;
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
62

Tools being used during the conformance testing.
At present NIST is seeking public comment on these draft test procedures. After this process, final
test procedures will be published on the website.
Canada – eHealth Collaboratory
Scope of the Labelling Procedure
Certification of Healthcare IT systems is at an early stage in Canada and an initiative of eHealth
Collaboratory (www.ehealthcollaboratory.ca).
The Collaboratory is on its own a joint initiative of Canada Health Infoway (Infoway) and the Centre
for Global eHealth Innovation (Centre). It provides vendors and purchasers of electronic health
records with access to services that support and accelerate the implementation of interoperable and
usable solutions.
The eHealth Collaboratory is an independant and separate entity. The Collaboratory has received
seed funding from the Canadian Government and uses assets provided by large health districts e.g. in
Ontario.
The initiative has the active support from the Canadian vendors, with ten of them contributing their
time and expertise during the Collaboratory’s first phase.
The Collaboratory’s services include:

testing EHR solutions for conformance to pan-Canadian standards for interoperability,
functionality, usability, security and privacy

usability assessment of EHR products

support for jurisdictions and other purchasers of EHR solutions as they seek to implement
secure, interoperable and usable solutions in the healthcare arena.
Rationale
The Collaboratory has been established to provide independent certification and procurement
support services to dramatically improve client’s ability to deploy standards-based, interoperable and
usable healthcare IT systems.
The eHealth Collaboratory has the ambition to be nationally recognised as the trusted authority that
provided and effective service offering enabling the Pan-Canadian EHR.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
63
Description of the Procedure
A first scenario of testing has been validated with a set of 6 vendors5 out of over 60 candidate
vendors. Most of the vendors were not able to comply with the initial set of requirements.
The test was focused on interoperability between the different system within a ‘virtual’
environment, realised in the premises of eHealth Collaboratory. For information on the procedure,
the
test
criteria
and
the
test
script/scenario
is
available
on
http://www.ehealthcollaboratory.ca/files/IntegrationService_SolutionAssessment.pdf
Interoperability tests are build around the use of standards, the standards accepted in Canada
through Infoway.
Interesting conclusions regarding the use of these standards are listed in the mentioned document.
Economic Model
No real documentation available except some indications that public funding seems to be the main
source of income. This might be linked to the “pre-operational” status of the initiative. Most the
services are not available yet.
Criteria
Hereby an overview of some of / of the functional requirements for the initial “interoperability tests”
done with the 6 vendors willing to invest time and resources:
1
The solution must provide single sign on.
The solution must enforce data synchronization both within the jurisdictional iEHR of participating
integrated systems and localized point of service systems.
The system must allow for user and patient context passing between systems.
2
3
4
5
The solution must support a unique patient key across all integrated systems.
6
7
8
9
10
11
12
5
Integrated systems are required to utilize the central client registry for patient identification.
Integrated systems are required to update the central client registry when changes to demographic
information are captured at the POS (Point-of-Service).
The solution must enable electronic collection and storage of medical history screening questions.
The solution must provide access to the patient’s allergy history.
The solution must provide access to the patient’s encounter history.
The solution must allow for electronic collection and storage of a mini-mental health assessment.
The solution must enable electronic capture of laboratory, diagnostic and medication orders.
The solution point of service systems need to generate a consolidated encounter summary and store the
summary in a location accessible to all integrated systems.
B Care, Katsi, Dinmar, MDI Solutions, Initiate, Sentillion
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
64
These criteria are clearly focused on the interoperability of IT systems within one given environment.
Testing Scenarios
The only testing scenario “available” for the moment is the one described in the mentioned
document. Hereby a summary of that scenario:
1) Patient arrives at the physician office and is registered. Demographics are checked against client
registry, found to be out of date and therefore are updated. Update goes to locally integrated system,
the provincial client registry and via publish/subscribe to the provincial clinical repository.
2) RN using e-form assessment tool completes pre-consultation assessment of medical history. Results
are stored back in POS demonstrating local system integration.
3) Assessment of injury is performed by MD demonstrating various capabilities of the POS.
4) A query is sent to the central clinical repository by the MD for updated allergy information.
5) The clinician switches programs to complete a mini-mental health assessment. The patient and user
context is passed between programs demonstrating the solutions CCOW capabilities.
6) Various lab and diagnostic orders are placed on the patient in the main POS system. (orders and
results messaging)
7) A dermatologist sees the patient. The EHR viewer is used to pull up a complete list of encounters for
the past year. An allergic reaction is diagnosed. Encounter notes are updated. This shows
query/response to the central clinical repository and CCOW.
8) The patient’s assessment is completed (x-ray, lab results are reviewed), various drug orders are
entered, the patient is given prescriptions and instructions, final notes are entered and encounter
results are sent to the central clinical repository.
9) The GP contacts the family physician in another location. They both view the encounter results in
the EHR viewer and discuss the encounter and recommended follow-up.
Lessons Learned
Collaborative conformance testing will begin in the second quarter of 2007, as indicated on their own
web site.
Some lessons, more especially regarding the need to remain “realistic” and linked to what’s feasible
on the market, are also part of the document mentioned.
France
Scope of the labelling procedure
The health authorities (Haute Authorité de Santé) issued in January 2007 a set of criteria and a
procedure for the labelling of “applications intended to assist the prescription” (LAP6).
6
LAP: Logiciel d’Aide à la Prescription
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
65
This initiative is based on the French Law of 13th of August 2004 regarding the social security
insurance and more especially article L161-387 of that low and the application decree article R. 16175.
The certification is not mandatory for providers of applications and surely not a condition for putting
on those applications on the market.
Rationale
The purpose of the certification is to promote functions and tools to:
 Improve security of the prescription.
 Ease the prescription for the prescriber.
 Reduce the costs of medicinal treatments at equal quantity.
Description of the Procedure
The certification by authorised certification organisations (Organismes Certificateurs) accredited by
COFRAC, French Committee for Accreditation, a public service. www.cofrac.fr.
Certification is based on the European Standard EN 45011 “General requirements for bodies
operating product certification systems“, completed with specific criteria.
Certification is based on:



A commitment to use a database of medicinal products from an editor that agreed with a
quality charter for so called „drug databases“.
A commitment of respecting some criteria that will not be tested during the certification
procedure.
The availability of an “operator” to perform, together with an auditor, the certification tests
during the audit, organised “on site”.
A test script, selected by lot, is used for the certification at the premises of the editor. All tests are
performed the same day. The auditor needs to document each deviation from the test criteria with
screen prints. Appeal is possible against a negative decision. Appeals are handled by a different team
or organisation.
The certificate is valid for three years, new versions of the application excepted.
Economic Model
The actual documentation does not contain any information on the economic model.
The redaction of the criteria and the test scripts is done by the health authorities (Haute Authorité de
Santé) and clearly funded publically.
„La Haute Autorité de santé est chargée d’établir une procédure de certification des sites
informatiques dédiés à la santé et des logiciels d’aide à la prescription médicale ayant respecté un
ensemble de règles de bonne pratique.“
7
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
66
The certification organisations are working as subcontractors of the health authorities. It is therefore
expected that costs will be paid by these authorities.
There are no indications for any payment to be done by the IT providers, the providers of
prescription tools or applications.
Criteria
A set of 69 test criteria has been defined.
14 criteria will not be tested by means of a test script. The providers need to sign a commitment of
compliance regarding these criteria.
The prescription oriented approach has a consequence on the test criteria: some criteria are not
directly related to the prescription as such nor to the medication management for the patient. Some
obvious as an example:

The application enforces the registration of the name, first name and date of birth of the
patient (if not yet available… when starting a prescription session).

Identifying data (name, first name, age and gender (if available) are permanently visible
during the prescription session.

Technical and commercial documentations are available.

The provider guarantees a helpdesk and maintenance, at conditions to be defined between
provider and user.

The provider offers an individual or collective training at installation of the application or of a
new version of the application.
These criteria are nevertheless “logical” for any IT application.
Testing Scenarios
Several “equivalent” scenarios have been elaborated. One of them, assigned by lot, is used for the
actual certification of an application.
Hereby an extract of one of these scenarios:
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
67
Lessons learned - Comments
The certification process did not start. Experts are actually validating the criteria as well as the test
scenarios.
Germany
Situation in 2010
In Germany the Kassenärztliche Bundesvereinigung (KBV), which is the National Association of
Statutory Health Insurance Physicians, is active in the certification of software for the statutory
health office computers. The German law regulates the transmission of data for remuneration and
gives the task of defining details of the procedure to the KBV. KBV offers manuals and tools for the
certification of components as well as complete software systems.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
68
Though the KBV focusses on administrative and reimbursement data, some administrative
applications require the use of encoded clinical data. For that purpose the KBV, defines, updates and
publishes a lot of classifications called Schluesseltabellen (literally: key tables) which are available to
the public via the web under http://www.kbv.de/keytabs/ita/schluesseltabellen.asp . Having these
tables published together with a prosaic comment for each table and each term, eases both
implementations and conformance assessments.
Beside certification of computer systems in the GP’s office, software within the domain of Disease
Management Programmes has also to be certified by KBV. This is done on the basis of general criteria
and disease oriented data descriptions in XML, a stylesheet in XSL, information about plausibility
checks, and transfer of data.
Norway
Situation in 2010
KITH is the Norwegian Centre for Informatics in Health and Social Care, founded in 1990. It is a
limited, not for profit, company with main focus on contributing towards coordinated and costefficient applications of secure information technology in the health sector. The company is owned
by the Ministry of health and Care Services, the Ministry of Labour and the Association for
Municipalities.
KITH is the national certification authority for EHR messaging in Norway. Furthermore, KITH is
responsible for the EHR standardisation in Norway. The company offers both testing (adherence of
software products to standards) and certification activities.
KITH has an automated test tool which vendors themselves can use to upload generated messages in
order to check for a correct syntax and (some) semantics. The test tool also offers to receive and
send messages over ebXML with application receipt. Messages can also be requested via a web
interface. Acceptance tests are conducted for sending and receiving of messages (either by test
cases, or by self declaration following by KITH inspection).
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
69
EuroRec Products & Services
Repository
Introduction
EuroRec produced and maintains a substantial resource with ±1600 functional quality criteria for
EHR-systems, each categorised and translated to European languages. EuroRec Use Tools help users
to handle this resource.
Figure: EuroRec Repository workflow
The figure above illustrates the way in which EHR quality criteria were extracted from source
materials and added to the repository, refined and indexed within it, and then subsequently
extracted by end users when compiling test plans and procurement specifications.
Workflow process that applies to EHR quality and conformance criteria
Original source documents were identified by the EuroRec Institute, national ProRec Centres or by
other international experts. These were formalised statements of requirement, functional
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
70
specifications, conformance criteria or test plans that applied in whole or in part to EHR systems and
preferably have been used within at least one jurisdiction, or at minimum had been subject to a
formal peer review process (such as a ballot). The goal was to prioritise those specifications that had
practical credibility, but it is also recognised that the implementation and adoption of EHR systems
(as opposed to organisation-based health information systems or setting specific clinical applications)
is relatively new: it is therefore expected that some areas of functional requirement will not yet exist
within validated instruments and that other source documents such as de facto and de jure standards
may be important to include. It is likely that some new specifications will be added to complete and
maintain the EuroRec repository.

Candidate original specifications were approved for inclusion by the EuroRec Editorial Board

The original statements within approved documents were first translated into English,
together with relevant section headings or category names, aiming at faithful translation
with no refinement or disambiguation (except as needed for the translation itself).

These translated original statements “Source Statements” (and their headings) were
imported into the repository, and indexed by each of the defined axes against high-level
terms. Because Source Statements may have a complex or compound content, or be loosely
expressed, this index was deliberately over-inclusive rather than precise. The aim of this
indexation was to assist with internal repository management and quality assurance. Being
useful to external end-users when searching the repository is not as such a goal of this
indexation. Each Source Statement references the original document from which it has come
(although the source document might not always be physically stored in the repository, and
might not always have been translated in full into English).

Each of these Source Statements was then be decomposed into one or more specific and
singularly focussed Fine Grained Statements. The language might have been re-expressed to
avoid ambiguity. On occasions multiple very similar statements or sub-clauses from the same
original source were combined in making these new Fine Grained Statements. Each Fine
Grained Statement references any primary Source Statement(s) from which it has been
derived.

Each Fine Grained Statement was indexed by each of the defined axes, using a detailed term
in each case. For some axes, multiple classifications were permitted, but the goal was to
balance sensitivity and specificity in this process, so that subsequent searches did not
overlook relevant statements but were also not overloaded with barely-relevant content.

Using the indexation as a means of reviewing and aggregating similar Fine Grained
Statements across multiple sources, a further set of new broader and potentially richer Best
Practice Requirements was defined. Each of these was also indexed, but such indexing might
have been straightforward as it might inherit the indices of the Fine Grained Statement(s)
from which it has been derived.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
71

Over time, some of these Best Practice Requirements will become validated through
adoption within test plans. In such cases a corresponding Test Criterion will be added to the
repository, and indexed.

With experience, certain groupings of Best Practice Requirements and subsequent Test
Criteria will be found suitable for given EHR system modules and care settings. These might
correspond to predefined queries on the repository based on certain index term values.
These groupings will be defined as EuroRec Profiles, and will be made available as
standardised sets to help foster consistent quality of EHR systems across Europe.

End users accessing the repository can choose one of the EuroRec Profiles or define a
customised search to identify relevant statements. These might then be exported so that the
end user can localise them if necessary, and incorporate them into procurement
specifications, test plans etc.

National ProRec Centres were involved in supporting local end users, re-translating these
exported statements, and working with vendors and purchasers to help validate and refine
the Q-REC materials.
Statistics
Some descriptive statistics of the number of fine grained statements, good practice requirements,
and their translations in several languages are presented in the two figures below (last update:
August 16th 2010).
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
72
Figure: Fine Grained Statements & translations
Figure: Good Practice Requirements & translations
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
73
EuroRec Tools
EuroRec has developed a set of tools for managing certification, documentation and procurement
services. The access to the EuroRec Use Tools requires a “license”. User name and password should
be requested at services@eurorec.org.
Figure: Home Page EuroRec Tools
Definition of the Tools

The EuroRec Composer™ enables the licensee to select the required Fine Grained Statements
that will be used in a certification session, a product documentation or a procurement document.

The EuroRec Certifier™ enables the licensee to structure the selected Fine Grained Statement of
a EuroRec Basket, completing them not only with aspects of importance within a given
certification but also with required reading information enabling a correct interpretation of the
criterion. The EuroRec Certifier™ also enables to include scoring of a certification session.

The EuroRec Scripter™ enables the licensee to write a certification or validation scenario for the
certification or procurement, linking to each of the scenarios the Fine Grained Statements linked
and to be validated individually during a certification or procurement validation session of a
product.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
74
The Composer, the Certifier and the Scripter are together the EuroRec Certification Suite.

The EuroRec Documenter™ enables the licensee to structure the selected Fine Grained
Statements of a EuroRec Basket in a way that they can be used in / merged with product
documentation, linking product description to the standardised descriptive statements.

The EuroRec Procurer™ enables the licensee to structure the selected Fine Grained Statements
of a EuroRec Basket, completing them not only with aspects of importance within a given
procurement document but also with required reading information enabling a correct
interpretation of the procurement.
The Admin function is only accessible to persons entitled to manage the repository.
EuroRec Seal
A selection of the EuroRec Functional Criteria has been published. IT-vendors can use this selection in
a validation process for their product. EuroRec issues a document called the EuroRec Seal when it has
been established that the product conforms to the Statement of conformance as claimed by the
vendor.
Seal Level 1
The EuroRec Seal Level 1 consisted of a basic set of 20 EuroRec Statements focusing on quality and
trustworthiness of electronic health data.
EuroRec Seal 2008 Documentation - English
Language: English
Version: 1
Month/Year: 04/2009
Status: Validated
Jos Devlies
09/04/2009
09/04/2009
Specifications:
EuroRec Seal 2008 Fine Grained Statements in English
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
75
All Selected Statements
1
GS001537.1 Each version of a health item has a date and time of registration.
2
GS001538.1 Each version of a health item has a user responsible for the effective data entry identified.
3
GS001539.1 Each update of a health item results in a new version of that health item.
4
GS001579.2
5
GS001593.2 Deletion of a health item results in a new version of that health item with a status "deleted".
6
GS001594.2
7
GS001598.1 A complete history of the versions of a health item can be presented.
8
GS001901.1 Each version of a health item has a date of validity.
9
GS001945.1 The system enables the user to designate individual health items as confidential.
Each version of a health item has a status of activity, e.g. active or current, inactive, history or past,
completed, discontinued, archived.
Each version of a health item has a person responsible for the content of that version. The person
responsible for the content can be a user or a third party.
10 GS002265.1 Each health item is uniquely and persistently associated with an identified patient.
11 GS002266.1 Each version of a health item is uniquely and persistently identified.
12 GS002268.1 Each user is uniquely and persistently identified.
13 GS002269.1
The system enables to assign different access rights to a health item (read, write,...) considering the
degree of confidentiality.
14 GS002281.1 All patient data can be accessed directly from the patient record.
15 GS002307.1 Each patient and its EHR is uniquely and persistently identified within the system.
16 GS002415.1
The system takes the access rights into account when granting access to health items, considering the role
of the care provider towards the patient.
17 GS002437.3
The system offers to all the users nationally approved coding lists to assist the structured and coded
registration of health items.
18 GS002672.3
The pick lists and reference tables offered by the system are the same for all the users of the same
application.
19 GS003952.1 The system does not display deleted health items, audit logs excepted.
20 GS003953.1
The system does not include deleted health items in clinical documentation or export, for audit purposes
excepted.
The EuroRec Seal Level 1 has been granted to four Irish GP EHR systems:

CompleteGP, version 2.1 of Eircom, Cork, Ireland

Health One, version 6 of Health Ireland Partners, Arklow, Ireland

Helix Practice Manager, version 1 of Helix Health, Dublin, Ireland

Socrates, version 1.5 of Technical Ideas.Com, Ballinode, Ireland
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
76
Certification was performed by the GP IT National Group in Ireland, in a way compatible with
EuroRec defined quality criteria and procedures.
The EuroRec Seal Level 1 has also been granted to the application “sense”, marketed by ITH icoserve
GmbH, Technology for Healthcare Technology for Healthcare, Austria.
Seal Level 2
The EuroRec Seal Level 2 consists of 50 statements:
All Selected Statements
1
GS001512.1 The system enables to link a role to a user.
2
GS001519.4
The system shall include the information necessary to identify each patient, including the first name,
surname, gender and date of birth. .
3
GS001523.3
The system enables the capture of all patient demographic data necessary to meet legislative and
regulatory requirements.
4
GS001531.2 The system displays all current health problems associated with a patient.
5
GS001537.3 Each version of a health item has a date and time of data entry.
6
GS001538.2 Each version of a health item identifies the actor who has actually entered the data.
7
GS001539.2 Each update of a health item results in a new version of that health item.
8
GS001544.4 The system supports the use of clinical coding systems, where appropriate, for data entry of health items.
9
GS001550.6 The system presents a current medication list associated with a patient.
10 GS001559.2 The system presents a medication history associated with a patient.
11 GS001573.2 The current medication list can be printed.
12 GS001577.3 The system provides a catalogue of medicinal products.
13 GS001579.2
Each version of a health item has a status of activity, e.g. active or current, inactive, history or past,
completed, discontinued, archived.
14 GS001590.2 The system presents a list of allergens with an active status.
15 GS001593.2 Deletion of a health item results in a new version of that health item with a status "deleted".
16 GS001594.2
Each version of a health item has a person responsible for the content of that version. The person
responsible for the content can be a user or a third party.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
77
17 GS001595.1 Each change of status of a health issue results in a new version of that health issue.
18 GS001598.2 A complete history of the versions of a health item can be presented.
19 GS001610.3 The system enables to document a patient contact.
20 GS001611.1 The system is able to present for one patient contact all the documentation associated with that patient.
21 GS001638.1 The system presents a history of the results for discrete lab tests.
22 GS001901.2 Each version of a health item has a date of validity.
23 GS001932.1 The system supports concurrent use.
24 GS001947.2 The system makes confidential information only accessible by appropriately authorised users.
25 GS002175.2 The system enables the implementation of a privilege and access management policy.
26 GS002182.1 The audit trail contains the registration of users logging in or out.
27 GS002184.1 The audit trail contains the registration of security administration events.
28 GS002198.2 Audit trails cannot be changed after recording.
29 GS002211.1 The system enables a user to change his password.
30 GS002243.1 Security service issues and operation of the system are well documented.
31 GS002265.1 Each health item is uniquely and persistently associated with an identified patient.
32 GS002266.1 Each version of a health item is uniquely and persistently identified.
33 GS002268.1 Each user is uniquely and persistently identified.
34 GS002269.1
The system enables to assign different access rights to a health item (read, write,...) considering the
degree of confidentiality.
35 GS002281.1 All patient data can be accessed directly from the patient record.
The system distinguishes administrators, privileged users and common users. Administrators assign
36 GS002287.2 privileges and/or access rights to privileged and common users. Privileged users assign privileges and/or
access rights to common users.
37 GS002300.2 The system is available in the languages required by the regulatory authorities.
38 GS002307.2 Each patient and his EHR is uniquely and persistently identified within the system.
39 GS002312.1
The system is able to make a distinction between patients with same name, first name, gender and date
of birth.
40 GS002415.4
The system takes the access rights into account when granting access to health items, considering the role
of the care provider towards the patient.
41 GS002437.4
The system offers to all the users nationally approved coding lists to assist the structured and coded
registration of health items.
42 GS002489.2 Data entry is only done once. Entered health items are available everywhere required.
43 GS002497.3
The system displays patient identification data (name, first name, age and sex) on each data entry
interface.
44 GS002582.2
The system displays, when prescribing a medicinal product, known allergies of the patient, if it does not
alert the user for a specific allergen.
45 GS002625.1 The system enables the user to modify patient's administrative data.
46 GS002638.1
The system distinguishes actual or active medication items from past medication items when including
and displaying medication items in lists or in a journal.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
78
47 GS002639.1 The system enables the user to modify health items, if legally admitted.
48 GS002655.2 The system has a timeout function, terminating a session after a configurable period of inactivity.
49 GS003787.1
The system has a consistent way to present clinical alerts, e.g. red color for abnormally and/or high lab
results.
A medication list presents at least the following elements: identification of the medicinal product
50 GS004729.2 (package), starting date, date of the latest prescription, dosing instructions (structured or as a textual
expression)
The EuroRec Seal Level 2 addresses different functional aspects of EHR systems:
Number
Index
Index Title
12
A00
EHR Data Entry related
9
A02
EHR Content related
8
A03
EHR Structuring Data
22
A04
EHR Display
1
A05
EHR Data Exchange Services and Record Interfaces
14
A09
EHR Generic Data Properties
7
A10
Medication Management
3
A11
Clinical Statements Management
3
A14
Shared Care
2
A15
Clinical Decision Support: alerts, reminders
7
A22
Demographic Services
1
A32
Laboratory Services
4
A6
Health Information System management
23
A7
Privacy and Accountability Services
6
A8
Technical Security Services
Most of the selected statements are indexed several times, being e.g. related to the display
aspects within medication management.
At present, 4 software packages have been granted the EuroRec Seal Level 2:

AxProFizioterapija, Audax d.o.o., Slovenia

AxProPatronaza, Audax d.o.o., Slovenia
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
79

AxProVrhniskiDENT, Audax d.o.o., Slovenia

CSC Clinical Suite, CSC Scandihealth A/S, Denmark
Release of the EuroRec Profile for EHR Compliant to Clinical Trial Requirements
In December 2009 EuroRec has released a profile identifying the functionalities required of an EHR
system in order to be considered as a reliable source of data for regulated clinical trials. The profile is
the result of a collaboration between EuroRec and the eClinical Forum (eCF) which brought together
global stakeholders from healthcare, pharmaceuticals, biotechnology, technology vendors and
regulatory organisations.
With EHR systems becoming increasingly prevalent in European Healthcare establishments, the
amount of data with potential use in clinical trials that is kept in an electronic format has increased.
Unfortunately, the extent to which today’s EHR systems conform to the strict guidelines governing
the use of electronic systems for clinical research is not known. Consequently, research sponsors are
often reluctant to rely on the security of data held within healthcare systems and this uncertainty is
contributing to the duplication of data entry to separate clinical trial systems and the ongoing
maintenance of paper records. The eCF have addressed this by producing a set of user requirements
that EHR systems should fulfill to enable the reuse of data for clinical research. This in turn has been
the basis for the definition of the new EuroRec profile.
The new EuroRec profile is designed to establish a baseline standard for the functional and
operational requirements for EHR systems to be compliant with clinical trial requirements. Having
this standard available will help EHR system developers know what to provide and healthcare system
implementers understand what to look for. Clinical trial sponsors and certification bodies will have an
important basis for evaluating EHR systems as source data repositories for clinical trials.
Details of the profile, including information designed to support use, are accessible from the EuroRec
website. It is also worth noting that this work has had a global scope. A sister profile, also developed
by the eCF project team, has been endorsed by Health Level Seven® (HL7®), a healthcare standards
organisation, which has also recently been approved as the healthcare industry's first ANSI (American
National Standards Institute) standard. As both the EuroRec and HL7 profiles draw upon the same
standard requirements for clinical trials, conforming to one will mean, in principle conformance to
both.
Medicine is being transformed by the introduction of EHRs. The growth of EHRs brings with it
important opportunities to improve the way data for clinical trials is handled. Healthcare,
patients, industry and regulators would benefit enormously from an environment for the re-use of
EHR data that could avoid today’s duplication of tasks and generation of paper. Such an environment
would undoubtedly introduce requirements for new functionality in both healthcare and clinical trial
systems and EuroRec will be monitoring these changing needs and introducing them into future
versions of the EuroRec Profiles for Clinical Trials.
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
80
European research projects
Past projects
Project acronym
MediRec
ProRec
Widenet
Q-Rec
RIDE
FP
FP3
FP4
FP5
FP6
FP6
Timeline
1994-1995
1996-1998
2000-2003
2005-2008
2006-2007
Topic
Lisbon Declaration (Recom. 9)
Creation of first ProRec centres
Creation of EuroRec
Creation of Repository & Tools
Roadmap for Interoperability
Project acronym
EHR-Implement
FP
FP6
Timeline
2007-2010
EHR-QTN
FP7
2009-2012
ARGOS
FP7
2010-2011
HITCH
FP7
2010-2011
Topic
National Policies for EHR Implementation in the
European area: social and organisational issues
Thematic Network on Quality Labelling and
Certification of EHR Systems
Transatlantic Observatory for Meeting Global
Health Policy Challenges through ICT-Enabled
Solutions
Healthcare Interoperability Testing and
Conformance Harmonisation
Current projects
Future projects
Project acronym
EHR4CR
Call
IMI 2009
call 2
Topic
Electronic Health Records for Clinical Research
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
81
Projects’ fact sheets
Q-REC
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
82
RIDE
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
83
EHR-Implement
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
84
EHR-QTN
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
85
ARGOS
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
86
HITCH
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
87
EHR4CR
Abstract
The EHR4CR (Electronic Health Records for Clinical Research) project aims to design and demonstrate
a scalable and cost-effective approach to interoperability between Electronic Health Record systems
(EHRs) and Clinical Research through multiple but unified initiatives across different therapeutic
areas, with varying local and national stakeholders and across several countries under various legal
frameworks. This unified approach will be made possible by both an EHR4CR business model and an
EHR4CR platform.
Working closely with the EFPIA partners, the consortium will confirm priority clinical trials scenarios,
such as patient recruitment, to be addressed and the requirements for these scenarios. The present
gap between EHR systems and clinical research systems to deliver these scenarios will be analysed,
which will direct the business model and the platform design.
The EHR4CR platform will:
-
-
enable trial eligibility and recruitment criteria to be expressed in ways that permit searching
for relevant patients across distributed EHR systems, and initiate participation requests
confidentially via the patients’ authorized clinicians;
support the feasibility, exploration, design and execution of clinical studies and long-term
surveillance of populations;
provide harmonised access to multiple heterogeneous and distributed clinical (EHR) systems
and integration with existing clinical trials infrastructure products (e.g. EDC systems);
facilitate improvements of data quality to enable routine clinical data to contribute to clinical
trials and vice versa thereby reducing redundant data capture;
enable clinical trials to be established and delivered more cost effectively at greater scale.
The platform will be implemented as a common set of tools and services that will be able to integrate
the whole lifecycle of clinical studies with heterogeneous clinical systems, including data extraction
and aggregation, de-identification and linkage, security, and conformance to regulatory
requirements.
The partners have sufficient experience of these challenges to understand their complexity. The
consortium will work closely with pharma partners to identify priority trials use cases and clinical foci.
They will take advantage of existing components and tools and prioritise the delivery of pragmatic
and affordably adoptable solutions over longer term research objectives.
The platform and the business model will support different key scenarios (related to the instruction
phase and implementation phase of clinical trials and other types of research):
•
•
•
•
hypothesis testing (provide a better understanding of real patient populations);
clinical trial feasibility;
population screening;
patient recruitment;
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
88
•
•
•
•
•
•
clinical trial execution: eCRF and EHR integration;
early detection of safety risks;
pre- and post- marketing safety monitoring;
treatment effectiveness and outcomes;
line extensions;
long term surveillance.
The EHR4CR project will primarily address patient recruitment and choose - together with the EFPIA
partners- the other priorities. The same holds for the careful choice of the several disease cases to be
included in the pilot demonstrations.
The organisational model, with inclusion of an independent Trusted Third Party (TTP), will also allow
for additional kinds of data transactions between different stakeholders and environments.
Part of the business model will be to define best practices as well as a specific profile identifying the
required functionalities and other characteristics of an EHR system in order to be considered as a
reliable source of data for regulated clinical trials. This should lead to a sustainable model and to the
quality labelling and certification of the EHR vendor systems. This type of certification of EHR systems
will, together with existing accreditation schemes for research units, accelerate the adoption of a
more harmonised approach throughout Europe and serve as a powerful means for ensuring
reliability and trustworthiness of the research partners (e.g. data providers) of the pharmaceutical
industry.
Consortium
1
AstraZeneca
2
The EuroRec Institute
3
Assero Limited (representing CDISC)
4
European Molecular Biology Laboratory
5
eClinical Forum Association
6
Athens University Medical School
7
Custodix NV
8
University College London
9
Kings College London
10 University of Dundee
11 Institut National de la Santé et de la Recherche Médicale – U872
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
89
12 Université de Rennes 1
13 Westfaelische Wilhelms-Universitaet Muenster
14 Friedrich-Alexander-Universitaet Erlangen-Nürnberg
15 Telematikplattform für Medizinische Forschungsnetze
16 Hôpitaux universitaires de Genève
17 Xclinical Gmbh
18 Assistance Publique Hôpitaux de Paris
19 The University of Manchester
20 MUW-POLCRIN
21 University of Glasgow
22 European Platform for Patient Organisations
23 European Association of Health Law
24 Heinrich-Heine-Universitaet Duesseldorf (representing ECRIN)
25 F. Hoffmann-La Roche Ltd
26 Eli Lilly
27 Sanofi Aventis
28 Merck KGaA
29 AMGEN
30 Johnson & Johnson
31 GlaxoSmithKline
32 Bayer Health Care
ARGOS is funded by the European Union. This is one of the seven
projects under the Pilot Project ‘Transatlantic Methods for Handling
Global Challenges in the European Union and the United States’.
90