Building FHIR Servers on Existing Application Databases A White Paper from Open Mapping Software Ltd R.P. Worden December 2013 rpworden@me.com Abstract: Fast Health Integration Resources (FHIR) is a new standard for exchanging healthcare information from HL7 International, which is rapidly being adopted by healthcare providers, suppliers of IT systems, and national initiatives. There is a growing requirement to enable existing healthcare IT applications as FHIR servers. Doing so can lead to improved levels of integration of healthcare applications, for a relatively modest effort. Adapting an application as a FHIR server involves several kinds of data transformation, to translate information between the application’s own internal data representations, and the FHIR representations. Developing these transforms can be time-consuming and error-prone. However, when the application data are stored in a relational database, the necessary transforms can be automatically generated from declarative mappings, rather than hand-coded, using an Open Source toolkit. With this mapping approach, an existing healthcare application can be enabled as a read-only FHIR server without writing any database-specific or resource-specific code – by defining database mappings and some other small configuration files. This takes a few days’ development effort per FHIR resource. This white paper describes the mapping approach, the open source tools, and their potential to accelerate the integration of healthcare applications. The Impact of FHIR Integration of healthcare applications cannot be achieved without standards for the exchange of information between them. Without such standards, there is a Tower of Babel in which the many healthcare applications cannot talk to one another. The development and application of these standards has consumed massive effort over the past 20 years, in organisations such as Health Level Seven (HL7) – the world-leading definer of such standards – and in the healthcare providers and IT system suppliers who apply them. Previous HL7 standards, particularly HL7 and CDA, are technically complex. This has made them unpopular both with the suppliers of healthcare IT software and the providers who use those systems. The Version 3-based standards have essentially only been deployed when mandated by national health IT programmes, where suppliers have been paid by the national programmes to implement them. This has not led to wide integration of healthcare data and systems. Most suppliers and providers avoid these standards as a source of extra complexity and cost, whenever they can. Fast Healthcare Integration Resources (FHIR) is a new standard for healthcare information exchange, developed by HL7. FHIR has recently become a Draft Standard for Trial Use (DSTU). There are key differences between FHIR and the previous standards. First, FHIR has been designed from the ground up with implementers in mind, so all the FHIR standards are accessible and easily implementable with widely available tools. The standard itself is completely open source. Second, FHIR has adopted an ‘80% principle’: that any feature can only be part of a core FHIR resource, if it is needed by more than 80% of the applications which will use that resource. Anything else is to be accommodated by an extension mechanism. This has avoided the feature bloat which beset the previous HL7 standards. Third, the FHIR development has been implementation–led, with all new ideas being tried out in reference implementations and servers. This has helped to avoid tricky features and retain the core simplicity of FHIR. As a result, any developer can learn the basics of FHIR in a few days, and can start to build working FHIR implementations in the same timescale. When developers learn about FHIR, they like what they see; they say: ‘Yes, I can do this’ and rapidly start doing so. This is in contrast to previous HL7 standards. Therefore FHIR is rapidly being taken up by suppliers, providers, and national initiatives. In England the new national e-Referrals system, for electronic booking of appointments, has adopted FHIR for its APIs. We anticipate that FHIR will become widely adopted at national, regional and local levels across the world – as IT suppliers buy into it, and as it becomes the easiest route to interoperability for the providers who use those suppliers’ products. However, FHIR does not need to achieve this multi-level adoption in order to be the best foundation for local interoperability - for instance, within one provider or small group of providers. If a single provider, such as a hospital, asks: “what is the best foundation for local IT interoperability within my organisation?” there are three possible answers: previous standards (such as V2, V3 and CDA), local ‘homebrew’ solutions, or FHIR. Already, FHIR with local extensions is the best of these answers. FHIR is much simpler and cheaper to deploy than the previous HL7 standards; it is better thought out than any homebrew solution is likely to be; it is within the skill-set of most developers; and it is a standard, increasingly supported by IT suppliers. So it already makes sense for any healthcare provider to base their local system integration strategy on FHIR. Using FHIR for Local Application Integration When a healthcare provider adopts FHIR for local integration, what steps do they need to take? FHIR is compatible with many interoperability paradigms and architectures, including REST, messaging, and SoA, so no major new architecture or infrastructure re-set is required. There is no need to rip and replace core applications by new FHIR versions. The important steps are incremental, businessled and clinical-led. These steps are: A. Decide, on business and clinical grounds, which applications most urgently need improved integration. B. Decide which FHIR resources these applications need to exchange in order to work better together (one of these is probably the Patient resource. What are the others?) C. Enable those applications as FHIR servers and clients, for the chosen resources D. Exchange and compare resources between the applications , in FHIR data formats E. Learn the business, clinical and technical lessons; initiate required changes to business and clinical processes This will not be a large ‘bet the hospital’ technology-led initiative; it is an iterative, business-led and clinically-led initiative, with the modest technical effort focused around step (C). The rest of this paper addresses step (C) – enabling existing applications as FHIR servers –and describes how it can be done with very modest cost, risk and elapsed time, using an Open Source toolkit. Most existing applications follow a three-layer architecture, of: 1. User Interface 2. Business Logic 3. Database (Relational). To integrate two such applications with FHIR, you need to build a FHIR adapter in front of each application. When application A is acting as a FHIR client, while B is a FHIR server, this looks like: Client A Server B User Interface User Interface Business Logic Relational Database FHIR Adapter FHIR Adapter Business Logic Relational Database Initially we assume that Application B is acting as a read-only FHIR server – able to respond to FHIR searches by sending the selected resources, but making no changes to its own data. Application A, which acts as the FHIR client, may need to update its own data. For application A, decisions about when to update data may need to involve the business logic and the user interface; but for Application B, they do not. The FHIR adapter for B needs just read-only access to the underlying database, if the adapter is designed in that way. We shall see that there are big cost advantages in doing so. Consider the sequence of events which occur when the client Application A requests some information from the server application B, using a FHIR RESTful interface. The simplest sequence is: 1. (Client A) compose and send a FHIR search request, on a resource in server B. 2. (Server B) translate the FHIR search request into its own internal search commands. This could, for instance, be an SQL query on its database. Run the query. 3. (Server B) translate the results of the internal search into FHIR data format. Send the resources to A in an AtomFeed bundle. 4. (Client A) translate the incoming FHIR resources to its own internal data representations, and use business logic to act on them. This involves three distinct pieces of data translation, between the FHIR representation and some internal data representation of an application. These are: Source Data Representation FHIR search request Server (B) internal data structures, returned by the search FHIR resource (XML or JSON) Target Data Representation Server (B) internal search instructions FHIR resource (XML or JSON) Client (A) internal data structures Although FHIR uses simple data structures as far as possible, the data structures of applications A and B may not be simple; and the intrinsic complexity of the healthcare domain mean that these three data transformations will not always be straightforward. All three are needed for even a simple interaction. Developing and testing these transforms may require significant effort and elapsed time, if they are developed in a procedural language without any use of automated tools. So even with FHIR, integration projects may not be straightforward, without using appropriate tools. The good news is this: By using the open source mapping tools from Open Mapping Software, all three types of transform can be automatically generated from declarative mappings, rather than being coded in a procedural language. This process is much less time-consuming and more reliable than developing procedural transform code. Using staff who are experienced in the mapping techniques and understand the database structures, any existing application can be enabled as a read-only FHIR server in a few day’s effort per resource type. To understand how this can be done, it is necessary to understand how the mapping tools work. Mapping and Transformation Toolset The mapping toolset from Open Mapping Software differs from most mapping and transform tools (such as Altova MapForce, or BizTalk Mapper) in one key respect. In most mapping and transform toolsets, the user defines direct mappings from one data structure (such as an XML tree, or a relational database) to another – typically using drag-and-drop across the two structures to make each mapping. Then the tools attempt to generate a transform between the two data structures, and often do not succeed (e.g. when there are structure clashes between the data structures – as there often are). When those tools do not succeed in generating a transform, you need to fill the gaps with procedural code, and you are back where you started. The mapping tools from Open Mapping Software (which will be called the Open Mapping tools) do not map directly from one data structure to another. Instead, they map several data structures onto one common logical model of the information – a UML class model, expressed in Eclipse EMF Ecore. Whenever two or more data structures have been mapped to the same logical model, the tools can automatically generate and run the transforms from any mapped data structure to any other. This has big benefits over the direct structure-to-structure mapping tools: Mappings are easier to make, because you do not need to bridge between two different complex data structures in one mapping operation. You only bridge from one complex data structure to a simpler logical model. If you have mapped N different data sources onto one logical model, the tools can generate all N*(N-1) transforms, from any data structure to any other. This saves a lot of effort, compared to making N*(N-1)/2 sets of mappings. Mappings are declarative and easily testable, with testing tools in the mapping toolset. Therefore the generated transforms are much more reliable. When the data structures differ markedly (e.g. a flat relational database, versus deep XML) the tools generate accurate transforms where other tools fail. The application of this mapping approach to FHIR is shown here: CDA XML Structure M = mappings B = Bridge M V2 XML Data Structure M FHIR class model (EMF Ecore) B FHIR Java reference implementation FHIR serialisation (XML or JSON) M Relational Database Structure The central logical model (which all the data structures are mapped to) is the logical model of a FHIR feed and resources – generated automatically from the FHIR specification. The lines marked ‘M’ denote mappings from other healthcare data structures onto this class model. The mappings are created manually and tested using the mapping tools. The tools include a bridge between the EMF Ecore class model and the FHIR Java reference implementation. The bridge is denoted by ‘B’, and is Java software supplied with the tools. The FHIR reference implementation can read or write any resource as XML or JSON. So if any data structure is mapped onto a FHIR resource class model, the tools can automatically generate and run transforms in both directions between that data structure and FHIR, either in XML or JSON form. This can be done for commonly used healthcare data structures such as HL7 Version 2 or CDA – or for any relational database. When an application’s data is stored in a relational database, data in the database can be transformed to FHIR (in both directions) using transforms generated automatically from database mappings. This is a fast and effective way to create the transforms in rows (2) and (3) of the table above. The other required transforms – from a FHIR search to an SQL query on the database – are also generated automatically by the tools, from the database mappings. Therefore all the transforms required for a FHIR adapter to an application with a relational database can be generated automatically from mappings, and need not be hand-coded. This is a major saving in the development effort to build a FHIR adapter. The screenshot below illustrates the sort of things you need to do, using the mapping tools, to develop database mappings onto a FHIR resource. The left-hand pane shows the structure of a typical relational database as a tree - with nodes representing the whole database, each table, records in tables, and columns in records. The righthand pane shows the EMF Ecore class model for a bundle containing a FHIR resource (in this case, the Patient resource) and the resources it references. A mapping is a declarative statement which means: ‘node A in the database represents feature B in the class model’ – for instance: ‘each occurrence of this record in the database represents an instance of the Patient resource class’. To make this mapping, you select the Patient node in the class model tree, select the appropriate ‘record’ node in the database structure tree, and make the mapping using a menu command. Simple mappings like this take a few seconds to make. An important aspect of mapping any database onto a FHIR resource is the data format conversions, required to convert from the database representation of some property, to the standard FHIR data formats. If these format conversions are simple translations of a small number of code values, a conversion table can be defined directly in the mappings. For more complex format transforms, you may need to write a small piece of Java code (to be invoked by the mappings) to make the conversions, if such code does not already exist in a library of common format transforms. This is the only procedural code you need to write to make a FHIR server. Once you understand the principles of mapping, it is straightforward to make and test the mappings. All the mappings required for one typical FHIR resource can be made in a few days’ effort (at most). Configuring a FHIR server Once you have mapped a database to a FHIR resource, it is a straightforward process to build a working FHIR server which uses that database to serve the resource. No further coding is required. You need only provide four simple configuration files to define the server. These are: 1. servers.csv defines the FHIR servers provided by the web service. For each server, it defines the required jdbc connection to the application database. 2. resources.csv defines the resources supported by each server. For each resource, it defines the mapping file which maps from the database onto the resource class model 3. searches.csv defines the FHIR searches supported for each resource 4. A small XML narrative template file per resource defines the format and content of an automatically generated narrative for the resource. Having built these configuration files, you assemble them together with the necessary java jar files and mapping files into the required folder structure, package it as a .war file, and it will run the FHIR service (at least under Tomcat). The process of defining the mappings and configuring the server is described in more detail in a companion paper: ‘Building a FHIR server on an existing application: Step by Step guide’ How the FHIR Server Works A demonstration of two FHIR servers configured in this way is available at http://worden.globalgold.co.uk:8080/FHIR/farm/demoServer.html. When one of these FHIR servers responds to a FHIR search request, the following happens on the server: 1. The FHIR search request is converted to an object query which uses only the language of the FHIR resource class model. 2. Using the database mappings, the object query is converted to an SQL query against the database, to retrieve the logical ids of all resources which satisfy the query conditions. 3. The SQL query is run to retrieve the resource ids. Another filter enforces any conditions which are not expressed in SQL. 4. A further SQL query is generated and run once for each resource id, to retrieve all mapped features of that resource 5. Using the mappings, the retrieved records for each resource id are converted into a FHIR resource instance, expressed in EMF Ecore. 6. The resource narrative is constructed from the resource instance and its narrative template file 7. The resources are packaged up into an AtomFeed bundle, and are sent to the client (using the bridge and the java FHIR reference implementation) The architecture which does this is shown here: Web server (e.g. Tomcat) Browser Client Application http FHIR MultiServer jdbc Relational Database 1 Relational Database 2 Configuration Configuration File Configuration File File 3 Relational Database 3 All this happens with no resource-specific or database-specific code, except possibly for a few small data format conversions. FHIR Servers and Clients with Update Capability So far we have discussed read-only FHIR servers, which can respond to searches and provide data in FHIR resource form, but which do not update their databases in response to update requests. If any application is to update its database, we would expect that the application’s business logic will be involved in deciding whether and when to update the database. So it is generally not possible to adapt an application as a read/write FHIR server without some coding in the business logic layer. Even in this case, mapping the application database onto the FHIR resource provides a significant benefit. Given the mappings, a generated transform can be run in the reverse direction, to transform an incoming FHIR resource into an XML tree version of all the changes which will need to be made in the database tables. So while the business logic of making the database update is not addressed by the mapping tools, the data transform aspect is addressed. This applies to applications acting as FHIR clients, as well as those acting as servers. FHIR and Data Quality When two or more applications have been enabled as FHIR servers for the same resource type – for instance, enabled to find and return Patient resources – it is then possible to search the same resource on all those servers, and compare the results. Comparisons are possible and easy because all resource data are returned in the same FHIR logical form, independent of which server returned them. It becomes easy either to inspect the data from two or more servers side by side, or to build software which assists in making the comparison. Issues of data quality and consistency across the different applications are easily exposed. The mapping toolset which is used to configure the FHIR servers includes a comparative query tool, which makes it easy to run these comparisons, and inspect and store the results. The process for doing this is as follows: 1. Compose a query in a simple object-oriented query language, which selects FHIR resources and displays some of their properties. Queries are expressed entirely in terms of the FHIR resource class models. 2. Connect the query tool to a number of application databases which have been mapped to a FHIR resource model. 3. Using the mappings, the query tool automatically converts the query to SQL retrievals against each database. 4. The query tool runs the SQL queries and translates the results to express them in terms of the FHIR resource logical model 5. The results are displayed in tabular form, which can be sorted on any columns, or saved as csv files/spreadsheets Inspecting these query results give a clear view of the data quality and consistency between the different application databases. An example query (using artificial data) is shown below. First, the query tool is connected by jdbc to two application databases, both of which have been mapped to the FHIR patient resource: The two databases are denoted by the codes A and B, which will appear in the query results. Next, a query is composed (or a saved query is retrieved), using only the language of the FHIR patient resource: select Patient.identifier.value Patient.name.given.value Patient.name.family.value Patient.birthDate where Patient.identifier.system = 'NHS' This query displays some basic data for all patients in the database who have an NHS number (the correct FHIR value of ‘Identifier.system’ for the NHS number has not yet been defined by NHS, so the string ‘NHS’ are used instead). The fields to display are defined by paths in the Patient resource. Next, the query is run against the two databases which have been connected to it (it may need to ask for user names and passwords for the databases), and the results are displayed as a table: Here, the query results have been sorted by NHS number, to group together the Patients from different databases with the same NHS number, and expose the data differences. (In this case the differences are rather extreme because of arbitrary test data; but in more realistic cases, the discrepancies are easily visible.) Duplicate records in one database would also be visible. In this way, enabling several application databases as FHIR servers can be a big step forward in understanding the issues of data quality and consistency between those databases. Depending on the expected impact of those data quality issues on costs, quality of care and patient safety, it may then be decided to address the issues and remedy the quality problems. If this remedy includes transferring patient data from one (more highly trusted) application, such as a Master Patient Index, to the other application databases, then the FHIR mappings can help here too. When two databases have both been mapped to one FHIR resource, the mapping tools can generate and run the transforms required to translate data from one database format to the other – to assist in the ETL process. FHIR and the NHS England Information Strategy Some key features of the NHS England information strategy are: ‘Connect all’ – existing healthcare applications can be retained, but must interoperate with other applications All applications should use the NHS number as the main identifier for patients. All applications should have Open APIs so that other applications can interact with them. It is recognised that some NHS trusts have a long way to go to achieve these goals of universal use of the NHS number and open APIs. There is a good fit between FHIR and this set of requirements. A practical way for any NHS trust to move in the direction of the NHS information strategy is the following: 1. Enable its key applications as FHIR servers, focusing particularly on the Patient resource (which exposes the NHS number if present), using the database mapping method described in this paper. The main cost, of making the database mappings, can be as little as a few mandays per application. 2. Enable key applications as FHIR servers for other important resources such as Conditions, Medications and Allergies – using database mappings to do so. 3. FHIR then becomes the open API between applications (as is already NHS policy for eReferral applications) 4. By reviewing Patient resources across applications, conduct an NHS number audit of the key applications 5. Use the FHIR interfaces and data transforms to populate the NHS number in those applications where it is poorly populated, and to address other data quality problems. A key benefit of this strategy (as well as being very cost-effective) is that it proceeds in small incremental steps, making the results visible at every stage. It does not require large technologydriven investments or long delays before it starts delivering results. This strategy at a local NHS trust level could be made even more effective, if NHS were to make the Personal Demographic System (PDS) and/or Spine mini-services available as a read-only FHIR server for the Patient resource – enabling any trust easily to compare its own Patient resources against the PDS patient resources, in the common FHIR format. Status of This Work The mapping tools, required to enable applications as FHIR servers by database mapping, are Open Source and are freely downloadable from www.OpenMapSW.com, under the Eclipse Public Licence. The extesnsion of these tools to handle FHIR is work in progress. Some features which are not yet implemented, but which we intend to implement in the near future, are: Resource extensions: we believe that at the level of the EMF Ecore model and database mappings, we can make resource extensions transparent – i.e. extension features will appear no different from core resource features. Types of searches supported: At present the range of searches supported is limited – e.g. we do not yet support date range searches or reference searches. This will extend incrementally with new releases. Documentation of the FHIR tooling: real soon now. Libraries of common data format transforms: will evolve with contributions from our users. We are currently working with a London NHS trust to provide FHIR APIs for their EHR system. This is work in progress, and we have so far developed a FHIR adapter (read-only, Patient resource) for their legacy PAS system to interface to the EHR. This is running on the test configuration of the PAS. Open Mapping Software can provide support and training in the mapping tools, and a FHIR mapping service, to develop FHIR mappings and configuration files for any application database. Please send any comments, enquiries or suggestions about this work to rpworden@me.com.