The Dutch Healthcare Authority (NZa) is
The Research Paper Series presents scientific
the regulator of health care markets in the
research on health care markets and
Netherlands. The NZa promotes, monitors
addresses an international forum. The goal
and safeguards the working of health care
is to enhance the knowledge and expertise
markets. The protection of consumer
on the regulation of health care markets.
interests is an important mission for the
Martin Smits, Jan Luijsterburg and Marcel van Ham
NZa
|
This paper reflects the personal views of
term efficiency, market transparency,
authors, which are not necessarily those of
freedom of choice for consumers, access
their employers. This paper is not in any way
and the quality of care. Ultimately, the NZa
binding the board of the NZa.
aims to secure the best value for money for
consumers.
TILEC
TILEC was born out of the recognition that
economics in research endeavours,
the traditional ways of organizing academic
even if they do not necessarily fall within
research - along faculty lines - are no longer
‘Law & Economics’ in the sense of the specific
adequate today. Researchers in law strive to
school of thinking which has arisen out of the
draw upon economics and yearn to work
work of US academics and is now well-
with economists, and vice versa.
established everywhere.
Furthermore, the outside world - market
actors, authorities, practitioners - has come
The mission of TILEC is:
to expect researchers from law and from
-
for participating researchers from the
Faculties of Law and Economics, to
premium on research that rests on both
provide support and to stimulate joint
disciplines. Given its excellent Faculties of
research activities, thereby enhancing
Economics and Law, Tilburg University is in
the intellectual climate for research at
an ideal position to meet the expectations
Tilburg University in the area;
of researchers and the outside world alike.
-
towards the outside, TILEC aims to belong
TILEC is meant to be the vehicle
to the top in Europe and to be recognized
for doing so. TILEC will be concerned broadly
as a leading centre in its areas of activity
speaking with the use of both law and
also in the US.
TILEC is affiliated with Tilburg University
Dynamics of Inter-Organizational Information Systems in
Health Care from an Actor Network Perspective
Martin Smits, Jan Luijsterburg and Marcel van Ham
2009
economics to work together, putting a
Dynamics of Inter-Organizational Information Systems in Health Care from an Actor Network Perspective
NZa. The NZa aims at short term and long
Corresponding author: m.t.smits@uvt.nl,
GGzE institute for Menntal health care, Eindhoven, Netherlands
GGZ Breburg institute for Mental health care, Breda, Netherlands
Center for Economic Research, Tilburg University, Netherlands
Martin Smits (1), Jan Luijsterburg (2), Marcel van Ham (3)
DYNAMICS OF INTER-ORGANIZATIONAL INFORMATION SYSTEMS
IN HEALTH CARE FROM AN ACTOR NETWORK PERSPECTIVE1
Abstract.
The health care sector is a huge industry in many Western countries going through fundamental
changes, with increasing needs for new Inter-Organizational Information Systems (IOISs) and
performance indicators that will help to increase transparency and performance of the sector. The aim
of this research is to understand the contextual dynamics involved in the implementation of a nation
wide IOIS in health care. We analyze the development of a large nation wide reporting system (IOIS)
in the mental health care sector in the Netherlands between 1998 and 2008, using Actor Network
Theory and analyze performance of the IOIS in 2007. The study confirms previous findings that IOIS
are built over years by layering new systems on top of old ones and that IOIS evolve over years as
result of interactions between actors.
INTRODUCTION
Key words: inter-organizational information systems; actor network theory
1
For over 25 years, inter-organizational information systems (IOIS) have influenced collaboration and
transactions between firms (Barret and Konsynski, 1982; Klein 1996; Steinfield et al, 2005). Recently,
researchers have shown that IOIS are built over years by layering new systems on top of old ones
(Lyytinen and Damsgaard, 2001; Rodon et al, 2008) and that the installed base of systems,
processes, culture, and stakeholder interests influence the path and success of IOIS implementation.
Implementation of new IOIS may therefore be regarded as improving and changing existing IOIS.
There is a need to identify success and failure factors for changing these large, costly, and valuable
systems (Payton, 2000; Subrami, 2004). Recently, the context of IOIS developments in health care
has changed because of the Google (GoogleHealth) and Microsoft (HealthVault) initiatives aiming to
manage health data in personally controlled health records (PCHRs). Google and Microsoft see
business opportunities and Fortune 100 companies in their role as employers see efficiencies and cost
savings when patients can securely store, access, augment, and share their own copy of electronic
health information (Mandi and Kohane, 2008).
This paper is published in the TILEC DP series (www.uvt.nl/TILEC) and has been submitted to ECIS 2010
Monitoring systems for health care delivery and health care status on the national level have been
developed in many countries (Arah et al, 2006; Westert and Verkleij, 2006) and are expected to
provide adequate indicators for different aspects of care (such as effectiveness, quality, and efficiency)
of the various sections of the health care system to one or more supervising agencies (Payton, 2000).
To fulfill all reporting requirements of the different actors involved, health organizations have
developed many automated and manual reporting systems (Mandli and Kohane, 2008). Attempts to
replace and aggregate old systems by a new IOIS is a complex effort since different actors may have
different objectives for implementation, use different health care performance indicators and therefore
may evaluate the success of an IOIS differently. The interplay between different actor objectives and
existing infrastructures on different levels complicates the identification of success and failure factors
for the development of inter-organizational information systems (Walsham and Sahay, 1999; Rodon et
al, 2008).
1
This paper focuses on the development of a monitoring IOIS in health care in the Netherlands
between 1998 and 2008. The IOIS was developed by a variety of organizations, including the
association of insurance companies, the national health care inspectorate, and the Ministry of Health.
The new system replaces several separate reporting systems, aiming to reduce the administrative
burden for mental health care organizations by reducing the numbers of external reporting relations
(IGZ, 2005). We analyze the step-wise development, implementation, and success of the system from
an Actor Network perspective.
The aim of this research is to understand the contextual dynamics involved in the implementation of a
nation wide IOIS to provide health care information. The theoretical relevance of this research is the
use of actor network theory as the basis for the analysis of the development of a large scale IOIS and
the linkage of ANT analysis to network effects and success indicators on different organizational and
network levels (Provan, 2007). The practical relevance of this research is that it may provide
guidelines for organizations to successfully participate in IOIS development and exploitation and to
aim for ‘optimal success’ in a given network setting among actors with a variety of objectives.
THEORY
This paper is organized as follows. First we provide theoretical background and describe our research
method and give an overview of the mental health care sector. Next we present our findings and end
with discussion and conclusions.
2
Actor Network Theory
We use Actor Network Theory (ANT) as the theoretical lens to evaluate the development and
implementation of inter-organizational reporting systems in health care. Other theories that may have
been used are industrial dynamics and network effects theory focusing on the standardization aspects
of implementing IOIS (Markus et al, 2006). We have chosen ANT because it tries to explain how
‘things’ evolve from a multi actor perspective on the interaction between human and non-human
(technology) actors (Walsham and Sahay, 1999; Sarker et al, 2006). We first summarize ANT and
define health care information systems and performance indicators.
2.1
ANT is a sociological theory developed in the 1980s by Latour, Callon, and Law (for an overview of
ANT in IS research, see Walsham and Sahay (1999) and Sarker et al (2006)). ANT is related to
structuration theory (Giddens, 1983) and systems theory (Ackoff, 1971). The focus of ANT is on how
people and objects are brought together in stable, heterogeneous networks of aligned interests (Law,
1991). ANT is distinguished from other network theories in that an actor-network contains not merely
people, but objects and organizations. These are collectively referred to as actors, or sometimes
actants, that together form a heterogeneous network consisting of both social and technical parts.
Moreover, the social and technical are treated as inseparable by ANT. ANT claims that any actor,
whether person, object (including computer software, hardware, and technical standards), or
organization, is equally important to a social network. As such, societal order is an effect caused by an
actor network running smoothly. This order begins to break down when certain actors are removed or
changed. For example, the removal of telephones, banks or the president may all result in significant
break-downs and changes in social order (home.aisnet.org).
A major focus of ANT when applied in a particular context is to try to trace and explain the processes
whereby relatively stable networks of aligned interests are created and maintained, or alternatively to
examine why such networks fail to establish themselves (Walsham and Sahay, 1999). ANT has been
applied to analyze the successes of large information systems implementations, such as hospital
information systems (Bloomfield et al, 1992), a nation wide geographical information system in India
(Walsham and Sahay, 1999), an inter-organizational harbor information and transaction system for
cargo handling in Barcelona (Rodon et al, 2008), and the implementation of a radiology network
system in a hospital (Cho et al, 2008).
ANT analysis is based on the distinction between two main phases or processes in the development
of an actor network: the Translation process and the Inscription process. In the translation process the
interests of a set of actors are aligned to each other or to a focal actor. Four steps can be
distinguished in the translation process (Rodon et al, 2008). The first step is ‘Problematization’ when
one actor frames a problem or an opportunity and attempts to persuade others that it is worth to
dedicate resources to it. The second step is ‘Interessement’ when actors become interested and
change their affiliations in favor of the new actor (triggered by opportunities or incentives). Step three
is ‘Enrolment’ when actors enter multi-lateral negotiations, followed by step four ‘Mobilization’ when
actors stabilize the network by making durable and irreversible relations.
During the Inscription process the agreements and interests get embodied into artefacts (such as
software, text, procedures). When such an artefact is inscribed into a piece of technology, the
technology becomes an actor imposing its inscribed program of actions on its users (actors in the
network). Appendix A includes an overview of ANT concepts defined in four articles published in main
IS journals (JIT, MISQ, EJIS, JMIS). The main ANT concepts and processes are summarized in Table
1, following Walsham and Sahay (1999) and the operationalizations of concepts by Rodon et al
(2008). Note that three ANT concepts (irreversibility, black box, and immutable mobile) indicate
irreversible situations and artifacts that result from translation and inscription processes.
ANT is suitable for studying the implementation of IOIS for three reasons (Rodon et al, 2008). First,
ANT helps to explore how actor networks are formed, hold together, or fall apart and emphasizes on
the process aspect of implementation. Second, ANT provides an analytical framework for studying
power processes within a socio-technical context. Third, ANT does not regard technologies to be
stable entities, but as evolving artefacts that change over time, while being passed from community to
community. Through the ANT lens, the implementation of an IOIS may be regarded as the
emergence, development, and stabilization of an actor network.
Description
Both human beings and nonhuman actors such as technological artifacts
Heterogeneous network of aligned interests, including people, organizations, and standards
Creating a body of allies, human and non-human, through a process of translating their
interests to be aligned with the actor network. Four steps in the process are:
- Problematization: one actor frames a problem or an opportunity and attempts to persuade
others that it is worth to dedicate resources to it
- Interessement: actors become interested and change their affiliations in favour of the new
actor
- Enrollement: actors enter multi-lateral negotiations
Through the ANT lens, an IOIS is a stabilized set of relations between actors, either humans or nonhuman artefacts (systems) and rules (laws, policies). ANT is related to contextualism theory which
states that useful research on (organizational) change should involve the continuous interplay
between ideas about the context of change (the various systems levels that might be relevant for
change), the process of change (the actions, reactions and interactions of the various interested
parties as they negotiate around proposals for change), and the content of change (the area where
changes takes place), together with relations between the three (Pettigrew et al, 1992).
Concept
Actor (or actant)
Actor network
Translation
process
Inscription
process
Immutable mobile: network element with strong properties of irreversibility and effects that
transcend time and place, e.g. software standards.
- Mobilization: actors stabilzize the network by starting durable and irreversible relations
New agreements and interests get embodied into artefacts such as software, texts,
standards, procedures. Artefacts have properties like:
- Irreversibility: the degree to which iti is subsequently impossible to go back to a point
where alternative possibilities exist
- Black box: frozen network element with irreversible properties
-
Table 1. ANT concepts (based on Walsham and Sahay (1999) and Rodon et al (2006)
We see two problems related to ANT. First, the key concepts in the theory are defined differently by
the various authors in the field (see appendix A). Second, previous authors specify no clear linkages
between “the two phases” and “outcome measures or effects of the phases on the network or network
participants”. For instance, Walsham and Sahay aim to evaluate the ‘implementation success of
nation-wide geographical IOIS in India’ (1999, p 40), but do not include success or effect measures,
but describe developments (inscriptions) on local and national levels. To overcome this problem we
have included outcome measures in our analysis to evaluate effects on the individual actor level and
on the network level.
2.2
IOIS and Health care performance indicators
We focus on the development of an inter-organizational information system (IOIS) that is intended to
support performance measurement and monitoring of mental health care on institutional, local,
regional, and national levels. Performance data are gathered by means of an IOIS that connects
health care organizations to regional and national platform organizations. An IOS is any networkbased information system that extends beyond traditional enterprise boundaries (Hong, 2002). IOIS
require specific IT applications and business process changes at both sides of business relations in a
business network. An IT application in one organization (an internal information system) might be
adapted for inter-organizational communications, but without requiring changes ‘at the other side’ of
the SC relation (Holland, 1995). IOIS vary from relatively simple data exchange applications to
external reporting systems, cash management systems, and extended enterprise resource planning
(ERP) systems (Clark and Lee, 2000).
Health care performance indicators play an important role for assessing the success of health care
information systems. A health care information system is considered to be successful if the output
consists of the right set of performance indicators in a reliable, timely, and complete matter Arah et al
(2006). Different actors require different indicators and will therefore evaluate the success of a system
differently. The success of an IOIS may be evaluated as the degree to which the system meets the
needs of different actors and as the degree to which it is able to report required performance
indicators.
METHOD
Health care performance assessment includes a large variety of indicators. Hermann et al (2000)
evaluate the American National Inventory of Mental Health Quality Measures and identify 86
measures in the following six categories: (1) treatment appropriateness (65% of all indicators); (2)
treatment continuity; (3) accessibility of care; (4) coordination of care; (5) disease detection and
diagnostics; and (6) disease prevention. Donabedian (1988) identifies three categories of health care
performance based on the structure, process, and outcome of health care. Structure indicators relate
to all tools and resources that are within reach of personnel and management in a health care
institution. Structure indicators include the entire organizational environment in which care processes
take place (material, finance, people and organization structure indicators). Process indicators relate
to all activities that take place between health care players (providers) and clients (consumers),
including indicators for clinical improvement of individual health and indicators for social and
psychological interactions between actors. Outcome indicators relate to the effects of the primary
processes on health and well being of clients (patients) and employees, including indicators of health
services.
3
We investigate how implementation of a large scale inter-organizational reporting system is shaped
through interaction with the health care context in which the system is embedded. This how-nature
question suggests a case study approach (Yin, 2003). More over, as our main interest is related to
change processes, we adopt an interpretive approach (Walsham, 1995). We analyze one (large) case,
being the development and implementation of a new national dataset of performance indicators and a
new inter-organizational reporting system for mental health care in the Netherlands.
Our research method consists of retrospective analysis of one case using a combination of in depth
qualitative, retrospective, and longitudinal ANT analysis of one IOIS (over a period of about 10 years)
and quantitative analysis of the success of the IOIS in 2007 (cross-sectional) by using a data set
including performance data of 64 mental health care organizations. In terms of qualitative data
analysis, the identification and development of ANT concepts, themes, and issues was achieved
during a one and halve year period (September 2007 – March 2009), as suggested by Walsham and
Sahay (1999), by careful reading and reflecting on the data and by frequent discussions between the
researchers. We used over 50 documents and publications covering mental health care in the
Netherlands between 1997 and 2008. Documents were gathered from internal archives of mental
health care organizations, and by desk and web research. Documents include reports from GGZ-Nl,
(mental) health research organizations (Trimbos institute, RIVM institute (RIVM, 2006)), and Ministry
of Health Care.
A final aspect of our ANT research method concerns our role as researchers. Following Walsham and
Sahay (1999) we identify our roles in this research. One author takes position in the role of
‘independent observer’ with mainly a descriptive objective. The other two authors take positions as
‘action researchers’ with active involvement in the change process, since they both work as
information managers in two mental health care organizations and participated in four national working
groups and platforms on the implementation of the IOIS.
Quantitative information on the success of the IOIS implementation come from the dataset JMV-2006,
semi-structured interviews with 12 operational managers from mental health care organizations, and
additional information on JMV from several publicly available reports. JMV-2006 is the “Yearly
Document Societal Accountability” and is the first dataset in the Netherlands that combines the scores
of all mental health care organizations in the Netherlands on 35 questions that cover 17 performance
indicators in three categories: effectiveness, safety and client-focus of care. This so-called ‘basic set of
performance indicators’ and the 35 questions have been designed and developed since 2005 by eight
large organizations: the national platform of health care insurers (ZN); the national platform of mental
health care patients (LPGGZ); the national institute of mental health care organizations (GGZ-Nl); the
associations of psychiatrists (NVvP), psychotherapists (NVP), psychologists (NIP), the national
platform of care and cure (V&V), the association of nurses in mental health care (FVidGGZ), the
Ministry of Health (VWS), and the Netherlands Health Care Inspectorate (IGZ).
The mental health care sector
Individual interviews were done (by telephone) in 2007 with 12 managers from 12 organizations,
randomly selected out of the 64 mental health care organizations. Managers were selected who had
been responsible for (part of) the input to JMV-2006. The interviews took about one hour each and
focused on the difficulties in providing the right data and answers to the 35 questions. For additional
information on the ‘basic set of performance indicators’ for mental health care we used the IGZ
publication on this topic (www.igz.nl) and a research report (Ham et al, 2007).
3.1
The health care sector is a huge industry in many Western countries. For instance, health care
expenditures in the US were nearly two trillion dollars per year in 2004 (Smith et al., 2006; CMMS,
2007). The health care sector in many countries represents more then 10% of the gross national
product, is knowledge and information intensive and employs over 10% of the total national workforce
(OECD, 2004). Performance of health care organizations is an important issue in many countries
(OECD, 2004). A number of factors have converged to establish a national agenda to increase
transparency and to monitor and improve the quality of health care. Rapid changes in the organization
and financing of care have put unprecedented pressure on health care delivery organizations to
reduce utilization and costs, leading to a need to ensure that quality is not adversely affected
(Hermann et al, 2000).
Mental health care spending in the Netherlands (17 million people) is about 4 billion Euros per year
(2003) and accounts for about 7% of the total health care spending. The numbers of patients treated
for mental health have risen from 638.000 (2003) to 757.000 (2005) (www.ggznederland.nl). The
mental health care sector consists of about 25 large, integrated institutions that provide many types of
care, and around 50 smaller organizations for specialized care, including care for addictions, institutes
for guided living (RIBW), child and adolescent care. The Ministry of Health in the Netherlands has
indicated several core themes for mental health care in 2008, such as prevention, chronic mental
health care, and forensic psychiatry (www.vws.nl).
Financing of mental health care organizations has changed substantially over the past years. All
financial revenues used to come from one (public) national source (AWBZ), based on a system of
yearly budgets and input-financing. The mental health care sector is now changing into output
financing (like the USA diagnosis related payments) by (private) insurance companies. These changes
have resulted in the introduction of various new (internal) information systems in mental health care
organizations to support the new financial billing rules and aiming to assess treatment effectiveness,
efficiency, and quality. Additionally, several inter-organizational information systems have emerged to
enable transactions between care providers, insurance companies, government agencies, and
platform organizations and associations.
CASE ANALYSIS
The national Act on Quality of Health Care Organizations in the Netherlands states “care providers
must systematically gather and store data on the quality of care….must evaluate how care processes
lead to good care….” In order to reduce the administrative burden for health care organizations, the
health care sector has implemented the “Yearly Document Societal Accountability” (JMV), replacing
the variety of separate inter-organizational reporting systems. JMV will ultimately include performance
indicators for the total health care sector in the Netherlands. Since 2006, the basic set of performance
indicators for mental health care is included in JMV. Since 2006, mental health care organizations
enter their performance data by answering an on-line questionnaire in the web-based application
digiMV. DigiMV was developed in 2004 by the Ministry of Health and is intended to be used by all
health care organizations.
4
IOIS developments since 1998
We first describe IOIS developments in mental health care since 1998, based on a qualitative,
longitudinal ANT analysis (4.1). Then we present the results of IOIS performance in 2007 (4.2).
4.1
We distinguish between six main actor groups in the mental health care sector, as listed in Table 2.
Mental health care is provided by about 100 care organizations (GGZ), with their umbrella
organization (GGZ-Nl). Rules and legislation on tariffs and care services are determined by the
Ministry of Health (VWS) and ministry-related organizations like NZA (supervises markets and
competition in health care), CVZ (determines the yearly tariffs for care), and IGZ (the health
inspectorate that supervises quality of care). Other actor groups are the representative platforms for
care providers (psychologists, psychiatrists, psychiatric nurses, etc.), the platform for all (about 20)
insurance companies in the Netherlands (ZN), and the platform of about 20 client representative
organizations in mental health care (LPGGZ).
The starting point of our IOIS analysis is 1998, the year in which the representative organization for all
mental health care organizations (GGZ Nl) was established and plans were initiated to create one
national mental health care information system (ZorgIS). Reporting systems in mental health care
already existed before 1998, but on a much smaller scale. In the seventies, the Dutch Mental Health
Inspectorate, in cooperation with the National Hospital Board, initiated the National Inpatient Registry
(NIR), which contained information on client admissions for most psychiatric hospitals and institutes for
treatment of addictions. NIR was used between 1978 and 1996 (2007). Mental health care
organizations had to develop internal systems to create data input for NIR. National umbrella
organizations would aggregate local information using systems like RIS/NIS, VERNIS and
LADIS/CADIS, which were all well-known acronyms in mental health administration in the 1980s.
Actor
Mental health care
organizations. (GGZ)
National institute of
mental health care
organizations (GGZ-Nl)
Ministry of Health (VWS)
Characterization
Provide specialized mental care to clients/ patients. Until 2007, the institutes were
paid (lump-sum) by the Ministry of Health, and since 2007 by insurance companies
(output based financing; new health insurance law).
Branch representative of all (about 100) mental healthcare organizations,
established in 1997 as a merger of specialized representative organizations in
mental health.
The Ministry has separate (independent) organizations to supervise quality of care
(Health Care Inspectorate (IGZ)), to supervise tariffs and competition (Care Authority
(NZA)), and to determine health services and tariffs to be provided in standard
health policies (Health Insurance Platform (CVZ)).
Branch representative of all (about 20) health insurance companies in the
Netherlands.
Psychiatrists (NVvP), psychotherapists (NVP), psychologists (NIP), nurses (V&VN,
FVidGGZ),
Branch representative of 20 patient representative organizations.
Overview of actors in mental health care in the Netherlands since 1997
National platform of health
insurers (ZN)
National Associations of
Professionals
National Platform of
mental health care
Patients (LPGGZ)
Table 2.
Stage 1: Emergence and decline of IOIS ZorgIS (1998-2003)
Figure 1 gives an overview of the IOIS developments in the mental health sector between 1998 and
2008. Mental health care institutes (left side of Figure 1) provide data input and the IOIS output is
created and used by the other actors (right side of Figure 1). The first stage of IOIS development on a
national scale for mental health care is between 1998 and 2005: the stage of the ZorgIS system.
ZorgIS contained information on care processes in mental health care organizations. The second
stage (since 2004) is characterized by the emergence of a national IOIS for reporting on diagnosis,
treatment, and costs (DIS). The third stage (since 2005) is characterized by the emergence of a
national IOIS for reporting on care quality (DigiMV). We now use ANT to analyze the interactions
between the actors and the IOIS developments.
4.2
2. DIS – IOIS
(since 2004)
Care volume
3. DigiMV - IOIS
(since 2005)
Care quality
1
3
4
4
2
GGZ Nl
4
GGZ
Data
Warehouse
(since
2008)
Yearly
Reports &
Analysis
NZA
Inspectorate
Ministry of
Health
Health
Insurance
Platform
CVZ
x Translation process ZorgIS
The mental health care sector in the Netherlands underwent substantial reorganization during the
1990s. Several hundreds, most small and specialized mental care organizations merged into about
114 larger organizations. These new organizations provided more types of care (such as addiction
care, outpatient and inpatient care) and integrated care for specific client groups (young, adult, and
elderly clients). In 1997, these mergers were followed by a merger of the originally fragmented
umbrella organizations into one national platform organization for mental health care (GGZ Nl). The
new organizations felt a “strong need for a common language and shared definitions of patient data
and care characteristics that could be used for epidemiologic research and benchmarking of regions
and institutes” (problematization). The care organizations declared that they “would contribute to a
new national registry, replacing three existing national registries” (interessement).
2
3
1
1. ZORGIS
IOIS (1998-2005)
Stages in IOIS development (rectangles denote systems, circles actors (see text)).
Mental Health
Care Institutes
(GGZ)
Figure 1.
In 1998, a project group was formed (known as ‘Output2’) consisting of representatives of the member
institutes (enrollement). Output2 developed detailed data definitions for institutional reporting and
national and regional benchmarking. Data definitions were based on Balanced Scorecard principles,
including client characteristics (address, sex, diagnosis codes when entering and leaving care), care
process characteristics (including the type and amount of care provided), and treatment data. This set
of data standards became known as the ‘Care Data Set’ (‘Zorggegevensset’).
All mental health institutes agreed to “gather the required data on the organizational level and to send
the dataset every three months to GGZ Nl”. As compensation, each institute that would deliver the
data in time would be exempted from their legal obligation to report waiting time data to the Ministry of
Health, because waiting time data were included in the set and would be reported by GGZ Nl
(mobilization). Actual use of ZorgIS started in 2000 and around 50% of the mental health care
institutes provided data (Wierdsma et al, 2007).
x Inscription process ZorgIS
Several irreversible, immutable mobiles were formed. First, information systems of the institutes were
modified to enable data entry and storage of the items needed according to the new definitions and
rules, and to generate the quarterly export file per institute. This file had to be uploaded in a new
national data entry application (second immutable mobile), which also checked for data consistency.
Implementation of the data reporting process by the institutes resulted in strict internal data definitions.
The data entry application also provided secure data transportation to the national data warehouse.
Third, a web portal was developed to provide data access and online analytical processing to member
organizations. The data warehouse became an instrument for GGZ Nl to respond to member
organizations and external parties. Also data extractions were made for research purposes.
The effects of the inscription process were disappointing. It turned out to be difficult for mental health
care institutes to create the quarterly input. Detailed analysis of data processing in one mental health
organization showed that 75 data items needed to be reported each quarter, requiring up to 770 (!)
internal data processing steps of which around 14% are manual procedures (Stratum, 2005). Typical
missing data in the reports are the ‘main complaint’, ‘reason for referral’, and ‘first diagnosis’. The
number of institutes that managed to provide (incomplete) data sets reached a maximum in 2003 of
53% of the member organizations (Kerncijfers GGZ 2001-2003, p 22). In particular the relatively small
and specialized institutes failed to report. Incompleteness and inconsistency of the data, mainly as a
result of limitations in registration processes and software applications used in the institutes, made it
difficult to use the data for reporting on the national level (Westert and Verkleij, 2006). It remained
impossible to calculate performance of specialized mental health care (youth, forensic, drugs
dependency, and sheltered housing) on the national level without using additional data sources
(Heijnen and Faber, 2005).
Stage 2: emergence of IOIS DIS (since 2004)
After 2005, the number of institutes delivering data to Zorgis dropped. Reasons for institutes to stop
investing in participation were (1) New obligatory financial reporting rules by the Ministry of Health
caused changes in the coding standards of the institutes that were impossible to combine with the
Zorgis dataset. The choice for the mental health care organizations and the umbrella organization
GGZ-Nl was to adapt or to stop using the system and standards, (2) Non-response by the institutes
had no legal or financial consequences, (3) Added value of the system for the mental health care
organizations was very limited. The system allowed each institute to choose five mirror institutes for
comparison, but such comparisons failed because most institutes failed to report. Attempts to enrich
the data by adding data on patient satisfaction failed, (4) the OLAP environment on the web portal was
difficult to use and the institutes did not trust data quality. Only few institutes used the portal and
researchers looked elsewhere (in the old regional case registers) for data.
4.3
x Translation process DIS
Costs of health care are notoriously difficult to control for governments and insurance companies.
These actors see the need for a new billing method based on standardized care services instead of
payment per treatment activity or per bed, and aim for market oriented health care reforms. “Mental
care in particular is regarded as a sector that lacks transparency” (problematization). The initiating
actor in these reforms is the Dutch government, with the Ministry of Health as the executing
organization. Changes were forced by law (the new care insurance law (2004) and the law on
diagnosis related treatment in mental health care (2003). Care institutes accepted increasing
transparency as an important goal, but in fact had no choice but to follow the new rules. Despite
criticism on the developments institutes anticipated on the developments by playing an active role in
the initial stages of the project by becoming ‘first user organizations’ (interessement) (Custers, 2007).
The Ministry of Health started an internal project group to initiate a new data model including all
relevant data to analyze mental health care activities and detailed diagnosis codes (the so-called DSM
IV standard). Software providers for the mental health care sector were included in the project group
to redesign care information systems in order to enable institutes to report on the care process as
required by the new DBC law (enrollement).
Analysis of data provided by a trial group of mental health care institutes resulted in an initial care
product definition, based on diagnosis groups and amounts and types of care provided to a client
within a one year period. The ministry announced that participation would become mandatory for all
mental health care providers. The data set was presented as a potential source of information for
research (mobilization). It was agreed (mobilization) that (1) the DBC Information System (DIS) would
contain information on the volumes of delivered care (DBCs) and the costs of care, (2) DIS information
would be anonymous, meaning that data cannot be linked to individual patients, (3) DIS information
would be provided to public and private (like NZA and CVZ) organizations to fulfill their legal tasks
(www.minvws.nl). It was further agreed that the mental health institutes would provide data on a
monthly basis.
x Inscription process DIS
The translation process resulted in several immutable mobiles. First, the implementation of a minimal
data set and a format in which the institutes must report on their care activities per January 2006. Data
definitions include 10 main treatment categories (and 336 detailed categories), 19 main diagnosis
categories (DSM IV diagnosis codes replacing the ICD-9 codes), and professional categories
(personnel types). Second, monthly reporting by the mental care institutes would only take place for
finalized completed (segments of) treatments of one year maximum, in stead of reporting on separate
activities (as parts of chronic treatments). Third, a validation process to be used for data entry was
defined that would guarantee completeness of information.
Stage 3: emergence of IOIS DigiMV (since 2005)
The three immutable mobiles have lead to subsequent changes in the internal systems (electronic
patient files) of the mental health care organizations because of changes in personnel allocation,
organizational structures and financial procedures (the threat of no payments if data in DIS are
missing!). As expected, several institutes are not able to fulfill the new requirements due to a lack of
internal systems and procedures. Also, it remains unclear how to deal with health care that is not
covered by the diagnosis-treatment combinations system (e.g. ‘patient mentoring’ and ‘nursing’) and
treatment of long-term psychiatric patients. There is a chance (risk) that existing regional information
systems may fill this gap.
4.4
x Translation process DigiMV
Data on the type and quantity of care as provided by IOIS DIS reveals to some extent what happens in
care organizations, but does not report on the quality of care. Developing quality indicators for health
care has been an important but difficult topic since many years and indicators for the various actors
involved are difficult to define (Arah et al, 2006). In 2003, the Ministry of Health developed a
framework of health care performance indicators on a macro (national) level, specifying indicators for
structure, process, and outcome. In 2002, the Health Inspectorate (IGZ) investigated to what extent
care institutes are using quality indicators to assess their performance. These two actors framed the
quality assessment problem in the mental health sector (problematization).
In 2005, GGZ Nl and IGZ initiated a project group in which also insurers, client organizations and
associations of professionals were involved, with the aim to define a set of outcome/ quality indicators
(interessement). The project group published a set of 28 indicators in 2006, divided in three
categories: treatment effectiveness, safety and client orientation (enrollement). An implementation
plan was made, in which estimation was made on what indicators reasonably could be reported by the
health institutes. This resulted in a set of 35 questions, referring to 17 indicators, about the
performances of the year 2006. Nine of these questions have to be reported per diagnosis groups,
distinguishing between eight groups (mobilization).
x Inscription process DigiMV
Reporting of the indicators was made an integral part of the new yearly document in which the Ministry
of Health gives a national overview of health and care activities. A web portal and an IOIS called
DigiMV were provided by the Ministry of Health, in which all Dutch care organizations can publish their
report every year. Reporting is mandatory for all health care providers as from 2008, and the data are
made public on a web site (www.transparantezorg.nl). The performance indicators for mental health
care have been used by insurers in their negotiations on purchasing contracts with care providers. The
initiative strongly stimulates the measurement of patient satisfaction, effectiveness of treatment in
terms of reduction of complaints and quality of life.
THE SUCCESS OF IOIS DIGIMV PER 2007
Since 2006, processes of evaluations and fine tuning of the 28 mental health indicators were started in
which all actors (Table 2) are involved. All actors, including the care providers support the
development and use of performance indicators and regard the indicators as useful and necessary.
Some problems were identified: (1) data delivery in the IOIS proved significantly more difficult than
had been expected, (2) not all data needed was available in the information systems, (3) data could
not always be extracted easily from the information systems, (4) several questions were not applicable
for all care institutes, (5) data definitions were open to different interpretations, (6) the breakdown of
treatments and clients by diagnosis group proved to be difficult and time consuming. We now analyze
the success and possible failure factors of DigiMV, by using the DigiMV dataset per 2007.
5
According to ANT, the success of an IOIS may be regarded as the degree to which the IOIS
successfully imposes its’ structure on the other actors. We evaluate the success of DigiMV in 2007 by
assessing the degree (percentage) to which the organizations are able to answer 35 standardized
questions related to 17 performance indicators. Like the Ministry of Health, we count a question as
being answered if an answer has been entered in DigiMV, even if the answer is not correct.
(1) Organizations that could
Percentage of Questions answered Average (sd)
(max = 35)
55.8% (27)
71.1% (23)
51.7% (28)
38.3% (23)
Table 5 shows the success rates of all 64 organizations and per type of organization. Integrated
mental health organizations score significantly better than specialized institutions. Integrated
institutions score better since they answer on average 19% (seven) more questions than specialized
institutions. Note that statistical analysis of the basic dataset is done by comparing descriptive
statistics only, since the data represent almost the total population (more then 80% of all mental health
care organizations in the Netherlands participate in the national reporting system). In total population
scores all differences between descriptive statistics are significant (and some differences may be
relevant). Below we report on the relevant differences in our findings.
N
64
24
25
15
Success of external reporting for different actor types.
not be categorized as integrated or specialized.
Actor type (type of mental health care
organization)
All
Integrated mental health institutes
Specialized institutes
Others (1)
Table 5.
64.3% (15)
Type of Internal System
Patient
Patient
records (3
questionnaires
questions)
(12 questions)
51.1% (9)
Activity registration
systems (17
questions)
46.9%
Incident
registration
(1 question)
85.9% (9)
Quality management
systems (2
questions)
Success rate of external reporting per internal data source.
47.9% (17)
Mental health organizations use five different types of internal systems (Table 6) to answer all 35
questions. Three questions relate to data in patient health record systems (paper based and
electronic), 14 to patient questionnaires (that the institutes use once or twice per year), 17 to
(treatment) activity registration systems, one to incident registration systems, and two to quality
management systems. We found the highest success rate (85.9%) for questions related to quality
management systems for internal organizational processes.
Success rate
average (sd)
Table 6.
The interviews with managers in 12 mental health institutes focused on the difficulties in answering the
35 questions grouped according to the 17 performance indicators (each indicator is assessed by one
to four questions). Most difficulties (43) were reported on effectiveness questions: on average 3.9
difficulties per question. Fewest difficulties were reported on customer orientation questions (1.6
difficulties per question). Most difficulties reported relate to ‘question is not relevant for our
organization’ (0.5 difficulties reported per question), fewest difficulties relate to ‘interpreting the data
definitions’ provided by the digiMV application (0.19 difficulties reported per question). Integrated
institutions score well on all performance categories, specialized institutions don’t score well on
effectiveness questions.
These findings in 2007 have resulted in agreements (mobilization) to improve the success of external
reporting by (1) allowing mental health organizations to skip questions that do not apply to their
situation, (2) improving the specification of care effectiveness indicators, and (3) improving the quality
of internal systems and data in the care institutes. It is expected that (2) and (3) will improve
‘automatically’ after further implementation of IOIS-DIS. In 2008, the actors agreed (mobilization) that
IOIS success would be assessed by analyzing the validity of the questions and indicators.
DISCUSSION AND CONCLUSION
In summary, these findings show that in 2007 (10 years after starting the first national reporting IOIS
ZorgIS), the mental health institutes can answer only 55% of all questions on performance, with an
even lower percentage for ‘correct answers’. The low percentage is (partly) caused by the yearly
changes and updates of the questions and the dynamics of the inter-organizational systems, and data
and communication standards. The findings also show that all actors agree to further improve the
national IOIS reporting systems for quality (DigiMV) and quantities of care (DIS).
6
This study aims to understand the contextual dynamics involved in the implementation of nation wide
IOIS in health care. We have presented a case study of the development of a national IOIS for
reporting and monitoring of mental health care over a period of 10 years. The process started with the
implementation of the IOIS ZorgIS by only two actors (the mental health care institutes and their
umbrella organization). Ultimately, after seven years, this IOIS was replaced by two follow-up IOIS
initiatives DIS (developed by the Ministry and insurance companies) and DigiMV (developed by seven
actors). Our study confirms the findings of Lytinnen and Damsgaard (2001) that IOIS are built over
years by layering new systems on top of old ones and the findings by Rodon et al (2008) that IOIS
evolve over the years as result of interactions between actors.
The analysis reveals that the relationships between actors, the data standards, and the reporting
processes that had been established in the first IOIS (translation and inscription processes) play an
important role in the development of the two new IOISs. First, they form the essential basis and
starting points for the translation processes for the new IOISs, and may therefore be regarded as
success factors. Second, they hinder quick development of a new IOISs because of they form the
difficult to adapt installed base, which indicates that they may be regarded failure factors.
Our study indicates that the development of a complex inter-organizational monitoring system may
start with multiple systems (the predecessors of ZorgIS), followed by a single system (like ZorgIS), that
evolves into sets of multiple interrelated IOISs (DIS, DigiMV). Our findings also indicate that the IOIS
situation in 2008 is not stable. One factor that creates instability is that since the end of ZorgIS, GGZNl and the mental health care institutes have lost their data source for analysis and reporting of the
care delivered. Therefore, in 2008, a new IOIS “GGZ data warehouse” was designed (see Figure 1) as
successor of ZorgIS. The new IOIS does not require direct input from health care institutes, but data
are copied from DIS and DigiMV and used to generate new quantitative information products. These
include “mirror products”, reports in which information presented by institutes are compared with other
institutes, theme reports on waiting times, personnel and specific types of care (youth, long-stay
clients), generic reports on the whole sector of mental health care, and on demand analyses.
Another factor that creates instability for the IOIS in 2008 is the dynamic context of health care,
leading to new information and reporting needs. Insurance companies and mental health care
institutes started to develop ‘Care-Intensity-Packages’, well defined care products that fit specific
patient needs within the new diagnosis standards. Instability for the IOIS in 2008 is also triggered by
the recent development of Consumer Quality Indices (CQI): standards for assessing the quality of care
from the consumer perspective (Schmidt et al, 2005). In 2008, CQIs were available for five care
domains (care insurance, home care, eye treatments, hip and knee treatment, disability care) and
another 25 CQIs were being developed, each CQI creating new needs for reporting by different actors.
A third factor for instability is the frequent changes of the performance indicators: in 2009, six out of
the 28 indicators have been fundamentally changed.
We found indications for new IOISs to be added (after 2007) to the national monitoring and reporting
systems. Further improvement of the DigiMV IOIS and the health indicators has resulted in 2008 in the
implementation of a new data entry module (web site) for mental health care institutes. This new
module is intended to serve as a database for ‘approved reports’, being the uploaded, evaluated, and
improved reports, that are made available for the other actors.
So, in conclusion, this study provides two important lessons for research: (1) contextual studies of
health information systems implementation processes can make important contributions to explaining
the implementation outcomes and (2) ANT and quantitative analysis of outcome is a useful approach
to help understand, structure, and present complex processes of IOIS development. Our research has
implications for practice. Managers within the health industry are advised to take contextual dynamics
into account before engaging in new health information systems implementations.
References
Ackoff, RL (1971). Towards a system of Systems concepts. Management Science, 17(11), 661-71.
Arah OA, Westert GP, Hurst J, Klazinga N (2006): A conceptual framework for the OECD Health Care
Quality Indicators Project. Int. J. for Quality in Health Care. Sep 5–13
Barret S., Konsynski B.R. (1982): Inter-organizational information sharing systems, MISQ 93–105.
Bloomfield BP, Coombs R, Cooper DJ, Rea D (1992): Machines and manoevres: responsibility
accounting and the construction of hospital information systems. Acc, Man, and IT (2) 4: 197-219.
Cho S, Mathiassen L, Nilsson A (2008): Contextual dynamics during health information systems
implementation: an event based actor network approach. European J of IS (17) 614-630.
Clark T.H., H.G. Lee (2000): Performance, inter-dependence, and coordination in business to
business electronic commerce and supply chain management, IT and Management, 1 85-105.
CMMS (2007): Centers for medicare & medicaid services, Office of the Actuary, National Health
Statistics Group, National Health Accounts (www.cms.hhs.gov).
Custers T, Arah OA, Klazinga N (2007): Is there a business case for quality in The Netherlands? A
critical analysis of the recent reforms of the health care system. Health Policy (82) 2:226-39.
Donabedian A (1988): The quality of care. How can it be assessed? J of the American Medical
Association (260) 12: 1743-1748.
Giddens A. (1984): The constitution of society. Polity Press, Cambridge UK.
Ham M van, Luijsterburg J, Brouwers E (2007): Evaluatiestudie basisset prestatie-indicatoren. Tranzo,
Tilburg University, NL.
Hermann R.C, Leff H.S., Palmer R.H. et al (2000): Quality Measures for Mental Health Care: Results
from a National Inventory. Medical Care Research and Review, 57 (Suppl 2), 136-154
Holland CP (1995): Cooperative supply chain management: the impact of inter-organizational
systems. J of Strategic IS. 4 (2) 117-133
IGZ (2005): Basic set of performance indicators mental health care and addictions care (in Dutch),
Inspectorate for Health Care, The Hague (see also www.zichtbarezorg.nl).
Klein, S. (1996). The configuration of inter-organizational relations. EJIS, 5 (2) 92-102.
Law J (1992): Notes on the theory of the actor-network: ordering, strategy, and heterogeneity.
Systems Practice (5) 4: 379-393.
Mandi KD, Kohane IS (2008): Tectonic shifts in the health information economy. N. Engl. J. of Med
(358) 16: 1732-1737.
Markus ML, Steinfield CW, Wigand RT (2006): Industry-wide information systems standardization as
collective action: the case of the US residential mortgage industry. MISQ (30) 439-465.
OECD (2004): Towards High-Performing Health Systems (www.oecd.org).
Payton FC (2000): Lessons learned from three inter-organizational health care information systems.
Information and Management (37) 311-321
Pettigrew, A. M., Ferlie, E., and McKee, L. (1992): Shaping Strategic Change: The Case of the
National Health Service, Sage Publications, London (UK).
Provan KG, Fish A, Sydow J (2007): Inter-organizational networks at the network level: a review of the
empirical literature on whole networks. J of Management 33: 479-516
RIVM (2006): Costs of Diseases (in Dutch). National Institute for Public Health and the Environment,
Bilthoven, Netherlands. www.kostenvanziekten.nl.
Rodon J, Pastor JA, Sese F, Christiaanse E (2008): unraveling the dynamics of IOIS implementation:
an actor network study of an IOIS in the seaport of Barcelona. J of IT (23) 97-108
Sarker S, Sarker S, Sidorova A (2006): Understanding business process change failure: an actor
network perspective. J of Management IS (23) 1: 51-86.
Schmidt A, Power M, Bullinger M, Nosikov A (2005): The conceptual relationship between health
indicators and the Quality of Life. Clinical Psychology and Psychotherapy 12: 28-49.
Smith, C., Cowan, C., Heffler, S., and Catlin, A. (2006). National health spending in 2004: Recent
slowdown led by prescription drug spending. Health Affairs, 25 (1) 186–196.
Steinfield C., Markus M.L., Wigand R. (2005): Exploring Inter-Organizational Systems at the industry
level of analysis. Evidence from the US home mortgage industry. JIT 20 (4) 224-233.
Stratum J van (2005): Effectiveness and efficiency of information processing for IOIS in mental health
care. Master thesis, School of Economics and Business Administration. Tilburg University.
Subrami M (2004): How do suppliers benefit from IT use in supply chain relationships? MISQ (28) 1:
45-73.
Walsham, G (1995): Interpretive case study in IS research: nature and method. EJIS 4 (2) 74-81.
Walsham G, Sahay S (1999): GIS for district level administration in India: problems and opportunities.
MISQ (23) 1: 39-66.
Westert GP, Verkleij H (2006): Zorgbalans, de prestaties van de Nederlanse gezondheidszorg in
2004. Bohn Stafleu Van Loghum, Bilthoven. ISBN 90-3134-830-9
Wierdsma, AI, Sytema S, Van Os JJ, Mulder CL (2007): Hebben we psychiatrische casusregisters nog
wel nodig? Tijdschrift voor Psyciatrie (49) 8: 569-573
Yin RK (2003): Case study research design and methods. Sage Publications, Inc, Thousand Oaks, (Ca), USA.
Appendix A: overview of ANT concepts used by different authors
Concept
Actor (or actant)
Actor-network
G. Walsham and S. Sahay
Both human beings and nonhuman
actors such as technological
artifacts
Heterogeneous network of aligned
interests, including people,
organizations and standards
S. Cho, et al
Any material, that is, human
beings or nonhuman actors
J. Rodon et al
Related actors in a
heterogeneous network of
aligned interests
punctualization
Translation
How actors generate ordering
effects by negotiating or
maneuvering others' interest to
one's own with the aim to
mobilize support
Problematization
Interessement
Enrolment
Creating a body of allies, human
and non-human, through a process
of translating their interests to be
aligned with the actor-network
Mobilize support by creating a
body of allies through
translations
Mobilization
Obligatory
passage point
Delegates/
speakers
delegate or
representative
Process of alignment of the interests of a set
of actors with those of a focal actor
First moment of translation in which an actor
frames a problem or an opportunity and
attempts to persuade other actors in the
network that the problem/opportunity is worth
dedicating resources to its solution
Second moment of translation in which 'other
actors become interested in the solution
proposed. They change their affiliation to a
certain group in favour of the new
actor'(Callon, 1986: 211)
Third moment of translation that concerns 'the
group of multilateral negotiations, trials of
strength and tricks that accompany the
interessements and enable them to succeed'
(Callon, 1986: 211)
Last moment of translation that consists of
stabilizing the actor-network by making
durable and irreversible relations
Situation that is fixed during problematization,
in which any actor with a stake in the network
would have to pass through in order to attain
his objectives
Delegates are actors who "stand in
and speak for" particular
viewpoints that have been
inscribed in them, e.g. software as
frozen organizational discourse
-
Embodied translations into a
medium or material
Treating a heterogeneous network as an individual actor to
reduce network complexity (law, 2003)
The process of the alignment of the interests of a diverse
set of actors with the interests of the focal actor [callon,
walsham)
The first moment of translation, during which a focal actor
defines identities and interests of other actors that are
consistent with its own interests, and establishes itself as
an obligatory passage point (OPP), thus rendering itself
indispensable [Callon].
The second moment of translation, which involves
negotiating with actors to accept definition of the focal
actor [Callon]
The third moment of translation, wherein other actors in the
network accept (or get aligned to) interests defined for
them by the focal actor [Callon].
A situation that has to occur for all of the actors to be able
to achieve their interests, as defined by the focal actor
[Callon].
An actor that speaks on behalf of (or stands in for)
otheractors [Callon, Walsham&Sahay].
betrayal
(and) Inscription
Sarker eta al
any element which bends space around itself, makes other
elements dependent upon itself and translates their will
into the language of its own (Callon and Latour 1981)
Heterogeneous network of aligned interests, including
people, organizations and standards (walsham&S)
Process whereby translations of one's
interests get embodied into technical
artefacts. That is, the way physical artefacts
embody patterns of use
A situation where actors do not abide by the agreements
arising from the enrollment of their representatives
[Callon].
A process of creation of artifacts that would ensure the
protection of certain interests [Latour].
Irreversibility
Black box(ing)
Immutable
mobile
The degree to which it is
subsequently impossible to go
back to a point where alternatieve
possibilities exist
A frozen network element, often
with properties of irreversibility
The degree to which it is
subsequently impossible to go
back to a point where
alternative possibilities exist
A temporary abstraction of a
network that acts as a single
unit so that the network efface
into one actor
Network element with strong
properties of irreversibility and
effects that transcend time and
place, e.g. software standards
A materialized translation that
can be interpreted in
essentially the same way in a
variety of contexts
Concept that captures the accumulated
resistance of an actor-network against
change; irreversibility also reflects the strength
of inscriptions
Process whereby an 'assembly of disorderly
and unreliable allies is … slowly turned into
something that closely resembles an
organized whole. When such a cohesion is
obtained we at last have a black box' (Callon,
1986: 131)
Degree to which it is subsequently impossible to go back to
a point where alternative possibilities exist [Walsham &
Sahay)].