Monitoring and
UNIT Evaluating District
Health Management
5
Information
Systems
Welcome to the last Unit of the Using Information for Effective Management of Health Care Services
I module. Unit 5 introduces the means to monitor and evaluate, as well as plan, district
health management information systems. You will be offered the opportunity to examine
a framework based on public administration concepts, and see how this framework could
provide the necessary holistic basis for developing sound management information
systems.
There is only one Study Session in this Unit:
Study Session 1:
Monitoring and Evaluating District Health Management Information
Systems.
Learning Outcomes of Unit 5
By the end of Unit 5, you should be able to:






Describe the fundamental components of DHMIS.
Explain the components of the DHMIS as a support- or sub-system.
Explain the components of the DHMIS as a management tool.
Use the sub-system framework to identify and explain common problems in
DHMIS.
Develop an overall monitoring and evaluation framework for a DHMIS, applying
your understanding of the relationships between the components of the DHMIS.
Apply the framework to various uses.
When you complete this Unit, you should be able to tackle and complete your
assignments. Before you go through the Unit, take a look at the final Assignment and
note what you will be required to produce.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
187
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
188
Unit 5 – Session 1
Monitoring and Evaluating
District Health Management
Information Systems
Introduction
Much has been written on the development and evaluation of District Health
Management Information Systems (DHMIS) in developing countries that. Even so, we
are faced with several shortcomings in these descriptions.



These descriptions and evaluations are usually based on a particular conceptual
framework. However, there is no adequate, widely accepted conceptual framework
that guides such descriptions and/or evaluations.
The literature commonly refers to specific aspects that influence the functionality of
DHMIS, but little success has been achieved in describing a comprehensive
framework for the holistic evaluation of these systems. Functions within the data
handling process or Information Cycle, e.g. the development and use of data
collection tools, indicators, reports, etc, are described and evaluated individually.
There is no overall guiding framework and no mention of the minimum infrastructural
requirements necessary to accomplish the smooth operation of the data handling
process, and consequently enable the performance of the above activities.
Moreover, the social nature of information systems and organisational interplay and
tensions, while investigated and described in certain contexts, there is no indication
of the basic infrastructural requirements needed to have a functional DHMIS in which
such social phenomena can arise and be managed.
Given this context, in this session we shall propose as a framework for planning and
evaluating the DHMIS a minimum infrastructural package required to operate a DHMIS
successfully.
Session Contents
1
2
3
4
5
6
7
8
Learning outcomes of this session
Readings
The fundamental components of district health management information systems
DHMIS as a support- or sub-system
DHMIS as a management tool
Using the framework to identify and explain common problems of DHMIS
Integrating the components into a monitoring and evaluation framework
Potential uses of the framework
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
189
9
Session summary
Timing of the session
This session could take you up to two hours to complete.
1
LEARNING OUTCOMES OF THIS SESSION
By the end of this session, you should be able to:




Describe the fundamental components of DHMIS.
Explain the components of the DHMIS as a support- or sub-system.
Explain the components of the DHMIS as a management tool.
Use the sub-system framework to identify and explain common problems in
DHMIS.
Develop an overall monitoring and evaluation framework for a DHMIS,
applying your understanding of the relationships between the components of
the DHMIS.
Apply the framework to various uses.


2
READINGS
The readings for this session are listed below. You are expected to read all of the
readings provided.
Reading
Publication details
Heywood, A. &
Rohde, J.
(2002). Using Information for Action. A Manual for Health Workers at Facility
Level. Pretoria: Equity Project: 92 - 96.
Lippeveld, T,
Sauerborn, R. &
Bodart, C.
(2000). Design and Implementation of Health Information Systems. Geneva:
WHO: 15 - 32 & 243 - 252.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
190
3
FUNDAMENTAL COMPONENTS OF DISTRICT HEALTH
MANAGEMENT INFORMATION SYSTEMS
In its simplest form, an information system is sustained or maintained by two internal
forces. The diagram below describes these:
DHMIS as a support- or sub-system
Pull Effect
DHMIS as a management tool
Push Effect
DHMIS as s Management Tool
The information needs of managers in the pursuit of sound management in a rational
management context create a pull effect on the DHMIS to supply information as
required. We have seen how the quality and relevance of the information that managers
base their decisions on informs good practice in the management of health services.
Thus, managers may exert a demand on the DHMIS for the satisfaction of their
information needs.
For managers to exert this pull on the information system, two requirements must be
fulfilled:
 An understanding of, and familiarity with, the role and utility of information in
health care management, and
 The existence of an information system that can satisfy the arising and changing
needs of the managers.
If managers are not empowered to use information in managing the health services, they
are not in a position to realise and fulfil their information needs. To use a driving analogy
that illustrates the point: A person reared in an environment where cars do not exist
would not necessarily realise the utility of vehicles. Similarly, if managers were trained
and obtained their experience in an environment void of an information culture, they
would be hard-pressed to realise the utility of information to achieve their desired ends.
However, sufficient skills transfer and exposure may expedite the creation of the pull
effect under discussion. This possibly argues for the inclusion of health information
elements in the curriculum of health professionals and management training
programmes.
DHMIS as a Sub- or Support System of the Public Health Care System
The second requirement noted above relates to the existence of an information support
system (or sub-system of the public health care system) that would supply what is
required to satisfy the pull effects arising from users. The DHMIS should be managed
well enough to ensure that it is able to satisfy the tension placed on it as a sub-system.
In order to manage this sub-system, it needs to consist of the same elements that any
other system or sub-system in the public health care context consists of. The discipline
of public administration suggests what those key elements of a DHMIS support system
might be. According to Cloete (a public administration specialist), the six generic
functions of public administration are as follows:
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
191






Policy and policy-making
Organisation
Staffing
Work Procedures
Financing
Controlling
If the DHMIS were conceived as a sub-system in the health services system, we would
expect these components to apply to the DHMIS, as it is a component of public
administration. If these components are sound and in place in the DHMIS sub-system,
then the sub-system should be strong enough to support the tension of the pulling effect
placed on it.
4
DHMIS AS A SUPPORT- OR SUB-SYSTEM
Regarding the six generic functions of public administration as core elements of the
DHMIS support / sub-system, makes it easier to identify and explain some of the
breakdowns that we experience in the functionality of health management information
systems. Without engaging in a long explanation, the following brief descriptions and
examples of these elements are provided:
Policy and Policy-Making
Policy is the legal framework in which public services are provided. Also, and maybe
more importantly, it illustrates the political commitment shown by a government. If clear
and specific policies relating to the DHMIS are developed and endorsed, they provide a
legal framework in which the DHMIS will operate. Some examples include: good faith on
the part of information officers would not be the only standard for determining that
reporting units submit data on time and in the correct fashion, uncooperative reporting
units could be dealt with legally and officially, all the ensuing generic functions of public
administration as it relates to DHMIS would be governed, and so on.
Organisation
This relates to the placement of information producers and users within the organisation
structure of the organisation. This ensures unity in command and creates a feeling of
belonging and of existing in an official and sanctioned capacity. This helps clarify the
roles of individuals in the operation of the DHMIS. Regardless of the background of the
information officers, they are at times expected to analyse information from disciplinary
specialities, provide sensible and descriptive reports, and even to produce indicators for
specific PHC sub-systems. In addition, all managers go straight to the information
officers with ad hoc information requests from time to time, which lead to the overloading
of these officers. Usually each these managers also see their requests as bearing the
highest priority. Rational organisation would counter many of these problems.
Organisation also relates to rational data- and information flows.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
192
Staffing
No sub-system can operate effectively without competent people. These people have to
be formally appointed, have the correct qualifications and/or skills, have a manageable
workload, receive sufficient training and support, etc. For instance, would the
development of the DHMIS in South Africa have been more uniform if all information
officers had a health background? Is it fair to see that certain areas have information
officers responsible for 13 to 20 facilities each, whereas in other areas information
officers are responsible for 90 to 105 facilities without assistance?
Work Procedures
If staff employed to manage the DHMIS do not know what their responsibilities are, they
would not be able to perform optimally, and therefore the DHMIS would not be able to
function adequately. Responsibilities related to the data handling process must be
assigned to the correct staff levels and categories. For instance, should the use of
information and reporting not be spelled out in the job descriptions of all health workers?
If work procedures relating to the DHMIS were better communicated and documented,
situations would not arise where managers expect information officers (without adequate
knowledge and skills) to fully analyse and interpret information. In addition, gaps
between development of the DHMIS and use of information for management between
different geographic and administrative areas would probably reduce, as more guidance
would be available.
Finances
Financial resources and procedures are required for the operation of any sub-system.
The operation of the DHMIS is highly dependent on material- and skill-resources.
Without the necessary financial means to accomplish objectives, the sub-system would
surely be ineffective. For instance, if information officers do not have access to a budget
to maintain computerised systems, purchase or print data collection forms, provide
training and reference materials, fund quarterly information review meetings, etc, the
DHMIS will sooner or later run into problems. In South Africa, information officers
generally do not have access to an allocated budget for the DHMIS. This leads to many
problems in maintaining the computers used for the software, which has knock-on
effects on availability of information for use to better manager the service.
Controlling
Controls have to be built into any sub-system in the public health care context; this helps
ensure accountability, a necessary component of public service. Information plays a
large role in ensuring accountability in public officials (especially managers controlling
funds); the DHMIS sub-system must also possess some controls. The TALI-tool has
been implemented in South Africa to serve as a guide and, to some extent, a control
mechanism. This tool makes sure that the activities relating to the data handling process
are fully functional and thus, the information / management tool is accounted for by all
involved in the DHMIS sub-system. The question also arises, who watches the
watchers? Why not indicators for the DHMIS or some other continuous monitoring
system?
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
193
This discussion has hopefully strengthened the applicability of the framework in
monitoring the functionality of the DHMIS as a sub-system. It also bears mention that all
the generic functions affect one another. For instance, policy should be formulated on
staffing levels and qualifications, budgeting procedures and content, etc. Similarly, work
procedures should be specified for appointing staff, budgeting, controlling, etc.
5
DHMIS AS A MANAGEMENT TOOL
The DHMIS provides an important tool for management in the form of accurate and
relevant information. To enable the generation of this tool, the functions relating to the
data handling process or Information Cycle - data collection, data quality, data analysis,
information reporting, and information use - must be carefully considered and monitored.
Some pointers to guide this are briefly discussed below.
Data Collection
The first question of the Information Cycle asks: “What do we collect?” The simple
answer is essential, accurate and timely data required for the management of the health
system. A process of analysis and their potential use for action must form the main basis
for the inclusion of the data elements we collect in the Essential Dataset (EDS). The
indicators that determine these data elements should be selected on the basis of
objectively determined and confirmed priorities. There are many ways to develop such
an EDS – we discussed this briefly in Unit 1. Once these data elements to be included in
the EDS have been agreed upon, data collection tools must be developed and always
be available at the appropriate points of collection. These tools must have certain
characteristics such as being simple, exclude overlapping, be cost effective, integrated,
relevance in terms of its type, take little time to complete, and be clear. As little time as
possible should be needed to complete a data collection tool – this is a question of
careful and thoughtful design. They should be easily aggregated and promote the
collection of accurate data. In addition, collection points must remain individualised
during the aggregation procedure so that accuracy of raw data from facility level can be
checked.
Data Quality
The most important aspect of data quality is accuracy. Mechanisms and tools that
promote data accuracy are crucial. Appropriate manual and computerised data accuracy
checking procedures are part of these mechanisms. Work procedures, design of all
tools, and training can contribute towards ensuring data quality, but in addition, tools that
enhance data accuracy can be developed, for example minimum and maximum range
values for each data element for each facility, outlier analysis tools, internal consistency
checks, monthly reporting logs for the reporting unit, etc. Computerised accuracy checks
and manual checks must be performed on a frequent and continuous basis. Thus, the
quality, existence and use of tools to check accuracy, correct errors and prevent errors
from reoccurring must receive attention.
Ease of data transmission from point of collection facilitates data quality. Additionally,
mechanisms that ensure that this takes place on time and continuously, help.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
194
Data Analysis
It is crucial that accurate data are analysed to produce information in the form of
indicators. These indicators must conform to certain characteristics such as being useful,
relevant, appropriate, valid, easily calculated, clearly defined, sensitive and specific. It
must also be ensured that the analysis takes place at the point of collection and at all
higher levels receiving the information. Frequency of analysis as well as work
procedures and training must also receive consideration.
Information Reporting
Reports must be clear and fulfil a need for knowledge as it arises from time to time.
Adequate reports of the correct sort must reach all officials who need information. These
reports must be provided on time and cover needs throughout the administrative
hierarchy by integrating all relevant pieces of information irrespective of its source.
Moreover, the correct officers must produce the reports to ensure sound interpretation
with the correct amount of detail.
Information Use
Information use must be defined and included in the design of any DHMIS. The agreed
upon definition will determine the products of proof, for enabling continuous monitoring,
that must be built into the system. Such products of proof need to be unbiased and
verifiable.
6
USING THE FRAMEWORK TO IDENTIFY AND EXPLAIN COMMON
PROBLEMS OF DHMIS
In the following example, the framework helps to explain some of the fundamental
problems identified in DHMIS and illustrates how the framework can help prevent these
problems.
Policy
DHMIS as Support- or Sub-System
 Reporting units are not regulated by policy to supply routine
data on time and therefore cannot be sufficiently provoked
to comply to data collection and reporting procedures.
 No policy guides the amount of facilities that each
information officer is responsible for, therefore a skewed
workload distribution results between varying geographical
areas.
 Policy on targets/objectives do not exist, therefore no
national reference point against which to judge performance
of specific areas and sub-systems exists.

Organisation

Data flow is not regulated and planned to channel through a
central point in the district, therefore district-level managers
do not have access to all information.
Integration of information systems does not occur, and this
lack of integration results in the situation where managers
are not able to access information relevant to their own
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
195



Staffing




Work Procedures


Finances




Controls

needs from a different sub-system or discipline, e.g. TB has
no access to nutrition data.
Unity in command does not exist resulting in the information
unit being inundated with requests for information from
managers that all view their requests as the top priority.
This situation results because all managers are viewed as
the superior to the information officer.
The information officer has trouble getting priority for use of
official transport to engage in support and training of facility
staff due to the fact that the support services available to
the information office are not specified and prioritised.
Certain districts perform much better since they made the
decision to employ information officers with a health
background that can easily attach value to the numbers as
opposed to other districts that opted for employing
administrative staff.
Certain districts employ fewer staff leaving them with a
higher workload than neighbouring districts.
Some information officers take a long time to produce good
results since they do not have adequate education /training,
because this was not a prerequisite for employment to the
position.
Districts may have different levels of data quality, reporting
and report compilation frequencies since the work
procedures of information officers have not been specified
and standardised.
Information officers are abused by management and tasked
with unrelated duties such as preparing slideshow
presentations for managers since the information officers
have no endorsed scope of responsibilities.
Health workers and managers do not collect or use
information since their roles in the DHMIS have not been
specified in their job description.
The information officer cannot capture data since his/her
computer broke down and no money has been budgeted
by, and for, the information department for equipment
maintenance.
No money exists for the information office to conduct a
workshop for the revision of the EDS.
Facilities run out of data collection tools (e.g. registers)
since no money have been allocated for the purpose.
The performance of information officers is variable since
there is no performance appraisal system specified for the
position.
Development of the DHMIS is delayed since information
officers are not required to prepare strategic and
operational plans for the DHMIS.
The sustained improvement of the DHMIS is not ensured
since no mechanism for continuous monitoring of the
DHMIS system exists.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
196


Data Collection



Data Quality


Data Analysis




Information Reporting



Information Use

DHMIS as a Management Tool
Managers do not use information since the data elements
collected do not reflect their priorities.
Data quality suffers since the dataset is too inflated and
troublesome for facility level staff to afford the necessary
time to capture data, resorting to fabricating the data
instead.
No information on finances is collected due to poor EDS
development.
Facility level staff report poor quality information since no
accuracy checks are performed on the data and no
feedback is provided to the facilities.
Data quality is not addressed because accuracy checks are
not applied.
Monthly returns from data collection points are received late
since the information officer has no mechanism for tracking
the submission of data within reporting timeframes.
No analysis is performed since the data elements identified
for collection did not consider information needs and
therefore had no indicator definitions defined.
Analysis is limited in value since inappropriate indicators
were defined.
Analysis is pointless since no benchmarks/targets exist.
Reporting is limited since managers have not specified
information needs, as they are unfamiliar with the indicators
available.
Reports are not used for management purposes because
they are not clear and purposively created.
Reports are limited in value since managers task
information officers without sufficient skills to produce and
interpret the information.
Managers are not required to account for the performance
of the indicators relating to their areas of responsibility.
Strategic- and operational plans are not linked to indicators.
Thus, the indicators are not reported to superiors.
Health care staff and managers have not been trained on
the utility and use of information. Therefore, they do not
possess the skills to employ health information to their
advantage in improving the health services.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
197
7
INTEGRATING THE COMPONENTS INTO A MONITORING AND
EVALUATION FRAMEWORK
The following diagram illustrates the conceptual framework and mentions a further
component:
Pull Effect
Push Effect
DHMIS as a support- or subsystem
DHMIS as a management
tool
The social nature of DHMIS






Policy
Organisation
Staffing
Work Procedures
Finances
Controls





Data Collection
Data Accuracy
Data Analysis
Information
Reporting
Information Use
The components of DHMIS explained in this session all take place within a social
context. Hence, human behaviour, organisational culture and external forces all
influence the development of DHMIS. Although this unit is not intended to elaborate on
the social nature of DHMIS and external forces influencing its design and operation, it is
important to acknowledge its influence.
8
POTENTIAL USES OF THE FRAMEWORK
As stated previously, the purpose of developing the model is to provide structure to
fundamental requirements of DHMIS, particularly in developing country contexts, and to
ensure its sustainability. Therefore, the following proposed uses, in their capacity to
enhance the sustainability and functionality of DHMIS:









Policy development for DHMIS operation and management
Institutionalising the DHMIS effort
DHMIS development planning
DHMIS routine management
Structuring interventions for DHMIS strengthening
Assessing DHMIS
Performing comparative studies on DHMIS
Structuring DHMIS personnel performance appraisal systems
Monitoring the performance of DHMIS
What difficulties can you think of in implementing and using this framework?
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
198
9
SESSION SUMMARY
You have reached the final session of the module. As a conclusion, we provided you
with concepts and ideas that you can use to plan, monitor and evaluate your district
health management information system, in order to enhance its efficiency.
We showed you the dual nature of DHMIS – as a sub-system for the support of health
care services in general, and as a management tool. As a sub-system it can be viewed
and evaluated through the same lens as any other sub-system, with particular attention
on the core components that it shares with any other sub-system, such as transport,
finance, health promotion, human resources, etc. These components are policy,
organisation, staffing, finance, work procedures and control. If these components are not
clearly defined, it is hard to produce high quality information that can be used as a
management tool.
As a management tool, an information system must undergo and consider numerous
processed that relate to the Information Cycle. These are data collection, collation,
accuracy checks, analysis, reporting and use. All of these processes must function
effectively at all levels of the administrative hierarchy.
Being aware of all these fundamental components of district health information systems
can assist one greatly in strengthening existing systems, evaluating them and designing
new DHMIS.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
199
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
200
USING INFORMATION FOR EFFECTIVE MANAGEMENT OF HEALTH CARE
SERVICES I
EVALUATION FORM 2006 - SCHOOL OF PUBLIC HEALTH, UWC
Please would you give us some feedback of your experience of this module. This will
help us to improve the learning materials for the next cohort of students. Thank you.
1. How helpful was this module in guiding you through the topic of using information
for managing health care services?
2. How did you find the level of the readings?
Too difficult
Appropriately challenging
Too simple
Please explain your answer.
3. Did the readings add to your knowledge and understanding of the topic? Can you
provide specific examples.
4. Was the content of these units useful to you in your work situation? Please
expand on your response.
5. Can you suggest any improvements to the way the sessions were written or
presented?
6. Was the timing guideline given at the start of the session accurate or helpful?
Please indicate if you took longer or less time to complete the sessions.
7. Did the module equip you to do the assignments? Please explain any problems
you experienced.
SOPH, UWC, Master of Public Health: Using Information for Effective Management of Health Care Services – Unit
5
201