Namibia National HIV/AIDS M&E System
Final Report on the MEASURE Evaluation Capacity Assessment
Consultation and Strategic Planning Process
This report prepared by Anne LaFond, Lela Baughman and Dawne Walker for the
Ministry of Health and Social Services (MOHSS) of Namibia
Funding for the capacity consultation assessment and report was provided by the
President’s Emergency Fund for AIDS (PEPFAR)
January, 2007
Table of Contents
Acknowledgements……………………………………………………………………..3
Introduction……………………………………………………………………………..4
Background……………………………………………………………………………..4
Methods for the Capacity Consultation………………………………………………...5
Description of HIV/AIDS Program and M&E System in Namibia…………………….8
Results of the Capacity Consultation Assessment,
M&E System Performance and Capacity Gaps……………………………………………..12
.
Capacity Building Priorities and Recommendations ………………………………………44
Annexes…………………………………………………………………………………50
This report was made possible by support from the U.S. Agency for International
Development (USAID) under terms of Cooperative Agreement GPO-A-00-0300003-00. The author’s views expressed in this publication do not necessarily reflect
the views of USAID or the United States Government.
2
Acknowledgements
The MEASURE Evaluation team would like to thank the members of the Response Monitoring and
Evaluation (RM&E) subdivision of the Directorate of Special Programmes (DSP), Ministry of Health and
Social Services (MOHSS) in Namibia for hosting the assessment team that conducted the HIV/AIDS
M&E System Capacity Consultation in Namibia. Our gratitude is extended to Sandra Owoses, Chief
Health Programmes, RM&E and her team - Kareng Masupu, (Technical Advisor), Efraim Dumeni.
(Statistician), Jennifer Lisotto, (Data Manager), Anna Jonas, Dalleen Witbooi, and Mary Mahy
(UNAIDS Technical Advisor), and Michael Mulondo of Namibia Network of AIDS Service
Organizations (NANASO), and Chair of the National HIV/AIDS M&E Committee--for guiding and
supporting our work. We would also like to thank the PHC Directorate of the MOHSS, the M&E Unit of
the Project Management Unit (PMU), Global Fund for AIDS, TB and Malaria (GFATM), Office of the
Prime Minister (OPM), Ministry of Education (MOE), Ministry of Women’s Affairs and Child Welfare,
Ministry of Regional and Local Government and Housing, National Business Coalition on HIV/AIDS
(NABCOA), Khomas RACOC, Ohangwena RACOC, Engela CACOC Eenhana Hospital, Ondobe and
Engela Health Centers for providing background information and insights for this assessment
consultation. We are also grateful to the Centers for Disease Control and Prevention in Namibia for
technical guidance and logistical support and to Claire Dillavou (Strategic Information Liaison, US
Government) for her assistance in all matters great and small. The President’s Emergency Plan for AIDS
Relief provided the funding for this assessment.
3
1. Introduction
1.1
In response to a request from the MOHSS for technical support to assess the capacity of the
national HIV/AIDS Monitoring and Evaluation (M&E) system in Namibia, MEASURE Evaluation
initiated a capacity assessment consultation and strategic planning process with the Response Monitoring
and Evaluation (RM&E) Subdivision of the Directorate of Special Programmes (DSP) in Namibia. The
overall aim of this process is to develop recommendations for strengthening the HIV/AIDS M&E
system. This M&E capacity building assessment is intended to support the operationalization of the Plan
for National Multi-sectoral Monitoring and Evaluation of HIV/AIDS and serve as a guide to all system
stakeholders on strategies for achieving designated improvements in overall performance of the
HIV/AIDS M&E system. The assessment will define performance objectives for the M&E system,
focused interventions and actions to address current gaps in capacity, and the roles and responsibilities of
system stakeholders in capacity building. Eventually, the results of this assessment can be incorporated
into a capacity building plan for the HIV/AIDS M&E system including a priority intervention areas, a
timeline and resource needs.
1.2
This report presents the outcomes from both, the first phase of the capacity consultation: a
scoping mission to orient the consultation team to the HIV/AIDS M&E system and its stakeholders, and
to work with RM&E to develop the final Terms of Reference and the second phase, which included the
National Stakeholder Workshop to complete the Checklist for Assessing and Monitoring Capacity of the
HIV/AIDS M&E System. This work was undertaken over an eight-day period in October 2006 and a five
day period in December 2006. The report contains background to the capacity consultation, methods
used during the consultation, results of the capacity assessment , key strategic issues related to overall
system development, and recommendation for action. .
2. Background
2.1
MEASURE Evaluation is a technical assistance program funded USAID that works with national
governments to strengthen monitoring and evaluation (M&E) systems. Since 1997, MEASURE
Evaluation has worked around the world to strengthen the capacity of host-country programs to collect
and use population and health data, with special efforts in HIV/AIDS M&E system development. The
MEASURE Program has two approaches to providing technical assistance in M&E to national AIDS
control programs: 1) providing Resident Technical Advisors who reside in country and work with the
national AIDS control programs, National AIDS councils, USG and its partners on a day-to-day basis
(i.e., Rwanda, Haiti, Zimbabwe, Cote d’Ivoire, Tanzania, Kenya and Nigeria, and South Africa), and 2)
providing long-term technical assistance from US-based MEASURE Evaluation staff to countries such as
Ghana, Guinea, and Kenya. Technical assistance for national AIDS programs has included: development
of national HIV/AIDS M&E frameworks and plans; refinement of national level indicators; development
of national M&E operational manuals; improving performance of national Health Information Systems
(HIS); training of national and international M&E officers in data production, collection, synthesis and
use; national M&E system capacity assessments; human resource planning for M&E, and development of
tools for data collection and use. The program also provides state-of-the art training in HIV/AIDS M&E
and capacity building for HIV/AIDS M&E organizations and units and has developed several M&E
system assessment tools.
2.2
This capacity assessment consultation was conducted in collaboration with Macro International
Inc., which is funded by HHS/CDC Global AIDS Program to provide technical assistance and capacity
building for HIV/AIDS M&E in developing countries. Collaboration between MEASURE Evaluation
4
and Macro International, Inc. on HIV/AIDS M&E system capacity building is supported by funding from
the President’s Emergency Plan for AIDS Relief (PEPFAR). MEASURE works in close collaboration
global development partners and with technical assistance organizations such as CDC/GAP and ORC
Macro in providing training and technical assistance in M&E capacity building. An illustrative list of
MEASURE Evaluation and ORC Macro technical assistance activities is found in Annex 1.
2.3
In March 2006, the Directorate of Special Programs (DSP) Response Monitoring and Evaluation
Unit (RM&E) requested MEASURE Evaluation assistance in conducting an M&E system capacity
assessment. MEASURE Evaluation recommended the use of an approach to capacity building and
assessment outlined in Building National HIV/AIDS M&E Capacity: A Practical Guide for Planning,
Implementing, and Assessing Capacity Building of HIV/AIDS Monitoring and Evaluation Systems (draft
April 2006). The Guide advocates a participatory approach to capacity building using self assessment and
a stakeholder-led strategic planning process. Several months later the RM&E and MEASURE Evaluation
scheduled a two phase capacity consultation that would consist of an initial scoping visit in October 2006
followed by a national stakeholders system self assessment workshop in December 2006. The end
product of the capacity consultation is a report of the capacity assessment and consultation with a
framework for a plan for improving the capacity of the Namibian National HIV/AIDS M&E system.
2.4
Terms of Reference for the Capacity Consultation Assessment are found in Annex 2. These
Terms of Reference represent a broad program related to information that is needed to inform capacity
building activities for the M&E system in Namibia. MEASURE Evaluation will contribute to meeting
some of the objectives of the Terms of Reference in the first two field visits, and has been asked to
provide guidance on the remaining tasks in their final report.
3. Methods for the Capacity Consultation Assessment
3.1
Observations on M&E system performance and capacity reported below are based on a desk
review of documents related to the HIV/AIDS M&E system and its products including previous
assessments, the Global Fund Gap Analysis, the National Strategic Plan on HIV/AIDS, the National
Multi-sectoral Monitoring and Evaluation of HIV/AIDS, and others. For a complete list of documents
consulted see Annex 3.
3.2
The capacity consultation was guided by the document Building National HIV/AIDS M&E
Capacity: A Practical Guide for Planning, Implementing, and Assessing Capacity Building of HIV/AIDS
Monitoring and Evaluation Systems. The methods for assessing and planning for capacity building
advocated in the guide are based on a wide range of experience in applying organizational capacity
building and M&E technical assistance techniques found in the literature and shared among practitioners.
The first lesson relates to the advantages of locally – led capacity assessment and capacity-building
planning that engages local stakeholders in analysis and problem solving (as opposed to an externally-led
assessment and planning process). Such an approach grounds the results in the perspectives and
experience of the stakeholders who are the most familiar with M&E system operations and weaknesses,
and encourages local ownership of problem analysis and problem solving. The second lesson relates to
the interaction between technical and organizational capacity in the M&E system. The guide encourages
analysis of capacity gaps beyond the failings of individual technical knowledge and skills to consider the
organizational and system context in which M&E techniques are applied. The capacity building solutions
therefore often go beyond the typical tendency to simply train people in M&E and consider the structures,
operations, and motivation of the organizations in which M&E staff work.
3.3
The ultimate goal of the M&E capacity building assessment consultation is to establish and
sustain a functional HIV/AIDS M&E system under the framework of the Three Ones (UNAIDS, 2005).
5
As the RM&E begins to outline a strategy for strengthening the M&E system, it is necessary to define at
the outset the basic functions of the Namibian HIV/AIDS M&E system and expected levels of system
performance. Once these parameters are agreed and clearly stated they can be used to guide national
stakeholders and development partners in their capacity building efforts. Capacity building priorities
should be linked to improving stated system performance objectives, and the success of capacity-building
efforts should be assessed by monitoring changes in system performance over time.
3.4
The Guide proposes five performance objectives for a typical national HIV/AIDS M&E system.
If a national HIV/AIDS M&E system is performing well it should be able to carry out the following
functions or actions to an acceptable standard:
„
„
„
Develop a national HIV/AIDS M&E plan
Implement the national HIV/AIDS M&E plan
Coordinate the assessment and upgrading of standardized M&E capacity of stakeholders, national and
sub-national HIV/AIDS M&E organizations, and M&E personnel
„ Produce high-quality data on a regular and timely basis
„ Communicate HIV/AIDS information to relevant stakeholders and facilitate the use of information
among stakeholders
It refers to the following conceptual framework or capacity map (Figure 1) to illustrate the relationship
between specific aspects of M&E capacity and system performance.
3.5
Many elements (resources, steps, abilities, behaviors, attitudes and actions) contribute to
HIV/AIDS M&E system performance. The three capacity domains in the framework (surrounding the
performance objective in the center) refer to the resources, organizational structures/operations, and
organizational culture that come together to enable the HIV/AIDS M&E system to perform effectively.
Following are definitions for each capacity domain:
Capacity Domain Definitions
Resources
Financial, material, human, and technical inputs to the
HIV/AIDS M&E system
Organizational Structures/Operations
Infrastructure, systems, and defined tasks or operations of
organizations working in the HIV/AIDS M&E systems
Organizational Culture
The way in which organizations function and factors that
influence organizational behavior
3.5
In preparation for the capacity consultation the MEASURE Evaluation team shared the guide
with RM&E to provide background on the process as well as other possible assessment tools available to
the global HIV/AIDS M&E community (e.g., Global Fund M&E Self-Assessment Tool and the
6
MEASURE Evaluation Assessment of Data Use Constraints). For examples of tools and capacity
building technical resources available to support M&E system assessment and strengthening see Annex 4.
3.6
During the first visit, the MEASURE Evaluation team conducted meetings and interviews with
individuals and groups representing the different M&E system stakeholders in the public and private/civil
society sectors. Annex 5 contains a complete list of organizations and facilities contacted. Since the
groups were often large, the team employed a series of open-ended questions to elicit information on
M&E experience (data collection and use), capacity issues, and suggestions for addressing capacity gaps.
The team traveled to Ohangwena Region for two days to meet stakeholders and observe information
collection and use in public sector and Mission hospitals and health centers, community level service
providers, and to meet regional and community AIDS Committee leaders and members. For the field trip
to the Ohangwena Region, the MEASURE team used a key informant interview guide. The team also
collected examples of reporting forms.
M&E System Performance and Capacity Domains
Organizational
Environment
Organizational
Structures/Operations
Resources
•
•
•
•
•
•
M&E System
Performance
Mandate/authority
Leadership
Human resources (skills mix)
Finances
Technical knowledge and skills
Supplies/equipment
Organizational
Culture
•
•
•
•
•
•
Motivation
Decision-making approaches
Consensus building
Clear and agreed roles
Commitment to share and use information
Coordination, collaboration, and networking
External
Environment
Figure 1
7
•
•
•
•
•
•
•
•
•
•
Organizational structure
Planning processes
Finance and budgeting
Human resource planning and development
Reporting
Data and information systems
Communication
M&E processes
Links among units in M&E system
Links between M&E system and other units
3.7
At the end of the scoping mission, the team introduced the conceptual framework above at a
stakeholder meeting and proposed the next step of implementing a self assessment of the national
HIV/AIDS M&E system using the Checklist for Assessing and Monitoring Capacity of the HIV/AIDS
M&E System found in the guide. Following a brief introduction to the process, the group decided to use
the checklist at a national stakeholder workshop during the next MEASURE Evaluation visit in December
2006.
3.8
During the return visit in December, the MEASURE Evaluation team focused on collecting
additional information from key informants through interviews and meetings and by conducting the
National Stakeholders Workshop to complete the Checklist for Assessing and Monitoring Capacity of the
HIV/AIDS M&E System. Annex 5 contains a complete list of the organizations contacted during this
visit. Annex 6 contains a sample questionnaire for key informant interviews. Stakeholder meetings were
held with individual organizations and the team talked to organization representatives about the current
status of their data production, data flow and data use. The MEASURE Evaluation team developed a
short questionnaire to structure the meetings.
3.9
Workshop participants included key stakeholders with knowledge of the M&E system. A list of
attendees can be found in Annex 7. The workshop began by introducing participants to the HIV/AIDS
M&E System Conceptual Framework for Capacity and Performance (see the following section) that
includes performance objectives for a typical M&E system The MEASURE Evaluation team then guided
participants through the checklist using group discussion to assess capacity of the system in different
areas or domains. Reponses were recorded, analyzed and used in this final report. When responses varied
for different parts of the M&E system, they were recorded as such. Comments, examples and
explanations for most answers were recorded for future reference and to assist in repeated applications of
the checklist. Annex 8 contains a summary of the results from the Checklist administered at the National
Stakeholders’ Workshop.
4. Description of HIV/AIDS Program and M&E System in Namibia
4.1
In 2004, the Government of Namibia published its Third Five-Year National Strategic Plan on
HIV/AIDS for the National AIDS Coordination Programme (NACOP). Previously, in 2002, under the
Second Medium Term Plan, the Directorate of Special Services was established in the MOHSS. Its
purpose was to design, manage and direct policy development, strategic planning, resource mobilisation,
co-ordination, facilitation, monitoring and evaluation of the national response across all sectors to reduce
the impact of HIV/AIDS, TB and Malaria. The Directorate has two divisions, one focusing on the health
sector requirements and the other on the multi-sectoral response. Figure 2 depicts the organogram for
NACOP.
4.2
The coordinating body for the HIV/AIDS M&E System – The Response Monitoring and
Evaluation Subdivision (RM&E) - was established within DSP in 2005 with the mandate to coordinate
the implementation of a national and multi-sectoral M&E system in collaboration with other sectors and
development partners. The relationship of RM&E to national HIV/AIDS coordinating units and M&E
system stakeholders is shown in Figure 3. Detailed descriptions of each stakeholder is found in Annex 9.
The subdivision currently has four government staff and two technical advisors supported by
development partners. The RM&E is assisted by the national HIV/AIDS multi-sectoral M&E Committee
that was formed in 2003. When it was established it was chaired by the MOHSS. Since 2006 the
Committee has been chaired by the Director of NANASO. As the Secretariat for the M&E Committee,
RM&E is in the process of securing permanent members for the M&E committee who are able to meet on
a regular basis to provide technical assistance in coordinating the M&E system. In 2006, RM&E along
with members of the M&E committee developed the Plan for National Multi-sectoral Monitoring and
8
Evaluation of HIV/AIDS. This plan was launched in October 2006 and includes steps to institutionalize
mechanisms to monitor the progress of the Strategic Plan through a national HIV/AIDS M&E system.
4.3
According to the Plan for National Multi-sectoral Monitoring and Evaluation of HIV/AIDS, the
mission of the National HIV/AIDS M&E system is to effectively lead and coordinate the M&E efforts of
all stakeholders in the multi-sectoral HIV/AIDS epidemic by:
• Facilitating the implementation of the Third Medium Term Plan (MTPIII)
• Monitoring and evaluation of the multi-sectoral national response
• Communicating and advocating key issues of the national response to stakeholders
4.4
The plan focuses on establishing an effective coordinated National Multi-sectoral Monitoring and
Evaluation system for HIV/AIDS to:
•
•
•
•
•
•
•
•
•
Ensure evidence based policies, plans and programs;
Systematically collect and use data to track progress and for informed decision making on the
key interventions;
Assess the impact by monitoring trends and explain changes in the levels of HIV/AIDS
prevalence over time;
Define a list of core indicators that will enable tracking of progress in the most critical areas
of the fight against HIV/AIDS;
Develop a data collection strategy that will enable the measurement of the core indicators;
Establish clear data flow channels between the different stakeholders in the fight against
HIV/AIDS;
Develop a strategy and mechanisms to ensure a correct dissemination of all critical
information amongst all stakeholders, implementing agencies, beneficiaries and the general
public;
Clearly describe the role of each of the stakeholders in the monitoring and evaluation of
HIV/AIDS programs;
Develop a plan for strengthening the capacity of all partners involved in the monitoring and
evaluation of HIV/AIDS programs.
Regional AIDS Committees (RACOC) and sub-regional AIDS Committees (DACOC or CACOC) were
established to coordinate and guide the response to the epidemic in the regions.
Figure 2 Organogram for National Response for HIV/AIDS
9
10
5. Results of the Capacity Consultation Assessment
M&E System Performance and Capacity Gaps
5.1.
The national HIV/AIDS M&E system in Namibia is in the early stages of development. The
Government and other stakeholders are working to establish the structure and formalize the many
functions and relationships required to fulfill the system’s overall vision: “to utilize effective, wellcoordinated monitoring and evaluation in the guidance of the national response to HIV/AIDS that will
lead to reduced HIV infections by the year 2009.” In spite of its youth, the system has made important
steps to lay the foundation for a well functioning and responsive multi-sectoral HIV/AIDS M&E system.
A number of documents published recently report on the progress of the HIV/AIDS M&E system.
However, many also list a series of outstanding gaps in system capacity and performance. This capacity
consultation and assessment builds on these observations and attempts to move the discussion forward
toward analysis of the underlying causes of these weaknesses and possible solutions.
5.2
To assess HIV/AIDS M&E system capacity and its influence on system performance this
assessment inventoried essential components of the system as well as the extent to which these
components are functioning effectively. As defined by the conceptual framework in Section Three,
effective performance relies on the availability of critical resources, organizational structures and
operations, as well the underlying organizational culture of the system (its values, practices, and
motivation for M&E). This section draws on a document review, key informant and group interviews,
field visits, and the results of the system capacity checklist to present the overall findings of the system
capacity assessment consultation. It is structured according to the five key areas of performance of the
framework, noting levels of performance and gaps in capacity.
11
5.3
Performance Objective 1
5.3.1
Rationale
Develop a national M&E plan
The Three Ones includes a commitment to building one national HIV/AIDS M&E system, with the
development of a national M&E plan. This plan should represent national ambitions for the HIV/AIDS
M&E system. Its production and periodic revision should be the responsibility of a specific team.
Through the processes of planning, the assigned team and its stakeholders should establish the
framework, standardized indicators, data flow protocols, time lines, and other parameters of the M&E
system. The plan should also spell out the roles and responsibilities of different agencies and partners and,
as such, represent a commitment to a shared vision for M&E in the country.
5.3.2
Performance
There are a number of ways to assess M&E system performance in the national HIV/AIDS M&E
planning. The discussion focuses on a few key areas of performance that provide a basic
understanding of national M&E system planning in Namibia. The table below suggests simple
indicators for these aspects of performance that can be adapted and used to assess changes in
performance over time.
Performance in national HIV/AIDS M&E planning
ƒ Existence of the national HIV/AIDS M&E plan
ƒ National HIV/AIDS M&E plan linked to national
strategic HIV/AIDS plan
ƒ Contents of national HIV/AIDS M&E plan meets
international standards
ƒ National operational HIV/AIDS M&E plan or
work plan
ƒ Regular review and revision of national HIV/AIDS
M&E plan
Indicators
ƒ National HIV/AIDS M&E plan exists and is linked
to National Strategic HIV/AIDS Plan
ƒ National HIV/AIDS M&E plan addresses
objectives and milestones of national HIV/AIDS
Strategic plan
ƒ National HIV/AIDS M&E plan includes M&E
framework, standard and harmonized indicator
definitions, defined data sources, data flow
diagram.
ƒ National HIV/AIDS M&E plan includes an
operational plan that includes system wide
capacity building strategies and interventions and
costing.
ƒ National HIV/AIDS M&E plan and operational
plan is reviewed annually and revised as needed
The Third Medium-Term Plan on HIV/AIDS for 2004-2009 (MTPIII) was developed to guide the multisectoral response to the HIV/AIDS epidemic in Namibia, and serve as a management and coordination
tool. The Namibia Plan for National Multi-Sectoral Monitoring and Evaluation of HIV/AIDS was
developed and launched in September 2006 to provide a plan for monitoring and evaluating the overall
progress and effectiveness of the strategies outlined in MTPIII. The content of the Namibia Plan for
National Multi-Sectoral Monitoring and Evaluation of HIV/AIDS is linked to the National Strategic Plan
on HIV/AIDS and reflects the main pillars of an M&E system. It includes a framework, indicators,
defined data sources and data flow diagrams and identifies system stakeholders and their roles. However,
indicator definitions are not precise and thus open to interpretation. There is no national operational
HIV/AIDS M&E plan with project costs for implementing the national M&E plan. However, efforts are
underway to develop and cost and operational plan in early 2007.
5.3.3
Capacity Strengths and Gaps
12
The development of the M&E plan is an important accomplishment that provides a foundation for the
overall performance of the HIV/AIDS M&E system. The ability to develop the plan has been influenced
by a number of key capacity strengths that can be built on to spearhead implementation and guide efforts
to update the plan. There are also a few capacity gaps that should be addressed to ensure that the plan
remains relevant and is updated to reflect progress in its implementation. These strengths and gaps are
described below.
Resources
•
The mandate and authority for developing a national M&E plan are explicitly stated in the MTPIII.
The RM&E was specifically established as the coordinating authority for HIV/AIDS M&E planning
and had used its position to lead a consultative planning process leading to the launch of the plan.
•
Stakeholders report that there have been adequate public sector financial resources to support the
development of the plan; however, resources for focused HIV/AIDS M&E planning in other sectors
are not sufficient.
•
The plan was informed by international technical guidelines and efforts to examine the current
performance of the Namibian HIV/AIDS M&E system. A variety of international and national
documents were reviewed and used to determine priority areas, indicators, and reporting mechanisms
and RM&E used external technical assistance resources to guide the plan.
Organizational Structure and Operations
•
Procedures to develop the plan involved stakeholder consultation and review. Missing is reference
to information needs and analysis of the link between data production and use of information in
informing HIV/AIDS programming and policies. There is limited focus on information use and no
plan or timeframe for communicating information to decision makers.
•
The national HIV/AIDS M&E plan includes a framework for the M&E system, defines data sources
and relationships between system stakeholders. It also introduces new quarterly reporting forms to
facilitate national level monitoring of the response to HIV/AIDS and internal and external reporting.
It does not include a timeline, a work plan, or estimates of the cost of developing and operating the
HIV/AIDS M&E system. However, RM&E is working with UNAIDS and GAMET to construct a
costed operational plan. Each of the line ministries has been charged with developing their own M&E
plan; however, few will have completed them in time to construct the national Operational Plan.
•
The national HIV/AIDS M&E plan indicates a need for capacity building to establish the M&E
system in some areas and enable it function better in others. However, it does not contain overall
performance objectives for the system and strategies for building capacity.
•
Strategies for rolling out the plan include convening meetings of key sectors to present the plan
contents, presenting the plan at regional meetings, and providing training on the basics of M&E to
selected groups (e.g., the line ministries). However, steps for updating the national M&E plan
(currently covers the period of 2006/7 through 2008/9) are not yet scheduled.
•
The National HIV/AIDS M&E plan will be reviewed at the same time as the MTPIII review.
Organizational Culture
13
•
The plan was developed in collaboration with key public sector, private sector, and civil society
stakeholders. However, some organizations that are expected to provide data for national and program
reporting and decision making reported that they were not consulted sufficiently on indicator choice.
•
The Minister of Health has endorsed the National HIV/AIDS M&E plan signally high level
commitment to HIV/AIDS M&E.
•
There is strong support among stakeholders for the HIV/AIDS M&E plan and recognition that it
can be used to shape working relationships and system development. Stakeholders also recognized
need to strengthen the M&E system across stakeholder organizations.
•
The overall vision of the plan is the establishment of an interactive, multi-sectoral system. However,
development of the M&E system is still seen as a series of functions rather than an effort to work
collaboratively towards common areas of M&E performance and evidence-based decision making in
the national HIV/AIDS program.
5.4
Performance Objective 2
5.4.1
Rationale
Implement the National HIV/AIDS M&E Plan
The implementation of the national HIV/AIDS M&E plan is a critical function of the M&E system. The
various public sector, private sector, and civil society organizations that collect, analyze, report and use
HIV/AIDS data are responsible for ensuring that the objectives of the HIV/AIDS M&E plan are achieved
and the actions included in the M&E Operational Plan are carried out in the time period specified.
Coordination of M&E system stakeholders is key in this process to ensure that implementers focus on
shared and agreed objectives and that resources are used effectively and efficiently.
5.4.2
Performance
There are a number of ways to assess M&E system performance in implementation of the national
HIV/AIDS M&E plan. The discussion focuses on a few key areas of performance that provide a basic
understanding of the implementation of the national HIV/AIDS M&E plan in Namibia. The table
below suggests simple indicators for these key areas; the indicators can be adapted and used to assess
changes in performance over time.
14
Performance in implementing the national
HIV/AIDS M&E plan
ƒ Reaching annual objectives, milestones and targets
of the national HIV/AIDS M&E plan
ƒ Adequacy of funding for the national HIV/AIDS
M&E system
ƒ Norms, procedures and values related to data
collection and information use
ƒ Adequacy of stakeholder support
Indicators
ƒ Percent of annual objectives accomplished
ƒ Percept of annual milestones reached
ƒ Percent of annual targets met
(Per national HIV/AIDS M&E plan)
ƒ National budgetary and external resource
expenditure for the HIV/AIDS M&E system is
between 5-10% of the national HIV/AIDS
Program budget.
ƒ Key partners have budgets for M&E activities
ƒ Standards for data collection and information use
are increasingly applied to system activities
ƒ Leadership/coordinating body meets regularly
ƒ Coordination and collaboration across partners is
evident
Implementation of the Namibian national HIV/AIDS M&E plan is in the early stages. The RM&E has
written its own 12-month work plan and important work and efforts are underway to draft a 5-year
strategic plan and a 2- year Operational Plan for the system. The Operational Plan will focus on system
develop and ensuring ongoing operations. It will detail the actions and responsibilities of all stakeholder
groups and time frame for implementation and estimates of resources needs and commitments. In
addition, the RM&E subdivision is in the process of establishing stronger linkages with key stakeholders
to engage them system development and implementing the plan. Orientation and support from RM&E
will include introduction and training in use of the quarterly reporting forms for the health sector, nonhealth public sector, and private/civil society sector and development of guidelines for design of M&E
systems, data collection and reporting that includes standard indicator definitions, guidance on data flow
and transmission, and reporting timelines. Although stakeholders were involved in the development of the
plan, the written document has not yet been circulated and discussed with all stakeholder groups. Thus,
among internal stakeholders, commitment to developing one national HIV/AIDS M&E system varies
from strong to feeble, and work remains to enable and enthuse stakeholders to play and active role.
Finally, development partners continue to provide financial and technical resources to develop the system
and support operations. However, the adequacy of total funding (internal and external) for the system will
only be determined once the Operational Plan is costed.
5.4.3
Capacity Strengths and Gaps
The overall performance of the HIV/AIDS M&E system in implementing the M&E plan is influenced by
several key capacity strengths and a number of capacity gaps. Establishing the RM&E unit and assigning
permanent staff to the unit has helped establish a clear mandate and focal point for responsibility for
coordinating implementation efforts. Establishing a national M&E committee and identifying key
stakeholders and involving them in ongoing efforts to define implementation strategies are positive
accomplishments. Further, implementation efforts are supported with public sector and development
partner funding. However, there remain important gaps that limit the system’s ability to fully implement
the plan and meet its objectives. These gaps are reported by capacity domain below.
Resources
•
In the national HIV/AIDS M&E plan, the RM&E is clearly mandated to coordinate the HIV/AIDS
M&E system. In practice, however, this mandate is not fully supported with legal or policy
documentation or high level endorsement and support for its role. It appears that there is a mismatch
15
between the responsibilities assigned to the RM&E subdivision and the authority and resources
available to fulfill them.
•
The mandate to implement M&E activities is less clear for partner organizations. Stakeholders across
all sectors have expressed an interest in and willingness to implement the M&E plan; however, there
are no formal agreements or policies in place that solidify their commitment. A fundamental concern
expressed by multiple partners is the lack of leadership and support for HIV/AIDS M&E at the
highest levels of the government.
•
The national HIV/AIDS M&E plan also designates several organizations or units in the public and
private/civil society sectors to provide leadership and coordination for HIV/AIDS M&E activities
(the RM&E, the National HIV/AIDS M&E Committee, RACOCs, and umbrella organizations
(NABCO, NANASO and Lironga Eparu).
•
Human resource gaps exist both in terms of quantity of staff available and skills mix. These gaps in
human resources capacity are central to system performance and the ability of the stakeholders to
carry out the national plan. Basic and advanced skills in areas of information collection, analysis and
reporting are lacking. Details related to technical needs are discussed under Performance Objective 4
that relates to data production. There are also critical gaps in human resource availability for overall
management and coordination. For example, although numbers of permanent staff assigned to the
RM&E continues to grow (two new people were added between October and December 2006),
several key staff positions are held by contractors and funded by development partners which makes
the unit vulnerable in terms of sustainability. Further, RM&E lacks a full complement of staff and the
overall skill set needed to fulfill its role. The RM&E team identified the following human resource
needs: research and surveillance, evaluation, HIS, multi-sectoral liaison and skills building, program
monitoring, informatics, communication, and writing/editing. Addition investment in coordination
and management for M&E is also required in line ministries and umbrella organizations.
•
Public sector financial resources funding for HIV/AIDS M&E is generally considered sufficient.
Commitments to increased staffing levels are slow but under discussion. Funding from development
partners is available, but tends to be defined by partners based interests and is limited by funding
cycles. For example, PEPFAR funding is obligated on an annual basis, and GFATM funds on a twoyear cycle which poses challenges for planning. Development partner respondents expressed interest
in funding additional aspects of M&E system development once needs and costs were clear defined.
Funding for M&E outside the health sector is just beginning to take shape. Typically, other line
ministries have no dedicated budgetary resources for HIV/AIDS M&E. Further, respondents report
that supplies and equipment such as information technology, software, and office space are only
partly adequate to fulfill data collection and reporting expectations.
Organizational Structures and Operations
•
The Plan for National Multi-sectoral Monitoring and Evaluation of HIV/AIDS designates several
organizations or units in the public and private/civil society sectors to provide leadership and
coordination for HIV/AIDS M&E activities. They include the RM&E, the National HIV/AIDS M&E
Committee, RACOCs, and umbrella organizations (NABCO, NANASO and Lironga Eparu). These
organizations, particularly the RM&E, are central to overall M&E system performance and are well
positioned to serve as the engines that drive capacity development in the sector.
•
Although clearly named as the designated authority for coordinating and operationalizing HIV/AIDS
M&E, the organizational placement of the RM&E sub-division poses challenges for implementing
16
the actions outline in the national HIV/AIDS M&E plan. The first challenge is the placement of the
RM&E subdivision in the MOHSS rather than the Office of the Prime Minister or another part of
the government with jurisdiction over line ministries. In many other countries, the M&E unit for
the multi-sectoral response sits in the National AIDS Council and receives support for its
coordination role from its position of seniority. Because of its placement in the DSP, the RM&E it
is difficult for the RM&E to made demands of other line ministries without sufficient support
from senior managers. As one respondent explained “there is a hierarchy of ministries… DSP has no
authority to coordinate other ministries; OPM is the only one that can.” RM&E is also requires
additional support in fulfilling its coordination role with respect to health ministry stakeholders.
•
The coordination role of the RM&E covers technical, operational, and political aspects of M&E
system development and coordination. Expectations surrounding their work appear to be ambitious
given human resources gaps. The current team devotes many hours to coordination and management
of HIV/AIDS M&E system activities - both development and operations - but is drawn into a wide
variety of activities without a clearly stated set of essential priorities. Although there is a work plan
for the subdivision, the team described their day-to-day work as responsive rather than pro-active and
strategic.
•
The establishment and formalization of the National HIV/AIDS Committee is a positive step in
strengthening the capacity of the HIV/AIDS M&E system. Terms of reference for the committee
have been documented, and include its objectives, membership composition, and frequency of
meetings. In general, the committee’s function is to support and advise the RM&E subdivision. It is
currently chaired by the head of NANASO but the Chairmanship will rotate among stakeholders.
Membership is still not finalized and the process for securing permanent members seeks to identify
appropriate representation from each stakeholder group and may involve identifying additional
members to serve on special sub-committees. Since it is not yet clear whether the committee serves in
an advisory or decision-making capacity, the selection of permanent members has been delayed. The
Secretariat is seeking a forum to facilitate lines of communication between RM&E and system
stakeholders as well as technical and other advice on the work of the unit. At the same time, RM&E
would like the Committee members to advocate for HIV/AIDS M&E in their organizations and serve
as a catalyst for improvement M&E system performance.
•
Umbrella organizations such as NANASO, NABCOA and Lironga Eparu (the latter was not
contacted during the capacity building consultation) are also expected to coordinate stakeholders in
the HIV/AIDS M&E system. They will assist the RM&E in data collection and reporting,
standardization and in information exchange with their respective stakeholder groups. Like the other
coordinating groups, they require additional investment in human, technical and organizational
capacity related to HIV/AIDS M&E and management
•
The regional and local structures (RACOC and CACOC) are potentially key instruments for
improving M&E system performance. Both DSP and MGLGHRD are expected to coordinate the
regional response and the RACOC structures. RACOCs, with their Regional AIDS Coordinators, are
expected to channel information from the regions to the central level. However, RACOC require both
organizational and technical skills building and direction to even begin to play this role.
•
The National HIV/AIDS M&E Committee provides one forum for partners in the HIV/AIDS M&E
system to communicate and exchange information. There are few other routine mechanisms for
information exchange and joint programming among M&E system stakeholders. Lines, methods, and
timeliness of communications among M&E system stakeholders are perceived as problematic and ahhoc causing late or miscommunication and undermines learning and productivity. For example, the
M&E Unit of the Global Fund PMU and RM&E do not meeting regularly or share work experience.
17
The PMU has developed and implemented supervisory and training approaches for strengthening
M&E capacity of their grantee which would be relevant to the RM&E subdivision. Similarly, there is
very little coordination between the PHC Division, MGECW and MOE in relation to OVC activities
and reporting, which has implications for the quality of data collected and reported. Formal linkages
and coordination between DSP and other ministries is also lacking. OPM expressed interest in
learning from the experience of NABCOA and private sector stakeholders. Demand for better and
more routine communication went as far as the regions, where respondents felt that linkages between
the RACOCs and CACOCs required strengthening
Organizational Culture
•
Respondents reported that culturally the M&E system, and the bureaucracy in general, is more
inclined toward planning than implementation. Implementation is “messy” and requires
considerable efforts in coordination, particularly for a multi-sectoral response. Respondents reported
that a lot of the decisions about implementation of the plan are made centrally. Although the RM&E
is beginning to reach out and develop relationships with stakeholders, more concerted efforts are
needed. Respondents felt that stakeholders across all sectors should be more involved in the process
of developing guidelines and policies for implementation of the plan, particularly at regional and local
levels. Involvement promotes ownership of the decisions and guidance, and joins stakeholders in a
common purpose.
•
Contributing to performance in implementation is the proliferation of ad-hoc requests for information
and actions that undermine regular, day-to-day execution of work plans. In particular, respondents
described the M&E system as more responsive to external stakeholder needs than internal needs.
•
Lack of clarification of roles and responsibilities of all stakeholders and agreements on modes of
working and expected products affects the pace and direction of system development and the shortterm focus of many actors. Many M&E system stakeholders have not yet seen the final national
HIV/AIDS plan and may be reluctant to move forward until their roles are discussed and agreed.
5.5
Performance Objective 3
5.5.1
Rationale
Assess and upgrade standardized M&E capacity of
stakeholders, national HIV/AIDS M&E organizations,
and M&E personnel
An important aspect of performance is commitment to and investment in strengthening the capacity of the
different components of the HIV/AIDS M&E system. Within the national HIV/AIDS M&E Operational
Plan there should be specific objectives and actions devoted to assessing local system capacity, directing
resources toward capacity building, and monitoring changes in capacity over time.
5.5.2
Performance
There are a number of ways to assess M&E system performance in capacity building of the national
HIV/AIDS M&E system. The discussion below focuses on a few key areas of performance that
18
provide a basic understanding system level capacity building. The table suggests simple indicators in
these aspects of performance that can be adapted and used to assess changes in performance over time.
Performance in national HIV/AIDS M&E system
capacity building
ƒ National HIV/AIDS M&E system capacity
assessment
ƒ Performance objectives
ƒ Performance objectives inform planning and
budgeting
Indicators
ƒ National HIV/AIDS M&E system has conducted a
capacity assessment
ƒ National HIV/AIDS M&E system performance
objectives are defined
ƒ National HIV/AIDS M&E system performance
ƒ
ƒ Capacity building plan or priorities
ƒ
ƒ
ƒ National HIV/AIDS M&E work force plan
ƒ
ƒ
objectives are defined and inform national and
annual planning and budgeting
National M&E system performance is assessed
against objectives
National HIV/AIDS M&E system has a plan or set
of priorities for developing system capacity
Capacity building plan or priorities include both
human and organizational capacity building
Stakeholders developed a national HIV/AIDS
M&E workforce plan
Percentage of milestones or targets reached in
national HIV/AIDS M&E workforce plan
The national M&E plan acknowledges the need to develop capacity of the RM&E subdivision, national
and regional programs across ministries and sectors, and implementing partners such as NGOs, CBOs,
FBOs, and facilities. Multiple efforts have been undertaken to assess and describe capacity, including:
Progress Report on The Third Medium Term Plan on HIV/AIDS (rapid SWOT analysis); Namibian
HIV/AIDS Situation and Gap Analysis, prepared for the Round 6 GFATM proposal; Namibia Health
Information System Situation Analysis, conducted by the Health Metrics Network; and the Situational
Analysis of the Health Management Information Systems, conducted by the University of the Western
Cape. Some steps have been taken to address findings from the HIS and HMIS assessments (described in
more detail in the section on data production, Performance Objective 4). However, there is as yet no
overarching capacity building strategy for the M&E system. It is expected that this capacity assessment
consultation will contribute to the development a capacity building strategy and action plan that can be
included in the upcoming Operational Plan. In terms of overall system performance in directing the
process of capacity building, there is a need to focus on setting capacity building priorities, fully
documenting capacity gaps in the health and non-health sectors, and at the regional and local levels, and
addressing capacity issues beyond human resource capacity to include organizational capacity.
5.5.3
Capacity Strengths and Gaps
The overall performance of the HIV/AIDS M&E system in terms of providing oversight and giving
direction to system-wide capacity building and performance improvement is influenced by some key
capacity strengths and a number of gaps. To its credit, the system has conducted a number of assessments
to better understand gaps related to specific components of the system. There is an expressed
commitment to addressing capacity gaps, and initial efforts to identify training needs have begun.
However, there remain important gaps that limit the system’s ability to improve performance in capacity
building. These gaps are reported by capacity domain below.
Resources
19
•
The mandate for capacity building emanates from the M&E plan and is clearly assigned to the
RM&E. RM&E must now be supported and equipped to fulfill this mandate.
•
The UNAIDS/GAMET technical assistance efforts will assist in building capacity and system
strengthening. In addition, several partner organizations are demonstrating their support and
commitment to capacity building. NABCOA and NANASO have conducted or plan to conduct
assessments of their own capacity. In addition, OPM has carried out an assessment of line
ministries’ level of activity related to the MTP-III and preparedness for HIV/AIDS M&E.
•
Respondent reported that there is public sector funding available for capacity building activities in
health sector, and specifically for RM&E, but funding levels are not considered sufficient. Funding
for system-wide capacity building or coordination of capacity building is not specifically earmarked
in the budget.
•
The national HIV/AIDS M&E plan does not specify activities, responsible organizations, timelines,
or performance indicators to monitor capacity building efforts. Further, capacity building strategies
have been narrowly focused on training as the main strategy and don’t encompass strategies such as
organizational development, communications, and linkages. Although need to build technical
capacity is acknowledged in the plan; a focus on organizational capacity is absent. Stakeholders
acknowledged that capacity building needs have not been fully quantified or defined
•
The RM&E subdivision has made good use of technical assistance resources that focus on capacity
building assessment and technical skills building. For, RM&E attended the 2-week Measure
Evaluation HIV/AIDS M&E course South Africa, obtained USAID assistance for a workshop on
Epidemiologic Projections, Demographic Impact & Resource Allocation facilitated by the POLICY
Project, and is working with UNAIDS/GAMET in operational planning and system strengthening.
•
There is little or no capacity to provide training in technical M&E among local Namibian
organizations. However, regional organizations, such at the University of Pretoria, and various
development partners have trained health and non-health staff in basic M&E, data collection,
reporting and use including: PACT, GTZ, GFATM, ITECH and others.
Organizational Structures and Operations
•
RM&E has taken the initiative to focus resources on assessing capacity and performance of the
HIV/AIDS M&E system and taking steps to develop a systematic approach to capacity building.
Since needs are vast and varied, stakeholders must set priorities based on immediate as well as longer
term needs. Performance objectives are useful for guiding capacity building and system strengthening
and help avoid ad-hoc acceptance of assistance as it is offered.
•
As noted above, there is a large amount of documentation on the HIV/AIDS M&E system and its
capacity. However, it is not clear whether this information is being used to guide capacity building
planning or for further exploration of capacity building needs.
•
The assessment team was not able to identify a written plan for human resource development for
the HIV/AIDS M&E system, although checklist respondents reported that one is available. The PHC
Directorate, ART and PMTCT systems have trained staff in basic data collection skills, data entry,
analysis, reporting and quality checking. The CDC advisor and senior data clerks continue to support
learning through supervision, regular meetings, and video conferencing. They are planning aggressive
training schedule in 2007 to orient staff in the new ART data capture system.
20
•
The Training Support Coordination subdivision in DSP reported efforts to track human resource
capacity needs and develop a data base of training events and trainees. Strategies for meeting human
resource needs in HIV/AIDS M&E are not in place for non-health sector organizations and nor are
they in place for regional and local levels coordinating committees. There is a need to define the
scope and scale of human resources training needs and develop a system-wide strategy for meeting
those needs.
Organizational Culture
•
The RM&E subdivision and key partners are committed to addressing capacity gaps in order to
strengthen M&E system performance. Key partners are motivated to contribute to building and
sustaining a functioning M&E system, but often lack the resources, training, and technical know-how
to do so. As one respondent said “there is excitement about M&E … when M&E-related meetings
are held, there is good attendance.” National partners are beginning to develop a sense of shared
purpose for M&E data, but need continued guidance on how to develop capacity within their own
organizations and contribute to the capacity of the system as a whole.
5.6
Performance Objective 4
5.6.1
Rationale
Produce high quality data on a regular and
timely basis
Central to the performance of the HIV/AIDS M&E system is the production of data and information from
the various health information systems and data collection activities that comprise the HIV/AIDS M&E
system. The national HIV/AIDS M&E plan maps out the various components of the system and the type
of data and data sources required for decision making and reporting. Organizations or units designated as
data sources, data producers or manager are responsible for ensuring that each information sub-system is
functioning effectively, or data collection activity takes place, that the data that flow from these
systems arrive on time, are of adequate quality, and are stored and shared effectively.
5.6.2
Performance
There are a number of ways to assess M&E system performance in the production of data. The
discussion focuses on a few key areas of performance that provide a basic understanding of data
production in Namibia. The table below suggests simple indicators for these aspects of performance
that can be adapted and used to assess changes in performance over time.
21
Performance in production of data
ƒ Completeness and timeliness of
reporting
ƒ
ƒ Data quality
ƒ
ƒ
ƒ
ƒ
ƒ Use of standard indicator definitions
ƒ
and reporting forms
ƒ Data production meets the needs of
ƒ
internal and external stakeholders
Indicators
Percentage of quarterly reports that are
complete
Percentage of quarterly reports that arrive on
time
Percentage of reports meeting quality
standards
Percentage of service delivery sites passing
data audits
Percentage of reporting units with procedures
to prevent double counting
Percentage of routine reports and surveys
that use standard indicator definitions as
defined in the national HIV/AIDS M&E plan
and guidelines
Percentage of national level indicators
reported
The collection of selected data for informing the national HIV/AIDS response in Namibia has
proceeded at a rapid pace. Data capture for ART and PMTCT in the public and private sectors is
improving rapidly and there are several current efforts to upgrade, streamline and integrate different
sub-systems and improve the completeness of reporting. Sentinel surveillance reports are produced
every two years and other key reporting demands have been met. The health sector continues to
upgrade infrastructure and add and train staff in data collection, rolling out the system steadily to each
region and district.
Since the national HIV/AIDS program in Namibia involves all sectors and includes the work of public
sector, private sector and civil society organizations, there are a large number of data sources
designated for monitoring the national response to the epidemic. For background, Table X lists each
data source in the national HIV/AIDS M&E system and the type of data each entity is expected to
produce per the National HIV/AIDS M&E plan. It also notes whether the data producer has its own
M&E plan, M&E staff, and data tools, providing a limited insight into its readiness to produce data as
planned. It was not possible to conduct an in-depth review of the strengths and weaknesses of each
information sub-system or data source during this assessment. However, as noted in the table, the
assessment was able to begin an inventory of data sources and basic capacity for data production. This
inventory forms the basis of what can become a routine capacity assessment or self-assessment, and
should be repeated in the near future by the M&E committee on an annual basis. In general,
information systems in the health sector are further developed than those in non-health sectors or
organizations. Thus, both health sector and non-health sector data producers will be considered
separately in the discussion.
Currently, no data are collected routinely to assess the overall performance of the HIV/AIDS M&E
system and specifically the production of data. Considering the performance areas noted in the table
above, but without data with which to construct indicators, this section presents four different aspects
of performance related to production of data based on documents, interviews and observation in the
field.
5.6.2.1 Completeness and timeliness of reporting.
22
The health sector currently collects the majority of data available to monitor and evaluate the national
HIV/AIDS response. The type of health data produced or data collection activities that the HIV/AIDS
M&E system is expected to undertake includes:
ƒ
ƒ
ƒ
ƒ
Facility-based routine service statistics on HIV/AIDS health service delivery
Population and facility-based surveys
Epidemiological and behavioral surveillance
Development partner monitoring data related to health activities
The MOHSS has recently completed analysis of the 2005 sentinel surveillance of HIV/AIDS
prevalence among ante-natal patients and a national Demographic and Health Survey (DHS) is
underway. The MOHSS also collects and reports routinely facility-based data related to ART and
PMTCT services, data on commodities required for HIV/AIDS care, and laboratory test results. Data
on voluntary counseling and testing (VCT) is collected by both the MOHSS and partners from civil
society and Faith-based Organizations (FBO). A range of program activity data are reported from
various public sector and civil society organizations.
Current gaps in health sector data collection activities include surveys or epidemiological surveillance
of HIV prevalence; behavioral surveillance; routine facility surveys of HIV/AIDS service providers
(that assesses quality of care). There have been some special studies and the RM&E is defining a
research agenda. The launch of a newly upgraded the Electronic TB Register (ETR) is expected in
early 2007.
The type of non-clinical data or data collection activities that the HIV/AIDS M&E system is expected
to undertake includes:
ƒ Health program data from civil society and private sector organizations
ƒ Sectoral monitoring data
ƒ National workplace survey
Although performance varies among non-health sector data sources (line ministries, umbrella
organizations, and civil society organizations), a large percentage are not yet able to produce the data
required to calculate and report the indicators included in the Plan for National Multi-sectoral
Monitoring and Evaluation of HIV/AIDS. For example, the Ministry of Education produces the Annual
Education Census that will in the future gather the necessary data for national level reporting on
HIV/AIDS activities. They are introducing regional HIV/AIDS reporting system and placing regional
HIV/AIDS coordinators in the field. In contrast, the Ministry of Regional and Local Government,
Housing and Rural Development (MRLGHRD) produces no data on HIV/AIDS program activity or the
work of the RACOCs.
5.6.2.2 Timeliness
There are delays in the completion and transmission of routine monthly reports from public sector
facilities to central level offices where analysis and aggregation takes place. However, the
introduction of data clerks for PMTCT and ART data systems has improved the timeliness of
reporting from the field. Program and VCT reporting is particularly slow because it comes through the
national HIS system. Currently aggregation and analysis of these data takes place at the central level
and reports are then transferred to development partners and other divisions in MOHSS that require
information. Routine program reporting of health data from the private sector and civil society is
generally timely but can vary from organization to organization. Currently data transfer from nonhealth sources to RM&E is on an ad-hoc basis related to reporting needs, capacity and requests.
Routine reporting from these entities to RM&E will begin with the introduction of the new quarterly
reporting forms (contained in the national M&E plan). The Division of Epidemiology produces an
23
Annual HIS report. Both 2005 and 2006 reports are available. Responsibility for producing the
Annual HIV/AIDS M&E Report is unclear. No such report has been produced to date.
5.6.2.3 Data quality
A third and important aspect of performance relates to the quality of data produced. The introduction
of standardized indicators in the Plan for National Multi-sectoral Monitoring and Evaluation of
HIV/AIDS provides a basis for assessing some aspects of quality. However, to date there are no national
guidelines established against which to monitor the overall quality of data reported. Individual subsystems such as ART and PMTCT use PEPFAR funds to employ specialized data clerks to support data
collection, monitor quality and address problems with incomplete or inaccurate reports. Both the
laboratory information system and the PMIS report efforts to monitor data quality and improve it.
However, few other data sources, whether health or non-health, report routine quality assurance activities.
Specific examples of poor quality data include: lack of consistency (two documents that report different
figures for the same indicator are found in national reports), missing data, and double counting. Given the
nature of patient monitoring data, which is dominated by simple aggregated numbers of people receiving
treatment and counseling, in areas with multiple providers, there are many reported instances of double
counting patients. Several respondents cited VCT data relating to the number of people receiving VCT
services to illustrate how the same patient has been reported by more than one data source. Health staff at
two clinics in Ohagwena Region reported sending the same VCT report to the national HIS, the GFATM,
and the Red Cross. Data relating to orphans and vulnerable children (OVC) also lacks standardized
indicators and a coordinated procedure for avoiding double counting.
24
Preliminary Inventory of Producers of HIV/AIDS Data - Namibia
Collaborating Partner
PHC Directorate, Division of
Epidemiology, Ministry of
Health and Social Services
(MOHSS)
Data and Data Systems
At all levels of the public
sector health system
Indicators to Report
per Plan
HEALTH SECTOR
• Several program
monitoring
indicators
M&E
Plan
HIV/AIDS
M&E Staff
Tools
Yes,
national
plan
HIS Director
New DHIS platform for
upgrading Health
Information System will
incorporate ART and
PMTCT.
HIS Annual Report
PMTCT
•
•
HIS/VCT
•
•
Percentage of infants
born to HIV infected
mothers who are
infected
Percentage of HIVpositive pregnant
women receiving a
complete course of
ARV prophylaxis to
reduce the risk of
mother-to-child
transmission
Yes,
national
plan
Number of people
completing the
testing and
counseling process
(pretest, counseled,
tested and post-test
counseled)
Number of new
clients treated for
Yes,
national
plan
Several
information
system staff
**
PMTCT data reported to
HIS subdivision but
managed separately from
HIS data.
Paper-based reporting
forms
Paper-based VCT
reporting forms
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
M&E
Plan
HIV/AIDS
M&E Staff
Tools
sexually transmitted
infection
ART
•
•
•
•
Percentage of health
facilities with drugs
for ARV/OIs in
stock and no stock
outs in last 6 months
Percentage of
women and men
with advanced HIV
infection receiving
ART
Percentage of adults
and children with
HIV still alive at 6,
12 and 24 months
after initiation of
antiretroviral therapy
Number and
percentage of health
workers who receive
post-exposure
prophylaxis
DNA-PCR
Yes,
national
plan
CDC
technical
advisor
ART data collection and
management system is
independent of the HIS.
2 analysts
and 1 data
clerk
(centrally)
Paper-based reporting
forms
Introducing new data
capture form for patient
18 data clerks monitoring (IMAI).
in the field.
DNA-PCR database.
Lab
•
Percentage of
transfused blood
units screened for
HIV
TB Registry
•
TB treatment success
rate
Percentage of TB
•
26
Lab information system
(METECH) since 2002
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
M&E
Plan
HIV/AIDS
M&E Staff
Tools
patients tested for
HIV
Pharmacy information
management system
(PIMS)
•
•
Faith-based hospitals
(public-private partnership)
A computerized patient
management and monitoring
system (CompuCare) operating
in four private practices and in
two Catholic Health Services
hospitals.
ƒ PMTCT
ƒ ART
ƒ VCT
ƒ Clinical care and patient
monitoring
•
•
•
•
•
Percentage of
women and men
with advanced HIV
infection receiving
ART
Percentage of health
facilities with drugs
for ARV/OIs in
stock and no stock
outs in last 6 months
Percentage of adults
and children with
HIV still alive at 6,
12 and 24 months
after initiation of
antiretroviral therapy
Percentage of infants
born to HIV infected
mothers who are
infected
Number and
percentage of health
workers who receive
post-exposure
prophylaxis
Percentage of
women and men
with advanced HIV
infection receiving
ART
Number of people
27
Several
information
system staff
Paper-based reporting
forms as in public sector
system.
Clinicians
trained in
data entry
and data use
Patient management
system.
Introducing new data
capture form for patient
monitoring (IMAI).
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
•
•
•
•
Directorate of Special Programs
(DSP), MOHSS
Response, Monitoring and
Evaluation (RM&E)
Subdivision
Aggregates data from
different sources for
national level reporting
Management research and
evaluation agenda
•
M&E
Plan
HIV/AIDS
M&E Staff
Tools
completing the
testing and
counseling process
(pretest, counseled,
tested and post-test
counseled)
Number of new
clients treated for
sexually transmitted
infection
Percentage of HIVpositive pregnant
women receiving a
complete course of
ARV prophylaxis to
reduce the risk of
mother-to-child
transmission
Percentage of health
facilities with drugs
for ARV/OIs in
stock and no stock
outs in last 6 months
Several program
monitoring
indicators
HIV prevalence
among pregnant
women
Percentage of
population
expressing accepting
attitudes towards
PLWHA
28
Yes
Team of
M&E
officers and
Technical
Assistants
Joins in supportive
supervision visits to
regions
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
•
•
•
•
Percentage of people
reporting the
consistent use of a
condom during
sexual intercourse
with a non-regular
sexual partner
Percentage of young
women and men
aged 15-24 who both
correctly identify
ways of preventing
the sexual
transmission of HIV
and who reject major
misconceptions
about HIV
transmission
Percentage of
women and men who
had sex with more
than one partner in
the last 12 months
Percentage of young
women and men who
have had sex before
the age of 15
Percentage of
women and men who
reported using a
condom the last time
they had sex with a
non-marital, noncohabiting partner,
of those who have
had sex with such a
29
M&E
Plan
HIV/AIDS
M&E Staff
Tools
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
•
•
•
•
partner in the last 12
months
Number of
vulnerable
populations (sex
workers, mobile
populations, etc.)
who have ever
voluntarily requested
an HIV test, received
the test and received
the results
Percentage of health
facilities with drugs
for ARV/OIs in
stock and no stock
outs in last 6 months
Number of
vulnerable
populations (sex
workers, mobile
populations, etc.)
who report always
using a condom
every time they had
sex in the last month
Percentage of
orphans and
vulnerable children
whose households
receive free basic
external support in
caring for the child
Ratio of current
school attendance
among orphans to
30
M&E
Plan
HIV/AIDS
M&E Staff
Tools
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
Directorate of Special Programs
(DSP), MOHSS
Collects data on training
events
•
M&E
Plan
HIV/AIDS
M&E Staff
Tools
that of non-orphans
Several program
monitoring
indicators
Plans to develop a
training database.
Several program
monitoring
indicators
Training Support Coordination
Subdivision
M&E Unit of the Project
Management Unit of the Global
Fund for AIDS, TB and Malaria
(GFATM), MOHSS
Yes
Program data from grant
recipients.
Data aggregated at central
level for GFATM reporting
3 M&E staff Data collection forms
(director,
assistant, data M&E supervisory
system
clerk)
M&E guidelines
•
ITECH
Pre and post- training
evaluations and contact
information for trainees
•
Civil society (NGOs, FB0s,
CBOs)
Data collection capacity
varies widely according to
experience and size.
•
•
Number of health
personnel/others
trained to deliver
ART/PMTCT/VCT/
Rapid
testing/TB/HBC
services according to
national/international
standards
Several program
monitoring
indicators
Several program
monitoring
indicators
Number of people
completing the
testing and
31
Training data base
(TIMS) not yet linked to
public sector
Mixed
Mixed
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
M&E
Plan
HIV/AIDS
M&E Staff
Tools
counseling process
(pretest, counseled,
tested and post-test
counseled) (SMA)
•
Lironga Eparu
HIV/AIDS Unit, Office of the
Prime Minister (OPM)
Several program
monitoring
indicators
NON-HEALTH SECTOR
Basic preparedness for
• Percentage of line
HIV/AIDS M&E
ministries reaching
80% of criteria
measuring
mainstreaming (e.g.
policy, annual plans,
guidelines, budget,
management
committees with
HIV/AIDS on the
agenda)
3 M&E staff
General
HIV/AIDS
focal person
Electronic reporting tool
for ministries to
complete.
Conducted survey
among line ministries to
determine progress in
activities outlined in
MTP-III
Developing a human
resources database.
data validation revealed
extensive data quality
issues
The Ministry of Regional and
Local Government, Housing
and Rural Development
(MRLGHRD)
Conducts regional visits
and collects
information/data on use of
housing funding
Oversees work of the RACOCs
Not yet prepared to collect
or aggregate HIV/AIDS
•
Percentage of
national, regional
and sectoral
management
structures with
comprehensive
HIV/AIDS plans
32
No
No
1 Volunteer
to HIV/AIDS
M&E
Committee
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
M&E data
•
•
RACOCS
Some data flow to regional
level and to national level.
In other cases data flow
bypasses regional level.
•
•
CACOCS
Program level records
reported to CACOC Chair.
•
which are financed
annually
Percentage of line
ministries reaching
80% of criteria
measuring
mainstreaming (e.g.
policy, annual plans,
guidelines, budget,
management
committees with
HIV/AIDS on the
agenda)
Several program
monitoring
indicators
Percentage of
national, regional
and sectoral
management
structures with
comprehensive
HIV/AIDS plans
which are financed
annually
Several program
monitoring
indicators
Percentage of
national, regional
and sectoral
management
structures with
comprehensive
HIV/AIDS plans
which are financed
33
M&E
Plan
HIV/AIDS
M&E Staff
Generally no
dedicated
M&E staff
No
Tools
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
The Ministry of Gender
Equality and Child Welfare
(MGECW)
Numbers and location of
orphans and vulnerable
children currently receiving
benefits (but not specific to
HIV/AIDS vulnerability)
•
In process of developing
OVC M&E system.
•
annually
Several program
monitoring
indicators
Percentage of line
ministries reaching
80% of criteria
measuring
mainstreaming (e.g.
policy, annual plans,
guidelines, budget,
management
committees with
HIV/AIDS on the
agenda)
Several program
monitoring
indicators
M&E
Plan
Yes
(OVC)
No
Tools
Database on numbers of
and location of OVCs
Database on gender
violence
Yes
The National Planning
Commission Central Bureau of
Statistics (NPC/CBS)
The Ministry of Education
(MOE)
HIV/AIDS
M&E Staff
Annual Education Census
•
OVC
Teacher attrition and other
education system indicators
In process of building
regional reporting system.
Anticipates full reporting
capacity early in 2007.
•
•
Percentage of
schools with teachers
who have been
trained in life-skillsbased HIV/AIDS
education and taught
it during the last
academic year
Percentage of young
people taught lifeskills based
HIV/AIDS education
in past 12 months
Ratio of current
34
No
Yes
Establishing database
Placing
regional
HIV/AIDS
coordinators
in the field
Last educational census
released is from 2003
New survey tool being
finalized to include
National HIV/AIDS
Indicators
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
•
•
Namibia Network of AIDS
Service Organizations
(NANASO)
Aggregates data from some
civil society organizations
and reports to EU and
others on health sector and
HIV/AIDS program data.
•
•
Namibia Business Coalition on
AIDS (NABCOA)
Conducted email survey
among member companies
on workplace HIV/AIDS
policies and practices.
Response rate 10%
Following up with face-toface surveys to increase
•
•
M&E
Plan
HIV/AIDS
M&E Staff
Tools
school attendance
among orphans to
that of non-orphans
Percentage of line
ministries reaching
80% of criteria
measuring
mainstreaming (e.g.
policy, annual plans,
guidelines, budget,
management
committees with
HIV/AIDS on the
agenda)
Several program
monitoring
indicators
Number of people
completing the
testing and
counseling process
(pretest, counseled,
tested and post-test
counseled) (SMA)
Several program
monitoring
indicators
Percentage of large
enterprises/companie
s (including line
ministries) that have
HIV/AIDS
workplace policies
and programs
Percentage of
35
Computerized system
for tracking data from
all affiliated civil
society organizations.
No
Plans to hire
dedicated
M&E staff
person
Conducting
Survey tool
Collaborating Partner
Data and Data Systems
Indicators to Report
per Plan
response rate.
•
•
employees in
public/private sectors
that have been
reached by work
place programs in
the past 12 months
Percentage of line
ministries reaching
80% of criteria
measuring
mainstreaming (e.g.
policy, annual plans,
guidelines, budget,
management
committees with
HIV/AIDS on the
agenda)
Several program
monitoring
indicators
M&E
Plan
HIV/AIDS
M&E Staff
capacity
assessment of
the
organization.
** Note: Blank cells represent Missing Information
Source: The Plan for National Multi-sectoral Monitoring and Evaluation of HIV/AIDS, interviews, documentation
36
Tools
5.6.2.4. Data flow
Health sector data for HIV/AIDS flows along several parallel systems and communication among
these systems for use in program management, monitoring and evaluation is limited and often
problematic. ART and PMTCT data entry is made at district level and transferred electronically or by
floppy disk to central level. Analysis is completed by the data clerks with support from the CDC
technical advisor and reports are sent to program managers, RM&E and others. VCT and program data
flow through the HIS system. Program managers are expected to analyze and report these data, but it
is often done by HIS staff, which delays the transmission to other stakeholders. Private sector
hospitals also report ART, PMTCT and VCT data to the PHC directorate, using the same data capture
and reporting forms but a different electronic system for storage and patient management.
Pharmaceutical and lab data flow through separate systems. Hospital and clinic based staff receive
limited feedback on the data sent up the system. They are contacted mostly to rectify problems with
data quality. Among non-health data sources, data flows or is expected to flow along sectoral lines or
to be aggregated and reported through umbrella organizations such as NABCOA and NANASO.
Currently data are reported on an add-hoc basis, while plans to introduce the quarterly reporting
system are being rolled out.
Ultimately data should be readily accessible to support decision making and reporting inside and
outside the MOHSS. However, data sharing among different stakeholder groups within the health
sector is not functioning effectively. In particular, data flow from the PHC Directorate to the
Directorate of Special Programmes (DSP) is often delayed or difficult to secure. Although DSP is
responsible for national level HIV/AIDS M&E and reporting, it must make a formal request to the
Directorate of Primary Health Care Services to release routine data, causing unnecessary delays. A
respondent reported a two-week wait to release data to put in the M&E report because key staff were
away. Access to data is also hindered by the proliferation of databases for storing different types of
HIV/AIDS data and limited access to these databases or interface among them. As one respondent
reported, information systems are independent and can’t “talk to each other.” The impact of delays or
obstructed data flow is that stakeholder needs for information are not met in a timely fashion or not at
all.
5.6.3
Capacity strengths and gaps
The overall performance of the HIV/AIDS M&E system in terms of producing quality data is influenced
by both capacity strengths and a number of gaps in system capacity. In its favor, the system has made
considerable strides in establishing new processes for data collection in a very short time. This progress is
in part due to national commitment to respond to the epidemic through establishing the Directorate of
Special Programmes and a subdivision devoted to M&E - the RM&E. In addition, the system has
received broad-based support from development partners in the form of technical and financial assistance
to ensure that prevention and treatment programs are able to monitor and report on program outputs such
as numbers of people receiving care. There are, however, important gaps in capacity that pose threats to
improving performance in the production of quality data. These capacity gaps are reported by capacity
domain below.
Resources
•
The mandate and authority among stakeholders to serve as data sources for the national
HIV/AIDS M&E system is not formally stated or clearly understood, particularly among nonhealth sector stakeholders. There is no policy document, legislation or memoranda of
understanding that obligate all M&E system stakeholders to produce data to monitor the response
and make data readily available to RM&E for national reporting and for supporting national level
decision making. The lack of mandate affects the sense of urgency and seriousness with which
stakeholders view their role in the HIV/AIDS M&E system. Coordination of stakeholders by
RM&E is excessively challenging without a clear mandate and authority.
•
The lack of human resources with M&E technical skills dominates discussions related to
HIV/AIDS M&E system capacity building in Namibia. Both the quantity of technically trained
staff and the overall combination of skills available in the system are undermining overall system
performance and data production in particular. Capacity and capacity building needs vary widely
between health and non-health stakeholders, between public and private/civil society sector, and
at different levels of the system. In the health sector numbers and skills of staff are higher.
However, in the MOHSS, a large number of M&E staff are supported by PEPFAR and Global
Fund and are not yet integrated into the national health bureaucracy. Most line ministries,
NABCOA and NANASO have not yet appointed an HIV/AIDS focal person or an HIV/AIDS
M&E focal person, or have staff with relevant M&E experience. Among NGOs/CBOs, technical
capacity varies by size and type of organization. A meeting with NANASO members revealed a
strong demand for technical support in M&E and a request for peer exchanges on techniques and
experience. As noted above, specific documentation of all human resources needs for HIV/AIDS
M&E is required to plan for capacity building. Plans are already underway to monitor the overall
human resource skills set through establishing a national training database.
•
There is also a high demand for leadership and coordination skills for the HIV/AIDS M&E
system. As noted above, the National HIV/AIDS M&E Committee, RACOCs, umbrella
organizations, and RM&E can play a role in coordinating the development and operation of the
system and guide stakeholders to collaborate effectively. In addition, it is critical that existing
leadership in line ministries and other organizations support efforts in M&E system
strengthening, to raise the profile of HIV/AIDS M&E in their organizations. The role of Regional
AIDS Co-coordinating Committee (RACOC) and Constituency AIDS Coordinating Committee
(CACOC) with respect to M&E is currently only vaguely defined but important for streamlining
data production and promoting data quality in decentralized settings
•
Finally, there is a need for increased access to state-of-the-art technical resources to help
strengthen M&E capacity and performance among all stakeholders, but particularly in the nonhealth sector groups. These resources include training, documents, manual, guidelines, and
forums for peer exchange and establishment of a core of knowledge and experience in this area
throughout the system.
Organizational structures/operations
•
The organizational structures for the HIV/AIDS M&E system exist but their placement and
the way they interact can undermine data production. The first challenge was mentioned
above: the placement of the RM&E subdivision in the MOHSS rather than the Office of the
Prime Minister or another part of the government with jurisdiction over line ministries.
Because of its placement in the DSP, the RM&E requires additional support for data system
harmonization and general coordination of data collection and reporting. The second
challenge is the separation of RME from the HIS unit that produces VCT data and houses the
PMTCT and ART data collection and reporting systems. In spite of existing in the same ministry,
the two directorates cannot easily share essential data without following special procedures for
approval and release of data. While the two M&E teams focus on operating and strengthening the
same M&E system, they do not meet regularly, conduct joint planning or regularly review
38
progress of the health sector HIV/AIDS M&E sub-systems. Both RM&E and HIV/AIDS
subdivisions reported the need to improve communication and collaboration with each other.
•
In the past two to three year Namibia has established a number of parallel systems to collect
and report HIV/AIDS health service and program data. Many of them, such as the ART and
PMTCT service monitoring systems, were developed in close collaboration with development
partners (in this case PEPFAR) and were constructed to run parallel to the national health
information system, allowing the system to meet reporting requirement of major donors. The
GFATM has also introduced an information collection and reporting system that focuses on
indicators required to assess grantee performance. This system also runs in parallel to the
national ART and PMTCT systems and the Health Information System. Dissatisfaction with
these multiple systems relates to duplication of effort in terms of reporting, increasing the burden
on field level staff and in program supervision. Both the M&E unit of Global Fund PMU and the
National AIDS Program conduct supervisory visits to support M&E in the field, often focusing
on collection and reporting of the same date. Parallel systems have also led to delays in transfer of
information for national reporting, lack of control and management of the system at local as well
as national level, and poor communication of data for use at field level. Finally, M&E staff that
work in ART associate themselves with CDC rather than with the MOHSS because of the central
role that CDC has played in developing and funding the information system.
•
The DSP and other partners in the National HIV/AIDS Programme recently launched two
initiatives to begin to harmonize and upgrade data capture for HIV/AIDS including the
adaptation of a WHO-recommended IMAI patient monitoring system that facilitates patient
monitoring as well as reporting on service outputs; and the upgrading of the HIS software
platform based on a system developed in South Africa called the District health Information
System (DHIS). It is intended that the DHIS eventually incorporate HIV/AIDS data currently
collected and managed through other systems (ART, PMTCT, VCT, and DNA-PCR).
Discussions are also underway to link data from the ART, pharmacy, and laboratory information
system.
•
Decision making in the M&E system is currently centralized. The RACOCs, regional health
offices, and implementing organizations have little or no role in defining practical steps to
strengthen the M&E system at the lower levels and streamline information collection and use.
Data currently flows to the central level with little or no analysis and interpretation for use in
program planning in the field. It is difficult to instill an appreciation for the importance of
HIV/AIDS data for use in programming if local level implementers and managers are not brought
into data management decisions.
•
The process of annual planning and budgeting for the M&E system is not based on achieving
objectives over the course of the year or channeling resources into areas where there are clear
resource gaps. In the self assessment checklist, stakeholders reported that they plan
independently, and in some cases simply cut and paste from last year’s plan rather than using data
to determine need and to set realistic objectives. Planning that fails to coordinate the work of all
the stakeholders allows each stakeholder to focus on its own interests rather than contribute
resources and actions to improving the system as a whole. Moreover, planning without objectives,
work plans and timelines that reference overall performance goals of the M&E system
perpetuates the independence of one part of the system from the other and allows ad-hoc requests
or interventions to dominate day-to-day operations rather than being guided by an agreed set of
goals. Respondents described the system as more responsive to external stakeholders than
internal needs.
39
•
Mechanisms for data quality assurance are clearly lacking throughout the system. Guidance is
required in terms of indicator standardization, reporting forms, timelines, data quality standards,
and a minimal skill set for M&E staff at different levels.
Organizational culture
•
Three areas of organizational culture are influencing performance in data production in Namibia. The
first is the ability of the system to motivate individuals and organizations to work collaboratively
and effectively to produce high quality data. Currently M&E is not a priority among high level
officials and support for the work of RM&E and others in establishing new structures and capacities
is lacking. The production of high quality data often depends on how the system values data and its
use in improving the national response. Expectations related to sound data collection, analysis and
reporting practices should flow throughout the system and be evident in the expressed demand for
data and the incentives directed at data collectors and mangers to produce high quality data. At the
individual level, motivation for producing high quality data depends on factors such as confidents in
one’s knowledge and technical capacity, accountability for producing quality data; and recognition
for having done so.
•
The second aspect of organizational culture that affects data production in Namibia is the lack of
clearly articulated and agreed roles and responsibilities among stakeholders. Although the Plan for
National Multi-sectoral Monitoring and Evaluation of HIV/AIDS outlines expectations for data
production from all data sources/stakeholders, the plan has not been discussed and negotiated with all
parties. Even where there is tacit agreement that an organization will support the production of data,
the capacity of the organization to perform its role and its understanding of what data production
entails requires further discussion. In particular, the non-health data producers and umbrella
organizations would benefit from support in strategic planning and capacity building for fulfilling
their contribution to monitoring the HIV/AIDS response.
•
Willingness and experience in sharing information is the third and final capacity gap that affects
data production. Reluctance or delays in sharing data within the MOHSS has prevented the RM&E
from producing aggregated national reports and slowed planning. There is a need for simple
procedures for sharing preliminary as well as final analyses that instills a sense of confidence that
data will not be used or released without having met agreed criteria.
5.7
5.7.1
Performance Objective 5
Communicate HIV/AIDS information to relevant
stakeholders and facilitate the use of information among
stakeholders.
Rationale
The ultimate goal of the HIV/AIDS M&E system is to inform policy making, program management,
patient care, etc. Assessments of performance must go beyond the production of data and information to
the use of evidence to guide decisions. This area of M&E system performance focuses on the availability
of data (in terms of appropriateness, relevance, and accessibility to stakeholders) as well as its use among
decision makers at all levels that work in or with the HIV/AIDS program, within the public and private
sectors. Information enables rational resource allocation, informs policy and program decisions, guides
daily operations in clinical and public health facilities as well as community and workplace programs.
40
The use of information should be a high priority for stakeholders at all levels of the M&E system and
there should be a high level of endorsement to coordinate the effective use of the information. Also, as
data use become more prevalent among stakeholders, interest in ensuring the quality of the data will
increase.
5.7.2
Performance
There are a number of ways to assess M&E system performance in relation to the communication and use
of data. The discussion below focuses on a few key areas of performance that provide a basic
understanding of system performance in Namibia as it relates to communication and data use. The table
below suggests simple indicators that can assist with monitoring the performance of communication and
data use that can be adapted to assess changes in performance over time.
Performance in communication and use of data
ƒ HIV/AIDS program communication plan completed
ƒ HIV/AIDS program communication system
functioning
ƒ Stakeholders are informed routinely about M&E
results related to HIV/ADIS programs
ƒ Decisions related to the HIV/AIDS program are
Indicators
ƒ Communication plan completed, includes
target audience, time table and format of
information
ƒ Dissemination activities have begun
ƒ Deadlines are being met
ƒ Regular briefings take place for HIV/AIDS
program managers and policy makers using
data/information produced from the
HIV/AIDS M&E system
ƒ M&E findings are used to write reports,
increasingly evidence-based
grants and other proposals for funding,
assessment of information needs for policy
formation, resource allocation, program
improvement, community mobilization and
advocacy
Currently, there is limited use of the data being generated by the M&E system. At the national level data
are being compiled to produce reports such as the UNGASS and PEPFAR reports and the MTPIII
Review. The pharmacy compares the data they generate against what the facilities are reporting to assess
gaps in data. NABCOA hosts three meetings a year where its members come and share information about
the performance of their HIV/AIDS workplace programs. In the near future, the University of Namibia
will be partnering with John Hopkins University and MEASURE DHS to offer a certificate course in
Behavior Change Communication using DHS data. At the constituency level, some of the home based
care programs are using the information that volunteers collect to motivate the volunteers. Facilities at
the regional/constituency level use data for order supplies and drugs.
Even though data are being used in Namibia it is not being used strategically. There is no
communication/information dissemination plan in place for the National Response to HIV/AIDS nor has
there been a stakeholder analysis to see what type of information they need and when they need it.
Reports that are produced are not routinely disseminated or used for planning and policy development.
Currently, the M&E committee and the RACOCs are not being used to their full potential in regards to
routine information sharing between stakeholders and evidence based decision making.
41
5.7.3
Capacity Strengths and Gaps
Resources
•
Limited human resources in the various line ministries and umbrella groups involved in the multisector response to HIV/AIDS contribute to the lack of use of data for planning and decision making.
Outside of Health, most ministries do not have a dedicated M&E person let alone an M&E unit to
address the demands of the M&E system, i.e. data production, reporting and use. Current staff do not
have enough time or understanding of how to analyze the data they have, translate the data into a
usable format and disseminate the information to those who would need it to make decisions.
•
Funding for communication and information use within the National Response to HIV/AIDS has not
been secured, which is an inhibitor to implementing information use activities. Without funding
materials cannot be produced and disseminated to decision makers to aid in their planning and policy
making process.
•
Besides human and financial resources, it is believed that stakeholders at the regional level and below
lack the appropriate software to analyze the data and communication systems to disseminate the
information once analyzed. Even if they had the appropriate software, a significant portion of the
staff lack the computer skills necessary to interpret and present data.
Organizational Structure and Operations
•
There is little to no coordination and collaboration across sectors involved in the National Response
to HIV/AIDS, which limits communication of information among stakeholders and, therefore,
inhibits use. Currently, a lot of data are being collected by various stakeholders in the system that
could be shared among the stakeholders to enhance planning for the multi-sectoral HIV/AIDS
response, but due to the lack of coordination and collaboration the data are not shared widely across
the partners involved in the multi-sectoral response.
•
Even sectors that have access to the data, do not use the data to plan their programs or to inform
policies as the quality of the data is questionable due to the lack of quality checks in the system.
Often times, those who report up only hear back about their data if significant information is missing
or if there is a glaring error.
•
Data currently being generated by the M&E system is driven by donor and national reporting
requirements, i.e. the UNGASS report, the MTPIII report and the PEPFAR reports, not by the
information needs of the country to make decisions. Though there is a lot of information being
reported up to the national level, DSP/RM&E does not use the data to plan and set targets for the
coming year rather they cut and paste from the current years plan. The lower levels (regional and
constituency) within the system see data production as something they have to do to report to the
national level about their programs. In the two regions the MEASURE Evaluation team visited, there
was some limited use of the data they were producing in regards to tracking ARV treatment
defaulters, ordering medicines, motivating volunteers, tracking health progress and identifying needs
beyond health. An inhibitor to use of information by the regional and constituency levels is that the
national level does not provide them with any reports or analysis on the data that they submit.
•
There, also, seems to be internal confusion around who is mandated to generate reports for internal
consumption. RM&E feels it should be HIS as they house the data, but the Plan for National
Monitoring and Evaluation of HIV/AIDS assigns DSP to produce the reports.
42
Organizational Culture
•
The M&E system in Namibia does not have a culture of transparency with information. The
leadership controls access to the data that is being collected and will only allow the release of
information if the PS has signed off on a request. Even internally there is no mechanism in place to
share the information with National Response to HIV/AIDS stakeholders without written permission.
If the information is not easily accessible, use of the information by the various stakeholders will be
limited. Leadership would need to change their policies and make the data more readily available to
stakeholders for there to be a significant change in the decision making process.
6. Capacity Building Priorities and Recommendations
As noted above, there is a range of possible factors that influence HIV/AIDS M&E system performance.
Performance depends on the technical abilities of the individuals that work in the system but also on the
structure, abilities and culture in the organizational environment that supports basic technical work such
as survey design, M&E planning, data collection and analysis, reporting and use of data in decision
making. This section presents a series of capacity building priorities and recommendations for
consideration by the HIV/AIDS M&E system stakeholders in Namibia. It begins with general
considerations on HIV/AIDS M&E system strengthening followed by recommendations for building
capacity in specific areas of the HIV/AIDS M&E system.
6.1
Secure National Commitment to HIV/AIDS M&E
6.1.1 Improving HIV/AIDS M&E system performance requires national commitment to and
appreciation of the value of information for ensuring an effective national response to the HIV/AIDS
epidemic in Namibia. Information enables rational resource allocation, informs policy and program
decisions. It guides daily operations in clinical and public health as well as community and workplace
programs and programs to protect vulnerable groups. The production and use of information should be a
high priority at all levels of the M&E system and NACOP, from the data collector in a facility to the
managers and coordinators of the system. It requires local investment in strengthening the M&E system
and high level endorsement of coordinated multi-sectoral action to collect and use information effectively.
6.1.2 As NACOP works to develop the mechanics of the M&E system and improve technical
performance, concurrent actions are required to raise the profile of the role of information in the national
response to the epidemic and convince stakeholders to invest time and resources in information systems.
The leaders and managers must set the tone, using advocacy and communication that focuses on these
two objectives. Target audiences for these messages are both internal and external to the HIV/AIDS M&E
system.
6.1.3 In addition to public advocacy for M&E there is a need to address legislation related to reporting
and using HIV/AIDS data and introduce practical changes in the management of data collection and use
among stakeholders groups.
6.1.4
Examples of steps to prioritize M&E and support M&E system capacity building include:
ƒ Introduce legislation that mandates the collection and use of information by all stakeholders of the
HIV/AIDS M&E system;
ƒ Incorporate responsibilities related to information collection and use into job descriptions of
HIV/AIDS program staff and managers;
43
ƒ Introduce incentives for producing quality data on a timely basis and using data routinely;
ƒ At the Regional level, introduce incentives for regional level review of HIV/AIDS program data
and technical support and training for RACOC level HIV/AIDS focal persons;
ƒ Develop a national strategy for communication and advocacy related to HIV/AIDS M&E that
stresses the urgency for system development and showcases the products and outputs of the M&E
system.
6.2
Improving National Planning for HIV/AIDS M&E National M&E
6.2.1 The Plan for National Multi-sectoral Monitoring and Evaluation of HIV/AIDS provides the
framework for defining the M&E system and its goals. However, additional guidance is required to
establish system structures and functions as envisioned in the plan. This assessment therefore endorses the
decision to develop an Operational Plan to guide all stakeholders in system development and to facilitate
its routine functions over the next two years. The assessment team recommends that the Operational Plan
incorporate clear performance objectives for the National M&E system, indicators for performance, and
steps to monitor performance over time. These performance objectives should be endorsed by key
stakeholders and used to guide stakeholders toward a common purpose. The Operational Plan should also
define key priorities for investment in human and organizational capacity.
6.2.2 Operational planning often includes the identification of resource needs and gaps. This type of
information will increase the ability of the RM&E and the M&E Committee to set objectives and targets
for system development and enable them to plan strategically for using financial and technical resources
to build capacity and improve specific areas of system performance.
6.2.3 To raise the profile of HIV/AIDS M&E within the line ministries outside of health and strengthen
the national Operational Plan, an M&E capacity assessment should be conducted that results in a costed
Operational M&E plan for each line ministry. The assessment should look at resources (human and
financial), data production, data quality and data use. Once these plans have been developed, they should
be presented to senior managers and program managers within the ministries. These plans should be used
to advocate for a specified budget for M&E within the various line ministries. Where training and
technical assistance are outlined in the plan, the line ministry can begin to secure resources to implement
the trainings or provide technical assistance through their own contacts or through working with RM&E
to identify technical assistance providers.
6.2.4 RM&E and/or the M&E Committee should institutionalize an annual performance review of the
national Operational Plan to monitor progress made towards its objectives. This review should involve an
open discussion of progress and gaps among all system stakeholders and focus on problem solving and
identification of strategies for addressing gaps in capacity.
6.3
Leadership and Coordination for HIV/AIDS M&E Performance
6.3.1 The RM&E subdivision is central to the performance of the HIV/AIDS M&E system.
Consequently, there is an urgent need for RM&E to define its core areas of business, prioritize tasks,
and assign responsibilities clearly to different team members or groups within the team. Prioritizing
activities will allow the team to move from a reactive mode of operating to a strategic mode. Priorities
include: technical standardization for the HIV/AIDS M&E system; coordination and communication
among M&E system stakeholders; prioritization of data collection and data use activities according to a
needs assessment of information needs; and capacity building planning and oversight for the HIV/AIDS
M&E system. Assigned roles should be directly linked to established priorities and defined skill sets (e.g.,
multisectoral coordination, communications, administration, capacity building, etc.)
44
6.3.2 In this context, it is recommended that the HIV/AIDS M&E Committee and RM&E define a
separate capacity building plan for the RM&E subdivision including both technical (data production,
quality and use) and organizational (leadership, management, communication, and advocacy) capacitybuilding interventions. Actions to build capacity of RM&E should be incorporated into the Operational
Plan.
6.3.3 An important aspect of RM&E’s role requires it to develop strong relations with the many
stakeholders in the M&E system. As the prime coordinator of HIV/AIDS M&E system development and
operations, it must introduce mechanisms for routine communication with stakeholders to maintain
relations and manage ongoing activities. Since the unit is new within the public sector bureaucracy, it is
functioning without the benefit of long-standing stakeholder relations and routine mechanisms for
working collaboratively normally found within well-established government units. Thus, as the RM&E
gradually becomes established with staff and resources, it must at the same time develop and negotiate its
role with respect to other stakeholders and introduce new practices and ways of working into an already
over stretched public sector. In addition, as part of the multi-sectoral response to the HIV/AIDS epidemic,
the remit of the RM&E extends beyond the MOHSS where it is housed to oversee the work of other line
ministries and organizations in the public and private/civil society sectors. This role presents particular
challenges since the DSP must cultivate new working relationships with a range of public sector and
private/civil society partners that work outside the health sector.
6.3.4 RM&E should take the lead in establishing regular meetings with each of the key stakeholder.
RM&E staff, as relevant to their defined role, should also establish the practice of routinely checking in
with stakeholders to discuss activities, needs, and progress. It is recommended that RM&E hire or assign
a particular staff member to serve as a multi-sectoral liaison officer whose role is to coordinate the
multisectoral partners.
6.3.5 Line ministries and other stakeholders also look to RM&E for technical assistance on a range of
M&E activities. Thus, priority should be given to building the capacity of staff in the subdivision to
identify and meet the needs of a wide range of system stakeholders and to provide or secure appropriate
technical assistance. The RM&E team, with input from the M&E committee, should outline objectives
for of its own capacity building plan and emphasize the importance of developing and institutionalizing
basic coordination functions and skills in a short period of time.
6.3.6 Although its organizational placement in DSP has been identified a key challenge to fulfilling its
role in coordination of M&E system development and operations, respondents were not optimistic that
this situation would change. To help increase its ability to coordinate and lead despite its organizational
status, it is critical to identify “champions” for the HIV/AIDS M&E system and the RM&E to elevate
their visibility and importance among the high level management within the government and secure great
support for M&E activities. Three possible strategies include:
• Enhance and leverage OPM’s role in the M&E committee in terms of helping define membership
and encouraging other ministries to fulfill their own roles in supporting HIV/AIDS M&E.
• Establish a stronger relationship with the Under Secretary of Health who has been identified as a
potential proponent of HIV/AIDS M&E.
• Strengthen and formalize relationships with the NAEC.
6.3.7 To nurture these relationships it is critical that the RM&E team with the help of the M&E
committee demonstrate to management the benefits of using data produced by the HIV/AIDS M&E
system and its relevance to program implementation and impact, policy making, and external resource
generation. Producing sound M&E products that are relevant to internal stakeholders and tailored to their
specific needs is critical to the process of building support for HIV/AIDS M&E. Communication of
45
M&E system products can be facilitated through newsletters, presentations at technical meetings, email
and informal channels.
6.3.8 The national HIV/AIDS M&E Committee can also play a critical role in supporting HIV/AIDS
M&E system development. It is important therefore to work quickly to clarify and institutionalize the
M&E Committee’s role, membership and activities. At a minimum, it is recommended that the M&E
Committee provide oversight for M&E system development and priority setting for capacity building, and
serve as a forum for decision making and communication. If it is determined that the committee should
play a role in determining HIV/AIDS M&E policy, it is recommended that it establish an executive
committee or similar structure with appropriate level membership that can lobby at high levels. A
separate sub committee populated by M&E experts and focal persons should handle technical issues,
serve as technical forum to guide and advise M&E activities across ministries and among other
stakeholders. Each sub committee should have its own goals and work plan with responsibilities and
timeframes clearly articulated.
6.3.9 Similarly, the two main umbrella organizations – NABCOA and NANASO – should play a role
in facilitating system capacity building for HIV/AIDS M&E. These groups should work together with
RM&E to clearly define their role in data collection, reporting and use and in coordinating the
development of M&E capacity among their members. In the process, each organization may wish to
exam its current capacity to undertake the coordination role outlined in the National HIV/AIDS M&E
plan and develop strategies to build its own capacity both organizational and technical. NANASO is
currently undertaking an organizational assessment to guide future development of the organization and
enable it to meet the needs of its members and fulfill its coordination and leadership role.
6.4
Regional Leadership and Coordination
6.4.1 The capacity of regional HIV/AIDS Committees (RACOC) to support HIV/AIDS M&E is
currently very limited. Yet, they can potentially play an important role in strengthening data collection
and use for responding to the HIV/AIDS epidemic in rural areas. Challenges include: centralization of
resources for HIV/AIDS M&E and the fact that regional HIV/AIDS resources are not earmarked for
M&E. Moreover, M&E capacity building needs in the regions have not yet been defined. Finally, the line
ministry that oversees the work of the RACOCs (MRLGHRD) only has a volunteer committee that has no
resources to support the RACOCs in developing their own capacity and the capacity of other stakeholders
in the regions. It is recommended that RM&E work with MRLGHRD to conduct a capacity assessment in
one or two regions and work with key stakeholder to define, fund and implement basic human and
organizational capacity building activities in HIV/AIDS M&E. Lessons learned from this pilot case can
be translated into work in other regions.
6.5
Harmonizing information systems and M&E system development
6.5.1 Harmonization of information systems is an important concern of M&E system stakeholders. In
the early days of the HIV/AIDS program, independent information systems were established which now
function in parallel. Dissatisfaction with these multiple systems relates to duplication of effort, delays in
transfer of information for national reporting, lack of local control and management of the system, poor
communication of data for use at field level, and other concerns. In Namibia, steps are being taken to
upgrade poorly performing systems, link information systems through data bases and routine
communication channels, and share information more readily. In the health sector, exciting work is
underway to integrate existing data collection and management systems and bring ART and other data
collection into one national system. Support for these efforts should continue as they expand into every
region, bring together the public and private sector, and work to bring data production under one shared
framework and set of standards.
46
6.5.2 Harmonization has four purposes: to allow NACOP to develop a comprehensive picture of the
national response to the epidemic; to promote standardized measurement of program outputs, outcome
and impact; to facilitate access to different types of information for all stakeholders (avoiding information
hoarding or hiding); and to promote efficient use of resources (reducing expenditure on parallel systems
and time spent aggregating data from a variety of sources). Harmonization can become a struggle when
there are several functional information systems on which people have come to rely, and harmonizing
poses a threat to the quality and timeliness of data production.
6.5.3 Coordination and harmonization of information systems go hand in hand and include
establishment and enforcement of standard practices in indicator definition, data collection, reporting
formats, data flow, data storage, and use. The Plan for National Multi-sectoral Monitoring and Evaluation
of HIV/AIDS provides a framework for harmonization. To operationalize the plan and give credence to
the commitment to harmonization, stakeholders must adapt data collection to meet national reporting
requirements and timelines, and reduce the burden on the M&E system of collecting information that is
not readily applicable to program level decision making or policy questions. In turn the M&E system
must provide guidance in the form of guidelines, training standards, audit tools, and technical support to
aligning existing systems to the overall national plan.
6.5.4 Data storage, warehousing and communication systems also require greater attention than they
are currently given so that the data produced from these improved systems can be used and shared
effectively and efficiently with all stakeholders. The assessment team learned how “databases proliferate”
in the HIV/AIDS M&E system but that none of them are used effectively to inform decision making
beyond where they are immediately housed.
6.5.6 In the health sector, there is a need for greater internal coordination with respect to technical
aspects of HIV/AIDS M&E and data production. It is recommended that DSP and the Epidemiology
Division work together to provide leadership in this area by forming a technical HIV/AIDS M&E
Working Group. This group should focus on issues of policy and practice as the HIV/AIDS M&E system
continues to expand, and provide technical leadership for all other stakeholders. The pooling of technical
resource between these two divisions to address common challenges and define outstanding questions of
policy and procedure is potentially beneficial for both groups and for the HIV/AIDS M&E system as a
whole. In is also recommended that RM&E and PHC institute routine data quality audits for assessing
gaps in health data quality and provide guidance to all M&E system stakeholders on approaches to
improving practices in data collection, analysis and reporting.
6.6
Building technical skills in HIV/AIDS M&E
6.6.1 Addressing the gaps in technical capacity in HIV/AIDS M&E requires an approach that balances
the development of individual skills with strengthening the organizational context in which M&E takes
place. Newly trained staff must be supported by their organizations to transfer their new skills effectively
into sound practices in data collection and use. Training programs should also reinforce the paradigm in
M&E practices which emphasizes the use of M&E for program improvement and decision-making rather
than data collection for reporting purposes alone.
6.6.2 The DSP has at least two sub divisions that are responsible for technical capacity building:
RM&E and Training. There are also two main umbrella organizations – NABCOA and NANASO – that
can play a role in technical capacity building for HIV/AIDS M&E. These groups should work together to
establish national standards for M&E skills develop and capacity building approaches, including training.
They should focus on the following steps to support workforce planning for HIV/AIDS M&E:
47
ƒ Each M&E system stakeholder group should quantify the human resources currently assigned for
HIV/AIDS M&E and levels of training, and assess gaps in M&E human resources.
ƒ RM&E and DPS colleagues should define minimum training standards for M&E, establishing
minimum skills sets for different M&E and program management positions;
ƒ RM&E and the Training subdivision should make standardized training and M&E resource materials
on HIV/AIDS M&E available through electronic formats. There are myriad of technical materials and
training programs available in the region and from global sources.
ƒ RM&E and the HIV/AIDS M&E Committee should develop a medium-term plan for the development
and maintenance of human resources for HIV/AIDS M&E.
6.6.3 Capacity building is most effective when its objectives are locally determined and led. Technical
assistance for assisting capacity building efforts must be managed strategically, taking into account the
range of experience and resources available in the region and globally. There is also a vast amount of
resources available on HIV/AIDS M&E that RM&E should tap into to supplement their capacity building
efforts.
6.7
Special Capacity Building Efforts for Non-health Stakeholders
Outside of the health sector, there is an urgent need to communicate the contents of the national
HIV/AIDS M&E plan and provide technical guidelines for stakeholders in data collection, indicator
construction, the use of appropriate data collection and data quality assurance approaches. In many line
ministries there is little or no experience with the kind of standardized M&E normally found in the health
sector and many do not know where to start responding to the demand for routine data. It is recommended
that RM&E assign one staff member to assess needs among non-health sector stakeholders and more
importantly build strong working relationships with these groups to help them define capacity building
needs and identify resources for fulfilling their data production roles.
6.8
Communicate HIV/AIDS Information and facilitate Information Use
6.8.1 The use of HIV/AIDS M&E data and information is currently not a high priority in the Namibian
HIV/AIDS M&E system. The overriding need to produced data has taken precedence over building
capacity and expectations around informed decision making. Efforts are needed at all levels of the system
to link data production to data use rather than to external reporting.
6.8.2 To further develop the HIV/AIDS M&E system’s capacity for data use, it is critical to develop an
understanding of the decision making environment and the mechanisms and opportunities available for
using HIV/AIDS data at all levels. RM&E and the HIV/AIDS M&E Committee should map the
information needs of stakeholders, the types of decisions they make and the frequency and location of
national and local decision making forums and events. These forums include planning meetings, policy
reviews, budgeting sessions and technical meetings. These maps can be compared to actual flow of
information through the system to look for missed opportunities for linking data producers with data users
(See Annex 10).
6.8.3 Once the information needs of the stakeholders are determined, a communications/information
use plan should follow. This communication and information use plan should include the following
sections:
• Goals and Objectives
• Stakeholder identification and needs analysis. Do the stakeholders effect or are they affected by
the information generated in the HIV/AIDS M&E system? What are the stakeholders
48
•
•
•
goals/motivations, power and influence on the HIV/AIDS system, importance to the system and
role within the system? What information needs to be communicated to each stakeholder,
method of communication and desired outcome of the communication?
Communication/information use activities based on the stakeholder analysis
Performance measures for the communication/information use plan with a timeframe for review
Action plan with deadlines
6.8.4 At the highest level there is a need to inform NAC and parliament of the importance of
establishing policies that mandate data collection and data use for HIV/AIDS programming and introduce
practices that will empower stakeholders to use data. At an operational level, planning for M&E should be
led by the demands of stakeholders and their needs for information not simply for fulfilling reporting
requirements.
6.8.5 RM&E should also consider conducting workshops for MOHSS and other leaders that focus on
evidence based decision-making and the value of using HIV/AIDS data in different decision making
areas.
6.8.6 There is also a clear need for training in data use and dissemination for M&E system stakeholders
(national and regional) to help them start to view information use as an integral part of their jobs and
enable data use to become a routine practice. The training should include how to analyze, present and
communicate data to different target audiences as well as how to share information, use it to make
practical decisions, and to build consensus. The workshops should be divided into a workshop for
national level stakeholders and those for regional level stakeholders with curricula that highlight the
specific information use needs of the different levels.
6.8.7 The M&E Committee meetings provide an excellent for opportunity to share information with
M&E system stakeholders. As the M&E Committee will assist RM&E with development of national
M&E strategies, the committee should review information produced by the M&E system on a regular
basis to inform these strategies and their own organizations. Sharing of information and discussing its
relevance to HIV/AIDS program planning and policy making should be a regular agenda item for M&E
Committee meetings.
6.8.8 At the regional level, capacity building is needed to develop ways to channel data to the RACOC
and enable committee members to interpret its relevance to regional HIV/AIDS programming. RACOC
monthly meetings provide an ideal platform for regional stakeholders to share and review data being
produced at the regional level and to reflect on reports from the national level.
6.8.9 Within the MOHSS, communication channels are often slow or under developed and there is a
reluctance to share information. It is recommended that RM&E develop an intranet site for storing and
sharing HIV/AIDS M&E system data, reports, information pieces and updates. An internal newsletter
with summaries of quarterly reports could also be circulated to all stakeholders to demonstrate the outputs
of the HIV/AIDS M&E system.
49
Annex 1
Introduction to MEASURE Evaluation and ORC Macro and
Illustrative List of MEASURE Experience in HIV/AIDS M&E
Capacity building and System Strengthening
MEASURE Evaluation is a USAID-funded project implemented by the Carolina Population Center at
the University of North Carolina at Chapel Hill in partnership with Tulane University, ORC Macro
International, John Snow Inc., and Constella Futures. Together, we work in consultation with USAID
Mission staff, government and nongovernmental partners, cooperating agencies working in country,
USAID Bureau of Global Health (BGH) staff and other donors to ensure the availability and use of
quality population and health data.
Our technical advisers and researchers combine extensive field experience and knowledge and are
innovators trained in public health, organizational development, demography, epidemiology, economics,
statistics, medical anthropology and clinical health sciences.
Technical activities of MEASURE Evaluation include:
•
•
•
•
•
•
Consultation on national monitoring and evaluation (M&E) systems and routine health
information systems (RHIS)
Strategic information planning for host country institutions
Secondary evaluation analysis of existing data
Training courses and masters programs in M&E in collaborating universities worldwide
Development and testing of M&E methodologies to support local programs
Development of tools to facilitate utilization of data for policy development, planning and
program management
The ultimate objective of collecting and analyzing
data is to inform and improve health program
decision-making and, ultimately, global health
outcomes. MEASURE Evaluation fosters the
demand for innovations in effective monitoring and
evaluation of data. Our guidance empowers our
partners to improve family planning, maternal and
child health, nutrition and the prevention of
HIV/AIDS, STDs and other infectious diseases
worldwide.
MEASURE Evaluation has produced more than 250
publications relating to monitoring and evaluation,
family planning and rural health, HIV/AIDS and a number of other topics. All documents are in the
public domain. Free materials are available for downloading at the MEASURE website.
50
Illustrative List of MEASURE Experience in M&E Capacity Building and System Strengthening
M&E Capacity Building Workshops
•
•
•
•
•
•
•
•
M&E capacity building workshop for National AIDS Control Council in Kenya
M&E capacity building workshop for National AIDS and STD Control Programme in Kenya
Training-of-trainers for the Nigeria National Response Information Management System for state
and district level professionals
Data Demand and Use workshop for Ghana AIDS Commission focal persons
Regional workshops on M&E of HIV/AIDS programs
Regional workshop on routine health information systems performance and information use for
health systems
Strategic Information and Monitoring and Evaluation Field Officer Orientation for USG and
UNAIDS M&E Officers
Strategic Information and Monitoring and Evaluation Field Officer Regional Meetings in South
Africa and Tanzania for USG M&E Officers and national counterparts
In-country Technical Assistance
•
•
•
•
•
•
•
•
•
•
•
•
•
Provide Resident Technical Advisors to Rwanda, Haiti, Nigeria, the Caribbean, Cote d’Ivoire,
Zimbabwe, South Africa, Swaziland, Kenya and Tanzania who provide technical support to
the national AIDS councils and/or programs
Participated with the Caribbean Health Research Council on team visits to six countries to work
on national M&E plans
Developed a Data Demand and Utilization Plan for Dominica, St. Lucia and St. Vincent to
incorporate into the national M&E frameworks
Worked with National AIDS programs in Dominica and St. Lucia to develop decision calendars
to help them use data more efficiently and effectively for making program and policy decisions
Provided technical assistance to the Trinidad and Tobago NACC for development of the
national M&E framework and Operational Plan
Provided technical support to Haitian district health offices in the use of the Haiti Systeme
d’information Sanitaire to enter monthly health facility data
Conducted an M&E capacity assessment for the Jamaican Ministry of Health
Assisted the Jamaican Ministry of Health in restructuring the M&E Plan and developing the
Operational Plan
Conducted a strategic information assessment in Swaziland
Participated in the development of the Rwanda National HIV/AIDS policy document, revised
HIV/AIDS strategic framework and the revised National HIV/AIDS M&E Plan
Facilitated and M&E Summit in Kenya to develop the National HIV/AIDS M&E Framework
Collaborated with the Kenya NACC to develop national data reporting tools for use by civil
society groups working in non-facility HIV/AIDS activities
Worked with the Kenya NACC and National AIDS and STI Control Program to identify
information needs and develop a National HIV/AIDS information use guideline
51
•
•
•
•
•
•
Provided technical assistance to the Nigerian National Action Committee on AIDS to develop,
pilot-test and scale-up the Nigerian National Response Information Management System
Provide technical assistance to the Cote d’Ivoire Ministry of Fight against AIDS in the
development of an M&E Strategic Plan
Provided technical assistance to the Directorate of Information, Planning and Evaluation in Cote
d’Ivoire to strengthen the HIS and develop HIV/AIDS data collection forms
Provided technical support to the Ghana AIDS Commission to develop and finalize the National
M&E Framework
Worked with the National AIDS Council in Zimbabwe on their National M&E Strategic
Framework for their Global Fund submission
Providing technical assistance to the National AIDS Council in Zimbabwe to develop workplans,
timelines and TORs for the roll-out of the M&E system
Related Publications
•
Bertrand J T, Solis M. (2004) Evaluating HIV/AIDS Prevention Projects: A Manual for
Nongovernmental Organizations
•
Brown, Lisanne, LaFond, Anne and Macintyre, Kate. 2002. Measuring Capacity Building,
MEASURE Evaluation, Carolina Population Center, University of North Caroline at Chapel Hill.
•
Foreit K, Moreland S, LaFond A. (2006) A Conceptual Framework for Data Demand and
Information Use in the Health Sector
•
Foreit K, Moreland S, LaFond A. (2006) Strategies and Tools for Data Demand and
Information Use in the Health Sector
•
LaFond, A. and L. Brown. 2003. A guide to monitoring and evaluation of capacity building
interventions in the health sector in developing countries. MEASURE Evaluation Manual Series
No. 7. Carolina Population Center, University of North Carolina at Chapel Hill.
•
LaFond, Anne and Frankel, Nina. 2004. Handbook on Capacity Building in Monitoring and
Evaluation, MEASURE Evaluation, Chapel Hill, North Carolina.
•
LaFond, Anne and Brown, Lisanne. 2003. A Guide to Monitoring and Evaluation of CapacityBuilding Interventions in the Health Sector in Developing Countries. MEASURE Evaluation
Manual Series, No. 7. Carolina Population Center, University of North Carolina at Chapel Hill.
2003.
•
LaFond, Anne, Traore, Baba, Eckhard Kleinau, et. al, 2003. Using data to improve health
services: a self-evaluation approach, CERPOD/USAID, in press.
•
LaFond, Anne, Brown, Lisanne, and Macintyre, Kate. 2002. “Mapping Capacity in the Health
Sector,” IJHPM, vol 7, Issue 1.
•
MEASURE Evaluation, CDC-GAP/ORC Macro, April 2006. Building National HIV/AIDS M&E
Capacity: A Practical Guide for Planning, Implementing, and Assessing Capacity Building of
HIV/AIDS Monitoring and Evaluation Systems, draft
52
•
MEASURE Evaluation, GFATM, PEPFAR, HMN, World Bank, RBM, Stop TB, Program or
Project-related M&E System Self-assessment Tool: Three Checklists for M&E System Selfassessment
•
Rodriguez M, Spohr M, Lippeveld T, Edwards M. (2005) Informatics Technology for Use in
HIV/AIDS Treatment in Resource-Poor Settings
•
UNAIDS, 2000. National AIDS Programmes: A Guide to Monitoring and Evaluation.
•
UNICEF. (2005) Guide to the Monitoring and Evaluation of the National Response for Children
Orphaned and Made Vulnerable by HIV/AIDS
Illustrative List of Technical Assistance in HIV/AIDS M&E conducted by ORC Macro
In addition to ORC Macro’s role on MEASURE Evaluation, the Atlanta office has conducted a
range of technical activities related to capacity building in HIV/AIDS monitoring and evaluation
under contract with the Centers for Disease Control and Prevention (CDC), Global AIDS
Program (GAP). Key activities include:
•
Development and delivery of a series of six skills-based trainings in M&E. A pilot and a
subsequent training were delivered to GAP headquarters staff in Atlanta, and four regional
trainings were held in Capetown, South Africa; Bangkok, Thailand; and Tobago for the
Caribbean region. Training sessions were provided primarily for GAP field office staff from
all 25 GAP countries and 3 regional offices, and were in response to the general need for data
and information from HIV/AIDS-related programs being implemented in developing
countries worldwide.
•
Conducting an M&E capacity assessment in Botswana to define and examine M&E capacity
and determine capacity building needs of Botswana. This involved identification of country
resources available for M&E activities, examination of capacity targets, identification of
country technical assistance needs for successful M&E capacity implementation, and
development of an action plan to secure technical assistance and build M&E capacity. This
work resulted in the following publication: ORC Macro. August 2002. Botswana Global AIDS
Program Monitoring and Evaluation Needs and Capacity Assessment Report. Funded by the Centers
for Disease Control and Prevention.
•
Providing support in developing capacity for strategic information (SI) and monitoring and
evaluation (M&E) in GAP-supported field offices and Emergency Plan focus countries,
including:
•
Developing, piloting, and conducting country-specific training. Training included
such topics as M&E 101; qualitative and quantitative methods; data utilization by
multiple stakeholders; data triangulation approaches; and use of M&E for program
management and program improvement. ORC Macro developed training materials,
including facilitator manuals with detailed instructions, participant manuals, and
exercises and activities. Training-of-trainers sessions and distance learning versions
are planned for the courses.
53
•
Planning logistics, developing content, and delivering regional meetings and training
workshops. These events included a 2-week orientation held in Atlanta, GA; a 4-day
regional meeting in the Southern Africa region, one in Thailand for the Asia region,
and another in Trinidad for the Caribbean region. ORC Macro also contributed to a 4day regional meeting in Tanzania for the East/West Africa region.
•
Recruiting and hiring SI/M&E officers. These officers were placed in field offices,
and ORC Macro provided them with ongoing technical assistance support.
•
Developing and maintaining a website (http://www.globalhivevaluation.org/). Using
this resource, SI/M&E officers and other stakeholders can obtain up-to-date
information on state-of-the art approaches to SI/M&E based on international
standards and practices; materials from key international agencies and contracting
organizations that support implementation for SI/M&E; a directory of all USG and
Joint United Nations Programme on HIV/AIDS (UNAIDS) SI and M&E field
officers; and supporting materials from meetings and workshops.
•
Maintaining and distributing a training calendar to key stakeholders. This calendar
shows all trainings provided or sponsored by USG agencies (U.S. Department of
Health and Human Services [HHS]/CDC, U.S. Agency for International Development
[USAID], U.S. Department of Defense [DoD], U.S. Census Bureau [BUCEN], Health
Resources and Services Administration [HRSA], and Peace Corps) and their
collaborating implementing agencies, particularly University Technical Assistance
Providers (UTAP), MEASURE Evaluation, and ORC Macro.
•
Participating in a multiagency working group on evaluation capacity building. These
activities involved the development of a framework for capacity building at
headquarters and national levels, including major input, activity, and output
monitoring indicators/measures and data sources for SI and M&E capacity building at
individual, organizational, and systems levels.
•
Conducting site visits. The purpose of these site visits was to document best practices
for SI capacity building and building and sustaining functioning SI units/teams.
Results of the site visits will be summarized into how-to manuals.
•
Producing GAP annual reports. These reports include indicator analyses for each
country; summary data for the program as a whole; country-by-country summaries of
accomplishments, challenges, and recommendations; a headquarters progress report;
and a cumulative summary of progress to date.
In-country Capacity Building, Technical Assistance and Training
•
Conducted M&E capacity assessment in Botswana
•
Planned and implemented strategic information and monitoring and evaluation field
officer orientation for USG and UNAIDS M&E officers
54
•
Provided training on fundamentals of M&E in Botswana
•
Convened strategic information and monitoring and evaluation field officer regional
meetings in South Africa and Tanzania for USG SI liaisons and M&E field officers and
national counterparts
•
Provided technical assistance on data collection and reporting for PEPFAR in Zimbabwe
•
Provided technical assistance on data collection and reporting for PEPFAR country
Operational Plan and Foreign Aid Operational Plan in Malawi
•
Provided M&E training to ministry of health staff in Guyana
•
Conducting evaluation of M&E training in Guyana
•
Assisting Zambia with development of training materials on HIV/AIDS M&E
•
Developing content for and convening training on HIV/AIDS health information systems
in Zambia
55
Annex 2
Namibia M&E Capacity Assessments: Terms of Reference
Soon after Independence, the National AIDS Coordination Programme (NACOP) was established in the
Ministry of Health and Social Services (MoHSS) in 1992. The mandate of the NACOP is to coordinate
the national response to the epidemic using the Medium Term Plan as its operating framework. This plan
encourages the involvement of all sectors and assists to mobilize resources for the response.
In 2002, Cabinet approved a new structure within the MoHSS which made a provision for a new
Directorate (Directorate of Special Programmes) to design, manage and direct policy development,
strategic planning, resource mobilisation, co-ordination, facilitation, monitoring and evaluation of the
national response across all sectors to reduce the impact of HIV/AIDS, TB and Malaria on the Namibian
population. The Directorate has two divisions, one focusing on the health sector requirements and the
other on the multi-sectoral response.
The Third Medium-Term Plan (MTP III) on HIV/AIDS for the years 2004-2009 serves as a guide for
sector response to the epidemic, as well as a management and coordination tool for all those involved in
the fight against the epidemic. The MTP III describes the roles and responsibilities of the various
mechanisms and committees at the different organisational levels of the national response.
The DSP has a Response M&E subdivision which is responsible for the coordination, development and
implementation of the M&E systems in all sectors implementing HIV/AIDS activities in the country. This
subdivision is also tasked with promoting and supporting the development and implementation of the
national M&E plan. This subdivision is staffed by Chief Health Programmes, Chief Health Programme
Administrator, Statistician, Data Typist (all government); M&E Technical Advisor (CDC) and Data
Manager (GF). This subdivision is supported by UNAIDS, GAMET, CDC, GF, USAID, EC, WHO,
UNICEFF and many other development partners.
The M&E committee was established in 2003, consisting of all stakeholders in the fight against
HIV/AIDS in the country. For some time this committee was functional, and is now being strengthened
under the chairperson ship of the umbrella organization for civil society. Its role is to coordinate and serve
as a consultative forum on strategic issues and to guide the RM&E subdivision.
As part of operationalising and supporting the development of adequate M&E systems in the country, the
subdivision would like to undertake assessment of the M&E skills and capacity in the country. The
assessment will be undertaken at all levels, that is, community, district, regional and national
Objectives of the M&E Capacity Assessment will be to:
1. To examine the existing infrastructure and capacity (at all levels) of the country’s HIV/AIDS
qualitative and quantitative data management (information systems including collection, storage,
retrieval , flow and QA);
2. To examine priorities, preparedness, practices and needs related to M&E in the country;
3. To identify existing HIV/AIDS M&E programmes, resources, best practices and gaps at national,
regional, district and sector levels;
4. To assess M&E training needs;
5. To assess the use of data to improve programme and make recommendations for improved data
utilisation
6. To determine the country’s HIV/AIDS information management systems needs that will enhance
RM&E efficiency;
56
7. To use the baseline data to develop a comprehensive and sustainable HIV/AIDS Response
Monitoring and Evalaution information system;
8. To develop technical assistance and capacity building action plan
9. To estimate the cost of M&E infrastructure and institutional capacity development
10. To initiate consensus building activities with stakeholders;
11. To generate a detailed report with clear recommendations for the development of M&E capacity
and practices within the HIV/AIDS response.
List of stakeholders:
™ Chair; NAC, NAMACOC, NAEC
™ DSP: Director and Deputy Directors X2, RM&E Staff (Head, Statistician, M&E Technical
Advisor, Data Manager)
™ Programme Officers: PMTCT, OI, VCT, TB, ART, PCR
™ Programme Officer HIS
™ Chairperson TAC
™ M&E Committee members (see attached list)
™ Civil Society Organisations: NANASO,
™ Private Sector: NABCOA, AMICALL;
™ FBO: CHS; ELCAP, CAA, LHS, CCN
™ Development partners: Chair partnership Forum, UNAIDS; UNICEF, WHO, CDC
™ PMU: Director & M&E Specialist
™ Other Public: OPM, NPC, MOE, MRLGHRD, MGECW;
™ Coordinator: Lironga Eparu
57
Annex 3
Documents Consulted
Gowes M, Reagon, C, Hedberg C, et,al, HIS Situational Analysis of the Health Management Information
System in the Namibian Ministry of Health and Social Services, School of Public Health, University of
the Western Cape, 2004.
Health Metrics Network, Namibia health information system situation analysis, undated.
MOHSS Namibia, Namibian HIV/AIDS Situation and Gap Analysis, Prepared for the Round 6 Global
Fund, HIV/AIDS, TB and Malaria Proposal, May 2006
MOHSS, Report of the 2004 National HIV/AIDS Sentinel Surveillance Survey, May 2005.
MOHSS, DSP/RM&E, Plan for Multi-sectoral Monitoring and Evaluation of the HIV/AIDS in Namibia,
2006/07 – 2008/09.
MOHSS, DSP/RM&E, Six Month Implementation Plan for the National Multi-sectoral Monitoring &
Evaluation of HIV/AIDS for the period: October 2006-March 2007.
Office of the Prime Minister, Table showing level of Activity of Line Ministries as per MTP-III, August
2006.
Republic of Namibia, Ministry of Health and Social Services, Directorate of Special Programmes, The
National Strategic Plan on HIV/AIDS: Third Medium Term Plan 2004-2009, March 2004.
Republic of Namibia, Ministry of Health and Social Services, National AIDS Coordination Programme,
Progress Report On The Third Medium Term Plan On HIV/AIDS, April 2004 - 31 March 2006
(draft)
Republic of Namibia, Ministry of Health and Social Services, Directorate of Special Programmes,
RM&E, Follow-up to the Declaration of Commitment to HIV/AIDS (UNGASS): Namibia Country
Report, January 2003 – December 2005, 31 December 2005.
UNAIDS/GAMET, Joint UNAIDS/World Bank (GAMET) Mission to Namibia, End of Mission debrief,
powerpoint presentation, Nov, 2006.
UNAIDS. 2004. “Three Ones” Key Principles.” Conference Paper 1. Washington Consultation
25.04.04. Geneva: UNAIDS. Accessed on UNAIDS.org, 9.18.06.
UNAIDS. 2005. The Three Ones in Action: Where Are And Where Do We Go From Here? Geneva:
UNAIDS. Accessed on UNAIDS.org. 9.18.06.
58
Annex 4
Illustrative list of M&E System Assessment and Strengthening
Tools 1
1. Building National HIV/AIDS M&E Capacity: A Practical Guide for Planning, Implementing, and
Assessing Capacity Building of HIV/AIDS Monitoring and Evaluation Systems (MEASURE
Evaluation, CDC-GAP/ORC Macro)
The purpose of the guide to Building National HIV/AIDS M&E Capacity is to assist national HIV/AIDS
programs to plan and implement HIV/AIDS M&E system capacity building, working toward the
development of a unified, standardized, and effective national M&E system. The guide provides a stepby-step approach to assessing performance gaps and guiding investment to improve data production and
data use for HIV/AIDS programs. It is designed for persons responsible for the M&E of HIV/AIDS
programs at the national and sub-national levels and their partners. Yet, M&E policy makers, managers,
and program staff alike will find it useful for guiding a comprehensive approach to building M&E system
capacity. The guide includes resources and tools for stakeholder engagement, facilitated discussions and
workshops, an assessment checklist, and templates for a capacity building plan. Working through this
process, participant countries and organizations develop a capacity building plan that includes: agreed and
measurable performance indicators for M&E systems, an inventory of both technical and organizational
capacity gaps, priorities for capacity-building interventions, and a monitoring plan for tracking capacitybuilding results over time.
2. Program or Project-related M&E System Self-assessment Tool: Three Checklists for M&E
System Self-assessment (MEASURE Evaluation, GFATM, PEPFAR, HMN, World Bank, RBM, Stop
TB)
The objective of the Program or Project-related M&E System Self-assessment Tool is to identify and
address gaps in the quality of data produced in an HIV/AIDS program or project. Results of the selfassessments are used to improve M&E and the quality of data generated to measure success of programs
or project activities. This tool can help all reporting entities under government programs and donor
projects assess the strengths and weaknesses of their M&E systems, including data collection and
reporting, highlighting areas for improvement that might require additional focus, funds and/or technical
assistance. The outcomes of the assessment are translated into an action plan to enable appropriate followup measures to strengthen M&E.
Checklist 1: Assessing the M&E Plan
This Checklist looks at the goals and objectives of programs or projects and how they relate to a
country’s National Strategy and M&E Plan, if they exist. The Checklist assesses the indicators
selected, their data sources, target-setting and availability of baselines. Special attention is given to
identifying whether parallel reporting systems are being set up for donor-funded projects (i.e., outside
the national reporting system). The Checklist looks at data dissemination and transparency, as well as
confidentially of sensitive data. Finally, the Checklist assesses the budget amount allocated to M&E.
Checklist 2: Assessing Data Management Capacities of the Program/Projects Management
Units
1
Note this list is not comprehensive. It represents only a few examples of tools and methods available for
HIV/AIDS M&E system capacity building.
59
This Checklist assesses the data management systems of the Program/Project Management Units
(PMUs) of national programs or donor-funded projects. The Checklist seeks to determine if the PMUs
possess the resources, procedures, skills and experiences necessary for M&E data management and
reporting. The Checklist assesses whether the PMU provides sufficient oversight, guidance and
support to sub-reporting entities and if feedback is provided to them on the quality of their reporting
and on program performance. Finally, the Checklist determines if reports are submitted on time,
complete and mistake-free.
Checklist 3: Assessing Data Reporting Systems per Program Area
This Checklist assesses the strengths of the data reporting systems, including the ability to report
valid, accurate and high quality data related to implementation of program areas. The three
questionnaires in this Checklist include one for health facility-based activities, one for communitybased activities (e.g. BCC for TB) and one for systems strengthening activities. Through these three
questionnaires, the Checklist focuses on data reporting systems that produce numbers related to: 1)
people reached/served, 2) commodities distributed, 3) people trained, and 4) service points/
facilities/organizations supported.
Each Checklist in the M&E System Self-assessment Tool contains a dashboard of findings and an Action
Plan for rectifying any gaps.
3. Data Quality Audit Tool (MEASURE Evaluation, GFATM, PEPFAR, HMN, World Bank, RBM,
Stop TB)
The Data Quality Audit Tool follows the content of the M&E System Self-assessment Tool (see above)
to assess in-country the data management systems related to HIV/AIDS, TB and Malaria – or other health
area. In addition, it provides the methodologies, tools and documentation necessary to sample and verify
the data quality of results at program or project sites for selected indicators. This tool is designed to be
used by an independent audit team.
4. PRISM Framework and tools (MEASURE Evaluation)
Performance of Routine Information System Management (PRISM) is a conceptual framework
encompassing six tools that aid in the assessment, design, monitoring, and evaluation of routine health
information systems (RHIS). This tool set analyzes the performance of RHIS by taking into account
behavioral determinants, technical determinants, and organizational/environmental determinants. Tools
include: RHIS performance diagnostic tool; RHIS Overview and Facility/Office Checklist; RHIS
management assessment tool; Organizational and Behavioral Questionnaire Tool (OBAT), the HIV/AIDS
inventory, and the HIV/AIDS records assessment tool. The tools cover data quality and information use of
any programmatic activity. The latter two are specific to HIV/AIDS.
5. Data Demand and Information Use (DDIU) Stakeholder Engagement Tool (MEASURE
Evaluation)
The Stakeholder Engagement Tool helps ensure that the appropriate stakeholders in decision processes
have been identified and involved. The tool provides a framework for assessing who the key actors are
and identifying their interests, knowledge, positions, alliances, resources, power and importance. It assists
with finding key areas of resistance or challenges to scaling up interventions and key areas of support for
the scale up.
6. DDIU Decision Use Calendar (MEASURE Evaluation)
60
The Decision Use Calendar provides a systematic approach for stakeholders to leverage data into more
productive decision processes. It encourages greater use of and demand for information by decisionmakers, identifying methods of more effectively using existing data, and providing a timeline for
monitoring progress in the decision-making process.
7. DDIU Decision Use Map (MEASURE Evaluation)
The Information Use Map assesses and identifies opportunities to improve data use and feedback
mechanisms for stakeholders across different levels. It provides a visual context for gaps and defects in
data sharing, premising mid-course improvements.
8. DDIU Assessment of Constraints (MEASURE Evaluation)
Assessment of Data Use Constraints is a rapid assessment tool designed to identify barriers and
constraints that inhibit effective practices in data use. These include organizational constraints, in which
effective and systematic processes are inadequate for sharing data in a way that promotes evidence-based
decision-making; technical constraints in data collection and computer/technological shortages; and
individual constraints.
9. M&E Capacity Building Training (MEASURE Evaluation)
The goal of the Monitoring and Evaluation Capacity Building Training is to strengthen the capacity of
participants to monitor and evaluate their programs by using information to make more informed
decisions. The training approach is highly interactive and focuses on building sustainable M&E systems
by building M&E capacity in key areas such as data use, data quality, and stakeholder engagement while
giving participants an opportunity to take existing information from their work and apply it to the
development of their own M&E work plan. The training is practical and focuses on the transfer of
individual learning back to the participants’ organizations. This 5-day workshop has also been adapted as
a two-day workshop for use at the organizational level for larger partners who have additional M&E
capacity building assistance needs and want additional depth of M&E capacity technical assistance. The
training program reinforces a paradigm shift in M&E practices which emphasizes the use of M&E for
program improvement and decision-making rather than for reporting purposes alone.
10. HIV/AIDS M&E Short Courses (MEASURE Evaluation)
MEASURE Evaluation offers two-week short courses in Monitoring and Evaluation of HIV/AIDS
Programs and Health Information Systems (HIS). These courses are offered in English, French and
Spanish. The M&E of HIV Programs course covers the fundamental concepts and tools for monitoring
and evaluating HIV/AIDS programs and includes field visits and group work developing sample M&E
plans. The target audience for this course includes national level M&E professionals and their
counterparts, assistants and advisors involved with the implementation of HIV/AIDS programs. The HIS
course includes comprehensive assessment and performance improvement of routine health information
systems (RHIS). The main objective is to provide rapid transfer of knowledge and skills in RHIS
performance strengthening at both the national level and sub-national levels based on the PRISM
conceptual framework. The course offers an on-line forum to exchange experiences and to interact with
experts to obtain continuous support. MEASURE Evaluation partners with in-country organizations to
plan and implement these courses, thereby building the individual M&E capacity of the course
participants while also enhancing the capacity of the partner organizations to conduct M&E and to offer
M&E training programs.
61
Annex 5
List of organizations, facilities and individuals contacted
Organizations and Facilities
Centers for Disease Control and Prevention (CDC), deputy/acting director, HIS
Directorate: Policy, Planning and Human Resource Development (DPP&HRD)
Directorate of Special Programmes, Training Subdivision
Directorate of Special Programmes, Response Monitoring and Evaluation Subdivision (RM&E)
Eenhana Hospital
Engela Hospital
Enegela Health Center
European Union (EU)
Global Fund to fight HIV/AIDS, TB and Malaria (GFATM)
INTRAH Health International
ITECH
Management Sciences for Health
Joint United Nations Programme on HIV/AIDS (UNAIDS)
Khomas Region Regional AIDS Co-coordinating Committee (RACOC)
Ministry of Health and Social Services (MOHSS)
Namibia Business Coalition on AIDS (NABCOA)
Namibia Network of AIDS Service Organisations (NANASO)
Ohangwena Region AIDS Co-coordinating Committee (RACOC)
Ohangwena Region Constituency AIDS Co-coordinating Committee (CACOC)
Onandjokwe Hospital
USG partners involved in lab and pharmacy
UNAIDS Joint United Nations Programme on HIV/AIDS
United States Agency for International Development (USAID)
Individuals
Contact Name
Sandra Owoses
Kereng Masupu
In Maiupu
E. Nghiitwikwa
M. Mulondo
Anna Kufu
Festus Ikanda
Amundaba Eloby
N.U. Ndilula
O.A. Ogundiran
L.K. Kahindi
L.N. Namupala
FV Shipunda
E. Hamutenya
Jennifer Lisotto
Madaline Feinberg
Dalleen Witbosi
Alexinah Muadinohamba
E. Dumeni
S. Magaz
Dinah Tsipura
Jennie Lates
Organization
DSP – RM&E
CDC/RM&E
CDC/MOHSS
MOHSS
NANSO
Omulongga Constituency
Omandaangilo Constituency
RMT
Engela Constituency
PMO
CDC/MOHSS
Eenhana Hospital
DPHCS
Eenhana Hospital
DSP RM&E
USAID
DSP RM&E
ITECH
DSP RM&E
MOHSS/PMU
MOHSS
MOHSS
62
Dawn Pereko
Esegiel Gaeb
Liz Stevens
Ivory Hairab
S. Geises
V.J. Henruert
H.R. Swartz
J. Basson
E. Makabanyane
E. Garises
Timotheus Gebhard
Francina Tijituka
Windeline Kausiona
David Sinombe
Jackson Wandjiva
I.F. Neis
A. Shaningwa
Elly Shaaniko
Evilene Hanson
Ripanga Muuandjuar
Anrean Shivute
Stella Cloete
Anna Ileha
Martha Angula
Elize Bock
Hilda Nakakuwa
Yide Putter
Emelda Kicham
Gabriel Benjamin
Sophia Shaningwa
Shakes Mugelus
Baitswew
P.N. Nelumbu
Florence Shivute
Lorna Kgluwapa
Emily Fillipus
Elizabeth Amungulu
Shiweua Oscur
Abel Ngato
Rick Bauer
Patricia Williams
Matha Mutilifa
Belinda Gaweses
Tuulikki Mule
D.M. Kaneudie
M. Liman
Peter van Wyk
Aina Heita
Mary Mahy
Godfrey Tuhaundule
Albertina Thomas
Claire Dillaveu
Albertina Thomas
Abel Ngato
MSH/RPM Plus
NIP
ITECH
MFMR
OFP
OFP
Ministry of Youth
MYNSSC
MME
MME
MOHSS
MOHSS/DSP
National Counsel
Safety and Security
MGECW
MOL&SW
Office of the President
Ministry of Information
Ministry of Information
Office of the Auditor
National Assembly
Auditor General
Justice
OPM/PSITM
Electoral Commission
Electoral Commission
EC/OSP/WOESS
OPM
Khomas Regional Council
Khomas Regional Council
HCCHP
Vision Life
MOHSS/Khomas Region
Church Alliance for Orphans
ACSA
Joint Compassion Keeper
Project Hope
DAPP – TCE
DAPP – TCE
Catholic AIDS Action
PACT
Elcin AIDS Action
Catholic AIDS Action
Elcin AIDS Action
Catholic Health Services
BES 3
NABCOA
NABCOA
UNAIDS
DSP – TSC
DSP – TSC
PEPFAR
DSP – TSC
DAPP/TCE
63
Brian Goercke
Amarcy Campbell
Godwin Chisenga
Shiley Magazi
Worio Shimomurk
Kaori Mizumoto
Emelda Ucham
Gottried Uaaka
Zelnadia Engelbracht
Perry Mwangala
Mark Damyson
Emad Aziz
Jeanette
Francena Dataykeni
S. Ndeikdyele
O. Kalume
Liberius Iipinge
Elizabeth Matrios
Joshua Kahikuata
Felicity Haingura
Sonja Poller
Tom Kenyon
Kristine Hamalwa
Albert Ndahalele
Ester Shatons
Hilja Metusalem
Jude Neurokike
Esegiel !Gaeb
V. Mulongeui
Ashipala
Anne DeKlek
Bjorn Christ
Ellah Munkanze
Magreth Hauge
Wilhelmina Kafitaa
Dawn Broussard
Liezel Wolmarans
Michael de Klerk
Ernst Mbangula
JHU/HCP
MRLGHRD
Catholic AIDS Action
PMU/MOHSS/GF
JICA Pretoria
JICA Pretoria
OPM/HIV/AIDS Unit
NIP
MG&CW
PMU/MOHSS/GF
CDC/HIS
Capacity Project
EU
Eenhana Hospital
Eenhana Hospital
Eenhana Hospital
Adibo Health Center
HIS
HAMU
HAMU
HAMU/GF
CDC
Eenhana Hopsital
Eenhana Hospital
Eenhana Hospital
Eenhana Hospital
MSH
NIP
MOL&SW
Foreign Affairs
OPM
Philippi Trust Namibia
URC
Ekahandja
DSP - TSC
CDC
SMA
HIS
HIS
64
Annex 6
Interview Guide for Line Ministries and OPM
Name:
Position:
Organization:
Date:
Interviewer:
1. Understanding of national reporting requirements
What role does the ministry play in national reporting on the HIV/AIDS Response?
2. Verification of data sources for national reporting
Indicator: % and Total # of workplaces that have been provided support to develop
workplace policies according to national code on HIV/AIDS in employment
Indicator definition (numerator/denominator)
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting:
Data storage
Indicator: % and # of workplaces that have established a condom procurement and
distribution system
Indicator definition (numerator/denominator)
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting:
Data storage
Indicator: % and # of workplace peer educators trained
Indicator definition (numerator/denominator)
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting:
Data storage
Indicator: % of large enterprises/companies (including line ministries) that have HIV/
AIDS workplace policies and programmes
Indicator definition (numerator/denominator)
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting:
65
Data storage
Indicator: % of employees in public/private sectors that have been reached by work place programmes in
the past 12 months
Indicator definition (numerator/denominator)
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting:
Data storage
- Request examples of data collection instruments
-
Request reports containing national level data / information
3. Summary of HIV/AIDS programming in the ministry.
4. Data collected and reporting on routine HIV/AIDS programming
- Is there an M&E plan for HIV/AIDS programming?
- Are there budgetary resources allocated to data collection for HIV/AIDS programs?
- Does the ministry receive technical and/or financial assistance for data collection and reporting for HIV/AIDS
programs?
- Are data collected routinely on these programs/activities?
5. What data are collected routinely on HIV/AIDS programming in the ministry?
Method of data collection:
Frequency of data collection:
Quality control
Frequency and method of reporting
Type of database used
Are data analyzed using standardized indicators?
Request reports containing information related to HIV/AIDS programming.
6. Coordination and management of HIV/AIDS programming data collection and reporting
- Staff assigned to data collection for HIV/AIDS
- Number of staff at central level
- Number at other levels (Reg/Consti)
- Qualifications and training levels
7. Please describe the steps in the process and who is involved in data collection, data quality checks, and
reporting (i.e., data flow; is this the same for all data/indicators)
66
8. Use of information for programming and policy making
9. Relationship to DSP and RM&E
- Mechanisms for interaction/frequency of interaction
- How data/information reported
- Use of data
10. Relationship to RACOC/CACOC to Regional Education Units
11. Relationship to Civil Society and Private Sector in terms of HIV/AIDS programming, data collection and
reporting/use
12. Capacity building needs: Technical vs Organizational
13. Role of National HIV/AIDS M&E Committee: effectiveness and potential future role
14. If time permits: Perceptions of national HIV/AIDS M&E system:
-
Challenges related to data collection and use for HIV/AIDS programming.
Successes
Suggestion changes or capacity building focus.
67
Annex 7
Attendees at the Namibian National HIV/AIDS M&E System
Stakeholder Workshop
Name
Organization
Godwin Chisenga
Michael Mulonda
Kering Masupu
E. Dumeni
Mary Mahy
Amarcy Campbell
Shiley Magazi
Sandra Owoses
Worio Shimomurk
Kaori Mizumoto
Emelda Iucham
Dalleen Witbosi
Claire Dillavou
Jennie Lates
Gottfried Uaaka
Zelnadia Engelbracht
Madaline Feinberg
Perry Mongala
Mark Damyson
Catholic AIDS Action
NANASO
CDC/DSP
MOHSS/DSP
UNAIDS
MRLGHRD
PMU/MOHSS/GF
MOHSS/DSP
JICA Pretoria
JICA Pretoria
OPM/HIV/AIDS Unit
MOHSS/DSP
PEPFAR
MOHSS Pharmaceutical Services
NIP
MG&CW
USAID
PMU/MOHSS/GF
CDC/HIS
68
Annex 8
Results for Checklist for Assessing and Monitoring Capacity of the HIV/AIDS M&E System in Namibia
Note: Xs that are on the line represent responses that were in between two categories
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
Capacity Domain: Resources
1.
Mandate/Authority
1.1
There are organizations or groups within
the national HIV/AIDS M&E system that
have the mandate and authority to carry out
the functions associated with the following
performance objectives.
Developing a national HIV/AIDS M&E Plan
Implementation of the National HIV/AIDS
M&E Plan
The mandate for planning is in the MTPIII
and reflected in organizational structure;
i.e., RM&E.
X
The role of RM&E is to implement the
National HIV/AIDS M&E Plan, not clear
about mandate among other stakeholders.
RM&E needs help with how to involve all
stakeholders in implementation of the Plan.
There is a willingness at all levels to
implement the plan, but there is no policy to
mandate the responsibilities of the various
stakeholders.
X1
Capacity building of the M&E system
Production of HIV/AIDS data/information
X
Capacity building is written in the plan, but
69
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
Explain
(Provide an example or
further details)
No, not
at all
Do not
know/
NA
there is no real authority to do it.
X
There is a mandate in certain areas to
produce data, which are specified in MTPIII.
Lab is the only area with a real mandate to
produce data for testing. HIS also has a
mandate.
Communication and use of HIV/AIDS M&E
data
X
There is a weak emphasis on the
communication and use of data.
The communication pieces need to be
tailored for their target audiences, so they
will be used more readily.
2.
Financial Resources
2.1
There are adequate public sector budgetary
resources devoted to HIV/AIDS M&E.
(Consider all performance objectives)
There are some resources dedicated to staff
for M&E, i.e. RM&E staff. Civil society does
not have a permanent M&E officer in place.
Developing a national HIV/AIDS M&E Plan
In terms of budgetary resources, pharmacy
is on the boarder yes partly/no not at all,
70
Statement
Answer with respect to
the M&E system
Yes,
completely
Implementation of the National HIV/AIDS
M&E Plan
Capacity building of the M&E system
Yes,
mostly
Yes,
partly
X
X
Production of HIV/AIDS data/information
X
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
local government is no not at all, and
gender and child welfare is no not at all. The
participants noted a need for a costed plan.
SI liaison indicated that PEPFAR would be
willing to help fill funding gaps if
documented. Integrated plan is needed.
The participants noted that the fact that
they have permanent staff demonstrates
that there are resources.
Communication and use of HIV/AIDS M&E
data
X
X
2.2
There are adequate financial resources
devoted to HIV/AIDS M&E from
development partners. Consider all
performance objectives.
Development partners tend to identify
areas they want to fund and the
government fills in the gaps. Most partners
are just starting to fund M&E.
Developing a national HIV/AIDS M&E Plan
X
Implementation of the National HIV/AIDS
M&E Plan
71
Need to develop a costed plan highlighting
what is currently covered, and presenting
the gaps to partners as potential areas to
Statement
Answer with respect to
the M&E system
Yes,
completely
Capacity building of the M&E system
Production of HIV/AIDS data/information
Communication and use of HIV/AIDS M&E
data
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
X
fund.
X
Outside of Health, finding resources for
M&E is difficult.
X
X
2.3
Financial resources devoted to HIV/AIDS
M&E cover needs for the next 3‐5 years.
3.
Human Resources
3.1
There is a sufficient number of staff in the
national HIV/AIDS M&E system. (Consider
both management and direct line staff at
different levels and in different sectors.)
X
X
The government funded staff are
permanent, but donor funding is short term,
i.e. PEPFAR is only for one year, GF is for 2
years.
There are permanent staff within RM&E,
but Some members of their team are
consultants, so not guaranteed.
M&E officers are non‐existent in other
ministries/partners (there are some plans
to add these role)
3.2
The National M&E Management Unit has
the right combination of skills among its
RM&E is new, so there are still technical
areas that need to be addressed
72
X
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
staff to function effectively. Below are
examples of a typical skill set of a National
M&E Management Unit. Consider the two
main categories – technical and
organizational.
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
Staff roles identified as needed/desired:
Research/surveillance
Evaluation
HIS
Focus on multisectoral
Program monitoring
Informatics
Editor/writer
Technical
„ M&E (general & HIV/AIDS specific)
„ HIV/AIDS planning and programming
knowledge
„ Epidemiology
„ Behavioral/social sciences
„ Statistics
„ Communication: packaging and
presentation of information
„ Information technology
„ Data base development and
management
„ Training (continued below)
Organizational
„ Political awareness/understanding
country context
„ Management
„ Financing and grant management
„ Conducting multi‐sectoral relations
„ Conducting donor relations
„ Facilitation
„ Consensus building
„ Advocacy for M & E
„ Publications
„ Using M&E to inform programs and
system development
73
Statement
Answer with respect to
the M&E system
Yes,
completely
3.3
The national M&E system and sub­
systems have the right combination of
skills among its staff to function effectively.
Below are examples of skills sets for a
national M&E system and sub­systems.
Yes,
mostly
Yes,
partly
No, not
at all
X
X
Explain
(Provide an example or
further details)
Do not
know/
NA
Partly for health, not at all for other
ministries, and not known for some
subsystems
Technical
„ Data collection
„ Data analysis and interpretation
„ Supervision and mentoring of data
collectors and users
„ HIV/AIDS planning and programming
„ Computer literacy (where applicable)
„ Data base management (where
applicable) (continued below)
„ Communication: packaging and
presentation of information
Organizational
„ Management
„ Coordination of partners and units
„ Consensus building
„ Advocacy for M & E
3.4
The national HIV/AIDS M&E system has
access to staff development resources in‐
country such as local universities or
training institutions and consultants.
X
74
There is not enough time to build capacity.
Staff who are trained/skilled do not have
time to train others. The need for CB is not
quantified.
Statement
Answer with respect to
the M&E system
Yes,
completely
3.5
4
4.1
The national M&E system depends on
external technical assistance to fill essential
local M&E posts.
Infrastructure/Supplies/
Equipment
The national HIV/AIDS M&E system has the
supplies and equipment required to address
all performance objectives.
Yes,
mostly
Yes,
partly
No, not
at all
Do not
know/
NA
X
X
This area needs work There is a need for
software and office space, and the need
varies widely at the regional/constituency
level. Pharmacy ranges from completely to
not at all.
A checklist of supplies and equipment might
include:
„ Data collection forms, registers
„ Reporting forms
„ Data storage (electronic and physical)
„ Information technology: hardware,
software, communication
„ Communication equipment
„ Communication systems (email,
Internet, phone)
„ Electricity supply
5
5.1
Explain
(Provide an example or
further details)
Leadership
There is an institution, team or individual
X
75
RM&E provides leadership, but they lack
Statement
Answer with respect to
the M&E system
Yes,
completely
5.2
5.3
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
that provides leadership in the national
HIV/AIDS M&E system.
clout due to where they are located in the
organizational structure. Not at all for
other parts of system (even in health)
The leadership for the national HIV/AIDS
M&E system has demonstrated a
commitment to achieving system
performance objectives.
Higher‐ups have not bought into M&E.
X
The leadership is effective in acquiring and
protecting resources for the national
HIV/AIDS M&E system.
6
6.1
Technical Knowledge and Tools
The work of the national HIV/AIDS M&E
system is informed by research and
technical resources (i.e., international M&E
guidelines, existing M&E assessment tools,
data use tools, models for M&E systems/sub
systems, documented experience, past
strategic plans for M&E, etc.).
62
The work of the national HIV/AIDS M&E
system is informed by data generated by the
M&E system.
X
X
Lack of leadership inhibits the system.
Implementation is inhibited the most.
Not even across all sectors
X
76
Statement
Answer with respect to
the M&E system
Yes,
completely
6.3
The HIV/AIDS M&E system has access to
and utilizes external technical assistance, as
required.
X
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
X
MOH completely, other sectors mostly.
Capacity Domain: Organizational Structures and Operations
7
Organizational Structure of national
HIV/AIDS M&E System
7.1
There is an organized authority for
coordinating and operationalizing
HIV/AIDS M&E.
7.2
There are gaps in the organizational
structure of the national HIV/AIDS M&E
system. For example, a lack of assigned
roles, lack of appropriate institutions, or
non‐functioning institutions)
X
RM&E
X
Lack of M&E unit in other sectors
77
Statement
Answer with respect to
the M&E system
Yes,
completely
8
8.1
Planning Processes for HIV/AIDS M&E
The development and review of the national
HIV/AIDS M&E plan involves the following:
„ Assessment/review of existing national
information collection systems and
organizational capacity
„ Assessment/review of national
reporting requirements
„ Assessment/review of information
needs for decision making
„ Holding stakeholder meetings
„ Developing frameworks
„ Selecting, developing or adapting
indicators and defining identify sources
of information
„ Planning for information use
8.2
The national HIV/AIDS M&E plan includes a
way to build technical and organizational
capacity.
8.3
There is a process and time frame for
national HIV/AIDS M&E planning, including
Yes,
mostly
Yes,
partly
Explain
(Provide an example or
further details)
No, not
at all
Do not
know/
NA
X
Planning for information use is not
institutionalized. There are also no
mechanisms in place for data audits and
quality control.
For the annual planning process, RM&E
tends to just reformat the previous years
report, rather than using data to plan. The
participants said that they would need
leadership to change this process.
X
X
X
Partly for technical capacity, partly/not at
all for organizational capacity
M&E plan is linked to MTPIII and will be
reviewed with MTPIII
78
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
revisiting and revising the plan periodically.
Annual planning in MOH differs from other
ministries.
8.4
The national M&E management unit leads
or conducts analysis to understand the
HIV/AIDS decision making environment.
This analysis might include:
„
„
„
X
Understanding the decisions that are
being made at the national and sub‐
national level that would affect HIV &
AIDS prevention, treatment and care;
There has been no assessment done of the
decision making process. Currently,
decisions are driven by national and donor
reporting requirements.
Did meet at beginning of planning process
for the development of the M&E plan to
understand needs.
Without leadership all stakeholders will not
develop together. RM&E needs to transfer
to another unit to succeed.
Where these decisions are being made
at the policy & program level; and
In general, other things will be prioritized
before M&E especially in other line
ministries.
Who are the key stakeholders involved
in making these decisions?
Most decisions about HIV are made at the
national level. It is difficult to get the grass
roots level involved esp. in decision‐
making.
Decisions made at the national level are
done without input from the lower levels.
The lower levels do not look at numbers for
planning, as they see them mainly as
79
Statement
Answer with respect to
the M&E system
Yes,
completely
8.5
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
requirement for reporting up.
Cost analysis of ART
The national M&E unit leads or conducts
cost analysis studies or reviews for
HIV/AIDS programs.
X
9
9.1
Decision­Making Processes
The leadership or coordinating group for
the HIV/AIDS M & E system meets regularly.
9.2
The leadership or coordinating group for
the HIV/AIDS M&E system reviews
progress related to the performance
objectives for the system.
X
It is a goal
9.3
Decision‐making related to planning,
operations, and strategies in the HIV/AIDS
M&E system involves consensus building
among key stakeholders.
X
Sometimes they have developed strategies
and informed stakeholders about them, but
no real consensus building. M&E
committee is a new group, consensus
building process is developing
9.4
National HIV/AIDS M&E strategic and
operational plans and procedures respond
to internal and external stakeholder needs.
X
RM&E Meets monthly
X
Mostly for external, partly/not for internal
X
80
Statement
Answer with respect to
the M&E system
Yes,
completely
10
10.1
Yes,
mostly
Yes,
partly
Financing and Budgeting
The HIV/AIDS M&E planning process
includes a budget and a plan to finance M&E
activities.
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
Completely/mostly at the national level.
Other sectors plan to do M&E work, but
have no M&E budget.
X
Planning process includes budget and plan
to fund M&E activities.
Finding resources for the M&E system is
not there. RM&E is in a better position than
the other line ministries.
10.2
Budget allocations for the HIV/AIDS M&E
system are aligned with system
performance objectives.
11
Human Resource Planning and
Development
The national HIV/AIDS M&E system has a
11.1
X
Health, HIS,
Pharmacy
OPM
81
There is a pooled budget for M&E, less
emphasis on data use and implementation.
DSP has different budget process than the
other ministries. They do not plan around
targets and objectives (as other ministries
do). Have not looked at objectives to do
planning.
Regional/
In health yes, but it is a new concept
Statement
Answer with respect to
the M&E system
Yes,
completely
strategy for meeting current and future
human resource needs.
Yes,
mostly
Yes,
partly
Explain
(Provide an example or
further details)
No, not
at all
national
Do not
know/
NA
local
because people are just beginning to realize
the need for M&E.
Civil society does not have permanent M&E
officers in place.
Space is an issue
12
12.1
Reporting
There are mechanisms in place at national
level for meeting internal and external
reporting requirements.
X
External
X
internal
There are databases and the HIS, and the
numbers for the indicators that need to be
reported are captured in these systems.
Participants noted that they do not always
get internal reports when needed
(mechanism is not working internally).
RM&E is not certain if they are supposed to
write the internal quarterly reports as HIS
has the data.
The HIS is currently being revised.
12.2
There are procedures for regional and local
level reporting to national level, both
program and financial.
X
82
Namibia is in a transitional phase in terms
of making international guidelines their
own.
Public sector yes. Civil society and private
sector not certain.
Statement
Answer with respect to
the M&E system
Yes,
completely
12.3
Reports from the national HIV/AIDS M&E
Unit are produced on time.
12.4
Data reported by the national HIV/AIDS
M&E unit are of adequate quality.
Yes,
mostly
Yes,
partly
No, not
at all
X
Explain
(Provide an example or
further details)
Do not
know/
NA
MTPIII progress report was not completed
on time.
X
There are no quality standards right now
and there are always gaps in the data.
Two different reports can have different
numbers for the same indicators.
12.5
National level reporting meets internal and
external stakeholder needs.
13
Data and Information Systems
Refer to All Data Collection Activities and
Sub­Systems in the national HIV/AIDS M&E
System
Data collection sub‐systems are in place and
are functioning effectively.
13.1
X
X
83
No internal HIV/AIDS report. Currently,
they do not know what the internal
information needs are.
Statement
Answer with respect to
the M&E system
Yes,
completely
13.2
Data storage systems (databases, data
warehouses) have been developed and are
maintained.
13.3
There is a system for extracting and
transferring data and information from data
sub‐systems and other data sources.
13.4
14
14.1
14.2
Yes,
mostly
Yes,
partly
No, not
at all
X
Routine mechanisms for quality assurance
related to data and information systems
exist and are applied.
Communication
There are mechanisms for communicating
the content of the national HIV/AIDS M&E
plan, policies, decisions, etc. to stakeholders.
X
The results of national HIV/AIDS M&E are
84
Explain
(Provide an example or
further details)
Do not
know/
NA
They are in place but they are not
maintained 100%. There are
inconsistencies in the data received.
X
There is a functioning HIS, but there are
problems getting data. RM&E needs to
write letters to the management to obtain
data from HIS.
X
Pharmacy employed a person to check the
quality of the data.
GF has quality checks in place for their data.
HIS is starting.
The Annual report, M&E plan and sentinel
surveillance report are disseminated to the
M&E committee. There is mainly passive
dissemination of information.
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
Explain
(Provide an example or
further details)
No, not
at all
Do not
know/
NA
communicated to stakeholders on a regular
basis.
14.3
The national HIV/AIDS M&E system
supports the linking of data to identified
policy and program issues.
Nationally they are planning without data.
There is limited use of data at the program
level
X
End up spending a lot of time planning and
producing data, but not a lot of time
implementing and using data.
14.5
National HIV/AIDS M&E data are
interpreted and presented according to the
needs of different stakeholders (e.g., policy
makers, program managers, development
partners).
15
Monitoring the national HIV/AIDS M&E
system
X
85
Mainly tailored to the health sector – one
size fits all model.
Currently, there is a lack of feedback loops
in the system.
Statement
Answer with respect to
the M&E system
Yes,
completely
15.1
Yes,
mostly
Yes,
partly
There are resources devoted to monitoring
the capacity and performance of the
national HIV/AIDS M&E system, including
human and financial resources.
No, not
at all
X
Explain
(Provide an example or
further details)
Do not
know/
NA
Quality checks have started.
They have done a SWOT analysis and an
assessment of the HIS, but nothing is
routine.
There are support visits to the field, also.
15.2
The national HIV/AIDS M&E system obtains
and uses data to monitor and evaluate its
units and activities.
X
15.3
There is a plan for monitoring and
evaluating the national HIV/AIDS M&E
system capacity.
X
Currently, they do not have trained
resources to build capacity.
Completeness within subsystems but not
national
Plan for monitoring
There are some internal assessments but
they are not routine.
In terms of planning, they tend to cut and
past from the previous year rather than
look at the data to plan. Need involvement
of leadership to change.
16
16.1
Links Among Units of M&E System
There is regular communication and
interaction among the agencies or
organizations within the national HIV/AIDS
M&E system.
16.2
There are coordinating mechanisms in place
86
X
M&E committee has been meeting
regularly. There have also been some
irregular meetings. HIS, NANSO meet
regularly.
X
This needs strengthening and is definitely
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
to facilitate work among different units in
the national HIV/AIDS M&E system.
17
17.1
something that the system would benefit
from. Regular meetings within DSP happen,
but not so much outside. NIP is
strengthening its link with DSP. The link
with TB is strong.
Links Between the national HIV/AIDS M&E
Unit and other units/organizations (other
subdivisions of the MOHSS, other line
ministries, development partners, umbrella
organizations, research organizations, M&E
Technical Groups)
The national HIV/AIDS M&E Unit has
formed linkages to other organizations to
support system development and
performance.
17.2
There is regular communication and
interaction between the national HIV/AIDS
M&E unit and other units/organizations.
18
Motivation
X
Not done systematically
There has been some discussion on how to
streamline, but nothing has happened to‐
date
X
Capacity Domain: Organizational Culture
87
Statement
Answer with respect to
the M&E system
Yes,
completely
18.1
18.2
Yes,
mostly
Yes,
partly
No, not
at all
There are incentives within the national
HIV/AIDS M&E system for achieving high
level performance in HIV/AIDS M&E.
Managers and staff working in the national
HIV/AIDS M&E system are committed to its
performance objectives.
18.3
The leaders of the national HIV/AIDS M&E
system perceive that capacity building
contributes to overall system performance.
19
19.1
Decision­Making Approaches
HIV/AIDS M&E system stakeholders are
involved in M&E system planning and
management decisions.
Explain
(Provide an example or
further details)
Do not
know/
NA
certificates
X
Varies a lot. Some people are put in charge
of M&E, but do not know what M&E is.
X
X
X
88
Statement
Answer with respect to
the M&E system
Yes,
completely
19.2
Decision makers in the national HIV/AIDS
program use M&E data to make decisions.
20
20.1
Consensus Building
There is an inventory of all stakeholders in
the national HIV/AIDS M&E system.
20.2
Reaching consensus among stakeholders of
the national HIV/AIDS M&E system is a
priority in strategic and operational
planning.
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
X
X
Not exhaustive
X
The committees are the consensus building
mechanism. There are so many partners,
though, that if a consensus cannot be
reached the government is deferred to.
89
Statement
Answer with respect to
the M&E system
Yes,
completely
21
21.1
Clear and Agreed Roles in the national
HIV/AIDS M&E System
There are written and agreed roles and
responsibilities for each unit and partner
involved in the national HIV/AIDS M&E
system.
21.2
The agreed roles and responsibilities for
each unit and partner involved in the
national HIV/AIDS M&E system are
reviewed on a regular basis.
22
Commitment to Sharing and Using
Information
There is commitment within the national
HIV/AIDS M&E system to sharing data and
information.
22.1
22.2
Yes,
mostly
Yes,
partly
No, not
at all
X
Explain
(Provide an example or
further details)
Do not
know/
NA
The roles and responsibilities are outlined
in national plan, but more needs to be done
to help stakeholders understand their roles.
X
X
Have not started implementing all of the
roles, so they can not review progress, yet.
Some in the health sector have been
reviewed.
Some information they are not allowed to
share. The current mechanism in place
prevents easy sharing. Have to go through
the PS.
There are mechanisms in place for sharing
information such as email, listservs,
X
90
Email
Statement
Answer with respect to
the M&E system
Yes,
completely
Yes,
mostly
Yes,
partly
No, not
at all
Explain
(Provide an example or
further details)
Do not
know/
NA
newsletters, and regular informational
meetings.
23
23.1
Coordination, Collaboration, and
Networking
Stakeholders in the national HIV/AIDS M&E
system are working collaboratively to further
HIV/AIDS M&E system performance.
X
91
The stakeholders are trying to work together,
but there are turf issues. Currently, there have
been no meetings for sharing
Annex 9
Descriptions of Stakeholders in the Namibian HIV/AIDS M&E
System
The National AIDS Committee (NAC) is the highest policy decision-making body, under Cabinet, on
matters related to HIV/AIDS. All M&E output reports will be submitted to this committee for endorsement.
The National Multi-sectoral AIDS Co-ordination Committee (NAMACOC) provides the leadership for
multi-sectoral and regional implementation. This committee will review progress made by the multisectoral
and regional structures and make recommendations to NAC.
The National AIDS Executive Committee (NAEC) will provide the technical leadership to the M&E
committee.
The RM&E subdivision within the DSP is responsible for the coordination of all multisectoral monitoring
and evaluation activities.
The National M&E Committee provides the leadership for multi-sectoral and regional implementation of
all M&E activities related to HIV/AIDS (annex ToR). The committee roles are to ensure a platform for
partnership, networking and collaboration between national level and local level stakeholders in monitoring
and evaluating various components of the MTPIII. It consists of representation from all stakeholders in the
national fight against HIV/AIDS.
Development partners are members of the national M&E Committee and many of the international
organisations that are present in Namibia play an important role in the monitoring and evaluation of
HIV/AIDS interventions.
The MoHSS Health Sector data are generated from health facilities and forwarded to districts and then
regions before being submitted to the Epidemiology division at national level. Therefore most relevant
indicators and information management need to be mainstreamed within existing M&E systems. It is also
important that health surveillance, service delivery and health systems M&E becomes better institutionalized
and linked to the national M&E system.
The OPM HIV/AIDS unit is responsible for coordination of work place programmes in the public sector,
therefore the M&E thereof. The unit to date has no (fulltime) focal point for M&E and it is urgent that this is
established and includes a budget for M&E.
The MRLGHRD has the mandate of serving as Government focal point for policy and operational matters
at the regional (RACOC) and constituency (CACOC) levels. It ensures the efficient operation and
coordination of the local authorities; social, welfare and community mobilization as well as the provision of
basic physical and social infrastructure. The HIV/AIDS unit is currently being developed. There is a budget
for staff and activities but the M&E component is yet not established.
The MGECW plays a leading role women and child welfare, more especially for orphans and vulnerable
children (OVC) and households affected by HIV/AIDS.
The NPC/CBS is responsible for the coordination of mainstreaming of HIV/AIDS activities in the public
sector. It also provides the framework for data collection and data management of national statistics in the
country.
The MOE is responsible for the education sector, more especially for introduction of HIV/AIDS life skills
programmes for children in school.
NANASO is the umbrella organisation for civil society organizations that play a crucial role in the response
to the HIV/AIDS epidemic in Namibia. Civil society organisations implement several HIV/AIDS
programmes at the national, regional, district and local levels. These organisations through NANASO
therefore play an important role in the monitoring and evaluation of the national response through the
activities they implement.
NABCOA is the umbrella organization for the private sector responsible for the workplace programme. The
private sector provides a large part of the clinical care and it has the responsibility of the provision of
adequate HIV prevention and care services for its workforce.
92
Lironga Eparu is the umbrella organization for support groups of People Living with HIV/AIDS (PLWHA)
nationwide. They have the responsibility to coordinate and support local PLWHA support groups and also
focus on psychosocial support, treatment support, and advocacy for PLWHA.
Currently, the three umbrella organizations described above which represent the largest proportion of the
national response to HIV/AIDS do not have established M&E units. It is necessary to build the capacity of
these umbrella organizations to include M&E focal persons supported by dedicated M&E budgets.
93
Annex 10
Sample Information Use Maps
Sample baseline Information Use Map
Adapted from final report, Strategic Information Assessment in Swaziland,
MEASURE Evaluation, January 2006.
When an information flow is mapped visually, deficiencies quickly become apparent. Large,
empty expanses of the chart tell the story. In sample (A), it is clear that insights from high-level
reports are not shared back with lower levels, and information is only being used to fill reports,
not to support evidence-based decisions for program improvements. The second map (B)
highlights potential improvements in the M&E system where feedback mechanisms can be
developed and where opportunities for increased data use can be identified.
(A) Existing Data Flows
(B) Potential Improvements to the M&E System
94
Information Use Map: Swaziland National HIV/AIDS Program
May 2005
Storage
Analysis
Reporting
NGOs
Government Facilities
Client data
collected in
electronic
patient record
systems
Client data
collected in
electronic
patient record
systems
Client data
collected
in registers
Regional
Compilation
Use
Opportunities
for Use
Client data
stored in
electronic
patient record
systems
Opportunities
for Use
Staff
compiles into
monthly
summary
sheets
Opportunities
for Use
Regional
facilities’
monthly
summary sheets
compiled
Opportunities
for Use
SNAP/MOHSW
NERCHA
Feedback Loops
Private Clinics
Data Collection
Data entered
into Access at
MOH Health
Statistics or
NERCHA
95
Reporting
to WHO or
GFATM
Opportunities
for Use
Sample expected (future) Information Use Map
For an M&E system for HIV and AIDS community-based data
MEASURE Evaluation, March 2005.
This Information Map describes a future scenario that would improve data utilization. This
map was developed with stakeholder consensus during a workshop in which desired
improvements in the M&E system were prioritized. In this scenario, information transfer is now
two-way, with feedback and quarterly reports being broadly shared across stakeholder groups.
The map also identifies ways to use data to monitor and evaluate programs, improve programs,
lobby for additional funding, influence legislation or share information with the media and the
public.
Information Use Map: Dominica National AIDS Program
Community Data
May 2005
Collection
Compilation
Storage
Analysis
Use
CAREC
GFATM
Receives
report and
gives feedback
National
Government
Receives
report and
gives feedback
Receives
report and
gives feedback
NAP
NGO/Education
data entered and
stored in
computer
Data
analyzed
Quarterly
report
prepared and
disseminated
Funding, enhance
programs, monitor
and evaluate, policy
and legislation,
media
Receives
reviewed report
and feedback
Funding, project
proposals, expansion,
planning, monitoring
and evaluation,
media, advocacy and
legislation
Receives
reviewed report
and feedback
Funding, project
proposals, expansion,
planning, monitoring
and evaluation,
media, advocacy and
legislation
Data stored
in locked
filing
cabinet or
secure
computer
MOE
NGOs
SCHOOLS
Reporting
Data
collected by
individual
NGOs using
developed
forms by
program
area
Individual
NGOs collate
data on
monthly basis
using form
Data
collected
using forms
Individual
schools
collate data
on monthly
basis using
form
Data stored
in locked
filing
cabinet or
secure
computer
96