EUROPEAN COMMISSION
DELEGATION OF THE EUROPEAN COMMISSION
IN NIGERIA
CO-FINANCING WITH CATHOLIC AGENCY FOR OVERSEAS
DEVELOPMENT (CAFOD), UK
FOR NIGERIA
Improving Primary Health Care for Rural Poor Communities in
Northern Nigeria
(Contract No. ONG-PVD/2006/119-131)
FINAL EVALUATION REPORT
Conducted by:
HIRAM Consulting Limited, PLOT 121 Golden Spring Estate, Duboyi District, Abuja
Report written by:
Adeniyi Olaleye,
Olayinka Falola-Anoemuah,
Patricia Suswam and
Doris Ogbang
This project is funded by
The European Commission
This project is implemented by
Catholic Agency for Overseas Development
(CAFOD) & Catholic Secretariat of Nigeria
1st February 2007- 31st January 2011
1|Page
TABLE OF CONTENTS
TABLE OF CONTENTS .........................................................................................................................................1
List of tables .............................................................................................................................................................4
List of figures ...........................................................................................................................................................4
List of Acronyms ......................................................................................................................................................5
Executive Summary..................................................................................................................................................7
1.1 Introduction ..................................................................................................................................................... 10
1.2 Objectives and purpose of the final evaluation: .......................................................................................... 12
2. Methodology...................................................................................................................................................... 13
3.0 Major Findings and Analysis of the Programme Outcomes ............................................................................ 16
3.1 (Objective 1): To increase the capacity of 63 Catholic-Church run primary health providers to manage
and deliver high quality, sustainable health care services in order to reduce the incidence of maternal and
childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and waterborne diseases.16
3.1.1 Training of Staff ................................................................................................................................... 16
3.1.2 Provision of Essential Capital Items and PHC Equipment ................................................................... 20
3.1.3 Technical support to PHC staff ............................................................................................................ 22
3.1.4 Provision of financial supports for running cost to selected facilities .................................................. 24
3.1.5 Development of a gender focal points network .................................................................................... 25
3.2 (Objective 2): To increase the level of community participation in, and ownership of primary healthcare in
order to promote sustainable, healthy lifestyles ................................................................................................ 26
3.2.1 Community mobilization activities....................................................................................................... 27
3.2.2. Establishment of Village Health Committees (VHC).......................................................................... 29
3.2.3 Training of Village Health Workers and TBA ..................................................................................... 30
3.2.4 Publication and distribution of IEC materials ...................................................................................... 31
3.2.5 Participatory Health Assessment by Rural Communities ..................................................................... 32
3.2.6 Awareness raising workshop on HIV & AIDS for religious leaders .................................................... 32
3.3. (Objective 3): Improved practice in primary health care as a result of increased collaboration, sharing,
coordination and learning between/amongst a diversified range of health actors and stakeholders ............... 33
3.3.1 Production of newsletters and documentation on project learning ....................................................... 34
3.3.2 Exchange visits to other PHCs within the programme ......................................................................... 34
3.3.3 Learning and sharing meetings ............................................................................................................. 36
3.3.4 Collection, production and dissemination of up-to-date health information materials ........................ 37
3.3.5 On-going/Regular dialogue with other PHC and HIV practitioners and stakeholders ......................... 37
2|Page
3.4.1 Creation of Health Information and Advocacy Unit in the Health Department of the CSN ................ 39
3.4.2: Facilitation and support to the CSN Health Think Tank ..................................................................... 40
3.4.3: Advocacy Training and Divulgation of CAFOD Advocacy Tool ....................................................... 40
3.4.4: Development and implementation of health advocacy strategies........................................................ 41
3.4.5: Active Engagement with State Health Decision Makers and other Non-State Key Actors ................ 41
4.0 Short term impact of the EC/CAFOD PHC programme ................................................................................. 42
4.1 PHC’s capacity to provide sustainable quality primary healthcare service ............................................... 43
4.2 Client Satisfaction ....................................................................................................................................... 44
4.3 Changes in health behaviour and incidence of communicable diseases ..................................................... 45
5.0 STRENGTHS, KEY CHALLENGES AND RECOMMENDATIONS ......................................................... 47
5.1 STRENGTHS OF THE PROGRAMME .................................................................................................... 47
5.2 KEY CHALLENGES .................................................................................................................................. 47
5.3 RECOMMENDATIONS ............................................................................................................................ 48
APPENDICES ........................................................................................................................................................... 51
APPENDIX 1: PROGRAMME RESPONSES TO RECOMMENDATIONS .................................................................. 52
APPENDIX 2: EVALUATION TERMS OF REFERENCE ........................................................................................... 58
APPENDIX 3: ANALYSIS OF THE PHC CHECKLIST ................................................................................ 64
APPENDIX 4: CLIENT SATISFACTION ...................................................................... Error! Bookmark not defined.
3|Page
List of tables
Table 1: Number of persons interviewed (planned versus actual) .................................................... 14
Table 2: Trainings/workshops conducted .......................................................................................... 18
Table 3: Capital items and equipment planned and distributed ......................................................... 21
Table 4: Program accompaniers and advisor recruited with appointments and disengagement dates23
Table 5: Summary of Village Health Committees activities and involvement of women ................ 30
List of figures
Figure 1: Actual number of persons trained versus targets ................................................................ 17
Figure 2: Sustainability and dependence of Facility on EC CAFOD Funding .................................. 25
Figure 3: Trend in number of clients for ANC and Delivery services in 6 selected facilities* ......... 44
4|Page
List of Acronyms
AIDS
Acquired Immune Deficiency Syndrome
ANC
Ante-Natal Care
ART
Anti-Retroviral Therapy
ARV
Anti-Retro- Viral
BCG
Bacillus Calmette-Guerin
CAFOD
Catholic Agency for Overseas Development
CHEW
Community Health Extension Worker
CSN
Catholic Secretariat of Nigeria
CSOs
Civil Society Organizations
CWC
Child Welfare Clinic
DPT
Diphtheria, Pertusis and Tuberculosis
DRACC
Divine-Love Retreat And Conference Centre
EC
European Commission
EC-ACP
European Commission Africa, Caribbean and Pacific
FCT
Federal Capital Territory
FGD
Focus Group Discussion
FGN
Federal Government of Nigeria
FMC
Federal Medical Centre
FMoH
Federal Ministry of Health
HBC
Home-Based Care
HIV
Human Immuno-deficiency Virus
HMIS
Health Management Information System
IEC
Information, Education and Communication
IMCI
Integrated Management of Childhood Illnesses
JCHEW
Junior Community Health Extension Worker
KII
Key Informant Interview
LGA
Local Government Area
MDG
Millennium Development Goals
5|Page
M&E
Monitoring and Evaluation
MoH
Ministry of Health
MVS
Multi-Vitamins Supplement
NGO
Non-Governmental Organization
NPHCDA
National Primary Health Care Development Agency
NSA
Non-State Actors
PLHIV
People Living With HIV/AIDS
PRA
Participatory Rapid Appraisal
PPRHAA
Peer and Participatory Rapid Health Appraisal for Action
SACA
State Action Committee for HIV/AIDS
SOP
Standard Operating Procedures
SPSS
Statistical Package for Social Scientists
TB
Tuberculosis
TBA
Traditional Birth Attendant
TOR
Terms of Reference
TT
Tetanus Toxoid
UN
United Nations
UNICEF
United Nations International Children Education Fund
USD
United State Dollar
VHC
Village Health Committee
VHW
Village Health Worker
6|Page
Executive Summary
In response to the challenges of primary healthcare delivery system in Nigeria, the European
Commission (EC) and Catholic Agency for Overseas Development (CAFOD) developed a programme
to improve the quality of Primary Health Care (PHC) services. The programme titled ‘Improving
Primary Health Care for Rural Poor Communities in Northern Nigeria’ (popularly called
EC/CAFOD PHC Programme) spanned from 1 February 2007 to 31 January 2011 and was
implemented in partnership with the Catholic Secretariat of Nigeria (CSN) and the 19 Catholic dioceses
that make up the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna. In terms of civil
administration, the programme covered 63 PHC facilities in 18 states and the Federal Capital Territory
of Abuja.
In March, 2011, CAFOD/Nigeria commissioned HIRAM CONSULTING LIMITED to conduct a final
evaluation of the EC/CAFOD PHC Programme. The main objective of this evaluation was to determine
the extent to which the programme has achieved its stated objectives. The final evaluation, which was
conducted between 7th of April and 30th of May, 2011, used a combination of qualitative and
quantitative methods to elicit relevant information from different sources. Ten PHC facilities (Ankpa,
Agagbe, Adikpo/Vandeikya HBC, Yakoko, Jada, Namu, Mandella, Fuka, Dongo Kurmi and Abuja
HIV) were selected for field visit through stratified and systematic random sampling methods. In all, a
total of 22 KII and 20 FGD were conducted. In addition, 53 clients exit interviews were conducted and
the facility check list was administered in 9 out of the 10 selected facilities (excluding Abuja HIV).
The key findings of the evaluation are presented under each of the four programme objectives as
follows:
Objective 1: To increase the capacity of 63 Catholic-Church run primary health providers to
manage and deliver high quality, sustainable health care services in order to reduce the incidence of
maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and
waterborne diseases.
To enhance the capacity of the PHC staff, a total of 27 training/workshops were conducted through
which a total of 809 participants were trained (79% achievement of the target). Generally, the training
contents were appropriate and relevant to the objectives of the EC/CAFOD PHC Programme. About
97.6% of various items and equipment proposed to be distributed to the PHC facilities (830 out of 850
proposed) were given out by the end of the programme, based on needs. Similarly, a total of 369 out of
the planned 810 visits (46%) were carried out by both Provincial and Technical Programme
Accompaniers within the programme implementation period. Among the 20 “vulnerable” PHC
facilities that were given financial support for running cost, the overall dependency on the EC/CAFOD
grant reduced from 38% in 2007, to 36% in 2008 and to 30% in 2009.
7|Page
There were both oral (through interviews) and documented evidence to show that the capacity of the
PHC facilities to deliver high quality PHC service has improved. PHC staff interviewed found the
trainings useful and reported that the knowledge and skills acquired had helped them to deliver quality
services to their clients. Noticeable improvement attributable to the trainings included: improvement in
data collection, record keeping and reporting; improvement in quality of drug prescription/use and
better obstetric practices, among others. The items and equipments provided also improved the quality
and efficiency of services being provided in the facilities.
Objective 2: To increase the level of community participation in and ownership of primary
healthcare in order to promote sustainable, healthy lifestyles
The data showed that 74 PHC staff, Health and HIV Coordinators were trained on Stepping Stones
methodology (82% achievement) while 50 PHC staff were trained as trainers (83% of the target).
Village Health Committees existed in 120 out of the 291 PHC stations with females constituting about
a third (34%) of the membership. Out of the 10 facilities visited, VHC were established and effective in
eight facilities. About 209 TBAs (66% of target) were trained on basic obstetric care, identification of
pregnant women at risk and refer same, use of IEC materials and participatory methodologies.
Similarly, a total of 81 participants attended the workshops on the Use of IEC and Participatory
Methodologies, representing 77% of the target. A total of 4 rural health assessments were carried out as
hands-on-training in Mandella, Zawan, Zambina and Zonkwa. On HIV awareness, 110 priests and
religious attended the awareness workshops of which about half of the participants were priests.
PHC facilities conducted HIV/AIDS awareness campaign and mobilization which created positive
impact through increased awareness and changes in behaviours of the community members and
resulted in increased community participation and ownership. There were reports that PHC facilities
produced IEC materials and translated some of the IEC into local languages. Thirty (30) local
authorities became more aware of key health issues for rural communities, suggesting 100%
achievement of the set objective.
Objective 3: Improved practice in primary health care as a result of increased collaboration,
sharing, coordination and learning between/amongst a diversified range of health actors and
stakeholders
Two drafts of newsletters were made but they were not finalised before the programme ended. Eleven
(30.6%) of the proposed 36 PHC facilities participated in the exchange visit for learning and sharing
with 65 staff as beneficiaries. Thirteen (13) meetings were held between programme/partner
representatives and other health stakeholders between February 2007 and January 2011 out of 15
planned (87% of the target). Key health information materials including HMIS forms and standards
treatment protocols were accessed from relevant government agencies, reproduced and distributed to
8|Page
the PHC facilities. The 10 PHC facilities visited reported that they had linkages and collaboration with
either or both local and state government departments of health.
The PHC staff commended the use of the exchange visit approach which exposed them to additional
knowledge and skills and enhanced their primary healthcare practices. The adaptation and use of the
FMoH data forms also enhanced the collaboration between government staff and the church health
teams.
Objective 4 : To build the capacity of the Catholic Church on health policy and advocacy issues in
order to engage with government and other key stakeholders in the health sector with the aim of
influencing the development and monitoring of pro-poor health policies in Nigeria.
Activities conducted under this objective included identification and training of advocacy champions in
the 3 provinces, meetings of the Health Think Tank in order to develop advocacy strategy for Catholic
Church and joint campaigns of health stakeholders on health issues. However, other planned activities
such as the creation of Health information and Advocacy Unit in the Health Department of the CSN,
development and implementation of health advocacy strategies and active engagement with state health
decision makers and other non-state key stakeholders could not be implemented due to various reasons.
Short Term outcomes of the EC/CAFOD PHC Programme
All of the 9 facilities (100%) visited offered a range of basic primary healthcare services including:
ANC, Immunization, Health education, Nutritional education, treatment of minor illnesses for adults
and children, HIV counselling and HIV testing. Six of the facilities (66.7%) offered labour/delivery
services while a third of the facilities (33.3%) provided postnatal services. Number of pregnant women
receiving appropriate antenatal care and number of deliveries assisted by health professionals and
trained birth attendants increased across the facilities. Majority of the PHC clients interviewed rated the
facilities high on all of the criteria used. Similarly, majority of the community leaders, TBA, village
health committee and PHC staff interviewed believed that the prevalence/incidence of most preventable
diseases has decreased in the community over the past few years.
Strengths, Key Challenges and Recommendations
The strengths of the EC/CAFOD PHC programme were identified to include the community -focus and
bottom-up approaches adopted in the design and implementation of the programme that was built on
the existing structure of the Catholic Church in Nigeria. The key challenges of the programme included
its weak monitoring and evaluation system and inadequate allocation of funds for M&E activities; high
staff turn-over; very wide coverage of the programme and non-existence (and usage) of SOPs and job
aids at the PHC level. Recommendations for future programme include setting up of an effective M&E
system at the programme start up, a review of staff recruitment process, a reduction in area of coverage
and focus on two or three related PHC services areas and capacity building for CSN and health
coordinators for monitoring and supervision of the programme.
9|Page
1.1 Introduction
In spite of her improved health status in the recent years, Nigeria still has one of the poorest health
indices in the world. For example, the life expectancy at birth in the country as at 2006 1 was estimated
at 46 years but it increased to 48.4 in 20102. Infant and under-five mortality rates were 99 in 2006 and
91.54 in 2010; and 191 in 2006 reduced to 138 in 2009 per 1000 live births3. These indices are worse
than the average for sub-Saharan Africa. The major causes of morbidity and mortality are preventable
and curable diseases. The World Health Organization4 ranked Nigeria’s overall health system
performance 142nd out of 191 member states. The high level of mortality in Nigeria is a reflection of
its weak and inefficient healthcare delivery system. Major challenges confronting healthcare delivery
system in Nigeria include poor medical infrastructure and inadequate capacity of healthcare providers
to manage and deliver high quality, sustainable health care services, especially at the primary
healthcare level which constitutes about 93.6% of formal health facilities in Nigeria5.
In response to the challenges of primary healthcare delivery in Nigeria, the European Commission (EC)
and Catholic Agency for Overseas Development (CAFOD) developed a programme to improve the
quality of Primary Health Care (PHC) service delivery in Nigeria. The programme titled ‘Improving
Primary Health Care for Rural Poor Communities in Northern Nigeria’ (popularly called
EC/CAFOD PHC Programme) spanned from 1 February 2007 to 31 January 2011 and was
implemented in partnership with the Catholic Secretariat of Nigeria (CSN) and the 19 Catholic dioceses
that make up the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna. In terms of civil
administration, the programme covered 63 PHC facilities in 18 states and the Federal Capital Territory
of Abuja.
1
UNAIDS/WHO. 2008. Epidemiological fact sheet on HIV and AIDS. Nigeria update. UNAIDS/WHO Working Group on
Global HIV/AIDS and STI. Switzerland
2
UNDESA (2009d) Barro and Lee (2010), UNESCO Institute for Statistics (2010a), World bank(2010g) and IMF (2010a):
Human development Report 2010
3
CIA World Factbook (20110.
4
WHO (2010) World Health Report. Geneva
5
National Primary Health care Development Agency (2007). Health Facilities Survey
10 | P a g e
The EC/CAFOD PHC programme was developed in collaboration with Catholic Secretariat of Nigeria
(CSN) in recognition of the efforts of the Catholic Church in Nigeria in primary healthcare delivery to
the poor people, especially those living in rural areas of northern Nigeria. The programme was built on
the premise of ensuring equal opportunities of access and improved quality of PHC services in rural
areas where there is little or no coverage by the public services. Improved service delivery is critical to
poverty alleviation which lies at the very heart of the EC programming and more broadly at the centre
of EC-ACP cooperation. Records show that Nigeria, despite being one of the highest producers of
petroleum in the world, remains a very poor country where 70.2% of the population live below the
poverty line of 1 US dollar/day. The country still features on the list of the United Nations Least
Developed Countries, ranking 142 in the UN Human Development Index (2010). Consequently, many
Nigerians are yet to have access to adequate services that meet their basic needs in areas such as health,
water and sanitation, and education.
The EC/CAFOD PHC programme was also in line with the Millennium Development Goals (MDGs).
The nature of the project and its key aim to improve health status of rural poor people via the
promotion of primary health was to contribute directly to the achievement of three MDGs, specifically:
MDG 4 - Reduce child mortality
MDG 5 - Improve maternal health
MDG 6 - Combat HIV/AIDS, malaria and other diseases
The programme also adopted gender sensitive approach in PHC practices to ensure that it contributes to
MDG 3 (Promote gender equality and empower women) and at the same time incorporated the building
of civil society networks of Non-State Actors (NSAs) and promote closer collaboration between civil
society and public authorities.
The specific objectives of the programme were:
1. To increase the capacity of 63 Catholic-Church run primary health providers to manage and
deliver high quality, sustainable health care services in order to reduce the incidence of
maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria
and waterborne diseases.
2. To increase the level of community participation in and ownership of primary health care in
order to promote sustainable, healthy lifestyles.
11 | P a g e
3. To facilitate, capture and share learning between Catholic Church, government, other private
PHC providers and other stakeholders in order to strengthen health networks and promote good
practice in primary health care.
4. To build the capacity of the Catholic Church on health policy and advocacy issues in order to
engage with government and other key stakeholders in the health sector with the aim of
influencing the development and monitoring of pro-poor health policies in Nigeria.
The EC/CAFOD PHC programme used holistic approach to healthcare delivery. On the one hand, it
made efforts to directly contribute to the improved provision and responsiveness of health services in
low-income communities, and on the other hand, it carried out activities geared towards building
advocacy and lobbying skills and initiatives for developing and monitoring policies that enhance equal
access to health services. The expected outcomes as a result of the interventions of the programme
included the following:

Strengthened capacity of 976 PHC staff in management and provision of a more comprehensive
range of effective and appropriate primary health care services that meet the health needs of
rural people living in northern Nigeria.

Community members benefit from healthier lifestyles as a result of behaviour change and
taking greater ownership of their own primary healthcare.

Improve practice in primary health care as a result of increased collaboration, sharing,
coordination and learning between/amongst a diversified range of health actors and
stakeholders.

The health structures of the Catholic Church are more pro-active in engaging with the
government, receive greater government recognition and support, and are able to contribute to
the formulation, implementation and monitoring of health policies, which reflect the needs of
the rural poor in Nigeria.
1.2 Objectives and purpose of the final evaluation:
The main objective of this evaluation was to determine the extent to which the programme has
achieved its stated objectives. The evaluation assessed the appropriateness and effectiveness of the
design and implementation of the programme and how the programme has built the capacity of 63
Catholic Church-run PHC providers to manage and deliver high quality, sustainable healthcare
services. See the Terms of Reference (TOR) for the evaluation in appendix 2. This evaluation will
12 | P a g e
enable CAFOD and its partners to take stock of achievements of the project and provide a learning
opportunity to improve future programming. The results of the evaluation will be useful as an advocacy
tool by the Catholic Church to mobilize resources locally for the continuity of the programme activities
and provide major inputs for designing of similar interventions in future.
2. Methodology
The final evaluation for EC/CAFOD PHC programme was conducted between 7th of April and 30th of
May, 2011. A combination of qualitative and quantitative methods was used to elicit relevant
information from different sources. The qualitative methods included focus group discussions (FGD)
and key informant interviews (KII) while quantitative method involved the use of semi-structured (exit)
interviews and facility checklist. The sampling procedure adopted involved stratified and systematic
random methods. To select the facilities to be visited for the evaluation, the 63 facilities were stratified
by provinces (Abuja, Jos and Kaduna) and a systematic process was used to select 10 facilities. At the
end of this process, the following facilities were selected: Ankpa, Agagbe, Adikpo, Yakoko, Jada,
Nassarawa, Kura falls, Malumfashi, Abuja HIV and Mafo fadiya. However, due to logistic reasons and
peculiar circumstances of some of the selected facilities, the following replacements were made:
Adikpo PHC with Adikpo/Vandeikya HBC; Nassarawa with Namu; Kura falls with Mandella;
Malumfashii with Fuka and Mafo fadiya with Dogon kurmi. The replacements were made purposively
based on advice from CAFOD staff to ensure coverage of all programme components while the
provincial spread of the sample was still maintained.
The range of persons interviewed with KII included CAFOD Programme Manager, CSN Director of
Church and Society, CSN HIV Coordinator, Diocesan Health Coordinators, PHC Managers,
Community leaders, Diocesan HIV Coordinator and FGD with group of PHC staff in 9 facilities, group
of VHC in 5 facilities and group of TBA in 6 facilities. In all, a total of 22 KII and 20 FGD were
conducted. In addition, 53 clients exit interviews were conducted and the facility check list was
administered in 9 out of the 10 selected facilities (excluding Abuja HIV). Table 1 shows the breakdown
of the interviews conducted.
13 | P a g e
Number of persons interviewed (planned versus actual)
Respondents
Method Planned
Conducted Remarks
CAFOD staff
KII
3
1
2 other key staff had disengaged
CSN staff
KII
2
2
The 2 persons interviewed were not the
staff who implemented the program
directly.
Diocesan Health KII
10
7
Shendam HC doubled as Namu PHC
Coordinators
manager, Vandeikya and Agagbe have
same HC while Jos HC was not available
for interview
PHC Managers
KII
10
9
Abuja HIV has no PHC manager designate
HIV Coordinator KII
1
At Abuja HIV
PHC Staff
FGD
10 groups
9 groups
PHC staff not available at Abuja HIV
Community
KII
10
4
Not available for interview at 6 sites
Leaders
(Abuja, Mandella, Jada, Namu & Dogon
kurmi). Community leaders at Jada and
Yakoko declined interview.
TBA/VHW/VHC FGD
10
6
Not available for interview at 4 sites
(Mandella, Abuja HIV, Fuka & Namu)
PHC Clients
Exit
90
53
Number able to interview among clients
interview
attending clinics on evaluation days.
Table 1:
In order to permit triangulation of information from multiple sources, collection of data involved three
basic approaches including: desk review of project documents, direct interviews and observations
during site visits. The evaluation process which lasted for about two months involved different stages
including:
a. Meeting with the CAFOD Programme Manager at CAFOD office in Jos on 07 April, 2011 for
briefing on the project activities, explanations on TOR, agreement on evaluation methodology,
sampling process and signing of Contract letter. The work plan and itinerary for the evaluation
were also adopted during this meeting. See annex 2.
b. Desk review of project documents and development of data collection tools: An extensive
review of relevant projects’ documents was conducted to familiarize with the project
implementation strategies, areas of focus, routine monitoring reports and findings of previous
reviews. The list of documents reviewed included: the Programme proposal and log-frame,
evaluation TOR, various reports (partners meeting, quarterly narrative, final narrative,
workshop /training), 2009 midterm review report and the report of 2005 CAFOD PHC review.
Based on the results of the review, a set of data collection tools was developed and shared with
14 | P a g e
CAFOD staff for comments and inputs. The tools were later finalized after incorporating
relevant comments/inputs from CAFOD staff.
c. Site visit and data collection: The actual site visits and data collection exercise were conducted
between 2nd and 13th of May 2011. Two evaluation teams were formed with each team
(comprising a consultant and a field assistant) visiting 5 facilities. Activities conducted during
site visit included KII and FGD interviews with different range of respondents as listed above,
exit interviews with randomly selected clients who received treatment at the facility on the
evaluation day using semi-structured questionnaire and assessment of staff, equipment and
commodities currently available in the facilities. All KII and FGD were tape recorded. Each
team moved from one facility to the other, ensuring that data collection was completed at a
facility before moving to another.
d. Data analysis and report writing: After the site visit, all data collected by the two teams were
collated. Quantitative data (exit interview and facility checklist) were entered into computer and
analyzed with SPSS and EXCEL respectively while the qualitative data (recorded KII & FGD
interviews) were transcribed verbatim and analyzed thematically. The evaluation report was
written in line with the format recommended by CAFOD in the TOR.
Limitation of the evaluation methodology
The methodology adopted for this evaluation was influenced by the following considerations:
1. The evaluation used more of qualitative method because there was no structured baseline
evaluation before the commencement of the EC/CAFOD programme. Hence, it was difficult to
assess level of achievement of some targets in quantitative terms because there were no
benchmarks on which comparison could be made.
2. The scope of the evaluation was limited in terms of sample size, range of persons interviewed
and activities carried out at the sites because very limited budgetary provision was made for the
evaluation.
3. Data collection at CSN and CAFOD offices was limited because most of the key staff that
implemented the programme had disengaged before the evaluation.
However, these limitations did not significantly affect the quality and findings of the evaluation.
15 | P a g e
3.0 Major Findings and Analysis of the Programme Outcomes
The key findings of the evaluation are presented under each of the four programme objectives in order
to assess the level of achievement of each objective and evaluate the relevance, effectiveness and
impact of interventions carried out.
3.1 (Objective 1): To increase the capacity of 63 Catholic-Church run primary health providers to
manage and deliver high quality, sustainable health care services in order to reduce the incidence of
maternal and childhood illnesses, and other preventable diseases, including HIV/AIDS, malaria and
waterborne diseases.
The first objective of the EC/CAFOD PHC programme was pursued using four main approaches: (1)
Training of staff (2) Provision of essential capital items & equipment (3) Provision of technical
supports to the PHC staff (4) Provision of running cost to selected facilities. Each of these approaches
is evaluated below:
3.1.1 Training of Staff
The programme proposed to conduct a total of 31 trainings/workshops where a total of 1030 persons
will be trained. The break-down of the training schedule showed that 63 PHC managers, 20 diocesan
health coordinators and 10 HIV coordinators were expected to be trained on managerial issues while
378 PHC health staff were to be trained on PHC best practices. The 2011 draft final narrative report
showed that a total of 27 training/workshops were conducted through which a total of 809 persons were
trained (note that in many instances, same person attended more than one training) representing 79%
achievement. Figure 1 shows the number of persons trained against what was planned while table 1
shows the breakdown of the types of trainings conducted and total number of persons trained.
16 | P a g e
Figure 1: Actual number of persons trained versus targets
Generally, the training contents were appropriate and relevant to the objectives of the
EC/CAFOD PHC Programme. Adequate number of days was allocated to each training and
delivery/facilitation methods adopted were appropriate for the audience. However, in some cases, the
training modules standards were higher than educational levels of some of the participants which
affected the rate of understanding and ability to step down the training to other staff. This was because
some PHC facilities in rural areas were unable to attract staff with required educational qualifications.
For example, Fuka PHC Manager reported that she sometimes found it difficult to get staff with
required educational qualifications to attend some trainings. This resulted in a situation where staff
with less educational qualifications was sent to attend some trainings or a situation where same staff
attending almost all of the trainings. It is important for the PHC and dioceses to develop strategies of
attracting staff with required educational qualifications to make trainings more effective.
During the interviews, all of the respondents including diocesan health coordinators, PHC staff
and managers confirmed that they attended different types of training under the programme either
directly or through a step-down. The most frequently mentioned among the trainings attended by the
17 | P a g e
respondents were trainings on HMIS, rational drug use, programme and financial management, gender
and health care provision and stepping stones methodology.
The respondents found the trainings useful and reported that the knowledge and skills acquired
had helped them to deliver quality services to their clients. The trainings had also improved the over-all
ability of the PHC to deliver sustainable quality primary health care services in their communities.
Some of the significant changes in practices of the PHC facilities attributable to the trainings were
enumerated by the PHC staff to include the following:
i.
Improvement in data collection, record keeping and reporting: All of the 9 PHC facilities
(excluding Abuja HIV) visited by the evaluation team had basic records (including registers for
ANC, Delivery, Postnatal, Immunization, HIV testing etc) in place and were updated. Assessing
the benefits from the HMIS training, a PHC staff said:
“When the project (EC/CAFOD Programme) started, there was no formal way of
writing a report. We had challenge in terms of knowing what to report because there
was no structured reporting format,...there was no data collection. After M&E (HMIS)
training a tool was provided which has the indicators which guided us in writing our
reports and helps to make it clearer and easier to put together” Another PHC staff
reported “The training of the health management information and PPRHAA has helped
me to be able to keep proper record and statistics of the patients here”.
ii.
Improvement in quality of drug use: All PHC managers interviewed confirmed attendance of
training on rational drug use. The training has changed the practice of poly-pharmacy to
accurate prescription based on diagnosis. Most PHC reported that they started using standard
manual for prescription after the training.
“There was training on rational drug use in Jos. Which help us a lot because we used to
just prescribe many drugs to patients thinking we are raising money for the clinic. But
we were made to understand that except in a critical condition, we are not supposed to
prescribe two antibiotics at the same time for a patient”- A PHC manager
Table 2: Trainings/workshops
S/N Name
training/workshop
1 Health management
information systems
18 | P a g e
conducted
of Type of participants
HC, PHC Managers,
HIV Coordinators
No
of Date
training
participants
conducted
86 June 18-21, 25-29;
July 2-5 2007
S/N Name
training/workshop
of
Type of participants
2 Health &
Organizational Strategic
Planning
3 Core Management
Function Training
4 Basic financial
management Training
HC, HIV
Coordinators,& PHC
Managers
Health Coordinator &
PHC Managers
HC, HIV Cord, PHC
Managers & Financial
Officers
HIV Coordinators &
PHC staff
93
HC, PHC Coordinators,
PHC Managers & PHC
Staff
HC, HIV & PHC
Coordinators, Provincial
Health Advisors & PHC
Staff
Both Male and female
Religious Leaders
84 Aug 11 -15 & Sept.
1 - 5 2008
5 Stepping Stones
Training of Trainers’
workshops
6 Rational Drug Use and
Treatment Protocol
7
Peer and Participatory
Rapid Health Appraisal
and Action (PRHAA)
8 Awareness Raising
Workshops on
HIV/AIDS for
Religious Leaders
9 Staff/Human resource
and team management
10 Natural Family
Planning &
Reproductive Health
11 Financial Sustainability
strategy training
24 Sept- 2November 2007
66 February 6 2008
81 April 1 - 17 2008
48 June 19 -19; July 7 17 2008
26 November 28 December 3 2008
110 January 14-15; 2021 2009
Health Coordinators,
PHC Managers and
Provincial staff
Nurses and women
community leaders
78 March 23-27; July 610 2009
HC, HIV Coordinator,
PHC Managers &
Health Advisor
Nurses, SCHEWs,
JCHEWs and TBAs and
VHWs
HC & PHC Health staff
61 April 20 - 21 2010
12 Training of Trainers
workshop on TBA and
community obstetric.
13 Gender and Health Care
Provision
14 Community TBA
VHWs and TBAs
Training
15 Drug Revolving Fund in PHC Managers
Primary Health Care
16 Stepping Stones
HC, HIV Coordinators
Awareness Workshop
& PHC Managers
Source: Compiled from Annex VI Final Narrative Report
19 | P a g e
No
of Date
training
participants
conducted
70 June 22 - 26 2009
50 June 14 - 18 2010
37 July 19 - 23 2010
195 July, Dec. 2010;
January2011
29 December 7 - 10
2010
74 October 22- 25;
Nov. 19 -29 2007
iii.
Better obstetric practices: PHC staff and TBAs reported that they now have improved
knowledge and skills in conducting antenatal care and taking deliveries. The training of TBA
has enhanced referral network between the TBA and PHC, thereby reducing maternal mortality
due to complications during home deliveries.
iv.
Other changes in PHC practices associated with the trainings received as mentioned by the PHC
staff and managers include: better project management style, effective pre and post HIV test
counselling, better financial management, improved working relationship and better leadership
styles, better drugs stock taking and use of essential drugs for prescription, cost effective drug
procurement and prescription for clients and effective community mobilization to utilize PHC
services.
However, despite the numerous benefits and positive changes in PHC practices associated with the
trainings/workshop conducted, the following limitations were observed:
a. Due to low educational level, some PHC staff could not benefit maximally from the trainings.
Some of them could not express themselves and their ability to assimilate training contents,
implement the knowledge or step it down to other staff is in doubt.
b. Many of the staff trained under the programme were no longer in the facilities due to high staff
turn-over in search of greener pasture. Some of the new staff employed to replace those that
resigned had not been trained.
c. Despite reported implementation of rational drug use by all PHC managers, no standard
protocol was sighted in consultation rooms of the PHC facilities visited.
d. Although HMIS was in place in all facilities visited, one of the facilities (Fuka) was using hand
ruled note books for essential records such as ANC, delivery, immunization, drug dispensing
and laboratory records due to stock out of standard forms and registers while Abuja HIV was
unable to produce their M&E forms and registers for sighting by the evaluation team.
3.1.2 Provision of Essential Capital Items and PHC Equipment
Another capacity building approach adopted by the EC/CAFOD PHC programme was the provision of
essential capital items and equipment to the PHCs. The 2011 final programme narrative report showed
that a total of 850 of 10 different items and equipments were planned to be distributed to some selected
facilities based on need. However, analysis in table 3 shows that a total of 830 items and equipment
were given out, representing 97.6% achievement of the target.
20 | P a g e
During the site visit, PHC managers and staff confirmed that various items and equipment were
received from CAFOD and were sighted by the evaluators. In all cases, the items and equipments had
been installed and were being used to deliver sustainable services to their clients. There were verbal
evidences to show that the items and equipments had improved the quality and efficiency of services
being provided in the facilities. Various instances of improvement in service delivery as due to these
items and equipments were mentioned by the PHC managers and staff. For instance:
i.
Solar panel installed at Namu PHC has increased the level of patronage of the facility. One of
the PHC staff reported that: “The solar energy has brought about a total change to the facility.
Patients are happy with the fact that there is 24 hours light in the hospital. So, they prefer to
come here”
Table 3: Capital
2
items and equipment planned and distributed
Items and Equipments
Pre
Plan Revised Number
Plan
vs
Implementation Prior
to distributed actual (%)
Plan
Implementation
Vaccine
Cold
box
60
60
53
88
carrier
Microscope
30
30
30
100
3
Adult weighing scales
30
30
30
100
4
Manual infant suction
machine
30
30
30
100
5
Autoclave
30
30
25
83.3
6
7
Electric refrigerators
Solar
Vaccine
refrigerators
Solar Lighting systems
Bicycles
Delivery and health
Kits
TOTAL
20
20
10
15
10
9
100
60
10
300
335
10
300
335
8
300
335
80
100
100
865
850
830
97.6
S/N
1
8
9
10
ii.
Provision of autoclave in Adikpo/Vandeikya HBC has improved sterilization procedure and
reduced the possibility of infections to patients.
iii.
Provision of Microscope in Fuka has improved the quality of diagnosis and enhanced ability of
the staff to attend to more patients.
21 | P a g e
iv.
Distribution of health (delivery) kits to Dogun/Kurmi PHC and TBA has promoted best
delivery practices, minimised infections and reduced maternal and neonatal mortality.
v.
Vaccine cold box carriers distributed to Dogun/Kurmi PHC have improved vaccination
activities as cold chain is adequately maintained and increased uptake of immunization.
vi.
Provision of bicycles has aided community outreach activities at Adikpo/Vandeikya HBC,
Dogon/Kurmi and Mandella and has enabled the PHC staff to take health services to hard-toreach communities.
Few challenges reported in relation to the provision of capital items and equipments include:
a. Late distribution of some equipment to the PHC. For instance, the last batch of bicycles was
distributed in April 2011 while some dioceses had not collected their allocations from CAFOD
office as at the time of this evaluation.
b. Some PHC facilities (for example, Mandella) did not understand the criteria for selecting
facilities that were given capital items/equipment and felt excluded.
Early procurement of equipment is recommended to ensure optimal use during the life of the
programme and criteria for selection of benefiting facilities should be clearly explained to all
stakeholders.
3.1.3 Technical support to PHC staff
The capacity building approach of the EC/CAFOD PHC programme also made provision for the
recruitment of 3 Provincial programme accompaniers, 2 Technical Programme Accompaniers for
HMIS and Finance, 1 Programme Administrator and 1 Health and Policy Advisor to provide technical
supports to the PHC staff in specialized areas such as HMIS, finance, gender and advocacy. Desk
review of project documents showed that three Provincial Accompaniers, two Technical
Accompaniers, one Programme Administrator and one Health Policy & Advocacy Advisor were
recruited in the course of the programme implementation. However, most of these staff disengaged
their services at different times before the end of the programme. Table 4 shows dates of recruitment
and disengagement of Program Accompaniers and Advisors. During the interview, the CAFOD
Programme Manager attributed the high technical staff turn-over to inability of some of the staff to
successfully go through probation period, personal reasons and securing new jobs towards the end of
the contract with the staff. As at the time of this evaluation, only the Programme Manager was still on
ground at the CAFOD office.
22 | P a g e
Table 4:
Program accompaniers and advisor recruited with appointments and disengagement
dates
First Appointment
S/N
1
2
3
4
5
6
7
Position
Provincial
Accompanier (Abuja)
Provincial
Accompanier (Jos)
Provincial
Accompanier
(Kaduna)
Technical
Accompanier
–
Finance
Technical
Accompanier - HMIS
Programme
AdministratorCAFOD
Health Policy and
Advocacy AdvisorCSN
Second Appointments
Date
of Date
of Date
of Date
of
Appointment Disengagement Appointment
Disengagement
April, 2007
January 2011
-
April, 2007
July 2009
January 2010
April, 2007
July 2009
-
April, 2007
July 2007
May 2008
June 2010
April, 2007
January 2008
May 2008
May 2009
April, 2007
November
2007
May 2008
January 2011
April, 2007
October 2007
August 2008
December 2009
January 2011
In terms of effectiveness and impact, this approach of capacity building appeared not successful. Most
PHC managers complained of not benefiting enough from technical supports of the Accompaniers as
they only had them in their facilities only a few times through-out the duration of the programme. Desk
review of the end of project narrative report revealed that a total of 369 out of the planned 810 visits
(46%) were carried out by both Provincial and Technical Programme Accompaniers within the
programme implementation period.
The reasons for the low performance of the Technical
Accompaniers could be attributed to high staff turn-over. However, both the PHC staff and managers
commended the Programme Manager for doing a good job in regular visiting and providing technical
supports and guidance in programme implementation and management. Staff recruitment process and
condition of service for technical staff should be reviewed for future programming in order to attract
and retain qualified personnel.
23 | P a g e
3.1.4 Provision of financial supports for running cost to selected facilities
Another capacity building approach adopted by the EC/CAFOD PHC programme was the
provision of running cost to 20 selected “vulnerable PHC clinics”. The aim of this approach was to
increase the financial capacity of the selected PHCs to continue to provide sustainable PHC services to
the community by giving them financial support to meet the overhead cost including cost of local
travel, drugs and consumables. Five out of the 20 PHC clinics (Abuja HIV, Agagbe, Ankpa, Yakoko
and Namu) that received financial support were visited by the evaluation team. The PHC Managers of
the five facilities confirmed that the financial support from EC/CAFOD Programme helped them to
carry out various activities (such as outreaches and community mobilizations) and meet their overhead
costs.
Performance indicators to measure success of the financial support approach stipulated that 75% of
partner PHCs improved their financial management and CAFOD’s contribution to core costs of
partners has reduced by 60%. The final narrative report revealed that the overall level of partners’
dependency on the EC/CAFOD grant reduced from 38% in 2007, to 36% in 2008 and to 30% in 2009.
The analysis further showed that 13 vulnerable PHC clinics were able to reduce their grant dependency,
two showed no difference in dependency while six showed increase in their dependence on the grant.
Generally, average dependence on EC/CAFOD funds by facilities was less than 50%. There was a
slight drop of dependence on EC/CAFOD from 49% to 44%, while facilities like Abuja HIV depended
solely on EC/CAFOD from the beginning to the end of programme. Other PHC facilities like Namu
dependency dropped to almost zero by the end of the programme.
24 | P a g e
Figure 2: Sustainability
and dependence of Facility on EC CAFOD Funding
3.1.5 Development of a gender focal points network
The programme planned to establish a gender focal point network at provincial level to work with a
cluster of PHC providers from each diocese to encourage the discussion and implementation of models of best
practice on gender and healthcare both inside the PHC facilities and in the outreach work of PHC staff. Gender
focal points were expected to play a specific role regarding the identification, development and implementation
of gender sensitive approaches in PHC practice which was to lead to the development of guidelines on models of
best practices on gender and health care in the specific context within which partners operate.
An attempt to develop and implement gender sensitive approaches in PHC practice started with the training of
37 PHC staff on Gender and Healthcare Provision in the context of primary healthcare to raise
awareness on gender issues. Realising the strong influence of men on nutritional intake of women,
some facilities reported that they targeted males (especially husbands) in nutritional education for
pregnant women as part of their outreach work. Some other facilities said they tried to be gender
sensitive by separating female and male wards. Few others reported that they ensured that staff of same
gender as the clients attend to some private medical issues.
However, there were no specific guidelines on models of best practices on gender and health care in any of
the facilities visited and there was no record to show that gender focal points were established at
provincial level. A PHC staff said: “There is no specific policy towards that issue but we are being
25 | P a g e
encouraged to ensure gender balancing between male and female to equally reflect in that”. The level
of gender sensitivity in primary healthcare delivery was low in the facilities visited. A PHC staff
reported that “Both male and female nurses receive labour and delivery without any complaints from
the patients”. There were reported cases of patients complaining about gender insensitivity in service
delivery in some facilities. A male JCHEW recounted his experience: “.... like in ANC I remember
some weeks back when this woman (referring to a female CHEW) was not around I was in charge of
this place, some women were complaining that why should a man come and see....... I tried to educate
them that either a man or a woman we both received the same training. That anything we discuss will
just end here they will not hear it anywhere. But I don’t think there is anything set aside to cater for
that problem”
The observed misunderstanding of PHC staff on gender issues and misapplication of the gender
sensitive approach are attributable to non existence of the guidelines on models of best practices on gender
and health care and weak follow up and technical support after the sensitization workshop. The development of
the guidelines on models of best practices on gender is urgently required to help the PHC in application of
gender sensitive approaches.
3.2 (Objective 2): To increase the level of community participation in, and ownership of primary
healthcare in order to promote sustainable, healthy lifestyles
It was expected that community members would benefit from healthier lifestyles by taking greater
ownership of their own primary healthcare and participate actively in its activities. To achieve the
second objective of the EC/CAFOD PHC programme which focused on community participation and
ownership, the following approaches were adopted:

Community mobilization activities

Establishment of village health committees

Training of village health workers and TBA

Publication and distribution of IEC materials

Participatory Health Assessment by Rural Communities

Awareness raising workshop on HIV & AIDS for religious leaders
26 | P a g e
3.2.1 Community mobilization activities
Community mobilization for health care services was concerned with encouraging people to feel
committed to working together for the benefit of themselves and their neighbours. To achieve effective
community mobilization, EC/CAFOD Programme conducted different trainings (including stepping
stones, IEC and Participatory methodologies and awareness raising workshops on HIV/AIDS and
Human Sexuality) targeting PHC staff, religious leaders and community members. This was to enhance
their capacity in community mobilization and create awareness on HIV/AIDS and primary health care
services (as reported in section 3.1.1). The trainings aimed at promoting healthier lifestyles, improving
communication and relationships, and bringing about behaviour change among the target groups and
the wider communities.
According to the EC/CAFOD programme log frame, 90 PHC staff, Diocesan Health Coordinators and
Diocesan HIV Coordinators were expected to be trained in the Stepping Stones methodology, 60 PHC
staff were to be trained as trainers of Stepping Stones methodology while at least 1200 community
members from 75 communities would participate in Stepping Stones workshops. However, desk review
of reports showed that 74 PHC staff, Health and HIV Coordinators were trained on Stepping Stones
methodology (82% achievement) while 50 PHC staff were trained as trainers (83% of the target). The
number of communities and community members that participated in Stepping Stones workshops could
not be ascertained. The log frame also listed the expected outcome of this activity to include 40%
increase in number of PLHIV support groups established in communities where Stepping Stones is
being used and 30% increase in number of volunteers working with PLHIV communities where
Stepping Stones is being used. The achievement of this outcome could not be assessed because there
were no baseline figures on the indicators.
Apart from the trainings, PHC facilities embarked on HIV/AIDS awareness campaign and mobilization
for other PHC services including childhood diseases, immunization for babies and vaccination for
pregnant mothers using Stepping Stones methodology. A PHC staff summarized community
mobilization activities as: ‘critical areas of positive health impact were nutrition, immunization of
children against preventable diseases, malaria prevention through the use of mosquito nets, quality
27 | P a g e
drinking water using water-guard, improved sanitation both personal (e.g. hand washing, bathing)
and environmental (by keeping surrounding devoid of over-grown grasses and stagnant waters)”.
Continuous engagement with the communities on HIV/AIDS has created positive impact through
increased awareness and changes in behaviours of the community members. For instance, the end of
programme narrative report indicated that families within Vandeikya/Adikpo HBC catchment now
provide care and support to members who are HIV positive and PLHIV no longer hide their HIV serostatus. Namu PHC also reported an increase in awareness of its local communities on HIV voluntary
counselling and testing and on prevention of mother-to-child-transmission. This was evident by 39%
increase in number of pregnant women tested (from 300 in 2008 to 764 in 2009). Increase in uptake of
HIV testing by pregnant women also has implication for the HIV status outcome of their babies
because those whose status was discovered to be positive will have access to preventive care/treatment
and they are most likely to deliver HIV free babies.
The community mobilization efforts had also resulted in increased community participation and
ownership. For example, Ankpa PHC reported that a community provided a piece of land and made
financial contribution to construct a PHC outreach centre where the PHC staff and VHWs can provide
primary healthcare services. Similarly, Bitako community under Yakoko PHC built a health post which
is now being used as mother and child clinic as well as providing other primary health care services.
Community enlightenment on environmental sanitation was conducted to improve the hygiene
practices of the community members and to reduce incidences of communicable diseases such as
diarrhoea, malaria, cholera and skin infections. During the interviews with the PHC staff, they
emphasized the successes they have recorded through sanitation campaigns to include improvement in
environmental sanitation of the targeted communities, better hygiene behaviour and safer lifestyle of
the people. Other achievements included community behavioural change such as increased utilization
of PHC services (e.g. immunization uptake, antenatal clinic attendance), access to quality drinking
water through the use of water-guard (courtesy of Catholic Relief Services), increased participation in
Village Health Committee and reduction in harmful cultural practices. The knowledge acquired from
the stepping stones training was very useful in community mobilization activities. Some comments
during FGD and KII were:
28 | P a g e
‘Community members now take the issue of sanitation serious. When we go back to some of these
villages we see that the hygiene level has increased’, a PHC staff at Fuka.
‘With the use of Stepping Stone approaches the Benue programme communities have reduced the level
of men cheating on their wives and wife inheritance is reduced due to improved knowledge of HIV’Adikpo/Vandeikya HBC member.
3.2.2. Establishment of Village Health Committees (VHC)
It was planned that every PHC would establish at least one VHC to represent and mobilise the local
community. Out of the 10 facilities visited, VHC had been established and were effective in eight
(80%) PHC facilities while two facilities (Mandella and Abuja HIV) had no VHC in place as at the
time of evaluation. The Mandella PHC Manager reported that establishing VHC was not possible
because the recent ethno-religious crisis had resulted in suspicion among community members. Abuja
HIV on the other hand established no VHC but worked directly with the community.
Interviews with the PHC staff and VHC members revealed that VHC, where they existed, had been
effective in representing and mobilising the local community to create awareness for and uptake of the
PHC services. Regular meetings were held between VHC and respective PHC staff to discuss
community health issues such as outbreak of childhood infectious diseases (e.g. measles, diarrhoea),
plan strategies for provision of specific health services like outreach immunization activities, decide on
community outreach dates and information transmission to the communities. Other functions that were
mentioned included the notification/reminder of the community members on dates of health post
clinics; and participation in tracking clients who need a follow up in the facility.
The PHC Managers and staff reported during FGD and KII interviews that VHC had been very helpful
in mobilizing community members to utilize healthcare services and provide feedback on clients’
satisfaction and complaints to the facilities. ‘The VHC members carry out some supportive activities
such as give health talk, support in weighing children during CWC and also confirmed that the VHC
provides information on outbreak of infectious diseases in the community’- Yakoko PHC manager
reported.
One of the expected outcomes of this activity was that there would be an increase in participation of
women in VHC and community health promotion initiatives. The desk review of reports showed that
29 | P a g e
there was an increase of about 45% between 2007 and 2010 in the proportion of women participation in
the VHC. The end of project narrative report further showed that VHC existed in 120 out of the 291
PHC stations (representing 41%) while females constituted about a third (34%) of the membership. The
average yearly meeting of the VHC was about 6 times (once in two months).
Table 5: Women
Abuja
participation in Village Health Committees
Number of Presence Composition of
outstations
of VHC
VHC
Male
Female
135
47
212
96
#
of
meeting
held
446
Kaduna
94
42
115
68
98
Jos
62
31
103
60
108
Total
291
120
430
224
652
Province
One major challenge encountered was the misunderstanding of the concept and functions of the VHC
by some members. There were reported cases of VHC not being active because they were demanding
that they should be paid whenever they hold meetings. VHC members should be adequately
enlightened to know that participation in the committee is a voluntary service (that attracts no monetary
gain) to engender community ownership of the programme. The concepts of voluntarism, dedication
and community ownership should be emphasised as part of orientation when new members/committees
are being inaugurated.
3.2.3 Training of Village Health Workers and TBA
Realizing the general acceptance and high patronage of TBAs in most rural communities in northern
Nigeria, EC/CAFOD PHC programme mobilized and trained a pool of VHWs and TBAs to improve
their skills on basic obstetric care, identification of pregnant women at risk and refer same, use of IEC
materials and participatory methodologies conducted in the local languages. Thirty six (36) PHC staff
and 14 TBAs were trained as trainers which later stepped the training down to 195 TBAs and VHWs
across 16 PHC facilities. The number of TBA trained represents a 66% achievement of the 315 target.
The training improved the knowledge and skills of TBAs in conducting safe home deliveries using
sterile procedure. The approach by which the PHC staff were trained to step it down to TBAs using
30 | P a g e
local dialect facilitated better understanding. In appreciation of the training a TBA in Agagbe said, ‘the
training has not only built our knowledge and skills but has given us confidence in our practice’ The
improvement in the skills of the TBAs as a result of the training were listed by both the TBAs and PHC
staff to include: improved knowledge on home delivery practices by the TBAs; identification of
complications during pregnancy, labour and delivery for referral to the PHC; decrease in maternal and
neonatal deaths; counselling of mother on antenatal services attendance; nutrition and appropriate
dressing in pregnancy and personal and environmental hygiene.
In addition to the training, VHWs and TBAs were supported with relevant items and equipment such as
health and delivery kits, IEC materials and bicycles to enhance their work in the community. The TBA,
VHW and PHC staff expressed appreciation and satisfaction with the supply of the health and delivery
kits during FGD and KII interviews. Most of the persons interviewed on the activities of TBA reported
that the training of TBA had improved the quality of their service delivery. A VHC member in Agagbe
reported that: ‘Before EC/CAFOD the TBAs were using the edge of a piece of unwashed bamboo stick
to severe the cord from the baby (mother) and dress the stump with sand. However, this is now history’.
3.2.4 Publication and distribution of IEC materials
The programme planned to train 105 persons on the Use of IEC and Participatory Methodologies and
produce Health/Training Kits and IEC materials that would be appropriate to the needs of the target
groups and translate the IEC materials into 4 local languages. The programme reports showed that a
total of 81 participants attended the workshops on the Use of IEC and Participatory Methodologies,
representing 77% of the target. Most PHC facilities visited during the evaluation exercise
(Vandeikya/Adikpo HBC, Fuka, Namu, Ankpa and Yakoko) reported that they produced IEC materials
and translated some of the IEC into local languages. Namu PHC was provided with technical and
financial supports by the programme to produce IEC materials on HIV/AIDS which was used during
World AIDS Day. The initiative and message came from the support group members during an
exercise on assessing the quality of life of members and quality of HIV programme.
In 2010, 500 copies of Health/Training Kits and IEC materials for TBAs developed by the National
Primary Health Care Development Agency were reprinted in English and distributed to 19 dioceses.
Thirteen PHC facilities namely Ankpa, Gambar, Namu, Vandeikya, Agagbe, Mutum Biyu, Takum,
31 | P a g e
Yakoko, Shuwa, Mafo Fadiya, Fuka and Malumfashi used the IEC materials to train VHWs and TBAs
in the communities.
3.2.5 Participatory Health Assessment by Rural Communities
The programme planned to complete 6 participatory rural communities’ health assessments to inform
the programme advocacy strategy and expected that at least 30 local authorities become more aware of
key health issues for rural communities. This was aimed to generate models of best practices within
PHC service provision and most importantly to identify key priorities to take forward in terms of
advocacy with local and state governments.
For quality results, a decision was taken during partners meeting to adopt the Catholic Secretariat of
Nigeria Peer Participatory Rural Health Assessment (PPRHAA) training methodology conducted for
Lagos province. This assessment tool involved two stages: the training of Diocesan Health
Coordinators from selected dioceses and second, the actual assessment of selected rural communities
and PHCs by the trained health coordinators and their staff.
The desk review of reports indicated that training for Diocesan Health Coordinators and PHC managers
was conducted in 2010. During the training, 4 rural health assessments were carried out as hands-ontraining in Mandella, Zawan, Zambina and Zonkwa. The assessment critical areas were: (i) potential
barriers to access (ii) satisfaction with care (iii) drug availability, quality and information (iv) staff
attitudes and behaviour; (v) environment and hygiene (vi) community participation in the affairs of the
facilities using focus group discussion methodology with the communities. Forty-four persons (20
males and 24 females) participated in the assessment. The end of project narrative report further
showed that 30 local authorities became more aware of key health issues for rural communities,
suggesting 100% achievement of the set objective. The key achievement of the hands-on-experience on
PPRHAA was that the participating communities were able to identify strengths and weaknesses of
health facilities in their communities.
3.2.6 Awareness raising workshop on HIV & AIDS for religious leaders
32 | P a g e
In order to effectively provide the required support to the PLHIV and their families, the EC/CAFOD
PHC programme adopted and conducted workshops for the priests and the religious on HIV and AIDS
to improve their capacity in contributing to the fight against the pandemic. The workshops’ focus was
on mobilising communities to create awareness on HIV and other health related issues. The expected
outcome of the awareness workshop was that 60 % of religious leaders attending the HIV and AIDS
awareness raising workshops become committed to fighting HIV and AIDS stigma in their
communities.
Desk review of the programme reports indicated that a total of 110 priests and religious attended the
awareness workshops of which about half of the participants were priests. Although there was no
record of the actual number of religious leaders demonstrating commitment to HIV/AIDS fights, some
of the priests who benefited from the workshop have reported demonstrable changes in their sermons
by using correct and sensitive messages and providing counselling to church members and
communities. The end of programme report quoted a priest from Kaduna Archdiocese who sent a text
message saying; “I have come to understand myself much better and I am responding with compassion
and empathy to those who come to tell me that they are HIV positive. Before I used to treat them with
some level of judgement and sympathy which was not helping me to help them as a spiritual
counsellor.”
Though the support for HIV activities was limited to 3 facilities (Abuja HIV, Adikpo/Vandeikya HBC
and Zawan), workshops and trainings related to HIV issues were extended to other facilities as well. It
is important to note that the PHC staff through their dedication and commitment to support the PLHIV
engaged the state and local governments through linkage and collaboration in the HIV activities. This
led to the PHCs (for example, Anpka, Jada, Namu, Fuka) benefitting enormously by having access to
HIV kits, cold boxes, mosquito nets and FGN forms for the returns of HIV test results.
3.3. (Objective 3): Improved practice in primary health care as a result of increased collaboration,
sharing, coordination and learning between/amongst a diversified range of health actors and
stakeholders
In order to effectively achieve objective three, the Programme Accompaniers conducted an initial
assessment of the health information needs of the project during visits to the dioceses. The findings of
33 | P a g e
the assessment provided a direction on areas of programme priorities. The approaches employed under
this objective were:

Production of newsletters and documentation on project learning

Exchange visits to other PHCs within the programme.

Learning and sharing meetings

Production and sharing of best practices

Collaboration between Church run PHC and public PHC.
3.3.1 Production of newsletters and documentation on project learning
The programme was expected to publish 3 newsletters as written forum for exchange of ideas,
knowledge and information. This was expected to be a continuation or follow-on of an already
produced bi-annual Newsletter by CAFOD partners. Two editions of newsletters were planned for year
3, with a third edition for the extension period. However, programme annual reports indicated that 2
drafts were made in the second year of the programme, they were not finalised before the programme
ended. Similarly, the 6 case studies on examples of best practice that were to be produced and widely
shared could not be achieved.
The challenge faced with the production of newsletters and case studies could be attributed to the high
staff turn-over and the fact that the task was not assigned to any specific staff or sub-group but
remained the responsibility of all the players in the programme. Such assignment should be assigned to
a specific committee made of staff members with required skills and commitment.
3.3.2 Exchange visits to other PHCs within the programme
Exchange visits approach was aimed to promote sharing, learning and to give the opportunity to the
programme supported staff to explore possibilities for further collaboration and networking among
PHC providers in the health system. It was expected that at least 36 PHC providers would take part in
exchange visits to other PHCs within the programme. For the PHC staff to achieve this, during the
February 2008 partner’s learning and sharing meeting, partners identified a number of opportunities for
34 | P a g e
exchange learning, which included community mobilisation and participation, TBA training and homebased care, HMIS, rational drug use and drug management .
The Programme Accompaniers also in year 3 introduced 11/2 days intra-diocesan sharing and learning
meetings with the PHC in each diocese as a part of diocesan monitoring visit. These meetings assumed
popularity hence the PHC staff appreciated the opportunity they had to practice in the real life context.
Eleven (30.6%) of the proposed 36 PHC facilities participated in the exchange visit for learning and
sharing with 65 staff and community members (VHC, VHWs and TBAs) as beneficiaries.
These best practices shared with the visiting teams included drugs and data management, community
and maternal and child health activities at Maiduguri diocese; a good community mobilization
methodology for health care activities and data collection at Yakoko PHC. A very laudable concept that
arose from the exchange visits was the collective initiative applied by the matrons and health
coordinators to carry out an objective assessment of Potiskum PHC. The outcome of the assessment
was technical supports provided to Potiskum PHC on reactivating the parish health committee,
encouraging community participation in health activities, quality data management and staff
motivation.
During the final evaluation process, Agagbe, Vandeikya, Fuka, Namu and Yakoko also confirmed their
participation in the inter and intra-diocesan exchange sharing and learning exercise. Agagbe PHC
visited Adikpo/Vandeikya HBC and Fuka; Vandeikya/Adikpo made exchange visit to Namu PHC and
Fuka PHC. The PHC staff highly commended the use of the exchange approach as they said they were
exposed to additional knowledge and skills which has enhanced their primary healthcare practice.
However, despite the opportunity provided by the exchange visits for PHC staff to share experiences
and learn better ways of doing things, only 36% of the target was achieved. PHC staff in some facilities
complained that they did not have opportunity of participating because they did not visit another
facility.
35 | P a g e
3.3.3 Learning and sharing meetings
The learning and sharing meetings approach was designed to provide the opportunity to identify,
develop, share and promote models of best practices in the EC/CAFOD programme. This was aimed to
focus on direct related health issues, health systems and management. Thirteen (13) meetings were held
between programme/partner representatives and other health stakeholders between February 2007 and
January 2011 out of 15 planned (87% of the target). The meetings included: 5 Programme
coordination, learning and sharing meetings; 4 National Diocesan Health Coordinators learning and
sharing meetings and 4 Provincial Health Coordinators learning meetings. Learning and sharing
sessions were held during these meetings with the health coordinators and PHC managers. Some of the
best practices shared during learning and sharing meetings were listed in programme reports to include:

The best use of data and plotting of disease trends and client progress each month on a wall
chart by Fuka, Ankpa, and Shuwa PHCs.

The involvement of community health workers as community contact persons in villages for
mobilization in Yakoko and Vandeikya PHCs. Ikot-Ekpene an ex-beneficiary also participated
in sharing their garnered experience.

Training of community TBAs by Ikot-Ekpene and the resultant maternal mortality reduction.
This huge success attracted the current PHC partners who requested for the training manual and
indicated to visit the facility for interaction with the TBAs and trainers.

Water filtration method carried out by Kuru was also one of the best practices that was
mentioned by all PHC staff during the final evaluation interview. However, the PHC staff and
VHWs in Makurdi, Minna, Kaduna and Shendam dioceses used the water-guard treatment
initiative to educate the clinic beneficiaries in purifying their water. This they said was free and
easily accessible since it was being leveraged from Catholic Relief Services.

Linkages and collaboration with Local and State governments, and other private health facilities
also proved to be best practices as these enhanced effective service delivery through easy access
to immunizations, TB and leprosy treatment, access to effective 2-way referral and continuum
of care in the communities and participation in trainings – shared by Mandella, Fuka, Agagbe,
Ankpa, Yakoko, and Vandeikya and Adikpo/Vandeikya HBC. The advantages of this practice
include cost effectiveness, continuous availability of services (e.g. immunization) and follow up
on clients through a 2-way referral etc.
36 | P a g e
3.3.4 Collection, production and dissemination of up-to-date health information materials
During programme implementation, key health information materials including HMIS forms and
standard treatment protocols were accessed from relevant government agencies, reproduced and
distributed to the PHC facilities. The Programme also took advantage of a workshop organised by the
Federal Ministry of Health in 2007 to access, reproduce and distribute the forms in 2008 to fast-track
the process for the PHC staff rather than allow them wait endlessly at the LGA offices for collection.
Standard Treatment Protocols were also accessed and printed for distribution to PHC staff to enhance
quality of service being provided to the clients. The protocols included: World Health Organization
(WHO) Good Prescription Manual for health care workers and the IMCI flow chart for use in training
PHC staff and display in outpatient department. A copy of syndromic management of Sexually
Transmitted Infections (STIs) was also obtained but there was no permission for re- printing and
distribution.
The adaptation and use of the FMoH data forms has enhanced the collaboration between government
staff and the church health teams through the remittance of data and other information. The 2010
annual programme report noted that the use of FMoH health forms had improved the data reporting rate
while the PHC mangers of Ankpa, Namu, Agagbe, Fuka and Yakoko reiterated during interviews that
government health staff appreciated their prompt submission of accurate data which is accorded
acknowledgement and adequate support through supplies of vaccines and TB and Leprosy drugs.
3.3.5 On-going/Regular dialogue with other PHC and HIV practitioners and stakeholders
The strategic location of the church-run PHC in the rural communities has further created avenues for
more collaboration with other health actors in the community such as private and community owned
health facilities. For record and reference purposes, one of the key indicators was to document such ongoing/regular dialogue with other PHC and HIV practitioners and stakeholders. In order to achieve this
objective CAFOD and partners have made series of visits to national and state government health
ministries and other stakeholders in health care. The programme annual report of 2009 indicated that
CAFOD made visits to FMoH in Abuja in June 2008, shared their training standards and requested for
37 | P a g e
materials on IMCI. Other visits were made to UNICEF office at Abuja and Jos university teaching
hospital’s gynaecological unit to seek and request for training materials for essential obstetric care and
IMCI. These visits helped in establishing linkages and raised awareness of the EC/CAFOD programme
among other key stakeholders. The approach also proved to be cost effective as some standard
documents were received and others were reprinted with the permission as mentioned in section 3.3.4
above.
On-going dialogue was very apt because of the need to ensure coordinated health services for the
clients within the catchment areas. Most facilities reported that collaboration with other PHCs in their
communities was through a 2-way referral and joint training. For example, Namu PHC organized a step
down training of rational drug use to staff of other PHCs in the community. Vandeikya, Agagbe,
Namu, and Fuka PHCs reported that they trained staff of other private PHCs on HIV counselling and
testing and also provided them with test kits.
‘We also collaborate with private hospitals and clinics ....... where we link through a 2-way
referral process for HIV pre/post counselling and we refer to them for ARVs and other critical
ailments’- A Vandeikya PHC staff.
‘Other clinics refer patients here for HIV counselling and testing, and the HIV positive patients
for Anti-retroviral drugs. Sometimes, we refer cases that are beyond our control to other
government hospitals’- A PHC staff at Fuka
It was confirmed during the final evaluation interviews with PHC staff and managers that all of the
facilities visited had linkages and collaboration with either or both Local and State Government
Departments of Health. These collaborations had led to some PHCs facilities being designated as
vaccine collection points or TB (Fuka and Yakoko) and leprosy (Jada) treatment centres. Some PHC
facilities (Fuka, Mandella and Ankpa) also benefited from such collaboration by receiving capital items
such as solar vaccine refrigerators, vaccine carriers and bicycles from their local Department of Health.
PHC facilities also maintained collaboration with their counterparts in government and private health
sector through attendance of meetings, sharing of information/documentation and promotion of joint
initiatives at the local, state and national levels. For example, Kaduna province organized a Health
Week in November 2008 focusing on malaria and other health management issues. The objective was
to enhance learning and sharing among PHCs. In attendance were 35 PHC providers from 7 dioceses of
38 | P a g e
the province. During the FGD and interview with the PHC providers, all the 10 sites expressed
satisfaction with the level of collaboration and linkages with state and local governments and other
private health stakeholders. The on-going dialogue clearly exhibits evidences of a healthy relationship.
It is cost effective (PHC received various items and knowledge at no costs) and has potentials for
sustainability and so should be continued even after the EC/CAFOD support.
3.4: (Objective 4) : To build the capacity of the Catholic Church on health policy and advocacy
issues in order to engage with government and other key stakeholders in the health sector with
the aim of influencing the development and monitoring of pro-poor health policies in Nigeria.
The fourth objective of the EC/CAFOD PHC programme was to build the capacity of the Catholic
Church in health policy and advocacy issues to influence the development and monitoring of pro-poor
health policies. The advocacy component of the programme was developed to meet the challenges and
gaps identified by the need assessment conducted on the primary health care delivery of the Catholic
Church prior to the commencement of the programme. The planned activities to achieve this objective
were:

Creation of Health Information and Advocacy Unit in the Health Department of the Catholic
secretariat of Nigeria

Facilitation and support to the CSN Health Think Tank

Advocacy training and divulgation of CAFOD advocacy tool

Development and implementation of health advocacy strategies

Active engagement with state health decision makers and other non-state key actors
3.4.1 Creation of Health Information and Advocacy Unit in the Health Department of the CSN
The EC CAFOD programme planned that creation of Health Information and Advocacy Unit in the
Health Department of the CSN should be one of the activities under objective four of the programme.
The unit was not created as a stand-alone but there were some activities carried out by CSN while there
was an Advocacy Advisor for the programme. The Advocacy Advisor was engaged on the programme
for less than a year. This really left a gap in the implementation of the advocacy component of the
programme. A lot of the activities for advocacy at the CSN level were on ad-hoc basis and there were
no detailed records of the events. This gave rise to the fact that some of the planned indicators for this
activity were not achieved. For instance, a health advocacy strategy was not defined and implemented
39 | P a g e
for the Catholic Church; the Health Information and Advocacy unit was not formally created and
thereby could not remain effective beyond the duration of the project. The Health Think Tank of the
Catholic Secretariat that was also meant to be producing widely circulated policy and position papers
on health services was not also functional.
It is recommended that this section of the programme should be managed and strengthened with
resources ploughed into recruitment and maintenance of skill staff to run the unit.
3.4.2: Facilitation and support to the CSN Health Think Tank
There was no record of formation of a Health Think Tank at the CSN. However, there were some
advocacy activities organized by the CSN while the engaged Advocacy Advisor was still in office. This
is also one of the gaps identified in the implementation of the EC/CAFOD programme by the CSN.
More efforts need to be put into planning such laudable strategy in future.
3.4.3: Advocacy Training and Divulgation of CAFOD Advocacy Tool
In the course of the implementation of the programme, one advocacy training was organised by CSN
out of the five planned. It was facilitated by CSN’s Health Policy & Advocacy Advisor, and two
consultants. It targeted selected diocesan health coordinators from the 9 Catholic provinces, selected
PHC leaders, and staff of the Catholic Secretariat of Nigeria. Two traditional rulers, a journalist and
one of CAFOD’s Programme Accompaniers also participated. In all, there were 28 participants made
up of 15 women and 13 men. The training methodology and content were technically appropriate and
useful in equipping the participants for appropriate advocacy actions at their dioceses, facilities and
communities. The participants also developed advocacy issues to be followed up after the training.
Evaluation of this component of the programme revealed that lack of continuity in the work of the
Advocacy Advisor constituted a major challenge to follow up of the activities. Hence, the expectation
of having at least 6 examples of advocacy initiatives with local/state authorities and evidence of joint
campaigns with other health stakeholders (as contained in the log frame) was not feasible at the end of
the programme.
In spite of these major challenges in implementing objective four, CSN HIV Coordinator during the
evaluation said some of the dioceses and facilities reported some level of positive outcome of their
advocacy activities carried out. For instance, Namu PHC became an ART outstation of OLA (Catholic)
Hospital Jos, a government approved ART provider thus Namu’s HIV clients (1,884 women, and 1,003
men) were currently accessing ART in Namu rather than having to travel 250kms to Jos every month;
40 | P a g e
Vandeikya/Adikpo HBC programme made a similar arrangement with the Federal Medical Centre
(FMC) in Benue State.
Bali PHC in Jalingo diocese also paid an advocacy visit to their local
government and it resulted in improving the infrastructure and provision of equipment to the facility
especially laboratory services.
This indicated that given adequate human resource, the advocacy strategy would have been more
effective in mobilizing better support for health services for the poor at the communities.
3.4.4: Development and implementation of health advocacy strategies
Since this activity was linked directly to the functionality of other components of the advocacy
strategy, like creation of a function Health Advocacy Unit with a Think Tank, regular advocacy,
trainings and meetings which were dismally implemented in the project, the development and
implementation of advocacy strategy for the Church was not implemented.
It is recommended that in programme planning, activities that are directly linked to other primary
activities should be planned as sub activities and not stand alone. Efforts should also be made to
ensure implementation of both sub and primary activities in order to achieve the desired results.
3.4.5: Active Engagement with State Health Decision Makers and other Non-State Key Actors
In the course of the programme implementation, two (2) health stakeholders meetings were held by the
CSN out of the five (5) meetings planned. The meetings aimed at increasing engagement with state
health decision makers and other key non-state actors in the provision of pro-poor health services at the
facilities and communities. A number of successes were recorded as a result of the advocacy meetings
and activities. These included:
i)
Health stakeholder meetings provided a valuable opportunity to highlight the contribution made
by the Catholic health services to health care delivery throughout the country, and especially to primary
health care. NPHCDA also expressed readiness to work closely with the Catholic health services and
to provide any necessary support. It also created a forum where the issue of the need for health systems
reform in order to discourage ‘skills flow’ and encourage ‘retention’ was discussed with government
actors.
ii)
In Kogi state, the dioceses of Idah and Lokoja increased their level of partnership with the
communities as well as with the state government. This resulted in the allocation of forty million naira
for the improvement of Catholic health facilities including a Catholic Midwifery Training Institution in
Idah Diocese; supply of drugs and the secondment of doctors. This was made possible because of
41 | P a g e
sharing of data with the government and making advocacy visits which demonstrated that the Catholic
Church contributed 60% to the human resources development in Kogi state.
CSN’s Health Policy and Advocacy Advisor was successful in lobbying the Federal Ministry of
iii.)
Health to include the Catholic health facilities in the distribution of HIV test kits. This resulted in the
allocation of 8,000 HIV test kits which were distributed to the health facilities most in need of these
items.
iv.)
The Shuwa PHC had close engagement with the local government and traditional chief of the
community to advocate for resource allocation to the centre. The Chief of Shuwa was invited to
participate in the advocacy training organized by CSN in Abuja in June/July 2009. After the training
he (the chief) became a strong advocate for the PHC with the local government authority and the
Adamawa state government, and this has resulted in increased support to the PHC, especially its
maternity services.
v.)
Namu PHC organised an advocacy visit to the local government chairman and the director of
Primary Health Care services to press for its inclusion in health resources allocation. This resulted in
the local government officials visiting the facility and assessing the services provided.
vi.)
Vandeikya/Adikpo HBC programme made a similar arrangement with the FMC in Benue State.
After active lobbying by local community leaders, the FMC agreed to provide ART access through
Vandeikya/Adikpo HBC, thus saving the programme’s HIV clients (399 women, and 216 men) a
200km journey to the state capital.
vii.)
Bali PHC in Jalingo diocese paid an advocacy visit to their local government. It resulted in
improving the infrastructure and provision of equipment to the facility especially laboratory services.
4.0 Short term impact of the EC/CAFOD PHC programme
Attempt was made to evaluate the impact of the EC/CAFOD PHC programme on:
a. PHC’s capacity to provide sustainable quality primary healthcare service
b. Clients satisfaction
c. Changes in health behaviour and incidence of communicable diseases among the community
members
42 | P a g e
The evaluation team also attempted to evaluate the EC/CAFOD programme performance indicators
related to the improvement in practice and capacity of the PHC to deliver sustainable primary
healthcare services.
4.1 PHC’s capacity to provide sustainable quality primary healthcare service
The facility checklist was used to assess the level of availability of basic staffing, equipment and drugs
in PHC context, as well as the level of clientele for various services in the 9 PHC facilities visited
(excluding Abuja HIV site). The objective was to determine whether or not the PHC had minimum
capacity (in terms of staff, equipment and drugs) required to deliver quality primary healthcare
services.
The programme performance indicator stipulated that 70% of partner PHC would deliver improved
quality and increase the number of services being provided. In terms of types of services being
provided, all of the 9 facilities (100%) visited offered a range of basic primary healthcare services
including: ANC, immunization, health education, nutritional education, treatment of minor illnesses for
adults and children, HIV counselling and HIV testing. Eight of the nine facilities (88.8%) offered STI
management, community mobilization and referral/follow up services while six of the facilities
(66.7%) offered labour/delivery services as well as family planning services. Only a third of the
facilities (33.3%) provided postnatal services while five facilities (55.6%) offered growth monitoring
services for children. See table 1 of appendix 3 for further details.
The assessment of medical equipments available in the facilities showed that basic equipments for
ANC and child welfare services were available in most of the facilities. About half of the facilities had
basic equipment for delivery. However, only two of the facilities had equipment for assisted vacuum
delivery while feeding tube for sick babies was not available in all facilities visited. About seven out of
the nine facilities had basic laboratory equipment. While all of the facilities had standard consultation
rooms, only two facilities had SOP or prescription manual. (See tables 2 – 6 in appendix 3 for full
analysis of equipment availability).
43 | P a g e
In terms of availabilty of drugs and supplies in the facilities, all of the facilities had antacids, vitamins,
general pain relievers, antibiotics and antiseptics in stock. Most of the facilities (8 out of 9) had antipruritus (anti-itching drugs) and vaccines in stock. Seven out of the nine facilities had vaginal passeries
(vaginal insertion drugs), anti haemorrhagics (Vitamin K for arresting bleeding), infusions (dextrose)
and blood tonics/tablets. Mineral supplements were in stock in six facilities while five facilities had
laxatives, anti-diarrhoeals, anti-fungals, labour inducing drugs and pain relief ointments. Table 7 in
Appendix 3 provides more details.
Evidence exists to show that the number of pregnant women receiving appropriate antenatal care and
number of deliveries assisted by health professionals and trained birth attendants had over the years
increased across the facilities. For example, in six facilities (D/Kurmi, Fuka, Jada, Namu, Mandella and
Yakoko) the number of pregnant women who received ANC increased from 2507 in 2006 to 3117 in
2010, representing 24% increase. Similarly, number of deliveries in five facilities (excluding Fuka)
increased from 171 in 2006 to 373 in 2010, representing 118% increase. Figure 13 below shows the
trend (2006-2010) in number of clients receiving ANC and delivery services.
Figure 3: Trend
in number of clients for ANC and Delivery services in 6 selected facilities*
Table 10 in Appendix 3 gives more details of the trend in clientele for basic PHC services across the
facilities visited
4.2 Client Satisfaction
The level of clients’ satisfaction with the quality of service being provided by the PHCs under
EC/CAFOD programme was assessed through exit interviews with 53 randomly selected clients who
44 | P a g e
had come to receive services at the selected facilities on the day of evaluation. The demographic
characteristics of the respondents showed that close to three-quarter (73%) of them were females and
majority (81%) were Christians. A quarter of the respondents were visiting the facilities for the first
time while others (75%) were returning patients. Two-third of the respondents were in the facilities to
seek healthcare services for themselves while 30% and 4% accompanied their children and spouses
respectively. About two-third (66%) reported that they did not wait long in the facility before they were
attended to, 23% said that they waited for between 1 and 3 hours while 11% waited for 4 hours or
more. More than half (54%) of the respondents rated the quality of services received at the facility on
the day of interview as good, 40% said it was very good and 4% described the service as excellent.
The returning clients among the respondents were asked to describe the quality of service in the
facilities based on efficacy of treatment, friendliness of staff, cost of treatment and drugs, waiting time
in the facility, availability of qualified staff and availability of drugs and equipment. The results
showed that overwhelming majority of the respondents rated the facilities high on all of these criteria.
(See details in appendix 4).
About 79% of the clients believed that the incidence of preventable diseases was decreasing in their
communities. Majority of the clients attributed the decrease to increase in uptake of immunization,
mass distribution of ITN, health education and hygiene campaigns by the PHC staff. Almost all (98%)
of the clients would recommend the facility to others.
4.3 Changes in health behaviour and incidence of communicable diseases
There were oral evidences to show that the community mobilization activities of the PHCs had resulted
in positive change in health behaviours of the community members which had led to reduction in
incidences of communicable diseases such as diarrhoea, malaria, including HIV. It had also led to
increase in uptake of basic obstetric and maternal and child welfare services.
The results of the interviews held with the community leaders, TBA, village health committee and PHC
staff showed that majority of them believed that the prevalence/incidence of most preventable diseases
had decreased in the community over the past few years. Although, there was no written evidence to
support their claims, the respondents explained the reasons for their belief in different ways:
45 | P a g e
“They are decreasing because the people are enlightened on taking their sanitation seriously.
The people are aware of the importance of proper hygienic lifestyle. Cutting of grass in their
surroundings has help reduce mosquitoes”- A PHC manager
“For me I think it is reducing because when I came two years back a lot of people were coming
in with fever and convulsion and other similar things like that and I think the issue of treated
bed nets government distributed to the community has also contributed as people are now
making use of the nets”- A JCHEW
“Why I said it has reduced is because the community is aware, we give them health talk. They
are also aware about their drinking water, personal hygiene at home, keeping their
surroundings clean. So, the area of malaria has drastically reduced”- A CHEW
“I don’t know about that. Sometimes because I stay in the pharmacy if I look at the record of
the drugs we give out the one of diarrhoea is still intact which means they are not demanding
for them. It is decreasing because of the good drugs we have for the treatment of such
diseases”- A Pharmacy Assistant
Similarly, the PHC staff believed that the reduction in the incidence of HIV infection in the community
is attributable to their community outreach, mobilization and awareness campaigns: A PHC staff said:
“Because of the awareness on HIV/AIDS, people are more careful about their sexual lifestyle
than before, HIV infection is reducing, the turn-up for HIV testing is high, people now know
that they can leave with an HIV infected person without contracting it. The people here now
know how to keep a hygienic lifestyle”- A CHEW
However, some of the respondents had different opinions. A few of the PHC staff said there was no
concrete evidence to show that incidence of some diseases such as malaria is decreasing. They
observed that incidence of malaria tended to fluctuate with weather conditions, increasing during
raining season. The statistics on disease pattern from the PHC records also showed a mixed pattern.
46 | P a g e
5.0 STRENGTHS, KEY CHALLENGES AND RECOMMENDATIONS
5.1 STRENGTHS OF THE PROGRAMME
 Community focus: The design and implementation of the programme focused on community
development and meeting the needs of people in remote communities who had little or no
access to government infrastructure.

Bottom up approach: The programme allowed the PHC and community members to make
decisions based on their needs and contexts.

Building on the existing church structure: The use of the existing church structure (CSN,
Province, Diocese and PHC facilities) helped in reaching large numbers of people in the
community and resulted in the Catholic Church institutional strengthening.
5.2 KEY CHALLENGES
i.
Weak Monitoring and Evaluation system for the programme: Lack of structured baseline
evaluation before the commencement of the EC/CAFOD PHC programme made the assessment
of the achievements of the programme in quantitative terms difficult since there were no
benchmark figures to some key performance indicators. Similarly, the data collection tools and
reporting formats were not structured to routinely monitor the progress of the key indicators as
listed in the log frame. Narrative quarterly report without a summary data on key indicators was
not adequate.
ii.
Inadequate funding of M&E activities: It was also observed that very limited budgetary
allocations were made for M&E activities, including the final evaluation exercise. Between
10% -15% of the total programme grant should have been devoted to M&E for effective system
that will demonstrate results and ensure learning.
iii.
High staff turn-over: There was a high staff turn-over at all levels of the programme
implementation. Technical Accompaniers and Health Advisors were recruited but not retained
for long period and there was also high staff turn-over at the PHC facilities. Most of the key
staff that implemented the programme at CSN, province and CAFOD levels had disengaged as
at the time of the evaluation. The amount and quality of data generated for the programme
evaluation would have been more if the staff were still available for interview.
iv.
Wide scope of the programme: The wide scope of the programme in terms of the geographic
coverage (63 PHC facilities spread across 19 northern states and FCT in Nigeria) and
47 | P a g e
intervention areas (all primary health care service areas) had negative implication for the
coordination, supervision and impact of the programme. There were reported cases of
communication problems and feedback system between CAFOD and partners, late
disbursement of funds, irregular monitoring and supervision visits and poor visibility of the
impact of the intervention due to large area of coverage.
5.3 RECOMMENDATIONS
Based on the assessment of each objective in section three, the following recommendations are
proffered to improve quality of future programming.
Objective 1:
1. It is important for the PHC and dioceses to develop strategies of attracting staff with required
educational qualifications to make trainings more effective.
2. Early procurement of equipment is recommended to ensure optimal use during the life of the
programme.
3. Criteria for selection of benefiting facilities for different supports should be discussed in a
participatory process with all stakeholders.
4. Staff recruitment process and condition of service for technical staff should be reviewed for
future programming in order to attract and retain qualified personnel.
5. The development of the guidelines on models of best practices on gender is urgently required to
help the PHC in application of gender sensitive approaches.
Objective 2
6. VHC members should be adequately enlightened to know that participation in the committee is
a voluntary service (that attracts no monetary gain) to engender community ownership of the
programme. The concepts of voluntarism, dedication and community ownership should be
emphasised as part of orientation when new members/committees are being inaugurated.
Objective 3
7. The challenge faced with the production of newsletters and case studies could be attributed to
the fact that the task was not assigned to any specific staff or sub-group but remained the
responsibility of all the players in the programme. Such responsibility should be assigned to a
specific committee made of staff members with required skills and commitment.
48 | P a g e
8. The collaboration and linkages with state and local governments and other private health
stakeholders was a cost effective approach (PHC received various items and knowledge at no
costs) and has potentials for sustainability. This should be continued even after the EC/CAFOD
support.
Objective 4
9. It is recommended that in programme planning, activities that are directly linked to other
primary activities should be planned as sub activities and not stand alone. Efforts should also be
made to ensure implementation of both sub and primary activities in order to achieve the
desired results.
Cross cutting issues
10. The importance of M&E to demonstrate programme achievement and learning cannot be over
emphasized. Future programme should make M&E a priority by setting up an effective M&E
system with relevant benchmark data, user friendly data collection and reporting formats that
routinely track programme performance indicators and allocation of adequate funds for key
activities such as monitoring visits and evaluation exercise.
11. It is recommended that the current staff recruitment system should be reviewed to identify
reasons for high staff turn-over, especially for highly skilled technical staff. Establishment of a
strong human resources unit and competitive working condition is recommended to ensure that
highly qualified staff are attracted and retained through-out the duration of future programme.
CAFOD should also provide technical support and guidance in staff recruitment process by the
partners.
12. It is strongly recommended that the scope of future programme should be limited to two to three
PHC service area(s) and few states. For example, future programme can focus on immunization,
drug provision and maternal and child health in four to six states for effective coordination and
maximum impact.
13. In order to further strengthen the Church structure and ensure an effective and sustainable
monitoring and supervision system, technical capacity of the CSN and health coordinators
should be built to perform this role with adequate fund allocation for the purpose.
14. The key to the sustainability of the programme in the community lies with the VHC. Selection
of VHC members should be done collaboratively to ensure that credible and dedicated persons
are chosen as members. The capacity of the VHC members should be strengthened to play
49 | P a g e
significant role in the design and implementation of the community mobilization activities of
the PHC and to be able to mobilize resources within the community to continue to implement
the activities after EC/CAFOD funding.
50 | P a g e
APPENDICES
51 | P a g e
APPENDIX 1: PROGRAMME RESPONSES TO RECOMMENDATIONS
PROGRAMME RESPONSE
RECOMMENDATIONS
Objective 1:
1.
It is important for the PHC and dioceses to
i.
develop strategies of attracting staff with
required educational qualifications to make
This will be implemented by CSN,
dioceses and PHC facilities.
ii.
trainings more effective.
CAFOD will build into its future
programme technical support to
partners on recruiting qualified
staff.
2.
Early
procurement
of
equipment
is
iii.
CAFOD
will
ensure
and
this
early
is
recommended to ensure optimal use during
observed
need
the life of the programme.
assessment for any capital cost is
done before the commencement of
programme.
3.
Criteria for selection of benefiting facilities
iv.
CAFOD will work with its future
for different supports should be discussed in a
partners and beneficiaries during
participatory process with all stakeholders.
baseline survey to identify specific
need and develop guidelines for
selection
based
on
available
resources and impact
4.
Staff recruitment process and condition of
v.
CAFOD and its partners will
service for technical staff should be reviewed
consider this in all its future staff
for future programming in order to attract and
recruitment
retain qualified personnel.
5.
The development of the guidelines on models
vi.
CAFOD will focus on developing
of best practices on gender is urgently
this
required to help the PHC in application of
programme with the partners
gender sensitive approaches.
52 | P a g e
guideline
in
its
future
PROGRAMME RESPONSE
RECOMMENDATIONS
Objective 2
6.
VHC
members
should
be
adequately
vii.
Dioceses,
PHC
facilities
and
enlightened to know that participation in the
Village Health Committees (VHC)
committee is a voluntary service (that attracts
will
no monetary gain) to engender community
members on their roles
ownership of the programme. The concepts of viii.
CAFOD will provide technical
voluntarism,
community
support to achieve this by sharing
ownership should be emphasised as part of
experiences of how it has work in
orientation when new members/committees
other places and the National
are being inaugurated.
Primary Health Care minimum
dedication
and
develop
and train
VHC
ward packages for Community
Development committee’s roles.
Objective 3
7.
The challenge faced with the production of
newsletters and case
studies
could
ix.
be
considered in future programme
attributed to the fact that the task was not
assigned to any specific staff or sub-group but
CAFOD and its partners will
learning
x.
CAFOD Nigeria will seek support
remained the responsibility of all the players
from the programme effectiveness
in the programme. Such responsibility should
and learning team in London for
be assigned to a specific committee made of
local and global sharing of best
staff members with required skills and
practices
commitment.
8.
The collaboration and linkages with state and
xi.
CAFOD and its partners are
local governments and other private health
committed and will continue to
stakeholders was a cost effective approach
keep this close collaboration with
(PHC received various items and knowledge
the
at
stakeholders
no
costs)
and
has
potentials
for
sustainability. This should be continued even
53 | P a g e
xii.
government
and
other
CAFOD will ensure linkage with
PROGRAMME RESPONSE
RECOMMENDATIONS
after the EC/CAFOD support.
government agencies and other
international and national agencies
so
that
they
understand
and
support CAFOD’s work in Nigeria
Objective 4
9.
It is recommended that in programme xiii.
Not
agreeing
with
planning, activities that are directly linked to
recommendation as most of the
other primary activities should be planned as
activities of objective/outcome 4
sub activities and not stand alone. Efforts
were directed at National level
should also be made to ensure implementation
advocacy (strategic levels) by the
of both sub and primary activities in order to
Strategic
achieve the desired results.
Catholic Secretariat of Nigeria
partner,
this
which
was
while outcomes 1-3 were mostly
linked to the primary activities
(operational
levels)
of
the
programme.
Cross cutting issues
10. The importance of M&E to demonstrate xiv.
CAFOD and its partners will in
programme achievement and learning cannot
future
be over emphasized. Future programme
baseline
should make M&E a priority by setting up an
findings,
effective
relevant
indicators from result of the survey
benchmark data, user friendly data collection
and agree on monitoring and
and reporting formats that routinely track
evaluation plans/tools before the
programme
and
commencement
allocation of adequate funds for key activities
implementation.
M&E
system
performance
with
indicators
such as monitoring visits and evaluation
exercise.
xv.
programming
conduct
surveys,
set
document
measureable
of
programme
Roles of each stakeholder in
monitoring and evaluation will be
agreed and documented.
54 | P a g e
PROGRAMME RESPONSE
RECOMMENDATIONS
xvi.
Programme budgets will include
adequate
monitoring
evaluation costs
and
and especially
collecting of beneficiaries voice on
change to quality of life and how
holistic the programme is in
meeting their needs
11. It is recommended that the current staff xvii.
Same as (V) above
recruitment system should be reviewed to
identify reasons for high staff turn-over,
especially for highly skilled technical staff.
Establishment of a strong human resources
unit and competitive working condition is
recommended to ensure that highly qualified
staff are attracted and retained through-out the
duration of future programme. CAFOD
should also provide technical support and
guidance in staff recruitment process by the
partners.
12. It is strongly recommended that the scope of xviii.
CAFOD
will
in
its
future
focus
on
three
future programme should be limited to two to
programme
three PHC service area(s) and few states. For
Strategic
example, future programme can focus on
specific emphasis on Maternal,
immunization, drug provision and maternal
newborn
and child health in four to six states for
Governance as it is linked to
effective coordination and maximum impact.
health and Gender
xix.
Health
and
child
with
health,
CAFOD will be guided by national
indicators
poverty
55 | P a g e
areas:
of
in
factors
Nigeria,
driving
most
PROGRAMME RESPONSE
RECOMMENDATIONS
disadvantaged
groups,
where
impacts can be made and shared
and the availability of funds
xx.
Three states with bad indicators
and one with good indicators will
be targeted for impact and cross
learning opportunities
13. In order to further strengthen the Church xxi.
structure
sustainable
and
ensure
monitoring
an
effective
and
and
supervision
CAFOD
will
work
more
at
strategic by building the capacity
of
the
catholic
Secretariat,
system, technical capacity of the CSN and
Provincial and Diocesan structures
health coordinators should be built to perform xxii.
CAFOD will build the capacity of
this role with adequate fund allocation for the
the local church and enable it
purpose.
generate local resources that will
support its development work.
This will be rooted in the catholic
social
teaching
and
CAFOD
LiveSimply principles of social
development approach
14. The
key
to
the
sustainability
of
the xxiii.
programme in the community lies with the
VHC. Selection of VHC members should be
done collaboratively to ensure that credible
and dedicated persons are chosen as members.
The capacity of the VHC members should be
strengthened to play significant role in the
design and implementation of the community
mobilization activities of the PHC and to be
able
56 | P a g e
to
mobilize
resources
within
the
Same as in action vii & viii above.
RECOMMENDATIONS
community to continue to implement the
activities after EC/CAFOD funding.
57 | P a g e
PROGRAMME RESPONSE
APPENDIX 2: EVALUATION TERMS OF REFERENCE
1) Background to CAFOD
The Catholic Agency for Overseas Development (CAFOD) is the official development and relief agency
of the Catholic Church in England and Wales, and part of the global Caritas network, a confederation
of over 150 Catholic aid agencies. It grew from an initiative taken by Catholic women and was set up
as a charity in 1962 by the Bishops of England and Wales, with the task of expressing the concerns of
the Catholic community for the needs and problems of the developing world. CAFOD’s mission is to
promote human development and social justice in witness to Christian faith and gospel values.
2) The Primary Health Care Programme
CAFOD has been working in Nigeria since the 1970s with Catholic Church as its major partner. The
Primary Health Care (PHC) Programme was launched in 1999 to provide support to a network of
Catholic- run Primary Health Care projects in the North of the country. There have been several
subsequent phases of the Programme, with a continued focus on strengthening the network of the
partners and building capacity on health related issues. Programme reviews have been undertaken in
2003, 2005 and 2009.
The initiative entered its current phase in 2007 based on the recommendation of the 2005 review. A
co-funding grant was secured in 2006 through the support of the European Commission, for a 3-year
programme entitled ‘Improving Primary Health Care for Rural Poor Communities in Northern Nigeria’
(Contract No. ONG-PVD/2006/119-131). Implementation began on 1st February 2007, following the
signing of a project contract between the EC and CAFOD on 20 December 2006. A further 1 year
extension was granted by the EC which ended in January 2011.
The purpose of the current programme is to strengthen Catholic Church Health structures, focusing in
particular on primary health-care facilities to enable them to provide high quality, sustainable
healthcare services that meet the needs of poor rural communities in the 19 dioceses that make up
the three northern Ecclesiastical Provinces of Abuja, Jos and Kaduna.
The specific objectives include:
i)
To increase the capacity of 63 Catholic Church-run primary Health care (PHC) providers to
manage and deliver high quality, sustainable Health care services.
ii)
To increase the level of community participation in and ownership of primary Health care in
order to promote sustainable, healthy lifestyles.
iii) To share learning between PHC providers in order to strengthen Health networks and promote
good practice in primary health care.
iv) To build the capacity of the Catholic Church in Health policy and advocacy issues to influence
the development and monitoring of pro-poor health policies.
In terms of civil administration, the programme covers facilities in 18 states and the Federal Capital
Territory of Abuja. In addition to the 63 primary health care facilities covered by the programme,
58 | P a g e
another important focus for capacity building is the management structures at diocesan and
provincial levels.
3) Purpose of the evaluation
The Evaluation will focus on the effort of the current programme in building the capacity of 63
Catholic Church-run PHC providers to manage and delivery high quality, sustainable healthcare
services.
The evaluators will assess the EC/CAFOD Primary Health Care Programme in order to:
a) Enhance accountability to stakeholders and European Commission.
b) Capture lessons to help improve CAFOD’s decision-making and work in responding to the
health needs of Nigerians
c) Guide future decisions on CAFOD’s long-term strategy in Nigeria.
4) Evaluation Criteria
The evaluation will assess the programme according to the following criteria:
i)
Relevance/appropriateness: Assess whether the programme inputs, activities and outcomes
are in line with local needs and priorities. Will seek to answer questions like:
 How effective were CAFOD, CSN and the 19 Dioceses from the 3 Northern Catholic
Ecclesiastical Provinces in assessing and analysing the needs and context?
 To what extent were past lessons or recommendations taken into account?
 How relevant was the programme in addressing the needs and priorities of the local
communities including specific gender needs and those affected or living with HIV.?
 Did the programme take the local wider issues and context into account (e.g. culture,
gender, etc.)?
 Were the staffing and management arrangements appropriate for effective
programme delivery, building the capacity of CSN and the 19 Dioceses from the 3
Northern Catholic Ecclesiastical Provinces, and strengthening the relationship between
CAFOD, CSN and the 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces
 Were intended beneficiaries able to participate effectively to inform programme
design and implementation?
 How could relevance and appropriateness be improved?
ii)
Sustainability: Assess whether PHC programme activities were carried out in a way that takes
longer-term and interconnected health problems into account. Will seek to answer questions
like:
 How has this programme built upon, supported and developed existing programmes or
structures (partner’s or government)?
 How has the programme supported or disrupted communities’ or partners’ ability to
support themselves?
 How does this programme fit with CAFOD’s plans for longer-term support in Nigeria
 How could sustainability be improved?
59 | P a g e
iii)
iv)
v)
vi)
vii)
Coherence: Assess whether there is consistency with relevant policies (such as National
Primary Health Care, National Health Policy, HIV and AIDS etc).
 To what extent have national health strategies been taken into account?
 Have the different aspects of the programme complemented or contradicted each
other?
Coverage: Assess whether the major population groups facing poor health indices are reached
and supported. Will seek to answer questions like:
 How successful was the programme in reaching the poorest, most vulnerable or
disadvantaged?
 Have programme activities reached areas not covered by other agencies?
 How could coverage have been improved?
Efficiency: Measures the qualitative and quantitative outputs achieved in relation to the
inputs and compares alternative approaches to see whether the most efficient approaches
were used. Will seek to answer questions like
 How efficient was this programme (effectiveness in relation to budget)?
 What factors affected the efficiency of the overall outcome? (E.g. political context,
logistics, working with local partners, staff capacity, monitoring systems, procurement
policies, transport, finance procedures).
 Were some locations or components more efficient than others, if so why?
 How could efficiency be improved?
Effectiveness: Measures the extent to which activities have achieved their purpose or whether
this can be expected on the basis of the outputs.
 To what extent did the programme achieve its goals and objectives? What are the main
factors that have facilitated or constrained the achievement of these?
 Have the levels of success varied between locations? What are the reasons for this?
 How effective were CAFOD’s, CSN’s and the 19 Dioceses’ systems of ongoing analysis
and monitoring?
 To what extent did CAFOD co-ordinate with other NGOs, CARITAS organisations,
partners, donors, government?
 To what extent were partners involved in field-based co-ordination mechanisms (with
other Caritas agencies, NGOs, the government), how effective were these?
 How successful has the programme been in delivering Primary health care outcomes in
accordance with WHO or Federal Ministry of Health PHC Principles? (Including the
Alma Ata declaration).
 How could effectiveness be improved?
Impact: looks at the wider effects of the Programme (social, economic, technical and
environmental) on individuals and groups (gender, age groups, communities and institutions).
Impacts can be intended and unintended, positive, negative, macro (sector) and micro
(household). Will seek to answer questions like
 Did CAFOD/CSN and the 19 Dioceses have wider effects on individuals and groups? If
so why did these impacts arise?
60 | P a g e



Have the impacts varied between locations, if so why?
How would the impact have been varied if CAFOD / CSN and the 19 Diocesan
programmes were different?
How could the impact of the programme be improved?
5) Outputs
The evaluation report (in MS Word format) should not be more than 30 pages and should articulate
findings, draw conclusions and make recommendations. The report (including annexes where
needed) will be in clear, plain English and will outline the main findings and recommendations.
The report will cover the whole EC/CAFOD programme components and must be submitted on or
before 31st May 2011.
The following headings are suggested for the report:




Executive Summary
Introduction
Methodology
Findings/Analysis/Outcomes addressing issues raised in the TOR. This could be organised by
reference to the Programme’s four objectives and then for each objective there could be an
analysis of the relevance and effectiveness of each activity (workshops, accompaniment visits,
etc)
 Conclusion and Recommendations with a section dedicated to drawing out specific lessons
with suggestions addressed to the relevant institution(s) for taking forward lessons learned
 Appendices, to include evaluation terms of reference, maps, beneficiary research and
bibliography. (All material collected in the undertaking of the evaluation process should be
lodged with CAFOD prior to termination of the contract)
The report and all background documentation will be the property of CAFOD (as the contracting
organisation) and will be disseminated and publicised as appropriate by CAFOD.
6) Intended users of the evaluation
 European Commission
 CAFOD particularly staff in Nigeria and ID
 CSN
 The 19 Dioceses from the 3 Northern Catholic Ecclesiastical Provinces
7) Key persons specification
It is anticipated that the evaluation will be conducted by a national consultant who will have the
following experience and skills:



At a least a Master degree in Public Health.
A good understanding of primary health care in the context of northern Nigeria
Familiarity with the current programme, the context and the health structures and services of
61 | P a g e
the Catholic Church
Sound experience in participatory review and evaluation methodologies
Excellent facilitation, analytical and report writing skills
Fluent written and spoken English
At least 5 years relevant experience of evaluating Health programmes, especially Primary
Health Care and working in Public health and development.
 Ability to analyse and synthesise in writing relevant information relating to public health data
 Ability to work respectfully with national NGO partners
Desirable:





Experience of working with Health programmes and faith based international agencies and
faith based national NGOs
8) Evaluation methodology
Approach
The evaluators will propose the methodology for the evaluation, however it should:





Use internationally accepted guidelines of evaluating health programmes
Ensure good representation
Use participatory approaches and enable feedback from participants
Include field visit to partners and beneficiaries
Organise a one day stakeholders consultative meeting
Timeframe
It is anticipated that the evaluation will require approximately 35 days work, in April to June 2011
Start Date
End date
Activity
7th April 2011
7th April 2011
Briefing on programme and agreeing on evaluation
methodology and timeline
8April 2011
12 April 2011
Desk review and development of evaluation tools
Protocol and Instruments Development
13 April 2011
12 May 2011
Programme Evaluation/Field trip/stakeholders
consultation meeting
First draft report writing
62 | P a g e
19th May 2011
19th May 2011
Submission of 1st draft report
20th May 2011
24th May 2011
Evaluation reference group discuss report
25th May 2011
25th May 2011
Feedback on draft report discussed with evaluators
26th May 2011
30th May 2011
Final Report writing
31st May 2011 31st May 2011
Submission of final report
Process










Initial meeting in Jos to review background information to inform the evaluation proposed
methodology
Write-up methodology and timeline
Desk based review of key documents
One day stakeholder’s meeting
Identify programme areas/partners to visit
Field visits – interviews/ focus group discussion with stakeholders: beneficiaries, CSN,
Diocesan Health Coordinators/PHC leaders
In-country presentation of preliminary findings to partners
Produce draft evaluation document
Presentation of draft report to CAFOD staff and reference groups to make comments
Incorporation of comments received and preparation of the final report
63 | P a g e
APPENDIX 3: ANALYSIS OF THE PHC CHECKLIST
Table1: Type of services being provided by the PHC facilities visited
Key Services
Facilities Providing Service
ANC
Total
Number
Agagbe, Ankpa, D/Kurmi, Fuka, Jada,
Namu,Mandella,Yakoko
Labour & Delivery
Agagbe, D/Kurmi, Jada, Namu, Mandella,
Post Natal care
Fuka, Namu, Mandella
Immunization
Agagbe, Ankpa, D/Kurmi, Fuka, Jada,
Namu,Mandella,Yakoko
General Lab screening Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu,
Vandeikya,Mandella,Yakoko
HIV counseling and Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu,
testing
Vandeikya,madella,Yakoko
Growth Monitoring
Ankpa, D/Kurmi,Fuka,Madella,Yakoko
8
% of
total
sites
89
5
3
8
56
33
89
9
100
9
100
5
56
Most of the 9 facilities visited provided the key PHC services. All of nine facilities (100%) provided
general lab screening, HIV counseling and HIV testing services. 8 of them (89%) provided ANC and
Immunization while 5 of the 9 (56%) provided labour and delivery services. Though
Adikpo/Vandeikya PHC did not offer ANC or take deliveries in the facility, they reported that they
offered ANC in the communities and refer deliveries to their affiliated clinic. Also, Yakoko offered
delivery services only in their affiliated hospital not in the PHC. Five of the facilities reported that they
offer growth monitoring services but on investigation, it was found that it was only weight and height
that were being measured. Only three of the facilities provided post natal services.
Table 2: Availability of basic items and equipment for ANC services
Key Equipment
Facilities Where Equipment is available
Sphygmomanometer
& Stethoscope
Foetal stethoscope
Vaginal examination
equipment
Immunization
equipment and vaccine
Couch with Pillow
Agagbe,
Ankpa,
D/Kurmi,
Fuka,
Namu,Mandella,Yakoko
Agagbe, D/Kurmi, Jada, Namu, Mandella,Yakoko 6
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 1
Vandeikya, Mandella,Yakoko
Ankpa, D/Kurmi, Fuka, Jada, Mandella,Yakoko
6
100
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, Namu, 8
100
64 | P a g e
Total
Number
Jada, 8
%
75
13
75
mandella,Yakoko
Growth
Monitoring Ankpa, D/Kurmi,Fuka,Mandella,Yakoko
equipments
5
63
Most of the 8 facilities that offered ANC services had the key ANC equipment available at the time of
visit. Of the 8 facilities, 8 (100%) of them had a stethoscope, sphygmomanometer and a couch with
pillow. 6 of them had a foetal stethoscope and immunization equipment with vaccine available in the
ANC room. 5 had growth monitoring equipment (weight and height scales only). However, only 1
facility (Dogon Kurmi) had vaginal examination equipment on ground.
Table 3: Items and equipment availability in the Labour Rooms
Key Equipment
Facilities Where Equipment is available
Total
Number
Bed with mattress
Foetal stethoscope
Vaginal examination
equipment
Running water
Mackintosh
Agagbe, D/Kurmi, Jada, Namu,Mandella
Agagbe, D/Kurmi, Jada
Agagbe, D/Kurmi, Jada, Namu, Mandella
5
3
5
% of
total
sites
100
60
100
Jada
Agagbe, D/Kurmi, Jada, Namu,Mandella
1
5
20
100
Of the 5 facilities that were offering labour and delivery services in the PHCs themselves, all 5 had a
bed with matress, mackintosh and vaginal examination equipment available in the labour room. 3 of the
5 had a foetal stethoscope in the labour room but only Jada had running water in the labour room.
Table 4: Items and equipments availability in the Delivery Rooms
Key Equipment
Facilities Where Equipment is available
Total
Number
Normal delivery kit
Equipment for assisted vacuum
delivery
Equipment for new born care and
neonatal resuscitation
Surgical set for minor procedures
like
episiotomy,circumcision
stitching
Lamp/torchlight for delivery
Mucus extractor for babies
Agagbe, Jada, Mandella
Agagbe, D/Kurmi
3
2
% of
total
sites
60
40
Agagbe, D/Kurmi, Mandella
3
60
Namu, 5
56
65 | P a g e
Agagbe,
Mandella
D/Kurmi,
Jada,
Agagbe, D/Kurmi, Jada, Mandella
Agagbe, D/Kurmi, Namu, Mandella
4
4
44
44
Table 5: Items and equipment availability in the Laboratories
Key Equipment
Facilities
available
Where
Equipment
is Total
Number
Reagents
for
essential
investigations
Binocular microscope
lab Agagbe, Ankpa, D/Kurmi, Fuka, 7
Jada, Namu, mandella
Agagbe, Ankpa, D/Kurmi, Fuka, 7
Jada, Namu, Mandella
Specimen containers (assorted)
Agagbe, Ankpa, Fuka, Jada, Namu, 6
Mandella
Refridgerator
Ankpa, D/Kurmi, Fuka, Jada, 6
Namu, Mandella
% of
total
sites
78
78
67
67
Table 6: Items and equipment availability in the consultation rooms
Key Equipment
Facilities Where Equipment is Total
available
Number
Sphygmomanometer & Stethoscope
% of
total
sites
67
Agagbe, Ankpa, D/Kurmi, Fuka, 6
Jada, Namu,
Thermometer
Agagbe, D/Kurmi, Jada, Namu, 6
67
Mandella
Vaginal examination pack
D/Kurmi
1
11
Couch with Pillow
Agagbe, Ankpa, D/Kurmi, Fuka, 6
67
Namu, Mandella
SOPS & Manuals
Ankpa, D/Kurmi
2
22
Trolley / wheel chair
D/Kurmi, Fuka,
Namu, 4
44
mandella
In the consultation rooms, 67% (6 out of 9) of the facilities visited had a sphygmomanometer,
stethoscope, thermometer, a couch and pillow. Less than half of them (4 out of 9) had a trolley or wheel
chair. Only Dogon Kurmi had a vaginal examination pack and only Ankpa and Dogon Kurmi had
SOPs and Manuals available. Jada however had a vaginal examination pack and a trolley/wheel chair
but these were kept in another room due to lack of space in the consultation room.
Table 7: Availability of essential drugs and supplies
Essential Drugs/Supplies
Antacids
Laxatives
Anti-diarrhoeal
66 | P a g e
Facilities Where Drug is available
Total
% of total
Number sites
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9
100
Namu, Vandeikya,Mandella,Yakoko
D/Kurmi, Jada, Fuka, Mandella, 5
56
Yakoko
Agagbe, D/Kurmi, Namu, Vandeikya, 5
56
Mandella
Anti-haemorrhages (vitamin Agagbe, D/Kurmi, Fuka, Jada, Namu, 7
78
K)
Mandella, Yakoko
Dextrose
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 7
78
Namu, Vandeikya,Mandella,Yakoko
Vitamins
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9
100
Namu, Vandeikya,Mandella,Yakoko
Anti-anaemia
(blood Agagbe, Ankpa, Fuka, Jada, Vandeikya, 7
78
tonic/tablet)
Madella, Yakoko
Antiseptics and disinfectants Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9
100
Namu, Vandeikya, Mandella,Yakoko
Labour inducing drug
Agagbe, D/Kurmi, Jada, Namu, Yakoko 5
56
Antibiotics (Various)
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9
100
Namu, Vandeikya, Mandella,Yakoko
General pain relievers
Agagbe, Ankpa, D/Kurmi, Fuka, Jada, 9
100
Namu, Vandeikya, Mandella,Yakoko
With respect to availability of drugs and supplies, most of the facilities had essential drugs in stock.
All the facilities had antacids, vitamins, general pain relievers, antibiotics and antiseptics in stock. 78%
had vitamin K (an anti-haemorrhagic), infusions (dextrose) and blood tonics while 56% had laxatives,
antidiarroeals and labour inducing drugs available.
Table 8: Availability of standard forms and registers (HMIS tools)
Key Documentation
OPD Register
ANC Register
Delivery Register
Drug stock card
HCT register
Facilities
Where
Registers/Records
are
available
Agagbe, D/Kurmi, Jada,
Namu,
Vandeikya,
Mandella
Ankpa, D/Kurmi, Fuka,
Namu, Mandella
Agagbe,
D/Kurmi,
Mandella
D/Kurmi, Mandella
Namu, Vandeikya, Yakoko
Total
% of total
Number sites
of sites
6
67
5
56
3
33
2
3
22
33
An assessment of the availability of data collection tools (forms and registers) across all the sites
visited left much to be desired, though for those who had, most of them were updated`. 6 of the sites
had an OPD register (all 6 were updated) 5 sites had ANC registers (all 5 updated). 3 sites had delivery
registers (all 3 updated). 2 sites (D/Kurmi and Mandella) had drug stock cards (both were updated).
Only Namu, Vandeikya and Yakoko had standard HCT registers and only that of Namu was updated.
Some other forms and registers were being kept but these varied widely across the sites. Fuka was
using hand ruled notebooks for their records.
Table 9: Availability of Staff
67 | P a g e
Key Staff
Community
Officer
CHEW
Facilities Where Staff is available
Health
D/Kurmi
Agagbe, Ankpa, D/Kurmi, Fuka, Namu,
Vandeikya,Mandella,Yakoko
JCHEW
Ankpa,
D/Kurmi,
Fuka,
Namu,
Vandeikya,Mandella,Yakoko
Nurse/Midwife
Agagbe, Ankpa, D/Kurmi, Fuka, Jada,
Namu
Medical
Records Ankpa, Namu, Mandella
officer
Pharmacy
Agagbe, Ankpa, Namu
technician/assistant
Medical officer or Namu, Mandella
NYSC doctor
Lab technician
Ankpa, D/Kurmi, Fuka, Jada, Mandella
Lab scientist
Jada
Lab assistant
Agagbe, Ankpa, Mandella
Total
% of total
Number sites
of sites
1
11
8
89
7
78
6
67
3
33
3
33
2
22
5
1
3
56
11
33
Most of the facilities had a CHEW and JCHEW available (8 and 7 of the sites respectively) 6 of them
had a nurse/midwife while 3 had a pharmacy technician and a medical records officer. 6 had at least a
lab staff (lab scientist/technician or assistant). Only Namu and Mandella had a Medical officer/NYSC
doctor and only Dogon Kurmi had a community health officer.
Trend in level of Clientele for basic PHC services
An analysis of the annual clientele for the key PHC services was done. However, data for some of the
years was not available. This was due to non-collection of the data by the consultant for some sites
(Agagbe and Ankpa) or unavailability of the data at the facilities). Analysis of the annual clientele for
ANC showed that for most of the sites, the clientele remained fairly constant over the years. Ankpa
however showed a marked increase (116%) in ANC clients in 2010 compared with 2006 while for
Agagbe, there was a decline of 50%. Clientele for labour and delivery services was also fairly constant
through the years. Mandella showed a steady increase over the years while clientele for Agagbe also
declined by about 50%. Post-natal care clientele for the 3 sites for which data was available (Fuka,
Namu, Mandella) showed significant increase through the years (over 100% for all 3 sites).
68 | P a g e
Table 10: Trend in clientele for basic PHC services (2006- 2010)
Key
Services
Year
ANC
Labor/Delivery
06
07
08
09
10
06
07
08
09
10
06
07
08
09
10
06
07
08
09
Agagbe
534
_
_
_
267
113
_
_
_
57
58
_
_
_
57
494
_
_
_
1698
Ankpa
2234 _
_
_
4834
_
_
_
_
_
_
95
3542
_
_
_
3783
D/Kurmi
121
84
79
410
296
34
23
93
61
110
_
_
_
_
_
_
_
55
37
36
Fuka
345
570
488
684
567
_
_
_
_
_
_
_
_
_
_
957
1151 944
995
999
Jada
85
95
90
75
87
10
15
8
6
5
60
117
59
78
128
1020 950
850
680
410
Namu
1260 1448 1110 1217 1348 _
85
103
64
82
260
240
210
280
360
98
104
126
211
120
Adikpo/
V/kya
_
_
_
_
_
_
_
_
_
_
166
182
_
_
_
_
_
HIV Testing
_
Immunization
_
1040 944
_
2541 271
Mandella 57
71
84
83
98
31
40
39
46
87
_
_
_
_
Yakoko
562
540
601
721
96
84
127
92
89
_
125
833
861
639
10
125
_
1026 1178 1238
1096 5310 8081 8820 4312 5069
For immunization services, clientele for most of the sites also increased significantly but Dogon- Kurmi and Fuka remained fairly constant, while
Jada’s clientele reduced progressively through the years. Of the 5 sites for which HIV testing data was available, Adikpo/Vandeikya HBC showed a
drastic drop in clientele in 2009 and 2010 while Yakoko conversely showed a marked increase in clientele through the years. Jada, Namu and Agagbe
remained fairly constant.
69 | P a g e
70 | P a g e