FACULTY OF HUMANITIES AND SOCIAL SCIENCES
DEPARTMENT OF DEVELOPMENT STUDIES
BACHELOR OF SOCIAL SCIENCE SPECIAL HONOURS DEGREE IN MONITORING
AND EVALUATION
TOPIC: A PROCESS EVALUATION OF WORLD VISION’S IGATE PROGRAMM IN
IMPROVING GIRLS’ ACCESS THROUGH TRANSFORMING EDUCATION IN
RURAL AREAS OF ZIMBABWE.
BY
ARTHUR CHIKANDE
L0201333H
MODE OF ENTRY: BLOCK (HARARE)
i|P ag e
Approval Form
The undersigned certify that they have read and recommended to the Lupane State University for
acceptance, a research entitled: A PROCESS EVALUATION OF WORLD VISION’S IGATE
PROGRAMM IN IMPROVING GIRLS’ ACCESS THROUGH TRANSFORMING
EDUCATION IN RURAL AREAS OF ZIMBABWE.
Name of Author: Arthur Chikande (L0201333H) MODE OF ENTRY BLOCK (HARARE)
Programme for which project is presented: A dissertation submitted in partial fulfilment of the
requirements for a Bachelor of Social Science Special Honors Degree in Monitoring and
Evaluation
SUPERVISOR Dr D. DUBE
Date………………………………
PROGRAMME/SUBJECT COORDINATOR
……………………………………………………
EXTERNAL EXAMINER
DATE: …………………
ii | P a g e
Signature………………………
DECLARATION
I Arthur Chikande , do hereby declare that this thesis is the result of my own investigation and
research, except to the extent indicated in the acknowledgements, references and by comments
included in the body of the report, and that it has been submitted in part or full for any other degree
to any University.
………………………
Student Signature
DEDICATION
To my mother, my mentor in every aspect of my life
iii | P a g e
………………………
Date
LIST OF ACRONYMS
IGATE
Improving girls’ through transforming access to education
BEAM
BEEP
CPC
CoH
CSGE
DEO
DFID
DiD
DSI
ECE
EGRA
EGMA
ERI
EWF
EFZ
FGD
FT
GEC
GBV
HH
HoS
HR
IGA
IGATE
ITT
KII
M&E
MC
MG
MoPSE
MoWAGD
Basic Education Assistance Module
Bicycle Education Empowerment Programme
Child Protection Committee
Channels of Hope
Community in Support of Girls’ Education
District Education Officer (now called DSI)
Department for International Development
difference-in-difference
District Schools Inspectors (formerly DEO)
Early Childhood Education
Early Grade Reading Assessment
Early Grade Math Assessment
Early Reading Initiative
Emthonjeni Women’s Forum
Evangelical Fellowship of Zimbabwe
Focus Group Discussion
Full-Treatment
Girls’ Education Challenge
gender-based violence
household
Head of School
Happy Readers
Income Generating Activities
Improving Girls’ Access through Transforming Education
Intent-to-Treat
Key Informant Interview
Monitoring and Evaluation
Male Champions
Mothers Groups
Ministry of Primary and Secondary Education
Ministry of Women’s Affairs, Gender and Community
Development
Wvz
World Vison Zimbabwe
List of Figures
Figure 1: Analysis of Respondents by the Education Level
Figure 2: Analysis of Respondents by Sex
Figure .3.Analysis of Respondents by Age
Figure4 : Aware of the programme activities
Figure 5. EGRA3 Scores of Treatment and Control Groups in PW Clubs
iv | P a g e
Figure 6 : Understanding the objectives of the IGATE Programme
Figure 7; Level of Service Appropriateness and Relevance of Services
Figure 8: Satisfaction of the program
Table list
Table 1: Evaluability Assessment Checklist
Table 2: Improving girls’ access through transforming education
Table 3:Comparative Table 3
Table 4 data collection matrix
Table5 Discussions
Table 6: EGRA Difference-in-Difference estimations for re-contacted girls
Table 7: Comparison of literacy outcomes by treatment status for re-contacted girls
Table 8: Change over time in literacy outcomes by treatment status for re-contacted girls
List of Appendices
Appendix 1: Structured Questionnaire for Beneficiaries
Appendix 2: Semi-structure Interview guide
Appendix 3: Summary of data collection methods and instruments
Table of Contents
CHAPTER 1.0 ............................................................................................................................1
INTRODUCTION ......................................................................................................................1
Introduction of the program.........................................................................................................3
1.2.1 Description of organization .............................................................................................3
1.3.1 Summary ...................................................................................................................... 10
CHAPTER 2: LITERATURE REVIEW ................................................................................... 10
2.0 Introduction ..................................................................................................................... 10
2.1 Theoretical issues on process evaluation .......................................................................... 11
v|P age
2.3 What is process evaluation? ............................................................................................. 12
2.3.1 Purpose of process evaluation ....................................................................................... 13
2.3.2 Evaluation questions answered by process evaluation ................................................... 13
2.3.4 What they measure........................................................................................................ 13
2.3.5 Methods that can be used .............................................................................................. 14
Evaluating a Youth Leadership Life Skills Development Program............................................. 16
2.4.2 Process evaluation: the filles éveillées (girls awakened) program for migrant adolescent
girls in domestic service in urban Burkina Faso ..................................................................... 16
Implications of the review ..................................................................................................... 20
Chapter summary .................................................................................................................. 20
CHAPTER 3: EVALUATION DESIGN ................................................................................... 20
3.0 Introduction ..................................................................................................................... 20
3.1 Evaluation objectives ....................................................................................................... 21
3.2 Evaluation questions ........................................................................................................ 21
vi. 3.2.1 Process for coming up with the Questions .............................................................. 21
viii.
x.
3.2.2 Standards by which program performance may be evaluated .............................. 22
3.3 Sampling .................................................................................................................. 22
3.4 Methodology and justification .......................................................................................... 25
3.5 Qualitative research ......................................................................................................... 26
3.5.1 Semi structured interview guide .................................................................................... 26
3.5.2 Observation checklist .................................................................................................... 26
3.5.3 Document Review......................................................................................................... 27
3.5.4 Focus group discussions ................................................................................................ 27
3.6 Quantitative Method ........................................................................................................ 28
3.6.1 Questionnaires .............................................................................................................. 28
3.7 Data Analysis Strategy ..................................................................................................... 28
3.8 Validity and Reliability .................................................................................................... 30
CHAPTER 4: DATA PRESENTATION, ANALYSIS AND DISCUSSIONS ........................... 31
4.0 Introduction ..................................................................................................................... 31
4.1 Discussions ...................................................................................................................... 31
.............................................................................................................................................. 32
vi | P a g e
4.2. Fieldwork Reflections: Demographic Characteristics of Respondents ............................. 32
4.3 efficiency and effectiveness of the IGATE Program ......................................................... 34
4.4. Aware of the programme activities .................................................................................. 36
Figure4 .................................................................................................................................. 36
4.5 Were the programme activities carried out according to the plan ...................................... 36
4.6 Understanding the objectives of the IGATE Programme .................................................. 47
Figure 6 ................................................................................................................................. 47
4.7 Relevance of Igate Programm ......................................................................................... 47
4.8 Quality and Quantity of personnel .................................................................................... 49
.............................................................................................................................................. 49
4.9Challenges being faced in implementing the Igate programme ......................................... 49
4.10 Chapter summary ........................................................................................................... 50
CHAPTER 5: CONCLUSIONS AND RECOMMENDATIONS .............................................. 51
5.0 Introduction ..................................................................................................................... 51
5.1 Conclusions ..................................................................................................................... 51
5.2 Main evaluation findings ................................................................................................. 51
5.3 Recommendations............................................................................................................ 53
APPENDICES .......................................................................................................................... 59
REFERENCES ...................................................................................................................... 54
vii | P a g e
ACKNOWLEDGEMENTS
I may fail to acknowledge everyone that may have helped me to make this thesis a success, may
you please forgive me if I do. My first gratitude is reserved for the Almighty God, the Horn of my
Salvation. He has made me the man I am today by his grace. To my parents, I am forever indebted
to you. They have been supportive of my academic career from the start. They have given me the
wisdom and strength to pursue these studies. Special gratitude goes to my mother, my mentor in
every aspect of my life. Thank you for being the strongest woman, and for being with me through
this journey.My gratitude also goes to Dr Dube Ncube for his patience throughout my studies at
Lupane State University. Without her guidance, this evaluation would not have been a success.
ABSTRACT
This study focused on the processes evaluation of World vision program in improving girls access
through transforming education in 10 rural district of Zimbabwe which are Gokwe South and
North,Lupane, Beitbridge,Nkayi,Chivi,Mberengwa,Binga,Mangwe,and Insiza. The study main
focus was educating institutionalized adolescent girls and early grade leaners, on different social
matters affecting them. Project activities were designed at preventing early drop-outs by making
and allowing environment that encourages girls that are already outside the system, to re-enroll
and complete their education. The writer utilized a mixed methodology approach. Semi structured
interviews were utilized to interview beneficiaries, project staff and the funders. A total of 10
interviews were interviewed and 80 questionnaires were distributed to beneficiaries. Interviewees
were purposively selected and 80 questionnaire respondents were randomly selected. The writer
viii | P a g e
also gathered data through document analysis and observations. The study assessed if activities
were being carried out according to plan. The study also assessed beneficiary satisfaction and the
improvements that could be done on the implementation of the IGATE programme..The program
also realized goal through discussions and lessons conduct by prominent people in Zimbabwe that
they look up to. In partnership with communities collaborating with the Ministry of Primary and
Secondary Education, the IGATE project focused on teaching parents about the importance of
education and schooling them to partake in their children’s education through literacy-based
activities. It is also of paramount importance to note that, the IGATE project aimed to make school
more accessible for children, especially girls, who live long distances from school buildings
through the provision of bicycles. The evaluation’s main objective was to evaluate whether what
was planned was being implemented on the ground. Similar evaluations on mentorship programs
being implemented in different regions were evaluated. The evaluation employed the mixed
research methods. Questionnaires were used for gathering quantitative data. Interview guides,
focus group discussions and observational checklist were used to gather qualitative data. The
research findings showed that the activities were being implemented according to plan and the
beneficiaries were satisfied with the programme. However, dropping out of some school girls were
highlighted as the major challenges being faced in the implementation of the programme.
Recommendations were offered for the smooth implementation of the IGATE Programme.
Key Words: Igate, process evaluation, adolescence girls, PW,Beep
ix | P a g e
CHAPTER 1.0
INTRODUCTION
This report presents the Process Evaluation of a project titled World vision’s Igate program in
improving girls’ access through transforming education in rural district of Zimbabwe. The
evaluation study focused on 10 rural districts of Zimbabwe which includes, Gokwe South and
North, Nkayi,Beitbridge Chivi,Mberengwa,Binga,Mangwe,and Insiza.. The Evaluation was
conducted primarily for the partial fulfilment of the study requirements for the Bachelor of
Social Sciences (Special Honours) degree in Monitoring and Evaluation undertaken by the
evaluator at Lupane State University (LSU).
A Process Evaluation is a systematic assessment of the programme for the purpose of improving
its design, its delivery, and the usefulness of the quality of services delivered to the consumer ,
examining how it is implemented and operates as planned (Rossi, Lipsey & Freeman 2004;
Saunders, Evans & Joshi 2005). It is a Formative type of evaluation aimed at giving direction on
how to improve project performance or best achieve its development goals. In this case, it was
conducted in order to: Examine the IGATE project implementation processes; Determine if the
activities of the IGATE are appropriate & whether they contribute to its Outcomes & Goal;
Determining how successful the IGATE Project follows its laid down strategy; as well as
specifically analyzing the project inputs, activities, and outputs and how they are working together
towards the set outcomes. Thus, without documentation and/or measurement of the project’s
adherence to an intended model, there is no way to determine whether unsuccessful outcomes
reflect a failure of the model or failure to implement the model as intended.
The writer carried out an evaluability assessment on the IGATE project before carrying out
process evaluation of the programm. Wholey (1994) asserts that the planning of any type of
project evaluation, which should lead to improved project performance, requires the evaluator to
make several decisions. Wholey (1994) states that it is vital to conduct an Evaluability
Assessment (EA) first prior to evaluation in order to assess whether a project is feasible,
appropriate and ready to be evaluated. The writer therefore conducted an evaluability assessment
1|P a ge
on the IGATE programme. The writer utilized the following checklist as guidance to the
evaluability assessment.
Table 1: Evaluability Assessment Checklist
QUESTION
Does the program clearly define the problem that it aims to change?
Has the beneficiary population of the program been determined?
Does the program have clear theory of change/logic model?
Is the results framework of the program coherently articulated?
Do the outputs, outcomes and goal follow results chain logic?
Are the objectives clear and realistic?
Are they measurable (quantitatively or qualitatively)?
Do they respond to the needs identified?
Do proposed program activities lead to goals and objectives?
(Source: Davis 2013)
2|P a ge
YES
NO
Introduction of the program
The evaluation’s intent was to track if the program of improving girls access through transforming
education being done by World Vision has achieved is intended objectives. Therefore, this chapter
is fixated on giving an insight of the Igate programme. The background of the organization, its
values, mission statement and the program description are going to be highlighted. A description
of the program itself was also given revealing the programme goal, objectives, outcomes, outputs,
activities and resources needed for the programme. The logic model and the program theory are
also going to be provided for the igate program
program. This evaluation will be limited to 10 rural district of Zimbabwe,
1.2.1 Description of organization
World Vision started operating in Zimbabwe in 1973, predominantly providing help to children’s
homes and providing relief to Zimbabweans in camps and institutions. In 1980, World Vision
changed to rehabilitation and small-scale development programs as the nation exiled and
previously displaced groups. Over the years World Vision has transitioned to large-scale
community development initiatives designed to address community needs, with a precise
emphasis on children, using participatory planning approaches and ensuring sustainability.
Currently, World Vision is the largest humanitarian organization in Zimbabwe, operating relief
and development projects across the nation benefitting more than two million people each year.
1.2.2 Description of the program itself (IGATE program)
The Girls’ Education Challenge (GEC) project “Improving Girls’ Access through Transforming
Education” (IGATE) was developed by World Vision Zimbabwe to help marginalized girls in
Zimbabwe’s poorest regions transform their futures through successful participation in highquality education. IGATE’s holistic theory of change is based on the contention that (a) enabling
key stakeholders to acquire knowledge, (b) strengthening existing and creating new communitybased systems, and (c) building the capacity of critical actors and structures are key elements for
changing knowledge, attitudes, and behaviors related to girls’ education and, eventually, to
changing realities on the ground. IGATE’s comprehensive and collaborative theory of change is
built on a lateral, multi-layered intervention approach using household/community- and school3|P ag e
based programming to support girls and their education. IGATE works directly with local
government ministries, parents, religious leaders, traditional leaders, school leaders, and the girls
themselves to increase knowledge and promote changes in traditional perceptions and practices
regarding gender to enable girls to attend school consistently and have equal conditions to
succeed academically. Enabling these key stakeholders to become agents of change is designed
to increase the sustainability and the impact of the interventions.
1.2.3Direct Target Group
The beneficiaries of the IGATE program are girls the primary and secondary school girls of
Zimbabwe rural areas.
1.2.4 Vision
For every child life in fullness
1.2.5 Mission
World Vision is an international partnership of Christians whose mission is to follow our Lord
and Savior Jesus Christ in working with the poor and oppressed to promote human
transformation, seek justice and bear witness to the good news of the Kingdom of God.
1.2.6Values
We value Integrity, Transparency, Honest, Compassion and Empathy
1.2.7 Goals and Objective
The Improving Girls’ Access through Transforming Education (IGATE) program seeks to identify
and lessen the obstacles and barriers that limit and hinder girls’ access to education , retention and
learning
outcomes
in
rural
districts
which
includes,Beitbridge,Gokwe
North
and
South,Nkayi,Lupane,Chivi,Insiza ,Mberengwa, Mangwe and Binga The project also aims to
educate institutionalized adolescent girls on different social matters affecting them. Project
activities are designed at preventing early drop-outs by making and allowing environment that
encourages girls that are already outside the system, to re-enroll and complete their education. The
program will realize this goal through discussions and lessons conduct by prominent people in
4|P ag e
Zimbabwe that they look up to. In partnership with communities collaborating with the Ministry
of Primary and Secondary Education, the IGATE project aims to teach parents about the
importance of education and schooling them to partake in their children’s education through
literacy-based activities. It is also of paramount importance to note that, the IGATE project aims
to make school more accessible for children, especially girls, who live long distances from school
buildings through the provision of bicycles. The project is working with schools in improving their
infrastructure so as to accommodate more pupils through the building of additional classrooms,
the building of housing for teachers in rural areas of Gokwe, and Chivi where it can be hard to
keep qualified teachers and the construction of sanitary and hygiene facilities, especially focused
on the needs of girls and children with disabilities. The program is founded UK Department for
International Development and objects to reach at approximately 101,448 marginalized girls from
467 schools located in four provinces in Zimbabwe.
1.2.8Outcomes
The project’s outcome is to increase knowledge, confidence and awareness to deal with the
different social challenges the girl child is facing. The program aims to have improved knowledge
on choosing a career, increased awareness in personal hygiene, increased awareness on sexual
health, increased knowledge on life after institutionalization, increased awareness and in dealing
with peer pressure and bullying, improvement in behavior and respect, improved time management
skills and improved knowledge on empowerment of a girl child. The project also aims make an
equitable school learning environment by working hand in glove with School development
Committees (SDCs) and Communities in Support of Girls’ Education (CSGE) groups which
contributed towards the development of school infrastructure sensitive to the different needs of
children.
1.2.9Outputs
As postulated by Rush and Ogbon (2010), yields are the immediate outcomes (products and
ventures) of the task which are to a great extent under the administration's control. The outputs
aimed by the program are lesson on literacy, financial skills conducted, lesson on career guidance,
education on personal hygiene, lesson on sexual health conducted, and discussion on life after
school conducted, counselling on relationships conducted and lesson on respect conducted.
5|P ag e
1.2.10Activities
Mayne (2009) characterizes activities as the undertakings did to execute the task and convey the
distinguished yields. These activities conducted in the program includes a lesson on financial
literacy, in accordance with the school curriculum of the Ministry of Education conducting a
lesson on career guidance and planning, conducting a lesson on Personal Hygiene, conduct a
lesson on sexual health, discussion on life after the institution, counselling on relationships, peer
pressure and bullying, teaching respect and tolerance and assisting discussion on abuse.
1.2.11Available Program Resources
The resources available for IGATE project includes human resources such as Field Officers,
District Coordinators, Enumerators, Monitoring and Evaluation Officers that offers training,
coaching to teacher enumerators among others. For financial resources there is going to be funds
which is going to be paid to the facilitators of the program.
6|P ag e
1.3 Logical model and program theory
LOGIC MODEL FOR IGATE PROGRAM
GOAL: IMPROVING GIRLS’ ACCESS TO EDUCATION AND LEARNING OUTCOMES
Table 2
Problem
Inputs
Activities
Outputs
Outcomes
-Marriages and
-Human Resource
-trainings on the
-lesson on career
-Improved
Pregnancy
I.e. Acting
effects of early
guidance
knowledge
Programs
child marriages
Manager, M&E
and the effects and
lesson on Basic
officer, Field
causalities of
literacy
Officers
pregnancy at a
financial
tender age,
management
and
then
on
negative
effects of child
marriages
and
unwanted
pregnancy
- conducting a
Long Distance
-Field officers
-Financial
lesson on sexual
-Literacy levels
health
are increased
-Provision of
Bicycles
Distance to school
is cutoff
resources
-Education
accessible
Shortage of
classes especially
-Literacy
in marginalized
improved
areas.
Builders
District
coordinators
-Construction of
is
School is more
classrooms
Learning is made
-the construction
easy
of housing for
7|P ag e
is
accessible
Financial
teachers in rural
resources
areas
Building materials
-construction of
improving literacy
Availability of
qualified
teachers
sanitary and
hygiene facilities,
especially focused
Increased
on the needs of
personal
girls and children
hygiene
with disabilities
1.3PROGRAM THEORY
A program theory according to Chen’s (1990), is a specification of what must be done to achieve
the desired goals, what other important impacts may also be anticipated, and how these goals and
impacts would be generated. A program theory is commonly assumption in a program under a
pertaining condition. Wilder Foundation (2009), expresses that a Program theory can often be
taken in a series of “if-then” statements. For instance, if a particular action is taken, then this would
happen.

If the resources specified, which are human, financial and material resources are available,
THEN the activities will be done.

If the activities such as, conducting a lesson on career guidance and planning, conducting a
lesson on personal hygiene, conducting a lesson on sexual health and basic literacy and
financial skills THEN the intended outputs will be achieved.
8|P ag e

If outputs which includes education on personal hygiene, lesson on sexual health conducted,
discussion on life after high school conducted, counselling on relationships conducted, lesson
on respect conducted, THEN the outcomes will be achieved.

If the outcomes such as provision of bicycles for learning, improved knowledge on choosing
a career and increased awareness in personal hygiene THEN, this will achieve the overall goal
of the project educate and facilitate a discussion with adolescent girls and young girls on
different issues that are affecting them.
v) Critique of Program theory and Model
The fundamental opinion of the programme theory and logic model highlighted above is that they
clearly indicate the relationship of the major problem and elaborate on solutions in a logical
manner but however they are challenges. One major obstacle that affects girls‟ education is related
to their self-defeating attitudes which compel them to have low aspirations about their lives largely
due to lack of local female role models and the environment that surround them. The IGATE Broad
Gender Analysis Study (2014) postulated that 8 structural inequalities at both district and
community level where top decisions making positions are dominated by men, deprived girls of
local role models who can mentor them and inspire their confidence and self-belief and aspirations
(CARE, 2014).More so provision of bicycles as alluded in the theory and model has caused major
problems in the sense that some parents in rural areas they do not value education henceforth
they use those bicycles for their daily modern routines therefore compromising the importance of
the project toward girl child education
The logic model assumes that if girls are taught on literacy n reality that not the case. The loan
sharks require certain collateral first for someone to get a loan and most of the beneficiaries they
don’t have all that. Also people learn from mistakes and no one is willing risk his wallet for
someone to make a mistake with it. Hence even though skills are now there the beneficiaries have
done nothing to change their status quo as entrepreneurship is concerned.
However, the logical program theory, make a clear, often visual, proclamation of the activities that
will bring about change and the results that are expected in a program. On one hand, a logic model
keeps stakeholders and participants in the program moving in the same direction by providing a
9|P ag e
common language and point of. For instance, the evaluator noted that, the program theory is
efficient in improving attendance and learning outcomes among the most marginalized groups
given the effects of a devastating regional drought which affected most parts of Southern Africa.
Findings indicate that attendances rates are increasing among girls in the lowest quintile of poverty,
hence showing the relevance of the program theory.
1.3.1 Summary
This chapter is the introductory chapter of the process evaluation of the Igate Program being implemented
by Word Vision Zimbabwe. The organization’s background, mission statement, vison and values were
highlighted. The program activities, outputs and outcomes were also provided. The logic model and
program theory were included. The following chapter is going to provide examples of other evaluations
which are similar to the Igate program.
CHAPTER 2: LITERATURE REVIEW
2.0 Introduction
This chapter offers a desk review of the available and current literature on educational programs
for adolescent girls and early grade learners within rural district of Zimbabwe. This section
provides the theoretical issues surrounding process evaluations, a review of similar
Evaluations and the implications of these reviews. These evaluations were compared paying
special attention to the methodologies adopted, the findings and the evaluation approaches used.
This also formed the basis of the methodology that is going to be adopted in this research.
10 | P a g e
2.1 Theoretical issues on process evaluation
This evaluation is based on theory of change. It is vastly appropriate for any academic work to be
directed by known theories as thoughts cannot be simply deracinated or taken from a vacuum.
Fourie (2007:230) portrays that a theoretical framework formulates the goals of theory, meaning
that before one can comment on how something works, he/she must first give a clear explanation
of the process. Theory of Change explains the process of change by outlining causal linkages in
an initiative that is its shorter-term, intermediate, and longer-term outcomes. The identified
changes are mapped – as the “outcomes pathway” – showing each outcome in logical relationship
to all the others, as well as chronological flow. The links between outcomes are explained by
“rationales” or statements of why one outcome is thought to be a prerequisite for another. A theory
of change is a structured approach used in the design and evaluation of social programs to explore
change and how it happens. It maps the logical chain of how program inputs achieve changes in
outcomes. • Building a theory of change with solid theoretical foundations and widespread buy-in
requires organizations to invest time and resources into a process with multiple steps and
participation at all levels of the program.
2.1 The Role of Key Concepts and Assumptions in Theory of Change
One problem identified in this review is the fuzzy use of language in ToC discussion. Terms such
as ‘theory’, ‘assumption’, ‘hypothesis’ and ‘linkages’ are used fairly interchangeably without clear
explanation. These terms lack clarity and often seem disconnected from their implications for
broader social science work. This is important since how these terms are understood may change
the way in which ToC is approached. Given their importance to the ToC discourse and to the JSRPTAF collaboration, the role of the terms ‘assumptions’ and ‘evidence’ was identified as in need of
deeper analysis. Within certain key pieces of literature, ToC is seen as providing practitioners with
an opportunity to engage more honestly with the complexity of change processes .Fletcher
(2009)postulates that ToC is seen more narrowly as extending the assumptions/risks column of a
logical framework. As becomes clear below, clarification on the role of the above concepts is
crucial if ToCs are able to accurately reflect the kinds of messy and unpredictable worlds they
attempt to engage with. At its best, ToC requires an engagement with wider social science theory
and research-based evidence. Such work is ultimately an attempt to describe and understand how
change happens in the world, and this is central to those practitioners engaging with ToC.
11 | P a g e
(Baethgen et al, 2003) denotes that, Perhaps most importantly, the confusion surrounding
assumptions feeds directly into what people expect ToCs to achieve. If we take the term
assumption literally, then ToC processes are simply about uncovering common beliefs, rather than
critically interrogating them. Broad application of the term ‘assumption’ could encourage a
superficial approach to ToC, rather than a nuanced attempt to engage with the complexity of
change processes. In this respect, it is problematic that assumptions are always seen as central to
ToC processes. If ToCs are to be taken as rigorous analytical pieces of work then the central
element of them perhaps should not be something that may lend itself to shallow analysis.
2.2 IGATE Theory of Change
Within IGATE’s Theory of Change (ToC), the school is the main base for programming, and
activities are spread from the school to key actors, centres, stakeholders, households, and to the
girls themselves. IGATE interventions were based on the belief that by working with girls,
mothers, and fathers as well as religious, traditional, and school leaders, all stakeholders’
knowledge and understanding of the importance of girls’ education would increase. This will
enable all stakeholders to be agents of change, to increase the impact, and to ensure the
sustainability of IGATE interventions once the project has ended. IGATE’s Theory of Change
(ToC) identifies nine contextual challenges and major barriers that must be addressed in order to
achieve the desired outcome of “girls who are empowered to stay in school and perform better.”
IGATE then addresses these barriers through school-based, multi-layered interventions. That is,
the school is the main base for programming, and activities are spread from the school to key
actors, centers, and stakeholders, households, and the girls themselves. IGATE interventions are
based on the belief that by working with mothers and fathers as well as religious, traditional, and
school leaders, stakeholders will increase their knowledge and understanding of the importance
of girls’ education, thus enabling them to be agents of change to increase the impact and ensure
the sustainability of IGATE interventions.
2.3 What is process evaluation?
Process evaluation enables an association to look at how it builds up its structures and its projects
so as to accomplish the results that the association expects to accomplish (Boothroyd, 2018). It is
an assessment that is led midlife of the venture where it looks for information with the goal of
understanding what will happen with the program (Brad Rose Consulting, 2015). For instance, if
12 | P a g e
the beneficiaries are getting the administrations they need. Hence, process assessments record the
procedure of a program's execute
2.3.1 Purpose of process evaluation
The key objective of process evaluation is to inspect if the association is executing and conveying
as prearranged, on the off chance that they are arriving at the intended beneficiaries and in the
event that they are delivering the foreseen yields (Boothroyd, 2018). Process evaluations inspect
issues around devotion of usage and the conveyance procedure. Procedure assessments are
additionally worried about how a mediation affects members, association, networks and effect on
determinants of results.
Process evaluation can likewise comprehend why a few projects neglect to be executed effectively.
Bartholomew (2016), gives a model that a program may have a sound program, however then need
constancy as the developers don't pursue what has been arranged. Woodworker (2019), underpins
this and explains that process evaluation gives data on why a program was fruitful or not and
whether a similar program can be imitated. Process evaluation, additionally enables the evaluators
to make significant qualification between execution disappointment and hypothesis
disappointment
2.3.2 Evaluation questions answered by process evaluation
Process evaluation questions are critical as they question how a program is functioning for
example, the number and the beneficiaries who are benefiting from the program, how individuals
are gaining access to the programs that are being offered. Process questions, “highlight the
effectiveness of community school components that can be improved or adjusted” (Institute for
Education Leadership, 2018). The solutions to process questions ought to be conveyed
systematically so that one can use the responses to regulate and improve program implementation.
2.3.4 What they measure
CDC (2018), affirms that process evaluation help center and give structures to an assessment,
control the assessment procedure, encourage basic leadership about assessment strategies to utilize
and advise discourses about how assessment results can be utilized to improve the program. These
process evaluation questions address the following issues
I. regardless of whether the program exercises were cultivated
13 | P a g e
ii. The nature of program segments
iii. How well the program exercises were executed
iv. How outside variables impacted program
2.3.5 Methods that can be used
Process evaluation can adopt both quantitative and qualitative methods. In some cases, mixed
method of both qualitative and quantitative research. It makes use of tools such as questionnaires.
Moreover, this method makes use of tools such as Focus Group Discussions, interviews and
unstructured questionnaires among others.
2.4 Review of process of evaluation on improving access to education among adolescence
girls
This section reviewed some similar evaluations on adolescent girls which included mentorship
programs. This will include the objectives, a review of the methodologies, the findings,
conclusions and lessons learnt. Each project will be reviewed separately.
2.4.1
Comparative Table 3
Name
of Program/pro
Evaluation
Methodolo
ject
objectives
gies
Population
-process
To
Council
evaluation:
refugee
organizatio
Findings
conclusion
Lessons
s
learnt
n
the
deliver -Focus
girls young
awakened
domestic
program for workers
14 | P a g e
migrant
with
adolescent
expertise,
-the
-
group
program
the
guides
was very beneficiari
-
in-depth
interview
guides
fruitful
and
allowed -there is
es to
has reached
a
need
for
be alternati
ve
been
and it was mentor
greatly
done
to
be
girls
in knowledge,
domestic
service
and
valued
social
in capital
according
and well- to
to
consulte
the d before
received
original
they
plan
have
urban
decrease
by
Burkina
their
beneficiari
started
Faso
defenseless
es, tutors,
the
ness
communit
program
and
increase
y
their
members,
prospects.
UNFPA
Evaluation
of
to
UNFPA evaluation
support
to was done to
adolescents
and
assess
the
youth performanc
2008-2015
e
of
the
program
from 20082015
(PREP
Personal
Providing
Responsibili
lessons
-document
Accessibl
Adolescent Improve
review
e
s have not reportin
-interviews
-FGD
observation
on-line
for policy programmi
ng.
on
ty Education teenage
-FGD
improve
work
policies
to
be
adopted
The
Retaining
Attenda
program
regular
nce
managers
attendance
at program
pregnancy
highlighte
(PREP)
prevention
d that they sessions
preferred
that
can be a
the challenge
flexibility
to meet the
intended
15 | P a g e
to
the
Program
Evaluation
and
advocacy
survey
-Interviews
g
n was used included in research
or
s
informatio been
of
the beneficiari
program
es
to evaluate interviews
The
Beneficiari
Empowerm Youth
the
evaluation
es acquired a control
ent
Leadership
skills for the
highlighte
new
Program
Life Skills
beneficiarie
d that the leadership
(YEP)
Developmen
s
beneficiari skills
t Program
incubation
es
program
confident
Youth
Evaluating a
life
of
the
FGD
were
enough
with
the
-lack of
group
moderat
ely small
sample
size.
leadership
qualities
gained
2.4.2 Process evaluation: the filles éveillées (girls awakened) program for migrant
adolescent girls in domestic service in urban Burkina Faso
Girls Awakened” program was envisioned to deliver refugee adolescent domestic employees with
services, to reduce the chances of the adolescents to be abused. Girls awakened program covered
of three key components for a girl-centered absorbed program: a protected space for adolescents
to meet, access to same-sex friends to create interactive organizations, and a female instructor.
Mentors for Girls Awakened needed to satisfy definite prerequisites. The mentors were also in
their mid-twenties, so as to be advanced enough to be a good example for the beneficiaries yet
young-looking enough for the beneficiaries to feel great belief in the mentors. The recipients of
the program were aged 11 and 16 for Bobo Dioulasso and aged 15 and 18 for Ouagadougou.
2.4.3 Methodology
16 | P a g e
The research made use of both qualitative and quantitative research. Focus group guided in the
evaluation. In-depth interview guides were also used and were written and administered by an
attaché and the Program Coordinator. In-depth interviews with the addition of FGDs were used
in the evaluation.
2.4.4 Findings
i.
Program Highlights: Overall, center gathering discourses and top to bottom meetings
uncovered that Filles Éveillées was generally welcomed and increased in value by businesses,
network individuals, and members. Everybody met commended the organization, and the
tutors for the program and for the improvement on the behavior of the adolescents. All
businesses persons who were engaged expressed that they would recommend and positively
prescribe the program to other people and would give other local laborers a chance to take an
interest later on.
ii.
The organization detailed that they desired they had additional period between the preparation
they got and the dispatch of the program to take into account the enrollment and preparing of
coaches. To aid coach enlistment, Association Tié procured an open commentator who
communicate the accessible situations over an amplifier mounted on a vehicle in the pertinent
parts.
iii.
By and large, the organization and tutors detailed that the notion of nontoxic spaces was a
helpful idea and that it functioned admirably. Oral understandings were highlighted with the
network and additionally those responsible for the space so as to guarantee the openness and
the length of the program. The protected environments were inquired about and chosen by
the executing association, and their utilization was consulted with network pioneers as well
as the proprietors of the space. Holy places, mosques, and different strict spaces were
maintained a strategic distance from so as to keep from partner the program with a religion
and conceivably estranging a portion of the members.
iv.
Overall, the educational program was regarded to be efficient and applicable to the
requirements of the members. One individual from Association Tié remarked specifically
that he valued the intuitive idea of the educational program and this connected with the
members. Coaches expressed that they found thepretends, models, and stories to be especially
useful in connecting with members.
17 | P a g e
v.
Employers and coaches expressed that home visits were most useful in gathering their
cooperation and urging them to enable their workers to take an interest. Network individuals
likewise stressed the significance of network pioneers' help of the program, and their quality
at network sessions.
2.5 Evaluation of UNFPA support to adolescents and youth 2008-2015
2.5.1Methodology
Data collection approaches “that were used in this research were document and literature reviews,
a detailed financial analysis and interviews with key informants, group interviews, focus group
discussions, survey of UNFPA country offices, direct observation in countries visited and an eround table (UNFPA, 2015)”. Based on investor mapping at all levels, 670 beneficiaries were
consulted of which 325 were teenagers and youth.
2.5.2Findings
i.
The evaluation conducted indicated that there was need for a comprehensive and inclusive
system to be adopted to include the youth in all the programs that were being conducted.
The program offered a space for the young people to benefit but for future they should be
incorporated.
ii.
UNFPA confirmed that many states were shifting tier priorities in order to work in
partnership to help the youth to be part of nation building programs.
A culturally sensitive approach should be adopted in dealing with issues of sexual and reproductive
health and reproductive rights of adolescents of the youth.
2.6 Improving and Mentoring Education Outcomes Middle School Students: A Program
Evaluation New York by Maerz, D (2015).
The evaluation offers an insight of a program based on overall student accomplishment in terms
of grades,the program was a pilot study that focused on attendance to lectures improvements in
grades and behavior. Therefore, the evaluation focused on the students’ who were already in school
18 | P a g e
to see if there were going to be any improvements. There was a control and intervention were
offered to the other.
2.6.1Methodology
Quantitative approach was employed in tis evaluation and quasi-experimental design was used.
Pre- and post-test data were collected concerning schoolchildren’s GPA, total absences, unexcused
absences, and behavior referrals. Changes in pre- and post-test data for mentors were calculated
and linked against those who did not benefit from the program.
2.6.2 Findings
i.
The performance of those who received the intervention decreased wile tat of those who
were not part of them increases. None of these results were statistically significant, the test
pattern indicates unpredictable results concerning the welfares mentoring has on
educational performance.
ii.
Absence f the students affected the performance of both students and mentors. In many
cases, the absence was unwarranted for and the students gave excuses.
The anticipated cane that the behavior and performance of the student was going to improve
because of the intervention is not what happened. This therefore means that the program was not
as effective as expected
2.7 Personal Responsibility Education Program (PREP) Evaluation (2018) UK
The program focused on adolescents who were at the verge of being affected by pregnancies which
was probably going to affect their performance in school as well. A social worker leads the
program after familiarizing herself with the students. Most of the beneficiaries of the program were
either homeless or in foster care. Two-thirds of the beneficiaries were either suspended or barred
from school. Poor attendance however, remained a major challenge.
2.7.1 Evaluating a Youth Leadership Life Skills Development Program
i. The program taught the students life skills that were going to be useful to tem in their day to day
life or for their survival especially since most of them were either homeless or in foster care. .
19 | P a g e
ii. Since it was an experiential education program it put a lot of pressure on the implementers to
show that the program was effective
Comparing the evaluations
The evaluations showed that those who are in vulnerable situations for example institutionalized
children, migrants and teenagers are prone to a lot of depression, drug use and other social ills.
This therefore has caused many challenges hence the introduction of these mentorship programs.
However, attendance still remains one of the major challenges. This evaluation in the Alabama
state showed that using a small sample size and lack of a control group can raise challenges of
attribution.
Implications of the review
The review of these similar evaluations mapped the way for deciding the evaluation questions and
the methodology that was adopted for the evaluation. More so, the lessons learnt, evaluation
objectives and the findings from the reviewed evaluations played a pivotal role in the development
of the following Chapter which is Chapter 3.
Chapter summary
This chapter evaluates and compares the similar evaluations that have been done before. The
common subjects for these evaluations are that they focused on improving education outcomes
programs for marginalized and adolescence girls. Some of the evaluations also focused on
vulnerable girls such as refugees. This literature review is going to form the basis for the
methodology to be used.
CHAPTER 3: EVALUATION DESIGN
3.0 Introduction
This section reviews the methodology that has been adopted for this evaluation design. The
evaluation objectives and questions were highlighted as they guided the methodology to be used.
The focus of this chapter is also going to be on the theoretical framework. The several data
20 | P a g e
collection methods and the amplification of the evaluation design will also be included in this
chapter and these will be hinged on the data analysis matrix.
3.1 Evaluation objectives
The evaluation is directed by the succeeding objectives:
i.
To identify the extent to which the Igate programm has achieved its intended objectives
and outcomes
To identify and lessen the obstacles and barriers that limit and hinder girls’ access to
ii.
education , retention and learning outcomes in rural area
iii.
The project also aims to educate poorest adolescent girls on different social matters
affecting them.
iv.
To assess if the Igate program is appropriate for children in rural areas
v.
To assess the challenges being faced in implementing the Igate program
vi.
To determine how the igate program can be improved.
3.2 Evaluation questions
The evaluation was hinged on the following evaluation question;
i.
Is the Igate program being implemented according to the plan?
ii.
How does the igate program contribute towards learning and transition outcomes for
the girls aged 6 -18 years
iii.
Is the Igate program relevant for the poorest girls in rural areas of Zimbabwe?
iv.
What are the challenges being faced in implementing the igate program?
v.
What can be done to improve the igate program?
vi.
3.2.1 Process for coming up with the Questions
vii.
The researcher came up with the evaluation questions through interactions with the
program implementers, the volunteers, staff members and social workers. Engaging
these people amplified their understanding and appreciation, and made effective use of
the findings when they would be available. The evaluator used two methods to
formulate the questions, firstly he used the programme theory to decide which areas
articulated by that theory, that are of interest and relevant to the particular program
evaluation and secondly was from the program stakeholders sand implementers. The
21 | P a g e
role of the evaluator was to outline the inquiries, raise issues that may be disregarded,
distinguish parts of program's activity and results that warrant request and outcomes
that warrant inquiry and to draw out concerns from stakeholders that can be translated
into questions that can be answered by evaluation research. The data was collected from
stakeholders through snowball sampling of key informant interviews and Focus Group
Discussions as well.
viii.
3.2.2 Standards by which program performance may be evaluated
ix.
The assessment questions pursued past execution, targets set by program managers,
master conclusion, pre-intercession pattern levels for target populace, conditions
expected without the program, cost or relative cost, needs of target populace, program
objectives and destinations, proficient models, standard practice, standards, lawful
necessities and morals, for example, social equity and value. These norms were
considered during the plan of the evaluation questions.
x.
3.3 Sampling
xi.
Purposive sampling
xii.
Purposive sampling may include concentrating the whole populace of some constrained
gathering or a subset of the populace. The evaluator chooses a sample based on whom
they think will be proper for the investigation (Fiske, 1996). The researcher targeted
those that she knew had certain pertinent information when conducting the interviews.
This was to save time as the researcher interviewed the program staff about the basic
day to day running of the project.
xiii.
Convenient sampling is the other sampling technique that was adopted in this
evaluation. The convenient sampling technique involves a sample size that is
convenient at a specific time. The researcher used convenient sampling to interview the
beneficiaries of the program since some of them were not always available due to the
duties they performed after their studies.
22 | P a g e
Table 4 data collection matrix
Data sought by question
Evaluation
Data Sources
Sample
size
Question
rationale
i. Is the Igate i. Qualitative data on if Monthly reports
Documents
program
Proposal,
being the program has been
Performance reports
applied according to plan
implemented
according to the
plan?
Document review on the
activities
being
and
such
as:
Workplans;
Activity
Reports;
Quarterly
Progress
Reports
done
against what was planned
Questionnaires
Purposively
selected
administered
program
officers,
mentors and volunteers
Questionnaires
administered to al he
program staff
ii. How does the
Qualitative data on how Focus
group Purposively selected 2
program the igate program is discussion guide
Igate
contribute
helping in learning and
group discussion with
the program managers,
towards learning transition outcomes 6-18
mentors,
and
and caretakers
transition years
rural
girls
in
volunteers,
outcomes for the Zimbabwe
poorest girls in
rural
areas
of
Zimbabwe?
iii. Is the igate Qualitative data on how Semi
program
the
program
structured Interviews
is interview guide on with
the
managers,
23 | P a g e
conducted
program
mentors,
appropriate
for appropriate
adolescents?
adolescents
for how the program is volunteers
appropriate
Quantitative data on how Focus
and
caregivers
group 2 FGD with the program
appropriate the program discussion to show if implementers,
is
the program is being volunteers
helpful
and
caregivers
2
FGD
with
the
beneficiaries
iv.
What are Qualitative
and Semi
structured Interviews
the
challenges quantitative data on the interview guide to with
being
faced
in challenges that are being highlight
conducted
the
program
the managers,
mentors,
implementing the faced
challenges that are volunteers
and
igate program?
being
faced
implementing
in caregivers
the
program
Focus
group
discussion guide to 2 FGD with the program
explain
challenges
the implementers,
being volunteers
and
caregivers
faced
2
FGD
with
the
beneficiaries
v. What can be Qualitative
and Semi
structured Interviews
done to improve quantitative data on the interview guide to with
the
program?
igate improvements
program
and provide information managers,
mentors,
recommendations given on how the program volunteers
and
for the program
24 | P a g e
the
conducted
should be improved
caregivers
Focus
group
discussions on how
to improve the igate
program
2 FGD with the program
implementers,
volunteers
and
caregivers
2
FGD
with
the
beneficiaries
3.4 Methodology and justification
Mixed research method was adopted in this evaluation. Mixed methods approach makes use of
both quantitative and qualitative techniques to gain a broad perspective on a problem. The most
effective mixed methods approach deliberately take advantage of the strengths of each approach in
context of a particular problem (Berent-Spillson, 2018). For instance, when one is conducting an
interview to discover how fulfilled understudies are with their examinations. One can utilize openfinished inquiries which help to learn things one never contemplated and increase new experiences.
Afterward, reviews can be utilized to test these bits of knowledge on a bigger scale. Streefkerk
(2019), affirms that it is likewise conceivable to begin with a study to discover the general patterns,
trailed by meetings to all the more likely comprehend the purposes for the patterns.
Mixed research method also makes use of triangulation which according to Schoonenboom
and Johnson (2017) seeks to:
1. Union, authentication, correspondence of results from various techniques;
2. Complementary looks for elaboration, upgrade, representation, explanation of the outcomes
from one technique with the outcomes from the other strategy;
3. Development tries to utilize the outcomes from one technique to help create or illuminate the
other strategy, where improvement is comprehensively understood to incorporate testing and
execution, just as estimation choices;
25 | P a g e
4. Initiation looks for the disclosure of conundrum and logical inconsistency, new points of view
of structures, the recasting of questions or results from one technique with questions or results
from the other strategy;
5. Expansion tries to broaden the expansiveness and scope of request by utilizing various
strategies for various request parts.
3.5 Qualitative research
Qualitative research is commonly more explorative, a grouping of research that is reliant on the
collection of behavioral or observational data that can be understood in a subjective manner. It has
a wide opportunity and is characteristically used to explore the causes of potential problems that
may exist Sis International (2018). The following tools were used for data collection in this
evaluation
3.5.1 Semi structured interview guide
A semi structured interview was utilized for subjective research strategies. Semi structured
interview is controlled with a frankly open arrangement which permits engaged, conversational,
two-way correspondence. Not all inquiries are planned and expressed initially. Most of the
questions are made during the interview, allowing both the questioner and the interviewee
compliance to go into sensitivities when required (Keller and Conradin, 2019).
A semi structured interview is a kind of meeting which the evaluator asks just a couple of
foreordained inquiries while the remainder of the inquiries are not arranged ahead of time. A
portion of the inquiries that the evaluator posed were free streaming while the other were raised
during the meeting (Zojceska, 2018). There are three major types of semi structured interviews
namely moderately, hybrid and combined interviews. The evaluator used semi structured
interviews as they allowed her to probe further for more details regarding the Igate program.
Furthermore, they allowed the respondents to express themselves which allowed the evaluator to
have knowledge of more information that they evaluator was not aware of.
3.5.2 Observation checklist
The researcher also employed the observation checklist for collecting data. Sauro (2015), states
that there is complete observation wherein the evaluator is a segregated spectator where the
evaluator is neither seen by the participants. The Hawthorne Effect as mainly been the critic for
26 | P a g e
observations. Many scholars explain that the participants change the way they behave if they know
they are been observed.
For this evaluation, the evaluator was an observer as participant here the evaluator is known and
familiar to the program officers and beneficiaries and in many cases, they know the evaluators
goals of the observer. There is some communication with the beneficiaries, but the interaction is
limited. The evaluator’s aim is to play a neutral role as much as possible. This is because the
researcher was once a volunteer at the institution and the children and mentors are familiar with
the evaluator. This method is important as it allows the researcher to gain an insight of how the
students and mentors interact with each other.
3.5.3 Document Review
This is a methodical strategy for assessing evaluation documents which included both printed and
electronic web material. Triad, (2016) attests that archive investigation necessitates that
information be analyzed and deciphered so as to inspire importance, increase understanding, and
create experimental learning. Reports contain content and pictures that have been recorded without
the evaluator’s intervention. In this evaluation, document review included documents from the
following sources:
i. Mentors’ activity reports for each mentorship session conducted
ii. Attendance register for each session
iii. Individual progress report card
iv. Igate progress report
v. Monthly progress reports
vi. Quarterly reports
3.5.4 Focus group discussions
Focus group discussion are mainly adopted in the social sciences and in this case, they were used
in qualitative approach. They are used to collect data from participants with similar needs or
characteristics. Despite the fact that the use of this strategy in qualitative research has been broad,
there are no basic consensus on the rules that have to be applied wen using them.
27 | P a g e
Baral, (2016), asserts that a focus group discussion comprises of a moderator who is the
interviewer and the participants who are usually an average of 8 to 12 participants. In this
evaluation a focus group discussion comprised of at least people especially the one for
beneficiaries. The focus group discussion of the program staff comprised of 10 participants as the
researcher also wanted to include the caregivers. In some cases, the researcher had to switch from
English to vernacular as some of the participants were not articulate English.
3.6 Quantitative Method
For quantitative research questionnaires were the data collection tool that was employed in this
evaluation.
3.6.1 Questionnaires
The research also relied on the questionnaires to source of information from the beneficiaries of
the program. As indicated by Pandey (2015), a questionnaire is a deliberate accumulation of
questions that are submitted to an examining populace from which data is obtained.Open ended
and closed questions were used to obtain both qualitative and quantitative data. The researcher
administered the questionnaire to the respondents and waited for them. The researcher did this in
order to translate from English to Shona if there was need. The researcher also did this to save time
as there was no need to return to collect the questionnaires. In order to ensure that the information
was not lost or tempered with, the researcher also made sure that the questionnaires were also
uploaded on Kobo Collect.
3.7 Data Analysis Strategy
Data analysis is the most crucial in research and it summarizes all the data collected in the research
(Durcevi, 2019). Weak data analysis produces inaccurate results that not only hamper the
authenticity of the results but also makes the results unusable (Bhatia, 2018). The reason for data
analysis is to change data or information into a response to the first research question. In this
research both quantitative and subjective information were produced, it pursued in this way that
quantitative and subjective examinations were finished.
3.7.1 Qualitative data analysis
Qualitative analysis was conducted to deduct the meaning from the text that the evaluator had to
derive meaning from the data collected through interview guide, focus group discussion guide and
28 | P a g e
the observation checklist. Thematic analysis was used to analyze the implementation of the igate
program
in
the
rural
areas
which
includes
Gokwe,
South
and
North
Chivi,Mberengwa,Lupane,Nkayi,Zvishane . Nvivo was also used to highlight the major themes
form the semi structured questionnaires that were used in the research. The questionnaires were
uploaded on Kobo Collect to avoid the information being misplaced.
3.7.2 Quantitative data analysis
Numeric information was broke down with utilization of tables, pie charts and different graphs to
demonstrate the recurrence of event through building up measurable connections between factors.
Data analysis was done on SPSS (Statistical Package for Social Sciences) and any disparities were
checked against every unique poll to guarantee exactness. Further to this, standard information
cleaning methods were executed, for example affirming every missing an incentive against unique
surveys, checking for information honesty, just as crosschecking anomalies on singular factors.
The information was then examined by creating recurrence tables and cross-organizations.
Qualities for quantifiable information had the information assembled into classifications that
reflected Evaluation Questions. Cross arrangement of factors were additionally used to indicate
relationship and how factors affected the other, for example relationship between loyalty of portion
and kind of incapacity.
3.8 Ethical considerations
It is important to take into consideration ethics in this evaluation so as to observe professionalism.
In this study, the evaluator prescribed to ethical considerations as the principles and guidelines that
govern the professional conduct of parties involved in this evaluation. The evaluator is going to
try and take the following into consideration when carrying out the research:
i)
Avoid asking the participants leading questions
ii)
use of vernacular language and in this case use Shona
iii)
avoid promising gifts to the participants to avoid getting misleading information
29 | P a g e
Ethical considerations shall be maintained first and foremost since they are the only way that can
carry out the evaluation. The evaluator sought for a clearance letter from the Lupane State
University to carry out the research. There was also the need to also seek for permission from the
rural distroct councils and from school heads. The evaluator also took into account 4 principles
which are included in research ethics that are informed consent, voluntary participation,
confidentiality and anonymity. In addition, other ethical considerations in research that will be
considered is informing the participants about the research. Before conducting the evaluation, the
respondents need to be informed and aware of what they are participating in. For those who prefer
to remain anonymous, pseudo names shall be used during the research. This will be done to avoid
serious violations of privacy and confidentiality rights of individuals. The issue of voluntary
participation is also going to be taken into account. The participants should not be coerced into
participating in the research. Throughout the study the researcher will need to maintain high level
of professionalism.
3.8 Validity and Reliability
Crossman (2019a), affirms that reliability is how much an estimation instrument gives similar
outcomes each time that it is used. For this evaluation, mixed research methodology which is the
utilization of qualitative and quantitative research was received to guarantee solid outcomes are
achieved. Legitimacy is the level to which the examination estimates what it is intended to think
about, and manages the reliability of the results (Crossman, 2019b). This assessment attempted to
gauge the contrasts between the gatherings that got the mediation and those that had not gotte
30 | P a g e
CHAPTER 4: DATA PRESENTATION, ANALYSIS AND DISCUSSIONS
4.0 Introduction
This chapter presents the evaluation findings, analyses and interprets them in relation to
answering the evaluation question and objectives of the evaluation. Quantitative data which
was attained from the questionnaires that the participants answered were analyzed using
Statistical Package Social Sciences (SPSS). Quantitative data was combined with that of
qualitative to make the evaluation findings credible, reliable and valid. The data gained from
using interview guide was analyzed using thematic approach. Pseudo names were used for both
the participants of the interviewees, focus group discussion and the questionnaires.
The topics that will be covered by the evaluation include:
i.
Effectiveness and efficiency
ii.
Are the beneficiaries aware of the programme?
iii.
Understanding the objectives of the programme
iv.
Relevance of the programme
v.
Satisfaction of the beneficiaries with the Igate Programme
vi.
The major challenges being faced.
4.1 Discussions
vii.
The evaluation made use of interviews, questionnaires and focus group discussions.
The program staff members, the tutors, beneficiaries and the guardians of the
children who are 5-15 years within the the 10 rural districts under Igate programm
were interviewed and some participated in the Focus Group Discussions. For the
purposes of discussion and presentation of the evaluation findings, participants of
this evaluation are going to be labelled as summarized by the table below:
Respondents
Description
Participant 1-30
These are the beneficiaries of the program who are with in the 10 rural
district of Zimbabwe
Participants A-J
These are the program staff that are implementing the project
Participant V-Z
These are the guardians of the 5-15 years within the Igate Programm
Table5 Discussions
4.1 How well were target groups reached?
IGATE was designed to reach the most vulnerable girls in some of Zimbabwe’s poorest
communities. IGATE districts were among the poorest nation-wide with low performance
rates. Based on end line findings from monitoring, quantitative survey, and interview data,
IGATE successfully identified and reached marginalized girls, families, and communities.
The project successfully reached parents via MGs and MC groups, VSL, CSGE, and SDCs.
The project successfully engaged marginalised girls via PW clubs, BEEP, and HR. Religious
and traditional leaders were reached in all communities via CoH. In addition, both boys and
girls in primary schools were reached via HR, as well as teachers in the treatment schools.
The project was limited in reaching girls recruited in older grades at baseline because IGATE
had very few secondary schools as treatment schools. Many older girls in the treatment group
‘missed’ the in-school treatments (such as Happy Readers) because they were either postprimary when IGATE began, or they moved into post-primary before the interventions
started at schools. (Also, Happy Readers books are written for lower primary grade-level
readers or those just beginning to read.) The timing of interventions for older girls thus most
likely affected learning achievement. WVZ also had trouble replacing out-of-school girls and
those who had migrated; thus, some of the marginalized out-of-school girls probably were not
reached.
4.2. Fieldwork Reflections: Demographic Characteristics of Respondents
4.2.1. Analysis of Respondents by Academic Level
Figure 4.1 below shows the level of education of respondents. 14 respondents constituting
19.7% only went to school up to primary level. Those under secondary were 10 constituting
14,1 %. Those who did not go to school at all were 47 constituting 66, 2 % of the total
respondents.
Analysis of Respondents by the Education Level
Source: (By Author)
32 | P a g e
Figure1
Source: (By Author)
Figure1 shows that most respondents are not educated at all.
4.2.2. Analysis of Respondents by Sex.
4.2.2. Analysis of Respondents by Sex.
Figure 2 shows that 20 respondents were males and 52 respondents were females.
(Source: By Author)
Figure 2 shows that most respondents were women.
4.2.3.Analysis of Respondents by Age
Figure 3 shows that respondents aged between 7-12 were 33.8%. The respondents between
the ages 8-10 were 25.35. and those above 13 years were 40.85%.
33 | P a g e
AGE
Age above
25.35, 25%
40.85, 41%
Years
Years
33.8, 34%
Figure 3
Source (SPSS 2020)
4.3 efficiency and effectiveness of the IGATE Program
This section addresses evaluation question number 1 which was focused on if the IGATE
program was being implemented according to plan. The project staff at World Vison and the
guardians were all interviewed and some participated in focus group discussions and answered
questions on the effectiveness and efficiency of the Igate Programm program. The evaluator
also made use of the work plans and program documents that were available.
i.
Program Staff
Of the 10-program staff that participated in the evaluation, all of them articulated that they
understood the activities that they were supposed to engage in. These are some of the responses
that were highlighted during the interviews and focus group discussions.
1. Tutoring the Adolescence at within the Rural district of Zimbabwe
2. Awareness campaigns on the advantages of volunteering
3. Awareness campaigns on the importance of adoption and foster care
4. Fundraising programs
5. Weekly and monthly report writing
6. Offering psycho social support
34 | P a g e
7. Counseling services
8. Mentoring the adolescents on different social issues
9. Life skills training
One of the interviewees who is a tutor explained that
At the end of the day the most important thing that we do is to ensure that these
children acquire skills and are able to navigate through different social issues
that are affecting them. We do understand that they do not have a strong support
system if they are at this institution. We try to bridge that gap and offer them
that. (Interviewee C, 2019).
Modi, etal (2016), supports this and highlights that the major challenge for marginalize d
children is a sense of belonging and attachment then becomes a problem. They will have a
challenge with trusting anyone as in most cases they would think that they would be left.
Therefore, they advocate for programs such as these so that the institutionalized children will
feel safe and have an environment to open up.
The evaluator managed to compare the activities that were mentioned with those that were on
the program documents. Most of the programs that were highlighted on the work plans were
conducted. The evaluator also noted that there were Igate Programm programs that were
supposed to be held every first Friday of the month. However, there were a few instances when
they were not conducted. The evaluator also noticed that there were some weeks when a guest
speaker was not available for the program.
35 | P a g e
4.4. Aware of the programme activities
Figure4
Source: SPSS, (2019)
60% of the participants explained that they were aware of the programs that were supposed to
be conducted. 40% of the respondents highlighted that they were not aware of the programme
activities that were being carried out at the Igate programme. Of these 40%, they included the
guardians as they explained that they are not involved in the day to day activities being carried
out at. One of them explained that
We just know that there are programmes that they have at World Vison,but other
than tutoring school work I do not have much knowledge of their programmes.
The children sometimes talk about the lessons they would have conducted.
(Interviewee Y, 2019).
The evaluation also showed that some of the younger beneficiaries were also not aware of the
programs and activities they were supposed to have.
4.5 Were the programme activities carried out according to the plan
The project’s impact on literacy outcomes shows different results for reconnected girls and for
the full endline sample of girls. As per the RCT design, the following sections report on the
reconnected (midline to endline) girls only.
Girls’ literacy outcomes for IGATE were measured by the Early Grade Reading Assessment
(EGRA) tests on letters and sounds (egra1), invented word reading (egra2), oral reading
36 | P a g e
fluency 1 (egra3), oral reading fluency 2 (egra4), and comprehension (egra5).1The letter and
sounds test was out of 100 questions, the invented word reading test was based on 50
questions; the oral reading fluency test 1 was based on 70 questions; the oral reading fluency
test 2 was based on 109 questions; and the comprehension test was based on five questions.
The first four literacy tests are scored based on the number of correct answers per minute.
The comprehension test is scored based on the number of correct answers. While results are
presented within the report for all five tests, the oral reading fluency tests are of greatest
interest to the FM, since scores from this test are to be compared to tests from reading
assessments the other GEC projects.
The EGRA literacy tests were administered to learners (either in schools or households) at
within the period of the Igate programm. All literacy tests were in English. The oral reading
fluency tests proceeded as follows: Enumerators asked the pupils to read passages aloud
(correctly and fluently); then they asked five comprehension questions to measure pupils’
reading comprehension skills. Enumerators asked learners in lower primary grades (baseline
Grades 1 to 5) to read one oral reading fluency (ORF) passage (ORF1 test), and they asked
students in higher grades (baseline Grades 6 to Form 2) to read two passages (ORF2 test) out
loud (correctly and fluently), but only if learners were able to read the first passage
satisfactorily.
The two passages from baseline were revised (equated) at midline and again at endline,
following standard EGRA operating procedures. The endline equating exercise was
conducted at Bluegum and Mwenembesi Schools in Gokwe South on November 2, 2016.
Twenty-two trained enumerators assessed 124 randomly sampled girls in Grades 3-6. While
the endline passages had the same level of difficulty as the midline passages, slight
differences in ORF mean scores signalled that equating strategies may be warranted. The
ORF mean scores for the first endline story was 2 percent less than the midline ORF1 scores,
while the ORF mean scores for the second endline story was 8 percent greater than the
midline ORF2 scores There were not significant changes to the DiD results when comparing
equated ORF scores with un-equated scores, so we have chosen to use the raw, un-equated
ORF scores throughout this report.
37 | P a g e
The estimation approach outlined in the outcomes spreadsheet uses a weighted endline target
literacy scores that is a combination of the EGRA ORF1 scores for Cohort Grades 1 through
5 and EGRA ORF2 scores for Cohort Grades 6 through 9. The weighted endline target
literacy score for EGRA ORF1 is 6.02. IGATE achieved an unadjusted, weighted endline
EGRA ORF1 score of 4.26, which is 70.76 percent of the target score. The weighted endline
target literacy score for EGRA ORF2 is 6.45. IGATE achieved a weighted endline EGRA
ORF2 score of -0.58, which is -9.05 percent of the target score. Together, the weighted
endline target literacy score from both ORF1 and ORF2 is 6.16. Using an unadjusted (fullsample) measure, IGATE achieved 46.08 percent of the weighted target score for literacy.
Utilizing only the girls who could be re-contacted, the adjusted estimate of achievement from
the single-variate DiD regression is 1.67 for ORF1 and -1.70 for ORF2, which is a 11 percent
achievement overall of the weighted endline target score for literacy.
Table 1: Summary of project performance on literacy outcome
Result
Details
Comments
Literacy
Beta: ORF1 – 0.09; ORF2 – 4.81
The Betas and p-values
result
p-value: ORF 1 – 0.97; ORF2 – 0.08
reported are adjusted using a
(Baseline
Target: ORF1 – 5.37; ORF2 – 6.37; Combined – 5.63
single-variate cohort analysis
Performance: ORF1 – 2%; ORF2 – 76%; Combined
DiD regression model. We use
to Midline)
– 23%
robust clustered standard errors
that are clustered on the school
(i.e., sampling point). The
cohort analysis reduces the
number of observations to
1,366 for the ORF1 test and
413 for the ORF2 test.
Literacy
Beta: ORF1 – 1.67; ORF2 – -1.70
The Betas and p-values
result
p-value: ORF 1 – 0.21; ORF2 – 0.31
reported are adjusted using a
(Midline to
Target: ORF1 – 6.02; ORF2 – 6.45; Combined – 6.16
single-variate cohort analysis
Performance: ORF1 – 28%; ORF2 – -26%;
DiD regression model. We use
Endline)
Combined – 11%
robust clustered standard errors
that are clustered on the school
(i.e., sampling point). The
cohort analysis reduces the
38 | P a g e
number of observations to
1,891 for the ORF1 test and
853 for the ORF2 test.
MWAI used a single covariate DiD regression to calculate the effect of the IGATE treatment
on EGRA scores, assuming the learning trajectories of the treatment and control groups
would have been the same in the absence of the intervention. To measure the impact of
IGATE on marginalised girls’ literacy, girls are tracked according to their baseline grade
cohort (Grades 1 through 9). Only girls who were connected from midline to endline were
included in the DiD analysis. There were no ceiling or floor effects for the literacy tests at
endline.
The inversion of the pattern for ORF 2 from midline to endline is worthy of more
exploration. The number of observations points to the large number of replacements
(approximately 21 percent of the secondary grade cohort was lost at endline). Whereas at
midline, a number of the girls had migrated, at endline, Grade 7 and Grade 8 girls had taken
their exams and had stopped attending school before data collection ever began. While
enumerators sought out these girls according to the protocol, it is not uncommon for Grade 8
girls to move and for Grade 7 girls to be away from home (e.g., visiting relatives) after taking
their end-of-year exams.
Table presents the results from a single-variate DiD estimation on the five literacy
assessments. From midline to endline, the IGATE treatment as a whole produced a
moderately significant effect for the EGRA5 (reading comprehension) only (p=.071). When
looking at change over time for both treatment and control groups, Table highlights
significant increases on four of the five assessments for the treatment and control groups,
suggesting both the treatment and control group scores increased at similar rates. Yet, the
control group did not change, and the treatment group significantly decreased from midline to
endline on ORF2 (EGRA4), causing a significant difference in ORF2 scores at endline. This
may suggest hat the ORF2 was too advanced for most girls in both groups, or that there was
confusion specifically on this assessment. Table shows that, for the EGRA5, the treatment
group made significantly greater gains than the control group from baseline to endline.
39 | P a g e
Table 6: EGRA Difference-in-Difference estimations for re-contacted girls
EGRA Difference-in-Difference Estimations
VARIABLES
Treatment
Observations
R-squared
(1
(2
(3
(4
(5
)
)
)
)
)
e
e
e
e
e
gr
gr
gr
gr
gr
a
a
a
a
a
1
2
3
4
5
-
0.
0.
0.
1.
1.
1
8
6
6
6
3
7
7
7
9
7
4
1
4
7
*
(0
(0
(1
(1
(0
.7
.8
.3
.6
.0
4
4
1
5
7
9)
6)
7)
9)
1)
2,
2,
1,
7
7
8
8
3
9
9
9
5
4
6
4
1
3
0
0.
0.
0.
0.
0.
0
0
0
0
0
0
0
0
0
0
1
0
1
2
2
2,
*** p<0.01, ** p<0.05, * p<0.1
Table 7: Comparison of literacy outcomes by treatment status for re-contacted
girls
Outcome
Treatment
Control
Variable
Mean
Mean
p-value
Sig
Treatment
Control
Mean
Mean
p-value
egra1 (letter)
11.469
12.462
0.111
16.653
16.800
0.828
egra2 (invent)
26.303
27.394
0.190
33.044
33.412
0.685
egra3 (orf1)
54.473
53.469
0.639
68.655
65.681
0.163
egra4 (orf2)
102.569
105.739
0.283
95.802
101.814
1.575
1.642
0.325
2.152
2.094
egra5 (comp)
40 | P a g e
S
0.034 *
0.407
Table 8: Change over time in literacy outcomes by treatment status for recontacted girls
Treatment
Outcome
Midline
Endline
Variable
Mean
Mean
egra1 (letter)
11.469
16.653
egra2 (invent)
26.303
egra3 (orf1)
egra4 (orf2)
egra5 (comp)
Control
Midline
Endline
Mean
Mean
0.000 ***
12.462
16.800
0.000 ***
33.044
0.000 ***
27.394
33.412
0.000 ***
54.473
68.655
0.000 ***
53.469
65.681
0.000 ***
102.569
95.802
0.006 **
105.739
101.814
1.575
2.152
0.000 ***
1.642
2.094
p-value
Sig
p-value
0.231
0.000 ***
Sub-group analysis
Analysing specific components in IGATE’s ToC reveals interesting findings that were not
seen for the sample overall using only treatment as a predictor and EGRA assessments as
outcomes. (T-tests of the difference in means for a variety of intervention and outcome
variables related to individual barriers outlined in the ToC are presented in Appendix B.) In
addition to analyzing the reconnected sample from midline to endline, the entire sample of
girls surveyed at endline was also analysed. The full endline sample included 3,113 girls
(2,836 reconnected, 230 substitute, and 47 girls who could not be matched). Analysing
endline data from all girls who participated (not just reconnected) adds nuance and detail to
how specific interventions influenced a variety of outcomes.
In terms of literacy outcomes, analyses of the full endline sample show that the treatment,
specifically PW, MG, BEEP, and HR interventions significantly influenced literacy. At
endline, girls in the treatment group were significantly more likely to report they like reading
than girls in the control group. In addition, girls with a PW club at their school (as indicated
on the child survey) were significantly more likely to score higher on the EGRA3 at midline.
In addition, girls who joined the PW club (as indicated on the child survey) were significantly
more likely to score higher on the EGRA3 at both midline and endline.
Figure 5. EGRA3 Scores of Treatment and Control Groups in PW Clubs
41 | P a g e
Sig
Reading Fluency, PW Girls
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
Midline
Endline
Control
Treatment
Source:
SPSS
42 | P a g e
in the past year. These Head Teachers also had girls from their school who were significantly
more likely to score higher on the EGRA3 at endline and report they enjoy reading at endline.
A HoS in Chivi described how attendance and the reading culture have changed due to
IGATE:
Yes, not only girls but even in the whole school there are great changes
due to this Happy Readers program. Now our children are showing signs
of being able to read fast and [with] understanding. . . . Across the board,
I have noticed a great change. The teachers are quite excited because it
has improved the zeal to read. The culture to read has improved greatly.
The HoS in Insiza also linked the reading competitions, Happy Readers, and reading camps
(run by the MGs) and teachers being trained to a large increase in literacy. Interviewees
valued the holistic, multi-pronged approach to boosting literacy.
In addition, participants reported that BEEP was positively associated with increased literacy.
At both midline and endline, girls who received a bicycle through BEEP or the IGATE
project were significantly more likely to score higher on the EGRA3. A Gokwe North male
parent also linked BEEP to increased literacy. This parent explained that their children take
Happy Readers home (similar to interviewees in Chivi), and that that BEEP shortened the
time students walked to school, which helped increase literacy:
There is a big difference now. I want to give an example of my own child. She didn’t
want to read at all. Now whenever she comes home early she takes her [Happy
Reader] books and reads. So, I think the BEEP intervention helped quite a lot because
children now have more time to read. So, they now have this big advantage because
they have a chance to read.
Effectiveness of the IGATE program by district
In addition to the treatment showing a significant effect on literacy comprehension (EGRA5)
for the sample overall, EGRA results were also analysed by district, and other important
nuances emerged at the district level, which highlight additional positive effects of IGATE.
compare literacy outcomes by district by treatment status and over time.
The overall DiD estimations showed significant effects for EGRA5. In addition, there were
also significant effects for certain districts on certain other tests, in addition to EGRA5. For
43 | P a g e
instance, from midline to endline, the IGATE treatment produced significant effects for the
letters and sounds (EGRA1) and comprehension (EGRA5) tests in Mangwe and Nkayi, in
addition to the EGRA5 test in Binga. When comparing treatment and control groups at
endline, treatment groups in Mangwe scored significantly higher than the control groups on
the letters and sounds (EGRA1), ORF1 (EGRA3), and ORF2 (EGRA4) tests. Qualitative data
from Mangwe also underscores the impact IGATE has had on girls’ education in that district.
One girl from the PW club said, “I thank the IGATE programmes because I am now going to
get educated and go on with my education.” The Mangwe DSI said, “[Happy Readers] is a
great intervention and if books are kept safe it will enhance the children’s ability and instil
the culture of reading . . . These interventions were carefully crafted. They complement each
other.”
Similarly, in Insiza, the treatment group had significantly higher gains over time ,than the
control group on the comprehension (EGRA5) test. The DiD estimation for the
comprehension (EGRA5) test in Insiza also showed that the treatment significant affected
literacy.
Comparing changes over time also revealed significant differences in rates of learning by
district (). On four EGRA tests (except ORF2), the scores for the treatment groups in Chivi,
Gokwe South, Insiza, and Nkayi all significantly increased (p<.10) from midline to endline,
oftentimes at a greater rate than the control group increases, suggesting that IGATE helped
the treatment schools ‘catch up’. Qualitative data from the treatment districts highlighted how
and why IGATE interventions positively affected literacy (and thus attendance) for
marginalised girls. For example, a HoS in Insiza linked the positive impacts of girls attending
reading camps (run by the MGs), participating in the Happy Readers program, and joining in
reading competitions, as well as the HR teacher training as contributing to increased literacy.
She said, “almost every teacher is involved…sometimes we carry out a competition in the
Happy Readers books and children like to do [them]…the books are being used by
everyone.” HR has helped schools offer a variety of literacy activities (not just during school
hours), which positively impacted attendance and, thus, literacy.
While the treatment group outperformed the control group in some districts, this was not the
case in all districts. Analyses by district showed that the treatment and control groups in
Gokwe North were significantly different . For the ORF1 (EGRA3) test, the control group
scored roughly 30 points higher than the treatment group at both time points. For the ORF2
44 | P a g e
(EGRA4) test, the control group scored roughly 45 points higher at midline, and 60 points
higher at These results strongly suggest that the control group was considerably different
from the treatment group, which calls into question the validity of comparing these two
groups. As was noted under ,limitation above, the bias that resulted from the random
selection of treatment schools was particularly pronounced in Gokwe North, where all control
schools are in the district capital, and all treatment schools are in rural areas . The findings
are not surprising: the treatment group in Gokwe North had fewer caregivers with some (or a
complete) secondary education ,more treatment girls had a disability and also more treatment
girls travelled 42 minutes or more to school and more treatment girls had a lower proportion
of full-time qualified teachers in their school than the control group. All of these factors can
negatively affect girls’ literacy and learning.
Nevertheless, Gokwe North interviewees noted the big impact of IGATE, particularly the HR
program, had on improved literacy. For instance, one male parent from Gokwe North
explained: “There is a big difference now. I want to give an example of my own child. She
didn’t want to read at all. Now whenever she comes home early she takes her [Happy Reader]
books and reads.” The DSI for Gokwe North noted that the Happy Readers program
complements and supports the two MoPSE nationwide initiatives to boost literacy: Early
Readers Initiative (ERI) for primary schools and Performance Lag Address Programme
(PLAP) for secondary schools. These two initiatives were also being implemented in the
Gokwe North control schools, which may be one reason the scores of the control schools in
this district were substantially higher.
The control group in Binga had a significantly larger increase over time than the treatment
group for the ORF2 (EGRA4) test. The treatment groups in Nkayi had significantly lower
gains than the control group for the EGRA4 test (-9.372 wpm). Similarly, the treatment
groups in Lupane also had significantly lower gains than the control group for the letters and
sounds (EGRA1) test (-4.035 wpm), as did the treatment group in Binga had significantly
lower gains than the control group for the ORF2 Thus, for instance in Lupane, both the
treatment and control groups increased their letters and sounds (EGRA1) test scores over
time, but the control group increased at a faster rate.
Contextual factors may be affecting these results. For example, in Binga, the most
marginalized and least developed of the IGATE districts, the treatment group had
significantly more girls with disabilities, more out-of-school girls, and a lower proportion of
45 | P a g e
full-time qualified teachers, all factors that could potentially negatively affect literacy. The
Happy Readers program was implemented in Binga pre-IGATE, and other organisations have
been intensely implementing this intervention since Binga is an extremely marginalised
district. Other initiatives that may have been affecting the control group include cash transfers
in Binga. Interviewees also reported issues with implementation of the IGATE Happy
Readers program in the treatment schools. For instance, in-school girls from Binga described
seeing the Happy Readers books and knowing where they were being stored, but teachers
were not using the books with students. One girl explained “[the books are kept] in a
matron’s storeroom…sometimes we ask to borrow the books, but they refuse to give them.
We do not know why they refuse.” A Binga HoS reported that only two teachers had been
trained on Happy Readers, and that the Happy Readers books were understood to be
supplementary and therefore kept in the library. Thus, interview data indicate that the literacy
scores of the Binga treatment group did not significantly increase, due, in part, to the Happy
Readers program implementation issues in that district. Qualitative evidence points to two
main implementation issues in those schools where the Happy Reader program had less of an
impact, thus a significant shortage of books negatively impacting the book to pupil ratio, and
a lack of teacher training on how to use the Happy Readers materials to support students’
literacy.
Finally, although quantitative data was not collected about boys’ literacy levels, qualitative
data indicate that MG and Happy Readers also have a positive impact on boys’ literacy. One
mother from Insiza stated, “Before the reading camps [run by MGs], my boy could not even
read but now he loves reading. He sometime takes the Bible and asks me to sit down and read
to me which is an improvement.” Another parent from Lupane stated:
‘I would like to point out that there has never been any discrimination
between boys and girls in terms of embracing these initiatives. Thus, boys
have benefitted in the same way as girls…Every child attends a reading
centre which is within their proximity, there is no discrimination…the
children’s perception about these initiatives is different from that of adults. I
say so, because when these interventions were introduced, the main target
group was girls. As parents, we perceive this as a girl’s project. However,
children see these interventions as programmes for both girls and boys.’
Therefore, community members saw IGATE interventions as positively affecting both girls’
and boys’ literacy.
46 | P a g e
4.6 Understanding the objectives of the IGATE Programme
Figure 6
Source: SPSS, (2019)
84.44% of the respondents indicated that they had an understanding of the objectives of the
study while 15.56% did not have an understanding. The evaluation showed that the guardians
did not have an understanding of the objectives of the Igate program. In addition, the younger
beneficiaries did attend the program but they did not have an understanding of the objectives
of the program.
4.7 Relevance of Igate Programm
The Implementers of The Igate Program view the activities of the activities of the Igate
Programm were well aligned to its goal especially the aspect of inclusion of financial
Literacy ,and issues of Hiv and Aids improved the accessibility education and other
mainstream activities. Awareness raising on education emerges as highly important for
empowerment and increased understanding about education outcomes. FGD
respondents concur with these views adding that it also addresses the expectations of
caregivers of the adolescent for instance it provides assistive devices, enables access to
education, provides inclusion knowledge, and skills for savings. Through awareness
campaigns, communities are increasingly embracing education to marginalized girls .
47 | P a g e
However, during the discussion, it was noted that some parents anticipated that the
project would give food, clothes, and also pay school fees for their children.
The extent to which the services/activities of the IGATE are appropriate and relevant to
the specific needs of Caregivers was also determined through the survey as presented in
Figure 4.6 below. The highest proportion were those who indicated that they were
moderately satisfied and comprised 34 % of the respondents; followed by 25% (high
extent) and 21% (very high extent). The findings are that most of the children find the
services and activities of the Igate appropriate and relevant to their specific needs.
45
40
35
30
25
20
15
10
5
0
Not at All
Low Extent
Moderate
Extent
High Extent
Very High
Extent
Level of Service Appropriateness and Relevance of Services
Figure 7; Level of Service Appropriateness and Relevance of Services
48 | P a g e
4.8 Quality and Quantity of personnel
The questionnaires administered showed that all of the beneficiaries of the programme
indicated that the quality of the personnel was generally good.
40
40
satisification of the Igate Programm
40
35
30
25
20
15
10
10
10
5
0
agree 92.50
disagre7.50
Series 1
Series 2
Column1
Source: SPSS, (2019) Figure 8
92.50% of the beneficiaries indicated that they are satisfied with the program and services being
offered with the Igate program. 7.50% indicated that they are not satisfied with the s Igate
program. Those that have highlighted some disgruntlements expressed that the major challenge
was time consciousness. They explained that sometimes they don’t go to school due to long
distance barriers and more so they spent more time in fields especially in the rain season.
4.9Challenges being faced in implementing the Igate programme

Girls reported that the challenges to their regular school attendance included the
following: many household chores and responsibilities including caregiving, safety
concerns (sexual or physical violence), being sent home from school for non-payment
of school fees, having to walk long distance to school (increasing their vulnerability to
sexual or physical violence), poor weather conditions, unsatisfactory school toilets
(especially problematic when girls were menstruating), and a lack of menstrual
hygiene materials.
49 | P a g e

A Male Champion in Lupane described issues and challenges related to men’s
involvement in the lives of their daughters when he stated,
Culturally it has been a norm that fathers do not talk to their girl children about
growing up and basically the future of the girl child. Hence, we [Male Champions]
encouraged fathers to take their time to talk to their girl children. As for the rights of
girls and women, it wasn’t easy at first considering the fact that there is this 50/50
issue, most men didn’t accept that at first as they thought that women wanted to
challenge them. However, during training we were taught that women should not take
advantage of their rights, the male figure will continue playing his role in the
household in respect of women rights while women respect their partners too. Also,
culturally – it was unheard of to accommodate the rights of women as men were the
only ones who were believed to have rights. However, they ended up understanding,
even though it was after some time; they understood that women and girls have rights
too, like men.

Cash and liquidity challenges, teachers were forced to spend many hours waiting to
get cash from banks (as banks reduced the daily withdrawal limit to $50/day),
reducing contact time between teachers and learners (as outlined in the WVZ quarter
14 and quarter 15 reports).

The Gokwe North District School Inspector noted some challenges to the
implementation of the Happy Readers program as follows, “we did some assessments
in one of our cohort schools but the challenge is untrained teachers who cannot
properly administer Happy Readers but we are talking to the Ministry so that they can
transfer some untrained teachers who will be replaced by trained teachers who can
administer the Happy Readers.” A Village health worker in Gokwe North commented
that, “At times a child is participating very well in class but we notice that there can
be a problem with the teacher who can be weak. This needs to be rectified.”
4.10 Chapter summary
In conclusion, this chapter has explained the various discussions that were provided
when conducting this evaluation. The data analyzed was adopted through the use of
interviews, questionnaires and focus group discussions. Both qualitative and
50 | P a g e
quantitative data analysis was done since the research employed mixed research.
Thematic analysis was used to analyze the data.
CHAPTER 5: CONCLUSIONS AND RECOMMENDATIONS
5.0 Introduction
The evaluation was aimed at conducting a process evaluation for the Igate program that was
being implemented By World Vision . the objective of this chapter therefore is to provide the
summary of the research paying special attention to the research questions and objectives, the
data collection methods and tools used, the evaluation findings and proposed recommendations
for the evaluation.
5.1 Conclusions
The following were the evaluation questions that were adopted.
i.
Is the Igate program being implemented according to the plan?
ii.
How does the igate contribute towards learning and transition outcomes for girls
in rural district of Zimbabwe ?
iii.
Is the igate program appropriate for Girls in rural areas of Zimbabwe?
iv.
What are the challenges being faced in implementing the igate program?
v.
What can be done to improve the igate program?
Mixed research method was adopted in this evaluation. Mixed methods approach makes use of
both quantitative and qualitative techniques to gain a broad perspective on a problem. Semi
structure interviews, questionnaires, observational checklist, secondary sources and document
review were the data collection tools that were used in this evaluation. Thematic analysis was
used for collecting qualitative data using Nvivo and on the other hand SPSS was used to analyze
quantitative data from the questionnaires.
5.2 Main evaluation findings
i.
Effectiveness and efficiency
The project staff, girls , caregivers
within the 10 rural district of Zimbabwe twere all
interviewed and some participated in focus group discussions and answered questions on the
effectiveness and efficiency of the igate program. Of the 10-program staff that participated in
the evaluation, all of them articulated that they understood the activities that they were
51 | P a g e
supposed to engage in. The evaluator managed to compare the activities that were mentioned
with those that were on the program documents. Most of the programs that were highlighted
on the work plans were conducted. The evaluator also noted that there were igate programs
that were supposed to be held every first Friday of the month. However, there were a few
instances when they were not conducted.
Are the beneficiaries aware of the programme?
60% of the participants explained that they were aware of the programs that were supposed to
be conducted. 40% of the respondents highlighted that they were not aware of the programme
activities that were being carried out at the Igate Programme. the younger beneficiaries were
also not aware of the programs and activities they were supposed to have.
Understanding the objectives of the programme
84.44% of the respondents indicated that they had an understanding of the objectives of the
study while 15.56% did not have an understanding. The evaluation showed that the guardians
did not have an understanding of the objectives of the igate program.
Relevance of the programme
The highest proportion were those who indicated that they were moderately satisfied and
comprised 34 % of the respondents; followed by 25% (high extent) and 21% (very high extent).
The findings are that most of the children find the services and activities of the Igate
appropriate and relevant to their specific needs.
Satisfaction of the beneficiaries with the Igate programme
The beneficiaries of the programme indicated that the quality of the personnel was generally
good. 92.50% of the beneficiaries indicated that they are satisfied with the program and
services being offered with the Igate Programm program. 7.50% indicated that they are not
satisfied with the Igate Programm program. Those that have highlighted some disgruntlements
expressed that the major challenge was time consciousness.
The major challenges being faced
52 | P a g e

Dropping out of some school girls were highlighted as the major challenges being faced
in the implementation of the programme. Recommendations were offered for the
smooth implementation of the IGATE Programme.

Cash and liquidity challenges, teachers were forced to spend many hours waiting to
get cash from banks (as banks reduced the daily withdrawal limit to $50/day),
reducing contact time between teachers and learners (as outlined in the WVZ quarter
14 and quarter 15 reports).

Girls reported that the challenges to their regular school attendance included the
following: many household chores and responsibilities including caregiving, safety
concerns (sexual or physical violence), being sent home from school for non-payment
of school fees, having to walk long distance to school (increasing their vulnerability to
sexual or physical violence), poor weather conditions, unsatisfactory school toilets
(especially problematic when girls were menstruating), and a lack of menstrual
hygiene materials.
5.3 Recommendations

A first recommendation is for all IGATE interventions to be implemented in the
target schools and communities, as quickly as is practicable while maintaining
sufficient staff and participant capacity and intervention fidelity to allow time for
the model to function as a whole. In particular, Happy Readers, with its expected
direct effects on reading skills, should be (and was) prioritised. It is important to
ensure, of course, that the multiple interventions can be managed by the
implementing teams and that they can be absorbed in the households,
communities, and schools in target areas. With vehicle and other resource
constraints that IGATE faces a thorough resource review is needed to facilitate the
implementation of the remaining interventions.

Consider scaling up BEEP. Given the role of distance in constraining girls’ access
to school, it is recommended that BEEP be scaled up fully, and that IGATE revisit
the question of which girls are considered for participation. In addition, it is
critical to ensure that support mechanisms (e.g., assistance mending flat tyres,
doing overall bicycle maintenance, and so on; and ensuring that girls retain
primary control of bicycles for use in getting to and from school) remain strong
and are fully in place.
53 | P a g e

The study’s findings support that a holistic approach to girls’ education support is
critical due to the interwoven nature of the barriers hindering or preventing girls
from going to school, staying in school and doing well in school. Having an
intervention that works with the girls themselves is also of critical importance

Inclusion of additional comparison groups for instance more , boys, would make
for a more robust study and would provide greater insight into the gender
dynamics that IGATE has demonstrated must be addressed.
54 | P a g e
REFERENCES
Leviton, L. C., et al. (1998). "Teaching evaluation using evaluability assessment." Evaluation
4(4): 389-409. Leviton, L. C., et al. (2010). Evaluability Assessment to Improve Public Health
Policies, Programs, and Practices. Annual Review of Public Health, Vol 31. J. E. Fielding, R.
C. Brownson and L. W. Green. 31: 213-233.
Mayne, J. (2009). Contribution Analysis; Addressing cause and effect, In K Ross, Morca.
[Online].
Available
at
https://www.researchgate.net/publication/303836616_Contribution_analysis_Addressing_cau
se_and_effect. Accessed on 16 April. 2019.
Poulin, M. E., et al. (2000). "The significance of definitions of success in program evaluation."
Evaluation Review 24(5): 516-535. Smith, M. F. (1990). "Evaluability assessment - reflections
on the process." Evaluation and Program Planning 13(4): 359-364.
Chen, H.(2010) The bottom up approach to intergrative validity, A new perspective forprogram
evaluation, Evaluation and program planning, 33(3 page 205—214.
Davies, R. (2013) Planning evaluability assessment: A tool for operating evaluation in social
change programs, vol. 9(4) page 453—469, Sage Publication, London.
Grant, A., et al. (2013). "Process evaluations for cluster-randomised trials of complex
interventions: a proposed framework for design and reporting." Trials 14(1): 15.
Hapanyengwi,O. (2013). A Synoptic View of the History of Special Needs Education in
Zimbabwe. Available on
http://www.thefreelibrary.com/A+Synoptic+View+of+the+History+of+Special+Needs+Educ
ation+in+Zimbabwea01073960231 [Accessed on 09/04/18].
Hall, R. (2002). Implementing inclusive educational practices through partnerships. South
African Journal of Higher Education 21(4):213-218.
55 | P a g e
Johnson, R.B. and Onwuegbuzie, A.J. (2004). Mixed Methods Research: A Reseach
Paradigm whose time has come. Educational Researcher, Vol 33(7), pp. 14-26.
Jourbish, M.F. et al (2011). Paradigms and Characteristics of a good Qualitative Research. In
World Applied Sciences Journal, Vol 12 (11), pp. 2082-2087.
Kaplan, A.S. and Garreth, K.E. (2004). The use of Logic Models by community based
initiatives. Evaluation and Programmes, Vol 28 (2005), pp. 167-172.
King, I. (2003). Examining Middle School Inclusion Classrooms through the lens of learnercentered principles. Theory into practice 42(2):151-157.
Kellog Foundation (2003). Logic Model Development Guide. Available online 10 April 2018
at www.wkkf.org.
Kellogg
W.K. (1998).
Evaluation
Handbook.
Retrieved
from
:
hhtp://www.wkkf.org/Pubs/Tools/Evaluation/Pub 770.pdf. on 08 March 2018.
Kusek, J.Z. and Rist, R.C. (2004) Ten Steps to a Results-Based Monitoring and Evaluation
System: A Handbook for Development Practitioners, Washington DC, The International Bank
for Reconstruction and Development/The World Bank.
Mashiri, P. (2000). The Socio-Cultural and Linguistic Aspects of Childhood Disability in
Shona Culture. Zimbabwe Journal of Educational Research vol. 12 No. 2. 2000 pp 170- 194.
Musengi, M. and Chireshe, R. (2012). Inclusion of Deaf Children in Mainstream Rural
Primary Schools in Zimbabwe, Stud Tribes Tribals, 10(2): 107-116 (2012). Available on
http://www.krepublishers.com/02-Journals/T%20&%20T/T%20&%20T-10-0-000-
56 | P a g e
12Web/T%20&%20T-10-02-000-12-Abst-PDF/S-T&T-10-2-107-12-279-Musengi-M/ST&T10-2-107-12-279-Musengi-M-Tt.pdf. [Accessed on 11/04/18].
Van de Ven P, and Aggleton P (1999). What constitutes evidence in HIV/AIDS education?
Health Educ Res 1999; 14:461-71.
Warinda, E (2011). Glossary of Commonly used M&E Terminologies. Monitoring and
Evaluation Series. ASARECA, Kenya.
Winter, E. and O’Raw, P. (2010). Literature Review of the Principles and Practices relating
to Inclusive Education for Children with Special Educational Needs. ICEP Europe. Available
on http://www.ncse.ie/uploads/1/ncse_inclusion.pdf. [20/04/18].
Weiss, H. (2005). Evaluation Theory or What are Evaluation Methods for? Harvard Research
Family Project. Vol 11(2)
World Bank (2000). The Logframe Handbook: A logical Framework Approach to Project
Cycle Management. Washington DC: The World Bank
W Thomas, G., Walker, D., and Webb, J. (1998). The Making of the Inclusive School. London:
Routledge.
UNESCO. (2009). Defining an inclusive education agenda: Reflections around the 48th
session of the International Conference on Education. Geneva: UNESCO.
United Nations. (1948). Universal Declaration of Human Rights. Geneva: United Nations.
UNDP (2009) Handbook on planning, Monitoring and Evaluating for development results.
Retrieved online 10 April 2018 at: http//www.undp.org/eo.handbook.
Valters, C. (2014). Theories of Change in International Development: Communication,
Learning, or Accountability? London: JSRP
57 | P a g e
World Bank (2011). Monitoring and Evaluation: Tools, Methods and Approaches.
Washington D.C: World Bank
58 | P a g e
APPENDICES
Appendix 1: Structured Questionnaire for Beneficiaries
Questionnaire on the implementation of the Igate Program being implemented by World Vision
Zimbabwe
My name is Arthur Chikande and I am carrying out a study in partial fulfilment of the
requirements for the Monitoring and Evaluation Studies which I am pursuing with Lupane
State University. The purpose of the study is to carry out a process evaluation of the Igate
Program. Your participation in this research is completely voluntary and your identity as a
participant shall not be disclosed. Completing this questionnaire is taken as a sign of consent.
SECTION A: Demographics
1. Gender:
Male
Female
2. Age
5-11
12-18
3. Level of Education
Primary
Secondary
SECTION B: Efficiency and effectiveness of the Igate programme
4.
Are you aware of all the programme activities?
Yes
5.
No
All the programme activities that you are aware of have been carried out. Agree or
Disagree?
59 | P a g e
Agree
Disagree
SECTION C: Quality and Quantity of Personnel
6. The personnel work well with you. Agree or Disagree?
Agree
Disagree
7. The quantity of the personnel adequate enough for Igate Program? Agree or Disagree?
Agree
Disagree
SECTION D: Beneficiaries satisfaction with the Igate Programme
8.
I am satisfied with the Igate Programm Igate b? Agree or Disagree?
Agree
Disagree
SECTION E: Challenges with the Igate Programme
9. Are there any challenges being faced with the programme? Agree or Disagree?
Agree
Disagree
SECTION F: Improvements on the Igate Programme
10. There is need for improvement on the implementation of the Igate Programme?
Agree
Disagree
Thank you
Appendix 2: Semi-structure Interview guide
Interview guide on the implementation of the Igate Programm Program being implemented
World Vision Zimbabwe My name is Arthur Chikande and I am carrying out a study in partial
fulfilment of the requirements for the Monitoring and Evaluation Studies which I am pursuing
with Lupane State University. The purpose of the study is to carry out a process evaluation of
the Igate Programm Program. Your participation in this research is completely voluntary and
60 | P a g e
your identity as a participant shall not be disclosed. Participation in this interview is taken as a
sign of consent.
Interview date…………………………………………
Interview number……………………………………
Interviewee’s profession……………………………….
Place of interview............................................................
SECTION A: Efficiency and effectiveness of the Igate Program activities.
1. What Igate programme activities are you aware of?
…………………………………………………………………………………………………
…...……………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
……………………
2. Which programme activities have been carried out and which ones have not been carried
out?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
………………….
3.
Do
you
understand
the
objectives
of
the
Igate
Programme?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
……………………
4. Are you happy with the personnel? Do the personnel work well with you and in what ways?
…………………………………………………………………………………………………
61 | P a g e
…………………………………………………………………………………………………
…………………………………………………………………………………………………
5.
Is
the
quantity
of
the
personnel
adequate
for
the
Igate
Programme?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
6.
Are
you
satisfied
with
the
Igate
services
and
in
what
ways?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
……………
……………………………………………………………………………………
7. Are there any challenges being faced in the implementation of the Igate Programme
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
8. Do you think there is need for improvement on the implementation of the Igate Programme?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
Thank you
Appendix 3: Summary of data collection methods and instruments
62 | P a g e
Data Collection Methods and Instruments
Data Collection
Type of Instrument
Type of Participant
Method
Parents/Head of
Household/Caregiver
Focus Group Discussions
In-school girls
In-school boys
District Schools Inspector
Qualitative
Head of School
Teacher
Key Informant Interviews
Church Leader
Male Champion
World Vision project staff
First Informant
Household
Caregiver
Child
Surveys
Head of School
Head of School
Teacher
Teacher
Early Grade Reading
Child
Quantitative
Assessment (EGRA)
Learning Assessments
Early Grade Math
Child
Assessment (EGMA)
63 | P a g e
Attendance Spot Check
Head of School
School Enrollment
Head of School
Thank you!
64 | P a g e