TERMS OF REFERENCE FOR AN INSTITUTIONAL CONSULTANCY
Project title: Evaluation on the contribution of the UNICEF school construction component
to the overall programme goals of access, quality and systems capacity
enhancement.
Section:
Basic Education and Youth Development
Key partner:
Ministry of Education, Science and Technology
Location:
Lilongwe, Malawi
Duration:
50 days
Start date:
TBD
End date:
4th December 2016
Closing date: Apply before or on 13 October 2015
BACKGROUND AND JUSTIFICATION
School infrastructure development component under the child friendly schools programme
of the current UNICEF MCO country programme has been designed to contribute chiefly to
two key components of the national Education Sector Implementation Plan (ESIP)
I.
improved access to equitable education
II.
improved and relevant quality education
The education sector in Malawi also aims to establish standards, norms and effective
processes for school improvement generally through the national Primary School
Improvement Programme (PSIP). In that framework, school construction is meant to enhance
participation, capacity, sustainability and efficiency of the sector. Access and quality of
education in Malawi has long been affected by an extreme shortage of classrooms. While
primary enrolment increased by 45% between 2004 and 2013, the number of primary
classrooms rose by only 12%. A recent study (USAID, 2014) identified classroom shortage as
one of the major factor leading to student absenteeism, repetition and dropouts. Analysis of
1
PET-QSD Survey data also confirms this. Hence, the target of the Malawian education sector
as laid out in the ESIP II (2013/14 – 2017/18) is to construct at least 20,000 classrooms to
meet increasing population of school age children.
It is within, and in response to, this context that the Basic Education and Youth Development
(BEYD) section seeks to meet the objectives of improved educational access and quality for
all children through a broad School Construction component contributing to both volume and
quality of school infrastructure in Malawi. School infrastructure development includes
multiple inputs such as building of classrooms, latrines and teachers houses as well as
contribution to the development of school construction norms and standards including
capacity for school construction. The proposed evaluation is important in assessing the extent
to which the School Construction Component enables UNICEF Malawi to contribute to those
broad objectives for key learner outcomes in education.
UNICEF has utilised the Child Friendly School (CFS) quality framework as guidance to meet the
MCO commitment to the education sector. This includes emphasizing the importance of
inclusiveness and equity for marginalised children. In the context of Malawi this has reflected
in a strong focus on girls’ education due to the many factors related to school environment
and infrastructure that hinder girls from enrolling, enjoying and completing basic education.
As an example girls drop out of school in large numbers when reaching puberty due to lack of
adequate water and sanitation facilities. The CFS framework also highlights the need to
consider school infrastructure as a holistic approach to improved learning environment;
including both hard-ware and soft-ware aspects.1
This evaluation will assess the extent to which the School Construction component of the
Child Friendly Schools Programme has contributed to a) the development of standards, and
capacity for their implementation within the wider BEYD goals of strengthening institutional
capacity and b) the learning environment.
PURPOSE OF THE EVALUATION
1
Hard-ware aspects include building of classrooms while soft-ware aspects include issues such as maintenance
and attitudes towards the school environment.
2
The purpose of the evaluation is threefold:
1) to determine school infrastructure contribution to the basic education programme
objectives of improving access and quality through child friendly school construction
and design
2) to assess effectiveness and efficiency of the inputs and processes
3) to identify lessons learnt and provide recommendations for future of school
infrastructure and other strategies in the new 2017-2021 country programme
The evaluation is to improve UNICEF’s accountability and provide for lessons learnt on schools
infrastructure projects. Findings from the evaluation will support UNICEF and its key partner
the Ministry of Education to make evidence based decisions for future programming and
planning within the field of education.
GOAL
To determine the contribution of the School Construction Programe as carried out under the
current Country Programme to the BEYD education programme objectives to improve
learner outcomes (access and quality) through child friendly education and capacity
enhancement.
SPECIFIC OBJECTIVES
1. To assess the contribution of the school construction component to the BEYD
programme objectives of improving access to quality basic education for all,
especially of the most vulnerable and girls
2. To assess the achievement of the school construction component in contributing to
the child friendly education approach in in focus areas and targeted schools;
3
3. To assess the effectiveness and efficiency of the inputs and process of the school
infrastructure project in meeting the above leaner outcomes, standards and capacity
enhancement objectives;
4. To identify lessons learnt and make recommendations for the new country
programme (2017-2021).
SCOPE OF WORK
Within the OECD evaluation framework (relevancy, effectiveness, efficiency, impact and
sustainability) the key dimensions to be evaluated will be threefold focusing on expected
outputs linked to learner outcomes; processes; and the chosen inputs and particularly the
child friendly education and the school improvement approach.
The impact evaluation is to focus mainly on 35 primary schools constructed in 5 districts of
Malawi under the BEYD Child Friendly Schools Programme between 2012 and 2015. Whereas
the type and level of construction and subsequent interventions vary between school sites,
the consultant will have to apply relevant and applicable assessment, examination and review
methods as appropriate in order to fulfil the outlined scope of work. Each dimension is laid
out below with key supporting questions and tasks intended to assist and guide (but not
limited to) the evaluator in his approach.
1. Assess the focus of the School Construction Component in line with the BEYD
education outcome objectives of increasing equitable access and also quality of
education where possible given that the country programme still has one year to go;

Were the programme objectives appropriate in the overall problem context;
needs and priorities?

What value (and for whom) did the School Construction add relative to other
education initiatives?

Given progress to date, are the School Construction objectives still relevant?
4
2. Examine efficiency, effectiveness and sustainability of the School Construction
Component inputs and processes within the child friendly school quality framework;

How well did the various activities transform the available resources into
results?

What measures were taken to ensure cost-effectiveness in procurement and
implementation?

What measures were taken to ensure effective financial implementation,
monitoring and reporting?

How well did the School Construction predict and react to risks?

To what extent did the project enable local participation and ownership of the
project?

To what extent will maintenance be sustained by local partners/beneficiaries
once the funding comes to an end?
3. Examine key outputs and standards in light of the child friendly quality framework

To what extent did the School Construction deliver expected results?

To what extent has the design enabled the improvement of the school
environment?

How well has the School Construction component contributed to the
improvement of standards and systems; especially in terms of institutional
capacity?

To what extent was progress made against key objectives of improving access
and quality of education?

Increased enrolment

Decreased drop out

Improved pass rates
4. Assess the extent to which the School Construction Component of the programme
has ensured capacity of key stakeholders through participation, training and
guidelines;
5

The evaluation will examine impact of the approach in terms of contributing
to establishment of systems/institutional capacity;

The evaluation will consider extent to which policies and strategies have been
established including cross-sectoral and gender responsive approaches;

The valuation will examine extent to which capacity of stakeholders and
ensured ownership including participation, maintenance and monitoring.
5. Review the overall safety and inclusiveness of the school environment, the level
ground and environment.

How well did stakeholders (head teachers, teachers, school management
committee members and students) in the targeted schools participate in the
process; by providing leadership harmonization, utilization and maintenance
of the built environment?

What are the key factors most likely to provide an understanding of success or
bottlenecks associated with the implemented interventions in the targeted
schools?
Improving access and quality education and achieving a child friendly school environment can
never depend on infrastructure or school construction alone. It relies on multiplicity of
interrelated in and out-of-school factors that are socio-economic and cultural in nature. When
attempting to evaluate the impact of certain factors or environment and not others it is
important to be able to apply methods that take into account and control for variables known
to be directly related to the quality of education. This can include variables such as availability
of qualified teachers, access to teaching and learning materials, socio-economic background
and other relevant factors within the context of Malawi. The methodology of the evaluation
will have to take this into consideration.
The evaluation shall include an overall performance rating for each of the above five
evaluation criteria (see scope of work) – on the basis of the following scale:
 Highly satisfactory (fully according to plan or better);
6
 Satisfactory (on balance according to plan, positive aspects outweighing negative
aspects);
 Less than satisfactory (not sufficiently according to plan, taking account of the
evolving context; a few positive aspects, but outweighed by negative aspects):
 Highly unsatisfactory (seriously deficient, very few or no positive aspects). Each
rating should be stated as part of the conclusions for each of the five criteria.
METHODOLOGY
The consultants are expected to present, in detail, their approach, methodology and tools,
with an action plan and time frame that addresses the expected outputs, with reference to
the overall and specific objectives as well as budget. The following are to be taken into
consideration:
1. The consultant will undertake a desk review2 of construction related education
programmes in Malawi to better understand and document the educational context;
including challenges and best practices that arise with school construction. The consultant
will have to be become well acquainted with all the materials which UNICEF provides. It
will have to draw up questionnaire and interview questions, coding plans, and how he/she
will analyse the data (analytical framework). This review will provide for a necessary
bottleneck analysis that explains the relationship between school infrastructure and key
domains of education quality such as access, demand, enabling environment and overall
learning outcomes.
2. Taking into consideration that there are numerous and multiple factors that can influence
the outcome of a programme a mixed method approach which uses both qualitative and
quantitative data collection and analysis is recommended. While quantitative methods
and analysis acknowledges the fact that there are objective realities that can impact
directly on particular outcomes. Qualitative methods and analysis acknowledges the fact
that processes, emotions and experiences are real and play a crucial role in the way people
2
The consultant to ensure all key documents are taken into consideration. The following specific documents are
particularly recommended: UNICEF strategies; Sector policy and strategies including ESIP documents; School
Improvement Programme documents; school quality standards including those for school infrastructure;
MCO/UNICEF country programme documents; Review and evaluation studies;, etc.
7
interact and make use of institutions, cultures and social structures. The balance between
the two methods is vital for improved understanding and learning of the subject. The
evaluation will therefore need to:

analyse fixed outcomes and factors through correlations and patterns
providing a quantitative analysis of impact and results (quantitative)

provide a deeper understanding of the outcome through in-depth interviews
and perspectives of the actual beneficiaries of the programme (qualitative)
Quantitative data collection and analysis: When choosing a quantitative data collection
method the consultant needs to consider that i) control groups were not establish for the
programme in the beginning of the programme ii) some school sites have ongoing
construction and interventions due to the fact that the country programme has one year
to go. The consultant will therefore need to choose a valid method3 that fits the scope of
work in order to be able to assess inputs, outcomes and impact of school construction.
Data is to be collected from 35 primary schools and should focus on capturing the
relationship between the built environment (school construction) and the following (but
not limited to) key education indicators:

Enrolment; boys and girls, OVC and SNE

Dropout; boys and girls, standards 5 and 8

Pass rates; boys and girls

Completion rates; boys and girls
It is important to note that there may be some challenges in data reliability and
disaggregation of data by gender, age group, etc. It is proposed that some of the risks
could be mitigated through triangulation, cross-referencing, focus group discussions and
other means.
Qualitative data collection and analysis: Whereas the evaluation is to assess processes of
work, its contribution to standards, norms and the success of capacity building along with
Methods such as ‘before and after’ comparison or ‘with-without’ comparison making use of regression
analysis are for example valid ways of performing impact evaluations.
3
8
attitudes and participation of stakeholders it is vital to make use of qualitative data
collection and analysis.
Data is to be collected through informative interviews and focus groups from all key
stakeholders4 at national, district and school level involved in the School Construction.
The consultant is also encouraged to make use of onsite observations in order to assess
maintenance, cleanliness and utilization of the built environment.
3. Ethical clearance - the consultants are expected to identify all relevant ethical issues (i.e.
consent from parents when discussing with children; gender related issues and issues of
power balance) from the proposed methodology and seek ethical clearance as
appropriate from the National Committee on Research in Social Sciences and Humanities
(NCRSH) in accordance with the rules and regulations of conducting research in Malawi.
4. Limitations – the consultant needs to take into consideration and provide workable
solutions to the following challenges and possible limitation of the evaluation:
a. A baseline survey was not undertaken .In utilising a before and after approach the
consultant (s) school take into consideration that some children and staff may not
have been at the school before the construction started.
b. Some school sites have ongoing construction, as such the evaluation criteria may
not be applicable in all schools
c. Control groups were not established in the beginning of the programme making it
difficult to compare outcomes. Hence, the consultant is expected to propose a
methodology that works around this issue.
The consultancy is expected to develop the above into a robust methodology and draw up a
work plan for visiting all relevant sites and stakeholders to collect all relevant and necessary
data.
SKILLS AND QUALIFICATIONS
4
Including MoEST, EIMU, and Education Managers at district and school level, Head Teachers, Teachers,
Community Members and Students.
9
A qualified team with at least 8 years relevant background and experience in programme
evaluation, research, extensive work with the education sector and school infrastructure is
expected to submit a proposal with a team of the following minimum expertise, and
background:
Education qualification and competencies:

Team leader

PhD in Education or the Social Science

8-10 years’ experience in programme evaluation within the field of
Education sector, or social development in Malawi or sub-Saharan Africa;

Proven capacity to conduct research at a national scale, including capacity
to work with national level partners as well as management, analysis, and
interpret large sets of data

Excellent analytical and writing skills.

Excellent communication and negotiation skills and fluency in English
written and verbal

Construction and/or Environmental Specialist (s)

Advanced or Master’s degree in Built Environmental Studies and
Psychology; Architecture or Engineering with a focus on schools
environment and links to learning of children;

8 years knowledge and experience in construction design and engineering
in Malawi or Sub-Saharan Africa with a focus on the comprehensive range
of school infrastructure, and school ground planning and development;

Skills in building economics, quantity surveying, and comparative cost
analysis and experience in value for money and technical audits;

Proven capacity to conduct research, the management, statistical analysis
and interpretation of large sets of data, at a national scale, including
capacity to work with national level partners;

Excellent analytical, writing and communication skills with. Fluency in
English both written and verbal is essential.
10
DISEMINATION PLANS
The final report will be shared widely with Government partners through the Ministry of
Education, donors and cooperating partners. The following activities will be implemented in
this regard:
1) Report will be printed and widely distributed to key stakeholders
2) At national level, report will be presented in all relevant technical working groups as
well as for development partners;
3) At district level. The report will be utilized for review and planning session including
at sub district levels.
4) Finally, the report will be used as part of knowledge sharing and management
process initiated by the basic education programme.
11
TIME FRAME
The following provisions outlines a detailed scope of works that the consultant is expected to
undertake. The evaluation will be undertaken between April and June 2015.
Task
1
Deliverable
Description
Timeframe
Inception
Prepare an inception report summarising how the 5 days
Report
consultancy has understood the tasks and
deliverables and how they propose to deliver on
the
consultancy,
i.e.
highlighting
the
methodology, the data collection tools and
analysis process and the presentation of results.
2
Desk review
The desk review should be structured to provide a 10 days
and
bottlenecks
analysis
that
presents
that
documentation relationship between school infrastructure and
3
of the field of
education in the form of access, demand, enabling
school
environment and overall quality such as learning
construction in
outcomes. The analysis will provide a background
Malawi
and support the findings of the evaluation.
Data collection
The consultancy is expected to produce a field
and Field
report after data collecting documenting process,
report
stakeholders met and key challenges. A
25 days
PowerPoint presentation will be done to a
reference group comprising UNICEF, Ministry of
Education, Science and Technology and other
relevant stakeholders for validation and
feedback.
12
4
Final report
1. Present draft findings in the form of a
and
report and a power point presentation to
presentation
UNICEF, Ministry of Education, Science
of findings
and Technology and other relevant
10 days
stakeholders for inputs, validation and
feedback.
2. Present final electronic copies of the
findings to UNICEF and Ministry of
Education, Science and Technology in a
‘user friendly’ report and power point
presentation.
The final report should include the following:

Executive summary (non-technical
including conclusions and
recommendation)

Introduction and background (including
the bottleneck analysis)

Methodology

Findings (both quantitative and
qualitative)

Discussions (wider context)
50 days
KEY RESULTS

Program review that results in evidence for future interventions and
programmes on child friendly school environment
13

Final report that will influence policy and programmatic decisions relating to
enhancement of quality education within Malawi at national, sub-national and
school levels
DELIVERABLES
Intermediary deliverables
1. Inception report and power point presentation to validate the methodology to be used
in the evaluation.
2. Field report after data collection and PowerPoint presentation for validation and
feedback.
3. A preliminary report highlighting major findings from the assessment and key
recommendations shared and reviewed.
4. Power Point presentation on the preliminary findings with UNICEF, government and
stakeholders
Evaluation report
The Evaluation Report which shall be compliant with the UNICEF - Adapted UNEG standards
(to be provided to successful consultants) should include - but is not limited to - the following
components:

Table of Contents

Executive Summary.

Background

Methodology including source of data, data collection, people and places visited

Quality and reliability of data

Findings

Conclusions

Recommendations
Appendices should include:

Terms of Reference

List of people interviewed
14

List of documents consulted
A tightly-drafted, to-the-point and free-standing Executive Summary is an essential
component. It should be short and not more than five pages. It should focus mainly on the
key purpose or issues of the evaluation, outline the main analytical points, and clearly indicate
the main conclusions, lessons learned and specific recommendations. The Executive Summary
shall include the Performance Rating of the main 5 evaluation criteria. Cross-references
should be made to the corresponding page or paragraph numbers in the main text that
follows.
The Recommendations should be the subject of a separate final chapter. Wherever possible,
for each key conclusion there should be a corresponding recommendation. The key points of
the conclusions will vary in nature but will often cover aspects of the key evaluation criteria
(including performance ratings).
Final deliverables
As final outputs three sets of documents will be required (electronic copies in Word & PDF)(1) Final evaluation report, (2) Micro data in soft copy
BUDGET
The consultant should provide an overall budget for the activity of the evaluation.
Payment will be best on deliverables as follows
•
Inception report and presentation 30%
•
Field report and presentation 20%
•
Draft report and presentation 20%
•
Final report and data sets 30%
Standard UNICEF procedures will apply for invoicing and all other financial management
requirements set out in the contract. Standard penalty clauses will also apply for late and
poor quality deliverables.
MANAGEMENT OF THE EVALUATION
15
The consultant will report to a reference group including the Chief of Education as well as
relevant government and civil society stakeholders. A reference group is responsible for
quality assurance of the evaluation at each stage providing the consultant with the criteria
for the evaluation of the quality of each deliverable. A technical team will provide required
logistical support, available research, documents, studies and evaluation relevant to the
exercise
Name
Title
Charles Nabongo
Chief of Basic Education
Chris Davids
Chief of PME,
Roisin De Burca
Deputy Representative
Mahimbo Mdoe
Country Representative
Signature
Date
Applications should be sent on or before 13 October 2015 to the following address:
hrmalawi@unicef.org
16