Evaluations of and
reflections on
university development
cooperation –
The Dutch case
Jolie Franke
Team Coordinator
Capacity Building Programmes
Barcelona, 9 December 2013
Dutch Programme for Higher Education Capacity
Building
NPT and NICHE focus on the sustainable strengthening of Higher Education
and TVET capacity in fifteen developing countries and fit in with the bilateral
cooperation policies of the Dutch embassies in four selected policy priorities
1) Water, 2) Food Security, 3) Sexual and Reproductive Health and
Rights and 4) Security and the Rule of Law (EUR 29 Million/year).
Strengthened education and training capacity must respond to labour
market and gender needs.
2
Evaluation practice





3
Regular internal project evaluations by project implementers: Requesting
and Dutch organisations
Incidental external project evaluations commissioned by Nuffic
Regular external country programme evaluations commissioned by Nuffic
Incidental external thematic evaluation, e.g. gender review of NFP and
NPT
Regular external programme evaluations commissioned by Netherlands
Ministry of Foreign Affairs: 2007 NPT; 2012 NFP, NPT and NICHE.
2012: External evaluation of NPT and NICHE

Commissioned by Ministry of Foreign Affairs NL:
o Accountability
o Learning
o Recommendations for successor programme

Executed by Ramböll:
o Relevance of NPT & NICHE
o Effectiveness of NPT
o Efficiency of NPT & NICHE
o Impact of NPT
o Sustainability of NPT
4
Evaluation methodology





5
Desk study: comparison of nine International Education Development
programmes (indepth: Sweden, Belgium, Finland and Norway)
Interviews and questionnaires of 16 selected projects
Visit to 6 countries and 34 requesting organisations
Statistical analyses:
o capacity development index
o Cost-efficiency analysis
o Cost-effectiveness analysis
Adaptation of 5 Capabilities concept: evaluation grid
o Capability to act
o Capability to produce development results
o Capability to relate
o Capability to achieve coherence
o Capability to adapt and self renew
Relevance

NPT and NICHE projects generally meet the respective needs on the partner
country level and on the level of the requesting organisations.
o Higher Education Sector:
 On the sector level as they contribute to higher learning reforms (even
more under NICHE).
 On the level of the requesting organisations a sector-wide approach as
undertaken by NICHE can strengthen the relevance of the projects by
avoiding “project islands”. This can create synergies between different
projects.
o Economic Sector:
 On the sector level NPT and NICHE are in line with the development
strategies of the partners.
6
 However, universities are sometimes not the main leverage to solve the
challenges within a certain sector. Here, a focus on the TVET sector, as
under NICHE, is more relevant.
Effectiveness on Southern Partner Organisations
• Often second
order learning
is not reached
NPT strengthens
human resources,
teaching capabilities
and infrastructure
NPT does not engage
in wider
organisational
change processes
• Top & middle
management is
often not
involved
NPT develops and
revises curricula and
courses successfully
NPT does not always
capacitate
organisations to
adjust to external
change.
• Training
mechanisms
are often not
institutionalised
• External
networks are
often not
established
7
Efficiency

Regional stakeholders (e.g. other universities in the region) added value
to an efficient project implementation and goal achievement

The roles distribution in the programme and project cycle are not clear
and cause inefficiencies

The demand-driven design in both programmes
o allows requesting organisations to articulate their demand
independently .
o causes inefficiencies regarding the time-involvement of the Dutch
organisations.

8
The tendering procedure has had a modest effect on the efficiency of the
programmes.
Impact: Employment According to Sector
Agriculture, food security
6%

Economy
26%
Education
17%
Energy
Engineering
Finance
4%
2%
Law
9%
Environment, resource management
Medicine/Healthcare
11%
5%
Nutrition
Civil society, democracy
7%
Water
7%
Crisis prevention
Other, namely
N=121
0%
The majority of NPT projects enable the
requesting organisations to increase the
availability of human power for the
specific sector.
o 96% of the NPT alumni are
employed
o 82% of the NPT alumni work in the
sector for which they have been
educated for
o Case studies showed however that
the absorption rate is not always
given.
o Case studies also confirmed that
most graduates gain employment
due to a shortage of adequate
qualified staff in the sector.
2%
5%
9
25%
50%
75%
100%
Sustainability in Organisations

The sustainability of the NPT
programme is not always
guaranteed because
o in some cases there is a
mismatch between the projects
and external circumstances.
o there is a lack of involvement of
the higher management.
o there is a lack of ownership.
o a lack of attention towards
reaching a critical mass in terms
of training and a loss of trained
personnel.
NPT strengthens
human
resources,
teaching
capabilities and
infrastructure
NPT does not
engage in wider
organisational
change
processes
NPT develops
and revises
curricula and
courses
successfully
NPT does not
always
capacitate
organisations to
adjust to
external change.


The investments into infrastructure
are highly sustainable as they are
used and maintained.
The general duration of the NPT
projects is an impediment to reach
sustainability.
10
Recommendations
1. Continue the NICHE approach and further strengthen the connection
to the Labour Market
2. Approach capacity development at requesting organisations in a
holistic manner
3. Define roles and responsibilities more clearly and improve
organisational and mutual learning mechanisms
4. Ensure future interest of Dutch organisations in the programme and
foster competition through improved incentive structures
5. Coordinate more effectively with other programmes
11
The Perspective of the Dutch Organisations
(Nikièma 2012)
 Past (before 2002): stimulating factors
 Prospects for long term cooperation
 Diversity of projects
 Opportunities for setting own agenda
 Quite “lean and mean” procedures
 Possibilities to input into processes from programme/project design
up to implementation
 Since 2002: hampering factors
 Limited/no room for setting own agenda or impacting the project
processes and directions
 More complexity and costs for acquisition, implementation and
administration of projects
Challenges for HE in realization of own ambitions
12
The Top 5 Motivation of Dutch Organisations to
participate (Ramböll)
Through our participation in the projects we can contribute to the
1%2%1%
development in the partner countries. (N=95; m=4.59)
32%
64%
1%
1%
The projects comply with the values and mission of our organisation.
3% 11%
(N=96; m=4.25)
40%
44%
1%
Through our participation in the projects we can develop networks with
1% 7%
relevant stakeholders. (N=136; m=4.13)
Through our participation in the projects we can increase our
1%2%
knowledge base in certain fields. (N=136; m=4.03)
Through our participation in the projects we can extend our reference
2%
list. (N=41; m=4.05)
8%
Does not apply at all
35%
18%
54%
24%
0%
Question is inapplicable
49%
Does not apply
24%
39%
25%
More or less
34%
50%
Applies
75%
100%
Applies fully
13
New NICHE phase started 1 July 2013

Roles: Nuffic more at distance; implementation based on trust

Simplified procedures; less reporting requirements

Holistic Capacity Development through 5 Capabilities approach

Bidders have more freedom to be creative in their bid: define outputs

Tender evaluation directly links price with quality

Focus on contribution to country programme

Trust - explicit notification - sanctions

Exploratory study on modalities of longer-term collaboration: pursuit of a
shared agenda which leads to mutual benefits
14
More information: www.nuffic.nl







15
www.nuffic.nl/niche
http://www.rijksoverheid.nl/documenten-enpublicaties/rapporten/2012/05/17/final-report-evaluation-of-npt-andniche.html
http://www.nuffic.nl/en/capacity-building/in-the-picture
Synergy in action: Coordination of cooperation programmes in higher
education and research
Conference proceedings: Shaping the future: New pwespectives on
Dutch cooperation in post-secondary education and training for
sustainable development
Complexities of gender mainstreaming in higher education capacity
development programmes
CD-ROM: Shared practice: Mainstreaming gender and labour market in
capacity development
Thank you for your attention – Questions?
Jolie Franke
jfranke@nuffic.nl
www.nuffic.nl
16