Synergy of Government and
Non-Government
Bodies’ Involvement in
Monitoring and Evaluation
Goal of M&E Process
The ultimate goal of M&E process is to
present to beneficiaries and stakeholders
a fair and unbiased perception of project
progress achieved. Progress measures
with the extent which the original project
goals have been achieved to.
Effectiveness of M&E process
Effectiveness of M&E process measures with:
• Consistency of the fundamental principles that
regulate project implementation and
organizational arrangements such as:
transparency; objectivity; centralization-vsdecentralization; delegation; accountability,
coordination-vs-communication.
• Capacity to promptly identify, analyze and adjust
whatever improprieties may occur.
• Capacity to provide an all-inclusive evaluation
perspective, ie, covering all the phases of the
project cycle.
Challenges to M&E Process
The mainstay of issues to be considered is secure M&E effectiveness
through involving outside, ie, non-government bodies.
Why is that topical?
1. Including outside bodies stands for an integral and autonomous
M&E structure which does not necessarily follow the administrative
line structures.
2. Unlike decision making M&E needs intensive communication on the
account of coordination, ie, on the account of rigid line
administration’s restrictions to communicate and share knowledge.
3. Including outside bodies stands for 1) large participation of
beneficiaries’ organizations; 2) building capacity at local level; 3)
increasing effectiveness and efficiency of M&E.
4. Therefore, it would allow for overcoming the deficiencies of the
centralized line administration without challenge it.
The presentation dwells on short falls and positive effects
of country led evaluation systems based on a survey of
monitoring and evaluation (M&E) procedures under two
types of projects being implemented by Ministry of Labor
and Social Policy:
• PHARE projects which are implemented by the EU PreAccession and International Projects Directorate with the
Labor Ministry (for the current financial year the PHARE
Program amounts to EUR 40 m round
• Social Investment and Employment Promotion
Project, joint development project of Labor Ministry and
World Bank of total capitalization EUR 66.7 m which is
being implemented by the Bulgarian Social Investment
Fund (semi-autonomous administrative body to the
Labor Ministry)
M&E Patterns: PHARE Program with the Labor
Ministry
EU Pre-Accession and International
Projects Directorate
Financial Department
- Financial Coordinator
- Technical Coordinator
- Financial Controller (Representative
of the State Agency for Internal
Financial Control)
Regional Level (28 Regional
Coordinators)
M&E Unit in charge of
operational monitoring
Beneficiary Level
M&E Patterns: SIEP Project
Steering Committee Based on TriPartite Principle
Executive Director
Advisory Services
Departments
Legal Advisor
Technical
Financial
M&E Coordinator
Local Level Monitoring – By Competitively Selected
Private Inspection Companies
Local Level – Communities and Municipalities
Is it possible to decentralize M&E
within the official structures?
Both M&E patterns evidence a high level of centralization. The M&E
arrangements follow the line administrative structures of the
respective official institutions.
No, because:
1. Both development projects and EU funded projects have been
designed, planned and implemented at the Government’s
discretion. They are Government led projects; they meet challenges
the Government reckons relevant; the respective Government
institutions have justified their relevance to the donors.
2. M&E is a multi-faceted process but financial/accounting audit plays
the leading role within it. By rule financial audit is centralized
because the official institutions which are in charge of project
implementation use their own centralized bodies for this purpose.
3. Individual data collection and data processing through an integrated
Management Information System requests centralization of
communication
Evidences
•
•
Mechanical division between financial monitoring and technical
monitoring is available. Financial monitoring is formal; technical
monitoring is substantial. Evaluating projects’ impact requests
integrity of both formal and substantial approaches. This is an
analytical work that requests specialized professional skills. The
SIEP Project Key Performance Indicators evidence this
requirement. (See next slide SIEP Project Key Performance).
There is no specialized body to monitor and evaluate the
effectiveness of each of the phases of the Project Cycle. The M&E
specialist/unit comes too late when in the implementation and
evaluation phases possible flaws of the previous phases may
make themselves felt.
SIEP Project MIS
SIEP Project MIS – Key Performance Indicators
Key Performance Indicators/Ключови индикатори за изпълнение
№
1
2
3
4
5
6
7
7.1
8
8.1
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Indicators/Индикатори
Reduced
L/T Poverty in the
Targeted Poor Communities
Намалена бедност в целевите
Number of Beneficiaries
Брой бенефициенти
Number
of Municipality
Infrastructure Microprojects
Брой микропроекти
заBased
общинска
Number
of Community
Microprojects
Брой микропроекти
за развитие
на
Number
of Municipality
SubProjects Completed
Брой на of
завършените
Number
Community общински
Sub-Projects
Completed
Брой на
завършените
общностни
Project
Cost
(BGN) – Additional
Resources to Poor Communities
Обща стойност
(лв)
Community
Project
Cost (BGN) –
Additional Resources to Poor
Communities
Общности - обща стойност (лв)
Contract Cost
Преки разходи
Community Contract Cost
Общности of
- Преки
разходи
Percentage
Poor
Communities/Municipalities
Targeted
Unemployment
Rate in Targeted
Poor Communities/Municipalities
Отношение на безработица в
Number of Beneficiaries
Брой бенефициенти
Cost/Beneficiary
Разход за бенефициент
Percentage
of Effective
Microprojects
Дял на ефективните
Number of Effective Microprojects
Брой финансирани проекти
Out of Them: Municipality MP
От тях за общини
Out of Them: Community Based
От тях за общности
Duration (month)
Продължителност (мес.)
Mandays
Човекодни
Number of employees
Брой заети
Cost/employee
Разход за един зает
Long Term Unemployed
Наети дългосрочно безработни
Labor Content
Съдържание
труд
Percentage
of на
Microproject
Wage
Payments Going to Unskilled
Workers %
Съдържание на неквалифициран
Average Gross salary
Средна брутна
заплата
Average
Gross salary
to National
Average %
Средна брутна
заплата към
Number
of Municipalities
Participating
Брой на of
участващи
общини
Number
Community
Based
Organizations Established
Брой на of
учредени
организации
на
Number
Partnerships
with Private
and Public Institutions Established
Брой на установени
Administrative
Costs партньорства
as % of Project
Expenditures
Административните разходи като
% от (satisfaction%)
разходите на проекта на
B.A.
Оценка на бенефициентите
(удовлетворение
%)of local
Level
of satisfaction
infrastructure (%)
Равнище наof
удовлетвореност
Satisfaction
local authorities’от
responsiveness
Удовлетвореност
Correspondance
to от
citizens'
priorities
Съответствие с приоритетите на
Project disbursment (loan)
Усвояване на заема
Average
Средни
Total
Общо
Targeted
Планирани
16.36
Accomplishment
Изпълнение
15.67
104.39
99,133,003.00
72.4
17.21
137.36
700,000.00
92.42
25.00
117.96
99,356,164.00
36.13
432,840.00
321.00
55.00
174.00
224,301.87
71,776,598.11
125,199.94
6,885,996.59
196,382.01
60,682,040.47
99,240.75
5,458,241.12
29.33
23.64
432,840.00
179.20
36.01
376.00
321.00
55.00
3.85
1,446.00
2,015.43
646,954.62
17.45
5,601.64
12,617.82
3.49
1,120.33
29.49
19.05
294.54
101.22
376.00
0.24
35,894,041.33
Evidences
•
•
•
Regarding individual data collection and on-site control M&E effectiveness is
additionally smothered by the centralized pyramidal project managing
authorities’ administrations.
M&E has not been fully institutionalized. M&E complements project
implementation rather than to be an autonomous process which bears on the
whole project cycle.
M&E system has not been comprised in an integrated Management
Information System (MIS) so far. The SIEP Project’s MIS made a substantial
progress with this regard where its M&E Module had been linked to the other
modules. The PHARE program is about to produce an integrated MIS too.
Thus, on line information on the project’s current status is available. But
generally, integration of individual projects’ MIS to line-ministries’ one is a
challenge far from having been satisfactorily tackled so far.
Centralization of information flows allows for expedient data processing and
minimizing data distortion due to vested interests and lack of professional
skills. The relevant principle to be followed with this regard is:
1) elaboration of as simple as possible measurable indicators (mandays
created, long term unemployed hired, wage earned etc.) to be covered with
individual data by local bodies; and
2) centralization of data collection and processing. Involving private
companies specialized in monitoring and control is strongly recommended at
the initial stage of data collection.
Alternatives
•
Radical Alternative. It takes stock from the fact that M&E has not been
fully institutionalized. Overcoming deficiencies of the now existing M&E
systems that have been evidenced by the above projects stand for an
autonomous M&E bodies where Government, Non-Government and
private sector are represented on equal footing. In this way final
beneficiaries would be much more effectively enabled to have a say on
project implementation. But the line-ministries should keep their lead
since the projects in question service primarily Government priorities and
budget commitments.
The M&E bodies would be involved in the whole Project Cycle along with
the official institutions.
The M&E bodies would develop flat administrative structure which is
conducive to more effective communication and individual data collection.
M&E should integrate financial and substantial (technical)
audit/monitoring.
Possible risk and weakness is the administration of integrated MIS and
coordination of data processing and analysis, in particular.
Alternatives
•
Realistic Alternative. It takes stock from the centralized way EU
and international development projects, mostly WB funded, have
been planned and implemented so far. This pattern of project
financing persists.
The only way to avoid the subsequent flaws is outsourcing.
Outsourcing M&E services with independent private bodies would
mitigate the negative effects of centralization. Primarily this would
provide M&E with own quasi autonomous administration which
would be free enough to set up formal relationships with
beneficiaries’ monitoring bodies.