Projects Evaluation in Pakistan
Dr. Fazli Hakim Khattak
Director General (Imp. & Monitoring)
Planning, Development & Reform Division
Islamabad, the 8th May, 2014
Projects Evaluation in Pakistan
Dr. Fazli Hakim Khattak
Director General
Planning Commission of Pakistan
Abstract
Government of Pakistan has been funding Projects and Programs for
socio-economic development since formulation of the first five years plan. Such Projects
falls in thousands with a cost of trillion rupees. Presently, 8000 projects and programs
are funded through development budget at federal and provincial levels. This is a huge
investment, and on completion requires transformation to regular budget on sustainable
footing to ignite the engine of economic growth in long run. Therefore, evaluation of all
these projects is required for assessment of ongoing and completed projects, programs,
policy and plan to make improvement in their design and implementation and stabilize
policy formulation by inbuilt mechanism of monitoring and evaluation. The Development
Assistance Committee (DAC) has already set a criteria in this regard for evaluation of
development assistance. It includes Relevance, Effectiveness, Efficiency, Impact, and
Sustainability.
In Pakistan, Projects Planning, implementation, monitoring and evaluation
are performed using the conventional methods through adhoc data collection (not
properly analysed) and without identification of the outcome and impact of such
investment. The tools used in the projects cycle are known PC-I, PC-II, PC-III, PC-IV &
PC-V. These are effective tools provided the adequate information is made available
through a reliable Management Information System (MIS). The software of Projects
Monitoring and Evaluation System (PMES) is of high worth in this regard to analyse the
data and extract results for identification of the outcome and impact help the policy
makers for a change in the related parameters to achieve the targeted level of economic
growth.
An Independent Project Evaluation (IPE) has been more effective in
evaluation of the individual project designed to achieve the specific objectives, within
allocated resources, in a prescribed period, following the approved plan of action within
the frame work of a broader program. In this case Vision 2025 provides the guiding
principles to achieve the targeted results through known indicators, and define its
connectivity with the national goals and objectives. These inventories may be reflected
in the Five Year Plans. However, a price tag of the plan matching with the physical
targets will further specify the development interventions. IPE is initiated and managed
by the project managers, and contributed by the independent external evaluators to
ensure quality assessment in the entire process of evaluation. Planning Commission of
Pakistan has adopted a mixed approach of monitoring and evaluation through Projects
Wing since 1983. A few projects were monitored till 2005. The Projects Wing was
upgraded in 2006 by investing Rs 360 million through the Project “Monitoring PSDP
Funded Projects (MPFP)”. This project has effectively delivered, despite the problems
and deficiencies in the inputs required for monitoring and evaluation including human
resources, equipment, training and refresher courses of the staff, and regular salary
packages etc. Planning Commission in its restructuring has emphasized on monitoring
and evaluation to maintain the worth of projects investment, and ensure sustainability to
support the engine of economic growth.
Projects Evaluation in Pakistan:
Introduction:
A national plan is an essential instrument to guide allocation of resources
through priorities setting within the country’s macroeconomic framework. Its basic
elements are individual projects programs and policies designed to achieve the plan’s
economic and social goals. However, the plan remains a conceptual document, a
composition of formulae, and a continuum of commitments, until it is implemented, and
the results accrued.
The number of projects included in the public sector development program
(PSDP) at the federal level alone was about four thousand in 1998. For the country as a
whole (federal and provincial), the number in a given year is around eight thousands
projects in the public sector. A similar number of projects are undertaken in the private
sector.
Once the plan is finalized by the national planning agency, responsibility
for its implementation primarily rests with the ministries at the federal level and with the
line departments at the provincial and regional levels. This is not to say that the
Planning Commission is absolved of all responsibility in this regard. National Economic
Council (NEC), the supreme policy making body in the economic sphere, bears
responsibility for monitoring and execution of its policies, its subsidiaries, named
“Executive Committee of the National Economic Council (ECNEC)” and the Planning
Commission also share that responsibility. By the same token, the Planning and
Development Departments in the provinces share the responsibility of implementation of
the development projects with the concerned line departments.
Monitoring and Evaluation:
According to ECNEC decision, the selected federal/provincial projects
costing Rs 50 million and above were monitored by the Projects Wing. The quantum
was usually monitoring about 50 projects, and produce evaluation reports of 10
completed projects was considered to be reasonable annually. However, a much larger
coverage of at least 300 projects for monitoring is desirable along with evaluation of 50
completed projects every year for which additional resources and institutional
framework was provided. The “MPFP” project was upgraded, and planning, monitoring
and evaluation (PME) cells strengthened/created in the federal and provincial
sponsoring agencies and a computerized MIS installed to provide a firm data base. This
could not nourish due to capacities problems related to qualification and experience of
the human resource, management, and updated soft wear etc.
The Planning Division through Projects Wing maintain close liaison with
the PME cells in the federal ministries and divisions and the provincial governments
help maintain a uniform monitoring and evaluation approach which facilitates interprovincial and inter-regional comparison. The division, in consultation with other
relevant agencies, also attempt to collect, refine and develop data collection instruments
and make efforts to these compatible with the changes in the development needs of the
economy. The data base and the MIS facilities have been created within the Projects
Wing, and available to the PME cells at all levels. These services are also available to
all agencies involved in project implementation.
The Projects Wing regularly shares monitoring and evaluation reports with
sponsoring/ executing agencies for taking remedial action. A quarterly report is being
submitted to ECNEC on the monitoring and evaluation of projects. Now the projects
status identified through M&E have been shared with the Public Accounts Committee
(PAC), Standing Committees of National Assembly, Senate, and the Prime Minister’s
Secretariat. This is a helping practice in identification of slow moving projects, resolve
the issues in projects of strategic importance, re-adjust priorities and allocation of
resources for mid-course correction in policy, plan as well implementation of the
projects.
Recognizing the need for upgrading the knowledge and skills of the
officers and staff engaged in planning, monitoring and evaluation tasks in various
ministries, provincial line departments and large corporations, a Planning and
Management Institute (PPMI) is being set up by the Planning Division. The Institute
offers preliminary and specialized courses to train and impart necessary knowledge and
skills to officers working in the areas like project formulation, appraisal, management,
monitoring and evaluation and macroeconomic planning. This is planned that all project
managers should complete a project management course from the Institute before
appointment. The Institute will strengthen the in-house capacity of ministries and line
departments and build up capacities in every major sector of the economy.
Institutional Framework (IF):
The Institutional framework of government decision making process is
rather complex instead to be absolutely clear. The IF embodies enforcement of project
cycle, particularly in projects formulation based on the national needs, and in evaluation
after completion of projects for correction in policies. The M&E is based on factual data
and national parameters. These are actually correlated with the development plans to
achieve the ultimate objectives of economic growth. The value parameters are actually
the national weights and factual parameters are government propensity to re-invest or
the marginal propensity to invest in the public as well as private sector. There are four
units which form the institutional framework given as under:
POLICY MAKERS
PROJECT OPTIMAL
UNDER VARIOUS
RANGES OF NATIONAL
PARAMETERS
FACTUAL PARAMETERS
(NATIONAL WEIGHTS)
PROJECTFORMULATORS
PROJETCS FOR EVALUATION
PROJECT EVALUATORS
RANGE OF VALUE
PARAMETERS
(NATIONAL WEIGHTS
FACTURAL PARAMETERS
RANGES OF VALUE PARAMETERS
PROJECTS CHOSEN UNDER VARIOUS
RANGES OF NATIONAL PARAMETERS
CENTRAL PLANNERS
(CPO)
Formation is crucially dependent on techniques of evaluation therefore the
feed back based on evaluation shall be reported to the policy makers for improvement
in formulation of policy and plan, for use in future.
In Pakistan evaluation also guides, transfer of projects to the regular
budget, where accountably and transparency in funds utilization through execution of
projects are the major objectives to assess and recommend for more value addition in
the projects and programs cycles. For such purpose, PC-III, PC-IV and PC-V
documents carries high significance. These instruments are depicted in the Project
Monitoring and Evaluation System (PMES) for use.
The PSDP size has grown up from Rs 113 billion in 2003-04 to Rs 540
billion. Following graph depict the PSDP size during last 10 years:
SIZE OF PSDP & NEEDS OF M&E
PSDP Allocation
600
540
500
Rs. Billion
400
350
335
300
300
280
270
200
219
204
180
148
100
113
0
Year
1
Source: Projects Wing, P&D Division, March,2014
Total projects completed and registered with the Planning Commission till
February, 2013 are 806, where 163 PC-IV (projects completion Reports) have been
compiled in the Evaluation Section as follows:
S.No
Years
1
2
3
4
2008-09
2009-10
2010-11
2011-12
Total
Projects
completed
190
245
212
154
806
PC-IV
Received
41
43
61
19
163
PC-IV
%
of
completed projects
22%
16%
29%
12%
20%
Source: MIS Projects Wing, Planning Commission of Pakistan, March, 2014.
The analysis of the above data indicates that Management Information
System (MIS) is the backbone of M&E which is weak, and not properly equipped with
regards to the qualified and trained staff. The up dated and accurate data is not
available with the MIS. The PMES soft wear however requires updation to meet the
targets through integrated result based monitoring and evaluation.
The evaluation record remained weak due to absence of the proper
mechanism and inputs. Following is status of the Projects/ evaluated during 2006-13:
PROJECTS EVALUATED
Achievments
45
42
40
No. of Projects
35
30
25
20
25
20
17
15
12
10
7
7
5
0
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
Year
11
Source: Projects Wing, Planning, Development & Reform Division
M&E Status:
Project Wing has been assigned the task of monitoring and evaluation
since 1983. Projects monitored were more than the projects evaluated during last 30
years (1983-2013). Exact criteria could not be determined for selection of the projects
meant for monitoring, and particularly for evaluation. Projects funded through PSDP
being sponsored and executed by 43 Ministries and Divisions monitored during last 10
years are following:
PROJECTS MONITORED
Projects Monitored
900
800
766
No. of Projects
700
618
600
587
500
578
474
421
400
200
325
290
300
256
156
100
0
2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-11 2011-12 2012-13
Year
10
Source: Projects Wing, Planning, Development & Reform Division
Analysis of the above trend shows that number of mega projects having
direct bearings on the economy has increased. The post devolution after 2010, PSDP
funded projects have declined to 1200 due to transfer of projects to the provinces. A few
projects related social development have been accommodated and adjusted in the
different Ministries/Divisions at federal level. The Evaluation unit of Projects Wing has
not delivered properly and stands beyond the targeted achievements. This unit has not
made the required contribution towards the economic up lift by improving effectiveness,
efficiency, and transparency in the execution of the Projects. The visible reason for less
delivery is non availability of the qualified manpower, and proper guidelines of
evaluation techniques. This has actually resulted in, imperfect outcome, and more of no
good use for improvement in policy formulation as well as implementation. Following are
the major issues being faced by the Evaluation System:
Issues:

Non-existence of evaluation policy, manual, guide lines.

Low demand side of M&E.

Lack of awareness about the importance of evaluation.

Improper maintenance of important project information record.

Deficiencies in project preparation and design.

Unused evaluation findings.

Insufficient capacity building of human resources.

Job insecurity.

Lack of clarity and capacity to integrate project within respective sector.

Deficiencies, in project preparation.

Shortcomings in PC-I with respect to its Monitoring & Evaluation indicators.
Recommendations:

Within six months of project approval, detailed design and costing should
be finalized and submitted to the competent authority. Implementation of
such project components, which require detailed designing, should be
started only when these have been finalized.

Financial phasing should be linked with the implementation schedule and
a realistic assessment made of the resource availability for the project.
The project should not be initiated unless adequate funding is assured.

In approving the standardized building designs, variation in climate,
topography and availability of local material should be taken into
consideration for economic and efficient use of resources.

The implementation schedule should be based on bar charts/ PERT/CPM,
and should form an essential part of every project document. The
schedule of rates used in estimating project cost should be regularly
updated by taking into account the market rates, instead of allowing
across the board premium on the schedule of rates.

Qualified Project Directors should be appointed in case of each project,
and should not be transferred during the currency of the project. The
Project Director should be delegated full administrative and financial
powers. These measures would improve management and help assign
technical and financial responsibility.

Plug-in the recommendations / findings of M&E into the Planning and
Development System through consideration by CDWP and ECNEC, and
also
direct
sharing
with
the
stakeholders
for
enforcement
and
implementation.

Development of M&E processes, procedures, tools and manuals.

Enhancement & up gradation of M&E information management system.

Ensuring the continuity and capacity building of M&E professionals.

Formal feedback processes for M&E activities.

Proper Coordination & close liaison among Federal and Provincial M&E
Cells.
References:
1. Dasgupta, Amartya Sen London School of Economics Guidelines for Projects
Evaluation, UN, 1972.
2. Stephen, Harvard University, UNID, 1972.
3. Guidelines for Project Evaluation, UN Pusercatin, UNIDO Viena, Newyork, 1972.
4. 7th Five Year Plan, 1988-93, and Perspective Plan 1988-2003, Planning
Commission of Pakistan, Islamabad.
5. Presentation on Evaluation by P&D Division, Conclave, 12th March, 2004,
Islamabad, Pakistan.
6. Pakistan MDGs Report 2013, Planning Commission of Pakistan, Islamabad.
89
7. Manual for Development, P, D & R Division (1997), Islamabad.
8. Project Management Guidelines, Planning Commission of Pakistan, 2004.
9. Projects Wing, Planning, Development & Reform Division, Islamabad, March,
2014.
The Secretary,
M/o Commerce,
Government of Pakistan,
Islamabad.
Subject:
REQUEST FOR ISSUANCE LETTER FOR THE EXPORT OF CEMENT
BAGS FROM PAK TO AFGHANISTAN.
Dear Sir,
It is humbly requested to kindly provide us one million cement bags, i.e.
(Lucky, Kohat and Best Way) for the transit of Pakistan to Afghanistan, as we are
already doing this on internal level, therefore we shall be thankful to you if our request
may kindly be accepted.
2.
In view of the above, it is, therefore requested that necessary permission
may kindly be given and issue a letter for export of cement bags from Pakistan to
Afghanistan. .All necessary requirements are completed in your aspects.
Thanking you,
Yours faithfully,
(Tallat Javed)
BRIEF FOR SEMINAR ON MONITORING AND EVALUATION
Planning Commission responsibilities also include monitoring and
evaluation of development projects/ programs, analysis of these reports and
presentation of findings to the NEC and ECNEC. Submission of monthly progress
reports of monitored Projects to Cabinet Division and implementation of policy decisions
pertaining to execution of projects is also part of our duties.
According to ECNEC decision, the selected federal/provincial projects
costing Rs 50 million and above were monitored by the Projects Wing. The quantum
was usually monitoring about 50 projects, and produce evaluation reports of 10
annually. However a much larger coverage of at least 400 projects for monitoring and
50 completed projects for evaluation were targeted annually.
The “MPFP” project was upgraded, and planning, monitoring and
evaluation (PME) cells strengthened/created in the federal and provincial sponsoring
agencies and a computerized MIS installed to provide a firm data base through MIS.
Web based Projects Monitoring & Evaluation System (PMES) is functional
for facilitation of all stakeholders for effective implementation of the Projects/ Programs.
The project titled ‘Monitoring of PSDP Funded Projects (MPFP) was
originally approved in 2006 by CDWP at a cost of Rs 131 million which later on revised
to Rs 360 million in 2008. Apart from strengthening of Monitoring sections of the Wing,
two new Sections i.e., Evaluation and MIS were created under the project.
Since inception of project 4000 development projects have been
monitored by the Projects Wing, upto June, 2013 which is 81% of the target. Ex-post
evaluation of completed projects have been conducted out as 124 projects which comes
about 62% of the total target. The main reason of shortfall in achieving the proposed
targets is insufficient provision of manpower under the project as well as against the
regular sanctioned posts.
The on-line connectivity of Ministries/Divisions was initiated during 200708. A total 36 Ministries/Divisions have been provided on-line connectivity upto June,
2013.
There has been a positive impact of monitoring on implementation status
of projects. Execution of delayed projects has been streamlined. As a result the element
of time/cost over-run has been minimized. Flow of benefits to the beneficiaries has been
improved. Efforts have been made to resolve the issues affecting the execution of the
projects for their timely completion and within the approved cost.
Experienced Officers/Staff were hired in the project. With the passage a
time, all of them have been well acquainted with the work and performing their duties as
per TORs. The Monitoring and Evaluation reports are prepared and circulated to all
concerned for mid course correction and improvement in policy formulation and
implementation of projects.
Dr. Fazli Hakim Khattak
Director General (SS)