COUNTRY
NAME OF MAIN AUTHOR
(DETAILED AFFILIATION1)
NAMES OF OTHER AUTHORS
(SHORT AFFILIATION)
PRESENTING AUTHOR NAME
SUGGESTED NEC SUB-TOPICS2
INDONESIA
ARIF HARYANA
NATIONAL DEVELOPMENT PLANNING
AGENCY (BAPPENAS)
ARIF HARYANA
MONITORING AND EVALUATION SYSTEM IN INDONESIA
1. Introduction
As part of the public sector reforms, Indonesia has been enacted The Law No.17/2003
Regarding the State Financing and Law No.25/2004 Regarding the National Development
Planning System. These laws provide regulatory framework on the implementation of
performance-based planning and budgeting, which demand performance information from
the monitoring and evaluation (M&E) results. M&E become important part of the
development management cycle (Figure I), since it provides information that present
accountability as well as input for the improvement of next planning and budgeting period.
Furthermore, the Government enacted Regulation No.39/2006 that provide legal basis for
conducting M&E and mandated the Ministry of National Development Planning/National
Development Planning Agency (BAPPENAS) to develop M&E system. In order to
implement the mandat, the Deputy of Development Performance Evaluation (DPE) was
establihed within BAPPENAS in 2007.
Figure I. PLANNING AND M&E CYCLE
1
National Development Planning Agency (BAPPENAS), Jl. Taman Suropati No.2, Jakarta 10310,
INDONESIA; tel./fax: +62(21)3192-7438; email: haryana@bappenas.go.id
2
Independence – Credibility - Use of evaluation.
The DPE has three substantive roles: to formulate and coordinate national development
M&E; to monitor and assess the annual plan (RKP) and medium-term (five-year) plan
(RPJMN); and to mantain the national development M&E partnerships.
The enactment of regulatory framework and institutional establishment for national M&E
shows the interest and commitment of the Government of Indonesia in M&E. However,
therev still many challenges in the implemention of M&E, at operational level as well as
policy level.
The purpose of this paper is to share information and lesson learned from the Indonesian
evaluation system and its implementation challenges.
2. M&E System in Indonesia
Laws No.25/2005 describes three time horizons of national development planning: long-term
plan (RPJPN, 2005-2025), medium-term plan (RPJMN, 5 years, the current RPJMN covers
2010-2014), and annual plan (RKP). However, GR 39/2006 only mandate to conduct M&E
on the RKP and RPJM. Both RKP and RPJM contains programs and activities that is
implemented by line ministries. All programs conducted by all ministries in all provinces are
subject to be monitored and evaluated.
e-Monev for More Timely Monitoring
The focus of monitoring on the implementation of RKP is to review the progress of
projects/activities that is conducted quarterly and to measure achievement of the targeted
output at year-end. It also identify the constraints and botlenecks if there exist in the
implementation. This information is collected from reports that is submitted quarterly by line
ministries and provincial government. Coordination meeting with line ministries as well as
provincial government is conducted at least twice a year to discuss the progress and find out
the way to solve the problems and constraints. The information from this mechanism is used
as input for next year planning process.
There were several problems and constraints in preparing the quarterly monitoring reports.
One of the main problems was low reporting rate lack of reporting promptness due to
difficulties in collecting and integrating M&E information from the program implementers
around the country. There was only about 30 percent of all the Line Ministries submitted the
reports in 2010. Reporting rate from provincial government was even less than 20 percent It
was surely not a good input for BAPPENAS to analyze the national development
performance toward performance based planning and budgeting. Another issue that causes
the low reporting rate is the reluctance of the Line Ministries to submit the report as there
was no punishment nor rewards for delivering the reports.
In order to improve the reporting rate, BAPPENAS has developed an online application
system, called the e-Monev. The application is aimed to assist line ministries in collecting
and integrating data, and submitting the report. Now, the application has been implemented
and the number of line ministries who submit the report substantially increase to 65 percent
in first quarter of 2013.
Another purposes of developing the e-Monev is to improve accountability. The e-Monev
provides page that contain basic infomation that can be seen by the public from everywhere.
The information includes both the progress of program/activities performance and budget
disbursement. The e-Monev also provides “traffic-light” notification color on the
performance progress to every program/activities, which is based on the gap between
achievement and target already set. The notification can be used as warning for line
ministries to improve the implementation of their program. For each Program/activities, the
notification color will be as follows:
-
Green: gap < 0%; the program performance meets the target;
Yellow: gap < 25%; needs more attention and efforts to achieve its target; and
Red: large gap (gap > 25%); potentially cannot achieve its target in the end the year.
The “traffic-light” notifications are eye catching and easy to be understood. Therefore, the
notifications might be a good way for BAPPENAS as well as the line ministries to monitor
the progress of all the development programs.
BAPPENAS keeps improving the e-Monev to significantly help the Line Ministries and the
local government in submiting their M&E reports. BAPPENAS is now developing the eMonev for local government, and it is by this end of the year and will be tried out in the third
quarter of 2013.
Based on information collected from the quarterly monitoring data, BAPPENAS
independently conducted evaluation of the government annual plans (RKP). The gap analyses
was used in the programs evaluation, that is comparing the output achieved by the program
to its targets. Moreover, there was also a quadrants analysis to compare each ministry
performance to others.
Evaluation of the National Medium-Term Plan (RPJMN)
The evaluation on the implementation of the RPJMN is aimed to measure the outcomes and
impact of programs implemented during five years implementation of the plan. The GR
No.39/2006 mandated to conduct M&E on RPJM at least once in the last year of the
implementation period of the plan. Basically, the evaluation on each program is conducted by
the line ministries as a self-evaluation and the report has to be submitted to BAPPENAS.
These reports and data from other sources became the particular input for BAPPENAS to do
the analyses and produced the integrated evaluation report
Eventhough there is no obligation to conduct mid-term evaluation (MTE), BAPPENAS has
conducted MTE of the current RPJMN in 2012. The purpose was to measure the progress
achievement on the outcome or impact of the development programs, especially those
programs that are considered as national priority. The MTE utilized gap analysis method,
that is by comparing the outcomes or impacts achieved until mid year of 2012 with the
planned or targeted outcomes and impacts. In addition, MTE also review the trends of
achievement in the last three years. Based on both gap and trend analysis on key performance
indicators we can make a judgement whether the programs already achieved its target, is on
track towards achieving its target, or lagged behind. Based on this judgment, we put the color
notifications on each performance indicators of program:
-
Green: target already achieved or on-track/on-trend to meet the target;
Yellow: needs more and efforts to achieve its target; and
Red: difficult to achieve its target in the year-end of the RPJMN.
The eye-catching color notifications is used in order to make it easier for policy makers to
grasp and understand the progress of the development programs.
For those programs which has performance indicators falls into the Yellow and Red color
notifications, The MTE also identified the constraints and problems that exist and proposed
solutions.
Figure II. Example of Analysis Format in MTE: Health Sector
The evaluation conducted by BAPPENAS can be considered as independent because
BAPPENAS is planning institution, not a ministry or institution that execute the development
programs. Therefore, the bias in analysis due to conflicting interest can be minimized.
BAPPENAS also free to use data from any independent credible sources, not only data from
line ministries reports. The use of credible data from independent sources, such as the
Statistics Indonesia (Badan Pusat Statistik/BPS) and national as well as international
independent institutions assure the credibility and quality input of the evaluation.
In practical, the information provided by the MTE was very beneficial for policy makers,
because the MTE was conducted at the right time. Performance information from MTE was
available just in time when the process of planning and budgeting for the fiscal year (FY)
2014 started. The MTE results was presented by the minister and discussed in the cabinet
meeting chaired by the President. As follow up of the meeting, the President deliver directive
policy to all ministers regarding the priority program and activities for FY 2014. The
President also instructed the three coordinating ministers to coordinate relevant ministries to
take steps needed to deal towards improving the performance of the indicators that fall under
Yellow and Red Notifications. This shows how important it is the M&E in the management
cycle of development process.
Public Consultation Forum For Planning and M&E
Government should provide a public sphere to give society access to all the public activities,
including in the planning and M&E practices. Article 2, Law No. 25/2004 requires and
provides space for public involvement in this area. The involvement is intended to capture the
aspiration of people and increase the sense of belonging of stakeholders. In practice, public
involvement is accomodate in the planning process. Civil society organizations (CSO) and
academia has been playing active role in the Musrenbang, discussion forum in the planning
process.
Eventhough there is no obligation to create formal forum for CSO to deliver their
views in the M&E, BAPPENAS held public consultation meetings since 2012, involving
participants from CSO and universities. Relevant CSO is invited to share and discuss their
views regarding specific issues in the meetings. The discussion is held with academia or other
independent parties as facilitators. The forum makes M&E results become more objective. It
is also expected to improve the current M&E system and methodology.
3. Closing Remarks
Indonesia has realized how important the M&E is. Therefore, Indonesia commits to
comprehensively implement it, both regulation and institutionalization part. With the
commitment, Indonesia has enacted GR No. 39/2006 and established the DPE within
BAPPENAS in 2007. However, there are challenges faced by Indonesia to improve its M&E
system and capacities.
The main challenge that is faced by Indonesia is to build the capacity of the M&E
practitioners. As a consequence, Indonesia needs to broaden its networking and partnerships
to share knowledge and take lesson learned from other countries in implementing a better
M&E system.
---oo0oo---