Evaluation of the Decision making process of the
Mexican federal budget.
MSD Model
AGUSTÍN CASO RAPHAEL
DIRECTOR ADJUNTO DE EVALUACIÓN DEL DESEMPEÑO
1
MACROECONOMIC INDICATORS
Mexico´s recent public finances reflect an era of fiscal responsibility
Over the past fifteen years, Mexico has undergone significant economic and
political reforms. This period has witnessed important improvements in the
economy and of public finances:
Formal employment generation (thousand of employments)
Aggregate supply and demand, 2012 (%)
Source: INEGI
Source: Instituto Mexicano del Seguro Social
MACROECONOMIC INDICATORS
Main numbers of the Mexican economy at the beginning of fiscal year 2013
Fixed Gross Investment (%)
Inflation (%)
Source: INEGI
Source: INEGI
Treasury Bonds (%)
Source: Bank of Mexico
Exchange rate (pesos per dollar)
Source: Bank of Mexico
MACROECONOMIC INDICATORS
Main numbers of the Mexican economy at the beginning of fiscal year 2013
Public deficit without Pemex investment (% of GDP)
Country Risk
Source: JP Morgan
Source: Ministry of Finance of Mexico
4
MACROECONOMIC INDICATORS
Mexico´s recent public finances reflect an era of fiscal responsibility…
Two decades ago, the situation of the Mexican economy was significantly
different:
 In August 1982, the debt crisis hit Mexico and the bailout came at a
very heavy cost.
 In 1995, the peso crisis had once again catastrophic consequences.
 The government embarked in a comprehensive economic policy to
foster economic growth with monetary stability.
The good economic and financial records have been achieved by means of:
 Strengthening the monetary system,
 Enhancing fiscal sustainability and the efficiency and effectiveness of
public spending,
 Improving the accountability of public expenditure at the national
and sub-national levels; and
 Implementing a sound Performance Evaluation System (SED)
BUILDING A M&E SYSTEM
Three stages of development
 Mexico’s path to a Performance Evaluation System (SED) can be divided into three periods:
 1970’s to 1990’s: Limited and isolated efforts of evaluation, concentrated in specific areas
of the public administration (e.g. education, agricultural policy).
 Late 1990’s: Budget System Reform; New Programmatic Structure (NEP), Integral Process
of Planning and Budget (PIPP); Evaluation of Social Policy.
 2006-2007: Institutionalization of M&E across the federal and sub-national governments.
Wider use of institutionalized, law binding, whole-of-government single evaluation system.
 The landmark 2006 Federal Budget and Fiscal Responsibility Law and the 2007 Integral Fiscal
Reform fostered the:
 Creation of the Performance Evaluation System (SED)

Legal framework for performance budgeting at the Sub-national and Local levels

Accounting and financial harmonization across all levels of government: Federal, Subnational and Local
6
BUILDING A M&E SYSTEM
Government Major Fiscal Reforms
It also established:
 A balanced-budget rule and modified the congressional budget
approval process;
 Principles and rules for the evaluation of investment projects;
 Required performance indicators to be included in the federal
budget;
 Called for the establishment of Guidelines for the External
Evaluation of Federal Programs; and
 Set out strict rules for controlling personnel and current
expenditure for a more efficient execution of public expenditure.
7
BUILDING A M&E SYSTEM
Government Major Fiscal Reforms
The 2007 Integral Fiscal Reform (IFR), a wide-reaching reform package:
 Induced tax reforms and modified the fiscal framework between
the states and the federal government:
 Established the framework for performance budgeting at the Subnational and Local levels.
The 2008 Government Accountability Act established accrual accounting and
financial harmonization across all levels of government: Federal, Sub-national
and Local.
 By 2014, we shall have accrual-based accounting registries,
account catalogues, and accounting manuals, at both the Federal,
Sub-national and Local levels.
8
THE SED
The government´s effort to improve the efficiency and effectiveness of
Public Programs
The SED relies in a comprehensive programming and budgeting architecture:
The Budgetary Program (BP) is the main link between programming,
budgeting and M&E (programmatic category for organizing, in a
representative and homogeneous way, the resource allocations of federal
program in charge of spending ministries and agencies in order to achieve its
objectives and goals).
The SED has two main components:
1. The evaluation of BPs and public policies by means of:
 Performance Indicators (strategic indicators) to measure program
outputs and outcomes,
 Program and policy evaluations, carried out as part of the Annual
Evaluation Program (AEP).
2. The evaluation of the management processes and public goods and
services delivery, by means of the Program for a Close, Modern
Government
9
PbR-SED
The relationship of the SED to the Budgeting for Results Strategy and the
NDP is represented in the following figure:
National Development Plan
6 years within a 20 to 25 years time frame perspective
•
•
•
Links budgeting with
the NDP, its objectives
and programs.
Strategic objectives.
• Logic Framework Methodology.
• Objectives, goals and indicators.
• New budgeting process and organization.
BUDGETING FOR
RESULTS (PBR)
Broad alignment: NDP
Sector Programs
Budgetary programs
"
"
Performance
Evaluation System
(SED)
Evaluation of
Budgetary Programs
and Policies
Budget Indicators
• Resource allocations based on results.
Evaluation of social
Development
programs
Program for a
Close & Modern
Government
ANNUAL EVALUATION PROGRAMME
IMPROVEMENT COMMITTMENS
10
PbR-SED
Stakeholders
11
PbR-SED
Budgeting for Results is tied to the National Development Plan…
The objectives that all the programs must achieve are identified by the National
Development Plan (NPD).
NATIONAL DEVELOPMENT PLAN
NATIONAL TARGETS / TRANSVERSAL STRATEGIES
PROGRAMS DERIVED FROM NDP
OBJETIVES
INDICATORS
STRATEGIES
TARGETS
ACTION LINES
BUDGETARY PROGRAMS
12
THE BUDGETING FOR RESULTS ANNUAL CYCLE
PLANNING
• Link of sector program to the National Development Plan
• Ministries and agencies strategic objectives (SO) setting
PROGRAMMING
•
•
•
•
Program structure authorization
Budgetary programs definition
MIR/LFM
Strategic and management indicators
BUDGETING
R
E
S
U
L
T
S
• Results based Budgetary appropriations
• PIPP link
SPENDING AND
CONTROL
• Administrative performance and public
expenditure improvements
FOLLOWING UP
• Monitoring reports.
• Following up indicators.
EVALUATING
(SED)
• Specific outcome commitments
and improvement of
administrative performance
REPORTING
ACCOUNTABILITY
• Public accounts
with results.
• Reports to
Congress.
13
THREE STAGES OF SED
The SED has followed a three-stage process:
a) SED Implementation
Actions taken during 2007 and 2008 to establish and implement the
principles, concepts, methodologies, guidelines, procedures and systems
that support SED operation.
The SED implementation phase was audited by the Federation Superior
Audit Office, audit branch of the Federal Congress, in 2010.
14
THREE STAGES OF SED
b) SED Consolidation
From 2009 onwards, the main goal has been to expand and improve the
quality of the SED, using performance information for planning,
programming, budgeting and resource allocation and for streamlining
budgetary programs.
The SED consolidation phase was audited by the Federation Superior
Audit Office, audit branch of the Federal Congress, in 2011.
15
THREE STAGES OF SED
c) Support to Sub-national governments
The Federal Government established a program, with the support of the
World Bank (WB) and the Inter-American Development Bank (IADB) to
implement PBR-SED in Sub-national governments.
PbR-SED diagnoses for the implementation of the SED in 31 states as well
as in the Federal District were performed in 2010 and 2012.
16
SUB-NATIONAL GOVERNMENTS: RELATIVE ADVANCE IN
SED IMPLEMENTATION (2010-2012)
0%
BC
MEX
JAL
DF
CHIS
PUE
QRO
SIN
NL
NAY
QROO
YUC
CHIH
MICH
HGO
SLP
TLAX
GTO
DGO
CAMP
TAB
COL
SON
MOR
ZAC
VER
GRO
AGS
OAX
TAMPS
COAH
BCS
10%
20%
30%
40%
50%
60%
70%
80%
Octubre (2012)
90%
100%
2010
17
Convenios con Entidades Federativas
PBR SED IN SUB-NATIONAL GOVERNMENTS
Budgeting for Results
Legal Framework
Planning
Programming
Budgeting
Exercise and Control
Training
Performance
Evaluation System
Evaluation
Performance
Indicators
Use of Information
for decision making
Transparency
Transparency in
general
Citizen Language
Budget
Transparency
18
PHASES OF THE SED – PERFORMANCE EVALUATION UNIT (UED)
SETTING
 Definition of public expenditure policy.
 Performance Evaluation System (SED) coordination.
 Rules and methodologies for underwriting performance improvement tools
at the institutional level.
 National Development Plan (PND) sector, special and regional program
dictamination.
 Performance evaluation and planning for results-oriented budget planning.
 Federalized resources (transferred to Sub-national and local governments)
M&E.
19
PHASES OF THE SED – PERFORMANCE EVALUATION UNIT (UED)
SETTING
 Performance information
mechanism (PIPP).
for
budgetary
decision-making
 Synthetic Model of Performance Information (MSD) improvement.
20
PHASES OF THE SED – STRENGTHENING TRANSPARENCY IN
BUDGET INFORMATION
Website creation with useful information on public spending.
Elements:
• Budget Integration information in one system.
• User´s easy access.
• Data Warehouse for budget information.
• Ministry of Finance portal linking all institutional systems.
www.transparenciapresupuestaria.gob.mx
21
MONITORING AND EVALUATION
Monitoring
Evaluation
• Continuous improvement of MIR.
• Annual Evaluation Program (PAE).
• MIR budgetary programs
coverage expanded.
• External and independent
evaluations.
• 75.7% of programmable spending
with performance information.
• 883 external evaluations since
2007.
PERFORMANCE INFORMATION SYNTHETIC MODEL (MSD)
The Performance Information Synthetic Model (MSD) is a tool for assessing the
performance of budgetary programs.
Purposes
 Budget decision-making based on Budgetary
programs.
 Information to improve objectives, indicators and
targets.
 Possible synergies or overlaps identification.
23
PERFORMANCE INFORMATION SYNTHETIC MODEL (MSD)
Evaluatio
n
Range
in%
Overall Performance
5
95-100
High performance
MSD?
4
80-94
Medium High Performance
3
55-79
Medium performance
The MSD is an instrument for budgetary evaluation
which gathers the main performance information of
budgetary programs, in order to know their trend.
2
30-54
Medium Low Performance
1
0-29
Under- performance
 What is the Performance Information Synthetic Model-
 What is the purpose of the MSD ?
That the agencies of the federal public administration:

Have elements for budget allocation decision making based on program results. Justify any proposed increases
or decreases in budget allocations

Use the information to improve the goals, indicators, targets and issues arising from external evaluations

Identify positive externalities and potential duplications in order to strengthen budgetary programs

Present information in a clear and simple language that allows citizens to know and understand programs
performance
 What is the overall assessment?
To obtain the overall performance rating of each budget program a value range from 1 to 5 is established, with 5
representing the highest performance
24
PERFORMANCE INFORMATION SYNTHETIC MODEL (MSD)
 What elements of evaluation does the MSD consider?
1.
Budgetary efficiency (2008-2011)

The efficiency in budget spending for each budget program for the past 4 years

The difference between the authorized and spent budget in order to determine the degree of under-spending or overspending
2.

Government priorities
Takes into account government priorities according to what is established by law
3.
Results from the analysis of the Indicators for Results Matrix (MIR) 2009-2011, taking into account three criteria:

Quality of the MIR according to an external evaluation

Ratio between the achievement of goals and the budget increase

Achievement of determined goals
4.
External Evaluations

It includes the assessment of external evaluators, taking also into consideration the lack of evaluations as an incentive
for contracting evaluations
5.
Recommendations of external evaluations (ASM)

The degree of progress in implementing the recommendations of external evaluations is taken into account.
25
PERFORMANCE INFORMATION SYNTHETIC MODEL (MSD)
 How is the overall assessment done?
For each budget program a score is recorded for each one of the items considered and a final performance rating
is obtained.
Budget (over/under
spending)
Government
Priorities
MIR
External
Evaluations
Recommendations of
external evaluations
10%
15%
25%
25%
25%
 Outcomes:
The Ministry of Finance convenes meetings with program managers in order to analyze performance results.
Information is analyzed by both parties in order to improve the budgetary cycle and agreements are reached. From
these meeting the next results have been achieved:

Improvement of the MIR

Proposals for new external evaluation

Improvement of performance information

Elimination of duplicated programs


Review of on going external evaluation
recommendations
Improvement of budgetary structure

Compliance with budgetary allocations

Compaction of programs

Resource re-allocation

Improvement of programmed goals

Increased performance transparency
BUDGETING PROGRAMME PERFORMANCE COMPARISSON 2011 - 2012
Comparative Performance
Low
Medium-Low
Medium
Medium High
High
Performance
Performance
2011
586
2012
643
High
Medium High
Medium
Medium Low
Low
0.0%
9.0%
31.1%
49.5%
10.4%
3.6%
26.1%
57.2%
15.2%
7.7%
27
Branch
Budgetary Program
Priority
(Art. 42
LFPRH)
20 Desarrollo Social
Subsidios: Sectores Social y Privado o Entidades Federativas y Municipios
Sujetos a Reglas de Operación
S048 Programa Habitat
Sí
S052 Programa de Abasto Social de Leche a cargo de Liconsa,
Sí
S.A. de C.V.
S053 Programa de Abasto Rural a cargo de Diconsa, S.A. de
Sí
C.V. (DICONSA)
S054 Programa de Opciones Productivas
Sí
S057 Programas del Fondo Nacional de Fomento a las
No
Artesanías (FONART)
S061 Programa 3 x 1 para Migrantes
Sí
S065 Programa de Atención a Jornaleros Agrícolas
Sí
S070 Programa de Coinversión Social
Sí
S071 Programa de Empleo Temporal (PET)
Sí
S072 Programa de Desarrollo Humano Oportunidades
Sí
S118 Programa de Apoyo Alimentario
Sí
S155 Programa de Apoyo a las Instancias de Mujeres en las
Sí
Entidades Federativas, Para Implementar y Ejecutar
Programas de Prevención de la Violencia Contra las
Mujeres
S174 Programa de estancias infantiles para apoyar a madres
Sí
trabajadoras
S176 Pensión para Adultos Mayores
Sí
S216 Programa para el Desarrollo de Zonas Prioritarias
Sí
S241 Seguro de vida para jefas de familia
No
Otros Subsidios
U008 Subsidios a programas para jóvenes
No
U009 Comedores Comunitarios
Sí
Desempeño de las Funciones
Prestación de Servicios Públicos
E003 Servicios a grupos con necesidades especiales
Sí
E016 Generación y articulación de políticas públicas
No
integrales de juventud
Provisión de Bienes Públicos
B004 Programa de adquisición de leche nacional a cargo de
Sí
LICONSA, S. A. de C. V.
Planeación, seguimiento y evaluación de políticas públicas
P001 Definición y conducción de la política de desarrollo
No
urbano y ordenación del territorio
P002 Definición y conducción de la política del desarrollo
No
social y comunitario, así como la participación social
P003 Actividades orientadas a la evaluacion y al monitoreo
No
de los programas sociales
P004 Desarrollo integral de las personas con discapacidad
No
Promoción y fomento
F001 Fomento del desarrollo de las organizaciones de la
No
sociedad civil
Proyectos de Inversión
K025 Proyectos de inmuebles (oficinas administrativas)
No
Administrativos y de Apoyo
Indicator
2013 Budget
2008-2013
Budgetary
Performance
Performance
Indicator Matrix
Annual
Evaluation
Program
Aspects
Susceptible of
Improvement
Budgetary
Program
Performance
Ranking
Branch General
MIR
MIR
45.1
1,086.8
MEDIUM
HIGH
HIGH MEDIUM HIGH
MEDIUM LOW MEDIUM HIGH
MEDIUM HIGH
MEDIUM
MEDIUM HIGH
MEDIUM
1
5
79
177
MIR
1,858.9
MEDIUM HIGH
MEDIUM LOW MEDIUM HIGH
MEDIUM HIGH
MEDIUM
3
157
MIR
MIR
414.1
150.8
MEDIUM HIGH
HIGH
LOW
MEDIUM
MEDIUM LOW MEDIUM HIGH
MEDIUM
MEDIUM LOW
MEDIUM LOW
MEDIUM
18
11
506
298
MIR
MIR
MIR
MIR
MIR
MIR
MIR
525.6
300.9
310.6
1,291.8
36,177.7
4,224.9
258.4
MEDIUM HIGH
MEDIUM
MEDIUM
MEDIUM
MEDIUM HIGH
MEDIUM HIGH
MEDIUM HIGH
MEDIUM LOW
MEDIUM LOW
MEDIUM
LOW
MEDIUM LOW
MEDIUM LOW
MEDIUM LOW
MEDIUM
MEDIUM HIGH
MEDIUM HIGH
MEDIUM
MEDIUM HIGH
MEDIUM HIGH
MEDIUM HIGH
MEDIUM
MEDIUM LOW
MEDIUM
MEDIUM HIGH
MEDIUM HIGH
MEDIUM
MEDIUM
MEDIUM
MEDIUM
MEDIUM
MEDIUM
MEDIUM
MEDIUM
MEDIUM
17
13
10
15
4
7
8
465
353
284
455
161
238
253
MIR
3,547.6
MEDIUM HIGH
MEDIUM LOW MEDIUM HIGH
MEDIUM
MEDIUM
6
231
MIR
MIR
MIR
26,000.9
6,631.4
400.0
MEDIUM
MEDIUM HIGH
LOW
LOW MEDIUM HIGH
LOW
MEDIUM
LOW
S/N
MEDIUM
MEDIUM LOW
S/N
MEDIUM
MEDIUM LOW
LOW
12
20
31
326
605
930
MIR
No
65.5
-
MEDIUM LOW
LOW
LOW MEDIUM LOW
LOW
S/N
S/N
S/N
LOW
LOW
23
29
793
899
MIR
MIR
273.0
247.8
MEDIUM
MEDIUM LOW
MEDIUM LOW
MEDIUM LOW
S/N
S/N
S/N
S/N
MEDIUM
MEDIUM LOW
16
22
457
734
MIR
1,477.1
MEDIUM
MEDIUM
MEDIUM
S/N
MEDIUM
9
279
No
18.7
MEDIUM LOW
LOW
S/N
S/N
LOW
27
874
MIR
1,042.5
MEDIUM
MEDIUM LOW
S/N
S/N
MEDIUM LOW
19
512
MIR
240.0
MEDIUM HIGH
MEDIUM LOW
S/N
S/N
MEDIUM
14
362
No
21.6
MEDIUM
LOW
S/N
S/N
LOW
25
816
MIR
49.6
MEDIUM HIGH
LOW
S/N
S/N
MEDIUM LOW
21
650
No
8.4
MEDIUM LOW
LOW
S/N
S/N
LOW
28
26
849
Thanks!
29