Using Performance Information
for Management and Budgeting
- Canada’s Experience
Lee McCormack, Executive Director
Results-based Management
OECD Presentation, May 3, 2006
Overview
 The Canadian context
 Problems, priorities and implementation challenges
 Lessons learned
2
The Canadian context
3
Canada is a big country with relatively few people
and a highly decentralized government
•
•
32 million citizens in 5 ½ time zones
Decentralized federation with 10 provinces and 3 territories
•
Provinces are equal to the federal government and have real power
•
Hundreds of federal-provincial agreements and federal
programming is weighted towards transfer payments
•
Over 90 departments and agencies in the federal government
– 46 Crown corporations
– 172 000 public servants
•
Eight consecutive federal surpluses
•
New Conservative minority government
4
The majority of federal expenditures
are transfer payments…
•
Total net expenditures in 2004-05 were $172.8 B, of which 56% ($96.8B)
were transfer payments (including major transfers & grants and
contributions)
Total Net Expenditures: $172.8 B
Public Debt Charges
17%
Grants and Contributions: $22.5B
Municipalities and
Local Govts
3%
Other Program Expenses
27%
Persons
19%
Non-Profit
Organizations
33%
Industry
13%
Grants and Contributions
13%
Major Transfers to Persons
(EI, OAS, GIS and
spouse's allowance)
22%
Transfer
payments
Major Transfers to other
levels of Government
(CHST, Fiscal
arrangements, etc)
22%
International
Organizations and
Foreign Countries
16%
Provinces and
Territories
16%
5
The federal expenditure management system is
also decentralized
Element
Responsibility
Developing an annual Budget and a multi-year
fiscal framework
Department of Finance
Establishing annual departmental reference
levels
Treasury Board Secretariat and
departments – approved by Treasury
Board
New policy or program approval phase
Privy Council Office supporting Cabinet
Allocating and reallocating to ensure alignment
with priorities and aggregate expenditure control
Departments routinely – plus three central
agencies in special cases
Seek Parliamentary approval of spending plans
Treasury Board Secretariat
Reporting to Parliament on spending plans,
actual expenditures, and results achieved
Treasury Board Secretariat and
Departments
6
The Finance department sets the Fiscal Framework
and manages the annual budget process
Budget Speech
Budget Bill
Main Estimates
Final Decisions on
Budget
February
Cabinet Review
of Priorities
Focus on the “increment”
No formal mechanism
requiring incorporation of
performance information
into Budget process
Jan-Feb
Aug-Sept
Not many targets
Cabinet Review
of Budget
Strategy
Dec-Jan
Release of Budget
Consultation Papers
(Fall Update)
Budget Consultation
Process
Oct-Dec
October
7
Treasury Board Secretariat monitors the ongoing
“A-Base” and seeks Parliamentary supply
Main Estimates
A-Base Perspective
Input to Budget
Interim Supply
Supplementary
Estimates and
Supply Bill
February
Jan-Feb
Pressure
Management
Dept Baseline
Funding
Oct-Jan
Substantial planning and
performance information given to
Parliament
Much performance information
available, especially on G and C
programs
Feb-Mar
Reports on Plans
and Priorities
Full Supply Bill
March-June
Room to develop more systematic
cyclical review
Supplementary
Estimates and
Supply Bill
Nov-Dec
Departmental
Performance
Reports, Canada’s
Performance and
Public Accounts
Aug-Nov
A-Base Perspective
Input to New Policy
Proposals
Aug-Nov
8
Canada has a relatively mature performance
measurement and reporting infrastructure
•
All major departments and agencies
– Have internal audit and evaluation units
– Publish their audits and evaluations
– Produce and explain to Parliament upon request their reports on plans and
priorities and departmental performance reports
– Are developing outcome frameworks that are consistently based across
government
•
Treasury Board Secretariat
– Reviews program and management performance, focusing primarily on G and C
programs
– Has a strong policy centre for results based management
– Produces an annual performance report – Canada’s Performance
– Assesses management performance across government, including results-based
management capacity
•
Office of the Auditor General
– Produces performance audits of departments and agencies
9
Although there is long results management history …
Management, Resources and Results Structures – regaining detailed program level knowledge
Today’s
Agenda
Possible Federal Accountability Act 2006 – cyclical evaluation of all transfer payment programs
Improved Reporting to Parliament – moving to whole of government planning and reporting
Management Accountability Framework – assessing management performance across government
Canada’s Performance – linking programs to societal performance
Results for Canadians – results-based management as a stated priority of government
2000
Transfer Payment Policy – performance frameworks and evaluations reviewed by TBS
Modern Comptrollership – investment in management practices and controls
Program Review – dealing with a large deficit through major expenditure cuts
Mid
90s
Departmental Reporting to Parliament – moving to results-based plans and reports on performance
Planning Reporting and Accountability Structures – unsuccessful try at linking resources to results
Early
to
Mid
80s
First Program Evaluation Policy – government-wide implementation
… some real challenges remain
10
Problems, priorities and
implementation challenges
11
Our agenda focuses on four main problems that are
limiting us from getting performance information into
our budget and management processes
•
•
To regain our detailed whole-of-government understanding of the
ongoing program base we are implementing the Management,
Resources and Results Structure (MRRS)
Agenda
To have better ongoing measurement, periodic evaluation and use of
performance information we are renewing the evaluation function
• MRRS
• Evaluation
•
To help Parliamentarians fulfil their role of holding the Government to
account we are transforming the existing reporting regime through the • IRPP
Improved Reporting to Parliament Project (IRPP)
• MAF
•
To ensure the right management incentives are in place we are
implementing the Management Accountability Framework (MAF)
12
MRRS
How do you regain a detailed understanding of
the program base?
•
Evaluation
IRPP
MAF
To manage its programs, the government needs to know:
–
–
–
–
What they are
How they align to intended outcomes
What they cost
What results they achieve
•
This should support more informed decisions on value for money,
program effectiveness and possible ways of improving
•
Treasury Board approved the Management, Resources and
Results Structure policy (MRRS) to remedy this situation
•
The policy is mandatory for appropriated organizations and entering
the second year of implementation
13
MRRS
The approach requires consistent, structured
information for every department and agency
Program Activity Architecture (PAA)
Evaluation
IRPP
MAF
+ Financial and non financial information
Program Activity Architecture
Department or Agency
M
Planned and Actual
Results
Departmental Strategic Outcome
D
D-1
D-2
D-3
Program Activity
Planned & Actual
Financial information
Required for all
elements and
all levels
of the PAA
Sub – Program Activity
Sub - Sub – Program Activity
D-4
Governance
Required for the
highest or “program
activity” level
Sub - Sub – Sub – Program Activity
Recoding of departmental information systems will be necessary
Central Expenditure Management Information System (under development)
14
MRRS
Evaluation
MRRS is the basis for planning, resource allocation
and reporting across the government
More logical &
consistent basis
for interaction
DEPARTMENTS
Management Tool
Improve program performance
Better alignment of resources & priorities
Easier reallocation
More evidence based reporting
Budget Office Function
Informed decisions and
investment choices based
on priorities and
value for money
Budget
Plan
Finance
Privy
Council
Office
IRPP
MAF
Common basis
for planning &
reporting inside
& outside
PARLIAMENT
MRRS
Reporting Tool
Stronger accountability
on spending and results
15
MRRS
Evaluation
Implementation faces four main challenges
•
IRPP
MAF
Cultural
– Some departments have had difficulty in adopting a performance
measurement culture
– Many tend to identify too many measures and actually measure too few
– Transparency leads to accountability and that can be scary
– Estimate it will take 3 – 5 years to establish a government wide
performance measurement framework
•
Institutional
– Departments are managed in vertical, organizational silos and some
are having difficulty converting to logic based program structures
– Over the last 2 years 30% of departments have made changes to the
basic program activity architecture – not unexpected
– Performance measurement capacity – demographic concerns
– 3-5 years to stabilize to long term and enduring structures
16
MRRS
Evaluation
Implementation faces four main challenges
•
IRPP
MAF
Definitional
What is a program?
How do you measure the effectiveness of corporate services?
How do you treat overheads consistently?
What is the lexicon of program descriptions? (dozens of other questions)
•
Technical
– Developing a system capable of collecting, sorting and linking
information
– Making sure it talks to other systems – annual budget, public accounts
and others
– Recoding of departmental systems is a large job – pilot projects
17
MRRS
The Federal Accountability Act, if passed, will
increase scrutiny of program performance …
Evaluation
IRPP
MAF
•
Auditor General to inquire into the use of funds by third parties under
any funding agreement with the federal department
•
Parliamentary Budget Office to provide analysis to Parliament about
the nation’s finances and trends in the national economy
•
Departments to evaluate, at least once every five years, the relevance
and effectiveness of its grants and contributions programs
… and this will put a strain on evaluation capacity
18
MRRS
There are some real challenges for the
evaluation function
Evaluation
IRPP
MAF
•
Quality, timeliness and strategic focus issues make it difficult to
use evaluation to support decision-making:
– often focused on small programs and not strategic
– can take too long to complete and are difficult to understand
– can be self serving when funded by program managers
•
Government wide capacity issues have made it difficult to deliver
– Lack of trained evaluators
– No consistent competencies for those who lead the
evaluation function
– Definition of the evaluation “product” hasn’t changed much in
20 years
19
MRRS
Step #1 is to renew our policy so that
evaluation is…
•
Better recognized as a core function of public management
•
Embedded in the decision-making process
•
Linked to budgeting and expenditure management
•
A neutral and independent function, aligned to government
management priorities
•
Timely in its production of a quality product
Evaluation
IRPP
MAF
20
MRRS
Evaluation
Step #2 is to zone in on implementation
•
•
IRPP
MAF
•
Make sure the function is properly resourced
Build capacity by re-establishing training and recruitment programs
and partnering with universities on certification
Ensure objectivity – evaluators fund evaluations, period
•
Redefine the “product” to promote relevance and better coverage:
–
–
–
–
•
•
Strategic policy evaluation
Impact evaluation
Value-for-money assessments
Implementation evaluation
Treasury Board Secretariat to assess quality and set out clear
competencies for Heads of evaluation and evaluators
Introduce a Government wide evaluation plan
21
MRRS
Canadian evaluators are ready for change but
they will face real implementation challenges
Evaluation
IRPP
MAF
•
Demographic - High turnover rates will need to be met with
targeted whole of government recruitment
•
Workload – Will be difficult to manage without a realistic plan
•
Skills – Will take time to rebuild training programs and partner with
universities
•
Jurisdictional – Continually reinforcing the distinction between
internal audit and evaluation
•
Relevance – Good quality, timely products need to be used to
inform allocation and reallocation decisions
22
MRRS
Parliamentarians want better public
performance reporting, ideally …
Evaluation
IRPP
MAF
• Simpler, more integrated information
• More context and analysis (trends, graphs, including reporting on
horizontal issues)
• High level overviews with the ability to drill-down to more detail
• Clearer logic between planning and performance reporting documents
• More balanced reporting
• Better links between programs, resources and results
• More clarity between parliamentary reports and estimates votes
23
MRRS
Whole of Government planning and reporting
is part of the solution …
Evaluation
IRPP
MAF
 Government could better communicate its overall “strategic plan”
 This “strategic plan” could be akin to a whole of Government
Report on Plans and Priorities:
– Outlining plans and priorities for the coming fiscal year
– Linking planned results to planned expenditures
– Electronic, in every parliamentary office
 Logically, a fall whole of Government performance report could
present the results and actual spending vis-à-vis such a Plan
… but this solution depends on a stable, whole of
Government framework
24
MRRS
This framework has been tested in Canada’s
Performance over the past few years
Evaluation
IRPP
MAF
25
MRRS
This agenda depends on better management
overall: the Management Accountability Framework
Evaluation
IRPP
MAF
Public Service Values
By their actions departmental leaders continually reinforce the importance of PS Values and Ethics in the delivery of results to
Canadians (e.g.: democratic, professional, ethical and people values).
Governance
& Strategic
Direction
The essential
conditions –
internal
coherence,
corporate
discipline and
alignment to
outcomes -- are in
place for
providing
effective strategic
direction, support
to the Minister
and Parliament,
and the delivery
of results.
Policy and Programs
People
Citizen Focused Service
Departmental research and analytic
capacity is developed and sustained
to assure high quality policy
options, program design and advice
to Ministers.
The department has the people, work
environment and focus on building
capacity and leadership to assure its
success and a confident future for the
Public Service of Canada.
Risk Management
Stewardship
Accountability
The executive team clearly defines
the corporate context and practices
for managing organizational and
strategic risks proactively.
The departmental control regime
(assets, money, people, services, etc.)
is integrated and effective, and its
underlying principles are clear to all
staff.
Accountabilities for results are
clearly assigned and consistent with
resources, and delegations are
appropriate to capabilities.
Services are citizen-centred,
policies and programs are
developed from the ‘outside in’,
and partnerships are encouraged
and effectively managed.
Results and
Performance
Relevant
information on
results (internal,
service &
program) is
gathered and used
to make
departmental
decisions, and
public reporting
is balanced,
transparent, and
easy to
understand.
Learning, Innovation and Change Management
The department manages through continuous innovation and transformation, promotes organizational learning, values corporate
knowledge, and learns from its performance.
26
MRRS
The Management Accountability Framework is
results based and demands attention
Evaluation
IRPP
MAF
•
All departments are assessed against a set of indicators including
the quality of Management, Resources and Results Structures,
program evaluation and reports to Parliament
•
Discussions with senior officials identify management improvement
priorities
•
Treasury Board Secretariat feeds this input into performance
appraisals for the most senior officials
•
So far, so good but there are some implementation issues:
– Process takes time
– Resource intensive
– Debates on ratings with departments
27
Lessons Learned
28
There are several lessons learned from recent
Canadian experience
•
•
•
•
•
•
•
•
There is no end point on results based management – persistence over
many years is required and you never get it “right”
You need central leadership to build capacity – someone with authority
must set the game plan, make and enforce policies and invest
A detailed understanding of the program base is essential – it is very easy
to lose and not easy to get back
Everybody knows about cultural barriers but don’t underestimate the
challenges with systems
You need a common whole of government planning and reporting
framework if you want to do real strategic planning and reporting
There is no substitute for evaluation but you need to give it some backbone
and support
Parliamentarians and the external auditor demand better public
performance information and this is good
There is no managing for results without sound management practices
period – you need clear expectations and annual assessment
29
Useful resources
•
Government of Canada - http://canada.gc.ca/main_e.html
•
Treasury Board Secretariat - http://www.tbs-sct.gc.ca/index_e.asp
•
Finance (Budget) - http://www.fin.gc.ca/fin-eng.html
•
Reports on Plans and Priorities (2005-06) - http://www.tbs-sct.gc.ca/estpre/20052006/deptlist_e.asp
•
Departmental Performance Reports (2004-05) - http://www.tbssct.gc.ca/rma/dpr1/04-05/0405dpr-rmr_e.asp
•
Canada’s Performance - http://www.tbs-sct.gc.ca/report/govrev/05/cp-rc_e.asp
•
Results-Based Management (MRRS, Evaluation, Improved Reporting to Parliament)
http://www.tbs-sct.gc.ca/rma/rbm-gar_e.asp
•
Management Accountability Framework - http://www.tbs-sct.gc.ca/mafcrg/index_e.asp
•
EMIS - http://www.tbs-sct.gc.ca/emis-sigd/index_e.asp
30
31