Evaluation
of the
International Actions
Theme of the
Clean Air Agenda
November 2010
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Report clearance steps
Planning phase completed
Report sent for management response
Management response received
Report completed
Report approved by Departmental Evaluation Committee
March 2009
May 2010
June 2010
June 2010
November 2010
Acronyms used in the report
ADM
APEC
APP
ARAF
AWG
CAA
CAA–RMS
CARA
CCME
CCS
CED
CESD
COP
CSLF
DFAIT
DG
DGMC
DGTLCC
DM
EC
EGTT
G&C
GHG
HMARF
IEA
IC
IPCC
ITL
LDCs
LEG
LULUCF
M2M
MEF
NGO
NOx
NRCan
OECD
PEMA
PIC
PM
Assistant Deputy Minister
Asia–Pacific Economic Cooperation
Asia–Pacific Partnership on Clean Development and Climate
Accountability, Risk and Audit Framework
Ad Hoc Working Group
Clean Air Agenda
Clean Air Agenda Results Management Secretariat
Clean Air Regulatory Agenda
Canadian Council of Ministers of the Environment
Carbon capture and storage
Clean Energy Dialogue
Commissioner of the Environment and Sustainable Development
Conference of the Parties
Carbon Sequestration Leadership Forum
Foreign Affairs and International Trade Canada
Director General
Director General Management Committee
Director General Theme Leads Coordinating Committee
Deputy Minister
Environment Canada
Expert Group on Technology Transfer
Grants and contributions
Greenhouse gas
Horizontal Management and Accountability Reporting Framework
International Energy Agency
Industry Canada
Intergovernmental Panel on Climate Change
International Transaction Log
Least Developed Countries
Least Developed Countries Expert Group
Land Use, Land Use Change and Forestry
Methane to Markets
Major Economies Forum on Energy and Climate
Non-governmental organization
Nitrogen oxides
Natural Resources Canada
Organisation for Economic Co-operation and Development
Pollution Emission Management Area
Policy and Implementation Committee
Particulate matter
i
Audit and Evaluation Branch
REEEP
SIDS
SO2
UNFCCC
EPA
Evaluation of the CAA International Actions Theme
Renewable Energy and Energy Efficiency Partnership
Small Island Developing States
Sulphur dioxide
United Nations Framework Convention on Climate Change
Environmental Protection Agency
Acknowledgements
The Evaluation Project Team was led by Linda Lee, under the direction of the Environment
Canada Evaluation Director, Shelley Borys, and included Dorota Geissel and Heather Low
from Foreign Affairs and International Trade Canada; Alan Amey and Annette Hollas from
Natural Resources Canada; and Rosemary Clarke from Environment Canada. The Team
would also like to thank all individuals who provided assistance to the project, including:



all interviewees who provided their insights on the International Actions Theme;
the expert focus group members, who provided critical analysis and opinions on the
Clean Air Agenda’s international accomplishments and challenges; and
all departmental personnel who responded to our inquiries and requests for
documentary evidence, and provided detailed comments crucial to the development
of this report and the associated appendices.
This report was prepared for Environment Canada by Stratos, Inc., in collaboration with
Marbek and with Environmental and Management Consultants.
ii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
TABLE OF CONTENTS
EXECUTIVE SUMMARY ..................................................................................................... V
1
INTRODUCTION........................................................................................................... 1
2
BACKGROUND ............................................................................................................ 2
2.1
2.2
2.3
2.4
3
Profile – International Actions Theme Program Components .................................... 2
Governance Structure for the International Actions Theme ....................................... 7
Resource Allocation for Program Components ......................................................... 7
Program Logic Model ................................................................................................ 9
EVALUATION DESIGN ...............................................................................................11
3.1 Purpose, Objectives and Scope of the Evaluation ...................................................11
3.2 Evaluation Approach and Methodology....................................................................13
3.3 Limitations ...............................................................................................................13
4
FINDINGS BY EVALUATION QUESTION...................................................................14
4.1 Rating of Findings ....................................................................................................14
4.2 Evaluation Issue: Relevance – Continued Need ......................................................15
4.3 Evaluation Issue: Relevance – Alignment with Government Priorities ......................19
4.4 Evaluation Issue: Relevance – Alignment with Federal Roles and Responsibilities..26
4.5 Evaluation Issue: Performance – Achievement of Expected Outcomes ...................29
4.6 Evaluation Issue: Performance – Demonstrated Efficiency and Economy ...............47
4.7 Findings for DFAIT’s Class of Contributions in Support of International Environmental
Programs 57
5
CONCLUSIONS...........................................................................................................64
5.1 Relevance ...............................................................................................................64
5.2 Performance ............................................................................................................65
6
RECOMMENDATIONS ................................................................................................69
7
MANAGEMENT RESPONSE ......................................................................................72
ANNEX 1 EVALUATION FRAMEWORK ...........................................................................78
ANNEX 2 SUMMARY OF FINDINGS.................................................................................85
LIST OF TABLES
Table 1: Resource Allocations by Department and Program Component ............................. 8
Table 2: Grants and Contributions Allocations by Departments and Program Component ... 9
Table 3: Ratings Legend .....................................................................................................14
Table 4: Departmental Priorities ..........................................................................................23
Table 5: PM Annex Planned Spending vs. Actual Spending, 2007–08 to 2008–09 .............52
Table 6: DFAIT Grants and Contributions, 2007-–2008 –to 2009–2010 ..............................60
LIST OF FIGURES
Figure 1: International Actions Theme Logic Model .............................................................10
iii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
iv
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Executive Summary
Scope and Objectives of the Evaluation
This evaluation was conducted as part of Treasury Board Secretariat requirements to
evaluate the program components grouped under the Clean Air Agenda (CAA) International
Actions Theme. The objective of this evaluation was to assess the relevance and
performance of the Theme, focusing on the fiscal years 2007–08 to 2009–10. The
evaluation covered all program components within the Theme over that period, including
Foreign Affairs and International Trade Canada’s (DFAIT’s) Class of Contributions in
Support of International Environmental Programs. Given the timing of the evaluation, which
occurred before the end of the implementation phase of this theme (ending 2010–11), the
focus was on program outputs and movement toward the achievement of early outcomes.
Theme Components
The International Actions Theme contributes to achieving the ultimate objectives of the CAA
by addressing air quality–related emissions whose sources originate from outside Canada,
and by ensuring that international actions on climate change and air pollutants are in line
with Canada’s interests. The Theme contributes to these objectives under four program
components: (1) the Particulate Matter (PM) Annex; (2) International Partnerships and
Negotiations; (3) International Financial and Other Obligations; and (4) the Asia–Pacific
Partnership on Clean Development and Climate (APP). These components are
implemented, with varying levels of responsibility, by four federal departments: Environment
Canada (EC), DFAIT, Natural Resources Canada (NRCan) and Industry Canada (IC). The
components involve activities intended to advance Canada’s international actions to improve
Canadian air quality and reduce greenhouse gas (GHG) emissions, and that are currently
funded through a $50-million funding envelope over the four-year period from 2007–08
through 2010–11.
Evaluation Methodology and Findings
The evaluation addressed five key issues: (1) the continued need for the program
components within the International Actions Theme; (2) alignment with Government
priorities; (3) alignment with federal roles and responsibilities; (4) achievement of expected
outcomes; and (5) demonstrated efficiency and economy. To examine these issues and the
specific evaluation questions related to each, the evaluation employed three main lines of
inquiry: (1) document review; (2) key-informant interviews; and (3) a focus group of experts.
The evaluation scope solely addressed Canada’s performance at the international level and
did not include its ability to meet Kyoto Protocol targets. In addition, the evaluation was
limited in its ability to consult with other countries’ negotiators and obtain their unique
perspective on Canada’s influence internationally.
Findings and Conclusions
Listed below are the five key evaluation issues, and, under each of those, the evaluation’s
findings and conclusions (numbered) are summarized.
v
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Issue 1: Continued Need for the International Actions Theme Programs
1. The program components and activities under the CAA International Actions Theme
are relevant and continue to be needed to achieve expected outcomes.
Issue 2: Alignment with Government Priorities
2. Activities are in direct alignment with federal government priorities.
3. Activities are aligned with the participating departmental priorities.
Issue 3: Alignment with Federal Roles and Responsibilities
4. Activities are aligned with federal roles and responsibilities.
Issue 4: Achievement of Expected Outcomes
5. With some delays, progress has been made in the achievement of outputs in
component areas. The following expected outputs have been produced:
• Assessed contributions have been delivered to international bodies.
• Canada’s Kyoto Protocol National Registry for carbon dioxide (CO2) emission units is
in place.
• Negotiations for a new post 2012 climate regime have been carried out under the
United Nations Framework Convention on Climate Change (UNFCCC).
• Policies, negotiating positions and advice have been produced on climate change,
although only initial research and early negotiating positions have been completed
for air quality components.
• There has been participation in a variety of international bodies and initiatives
outside of the UNFCCC, and support has been provided to foster partnerships. For
example, a range of initiatives such as the Expert Group on Technology Transfer
(EGTT) and Least Developed Countries Expert Group (LEG), as well as contribution
programs, have built capacity, increased understanding in areas such as adaptation,
advanced discussions on key issues in negotiations, and advanced clean
technologies.
• Formal and informal information sessions have taken place with other levels of
government, Aboriginal groups and domestic stakeholders to support climate change
activities.
However, there were significant delays in achieving certain activities and outputs:
• The National Registry for CO2 emission units was approximately six months late.
• Many activities under the APP were not initiated in the first two years.
• Work on a PM Annex to the Canada–United States Air Quality Agreement has not
proceeded as planned.
6. Some progress has been made in the achievement of immediate outcomes, with
achievement of the following expected immediate outcomes:
• Canada has complied with its financial and other obligations under international
climate change treaties and agreements.
• Canada’s positions with respect to the post-2012 international climate change
framework have been clearly and consistently communicated (within the context of
the UNFCCC) in a range of other multilateral and bilateral fora, and with strategic
allies such as members of the Umbrella Group (a loose coalition of non–European
Union developed countries). In addition, a number of these positions are reflected in
vi
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
the Copenhagen Accord, although key gaps remain. However, it is difficult to
attribute the advancement of these interests directly to Canadian international
activities under this theme, particularly given the multiple factors involved in
international negotiations.
• There has been improved information sharing and more formal engagement
mechanisms with provinces and territories, but enhanced consultation activities have
not necessarily resulted in increased consideration of stakeholder interests in
Canada’s positions. For the PM Annex, this evaluation found no evidence that
stakeholders were consulted directly and it was not clear how Canada’s position was
developed or how the views of domestic partners and stakeholders have had an
influence or been meaningfully reflected.
• There has been a small increase in the awareness of trade-related opportunities
pertaining to climate-friendly technologies.
• While bilateral and multilateral relationships have been advanced through voluntary
(i.e., non-assessed) contributions, and contributions have contributed to a shared
understanding of issues and advancing Canada’s position abroad, there is no
evidence yet of a return on investment in terms of GHG reductions.
7. No significant unintended outcomes were identified, but opportunities are emerging
from the UNFCCC process to enhance Canada’s export opportunities and
partnerships for clean technology.
8. External factors have influenced success and delayed the achievement of outcomes.
Key factors include: changes in Canadian policy direction and positions; higher than
anticipated demands regarding international climate change; challenges within the
UNFCCC decision-making process; and staff turnover and Ministerial changes.
Issue 5: Demonstrated Efficiency and Economy
9. There are a limited number of more economic and efficient means to achieve
objectives, and such means have not been the focus of program implementation.
However, there are options in terms of how Canada uses its voluntary contributions
and how to best manage the APP Secretariat.
10. Some options to improve efficiency have been implemented. Particularly beneficial
has been the establishment of a Chief Negotiator and Ambassador for Climate
Change in the federal government.
11. There has been progress in program implementation as planned, but there have
been significant delays in some areas. Initial lack of progress for the APP and
negotiating the PM Annex led to under-spending during the first two years. However,
there have been higher than expected demands in the International Partnerships and
Negotiations component.
12. The Horizontal Management and Accountability Reporting Framework (HMARF) is in
place and is facilitating government-wide reporting of financial performance, but is
not yet supporting decision making at the program level.
13. Performance data are collected and reported, but not fully across all program
components; and such data are not used by managers.
vii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Recommendations
The following recommendations were developed based on evaluation findings and
conclusions. These recommendations are directed to the responsible Assistant Deputy
Ministers of the International Affairs Branch at EC, the Global Issues Branch at DFAIT, the
Energy Sector at NRCan, and the Industry Sector at IC, and the Associate ADM of the
Environmental Stewardship Branch at EC.
The following recommendations were developed to continue improving and advancing the
CAA International Actions Theme.
Recommendation 1: EC, in consultation with other participating departments, should
review and implement options for enhancing coordination and management of
program components in order to improve the strategic direction and effectiveness
for the Theme overall.
Although there is some overall coordination across departments, each program component
within the International Actions Theme is essentially implemented and managed separately
by the department(s) responsible. This theme is not managed or implemented as a “theme”
as originally articulated. Therefore, areas for improving possible synergies and connections
between the goals of program components and departmental mandates should be explored,
and formal mechanisms should be put in place (e.g., regular senior-level meetings across
participating departments) to ensure that shared objectives for this theme and the CAA
overall are met.
Recommendation 2: EC should consider the costs and benefits of integrating PM
Annex program activities with Clean Air Regulatory Agenda (CARA) activities.
Streamlining all air quality activities together within the Department can lead to a more
coordinated and coherent approach to planning, completing and reporting on the results for
initiatives related to PM. Under the existing CAA structure, the majority of air quality
activities are funded by sources within CARA. Since PM Annex activities are more closely
aligned with the domestic objectives for CARA than to international climate change
activities, EC should explore the appropriateness of integrating PM Annex resources and
activities with those of CARA. This integration should facilitate the coordination of all work
related to PM at EC.
Recommendation 3: EC should improve the effectiveness of its engagement with
provinces, territories and other domestic stakeholders for the PM Annex, by
considering the most appropriate formal mechanisms for consultation that will
provide strategic direction for the future.
There was little evidence that the provinces, territories and other domestic stakeholders
were engaged as part of the PM Annex’s negotiations process. Given the role that these
groups play in implementing clean air and clean energy programs and technologies,
regional partners and other levels of governments should be engaged to enhance
opportunities and articulate region-specific issues and context to inform priorities and
negotiating positions for the Annex. Because the Annex experienced some delays in
implementation, it is imperative to develop effective processes that will enable the Annex to
move forward efficiently toward achieving its expected outcomes in the future. EC should
therefore review existing processes for consultations and consider formal mechanisms for
viii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
implementation of consultations that will provide strategic direction for the Annex’s
negotiating positions.
Recommendation 4: EC, DFAIT and NRCan should develop an overarching strategic
framework to guide the selection of international climate change voluntary
contributions to international bodies and programs.
Within the International Actions Theme, there are multiple and sometimes conflicting goals
assigned to Canada’s voluntary (non-assessed) contributions to international bodies and
programs, which makes it difficult to determine the cohesiveness of these contributions in
meeting broader thematic objectives. While some program components allocate funds using
specific selection criteria based on their specific objectives, some components allocate
funding based on opportunity rather than proactively. Therefore, an overarching framework
should be developed to guide the selection of the International Actions Theme’s voluntary
contribution projects. This framework should articulate the overarching strategic priorities,
criteria for selection, appropriate terms and conditions, and broad processes to review,
select, approve and monitor/oversee contribution projects.
This recommendation’s aim is to ensure that all funds are allocated to those initiatives that
can best achieve the Theme’s broad expected outcomes. This overarching strategic
framework should apply to all voluntary contributions in the Theme, to enable a more
integrated process for soliciting, reviewing and approving proposals based on common
expected outcomes and priorities (e.g., GHG reduction potential, trade-development
potential, or value in developing partnerships, or some combination of these or other
objectives). The intention is to have a Theme-level framework and a set of priorities that
govern all voluntary grants and contributions programs under the Theme. However, this
framework is intended to guide, not replace, the component-specific processes that are
already in place for program components.
To support international partnerships and negotiations, this contributions strategy should
also consider the goal of developing strategic alliances with developing countries,
particularly with Least Developed Countries (LDCs) and Small Island Developing States
(SIDS), to better understand the potential adverse effects associated with climate change
(e.g., adaptation in SIDS and LDCs), thereby building on commitments to support the
poorest and those most vulnerable to climate change. Engagement with other federal
departments not included in the International Actions Theme (e.g., the Canadian
International Development Agency) may be beneficial.
Recommendation 5: EC, NRCan, DFAIT and IC should consider how APP program
activities and outputs can be used to provide trade opportunities for the
development, diffusion and deployment of clean technologies.
The initial portfolio of APP projects focused less on technology deployment and more on
capacity building and technology research. However, opportunities for Canadian businesses
to develop and promote clean technologies internationally are emerging as large developing
countries articulate their climate change actions and technology requirements to reduce
GHG emissions. In order to maximize these opportunities (e.g., enabling technologies for
smart-grid development in countries looking to improve energy efficiency), federal
departments should explore how the International Actions Theme’s voluntary contributions
can be used to increase support for these emerging trade opportunities for Canadian
businesses.
ix
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Recommendation 6: EC, DFAIT, NRCan and IC should develop and implement
mechanisms that will ensure the ongoing collection of meaningful and accurate
performance measurement data.
Given that this evaluation occurred early in program implementation, the achievement of
immediate outcomes was assessed at this time. Ongoing performance measurement data
are required to assess progress toward achievement of intermediate and final outcomes in
subsequent evaluations. However, some program components under this theme do not
have a clear strategy for collecting ongoing performance information. As well, the
performance measurement data collected as part of the HMARF are not used by, or seen as
useful for, program managers. Therefore, EC, in consultation with participating departments
and working with the Clean Air Agenda Results Management Secretariat (CAA–RMS),
should enhance the performance measurement strategy for this theme, including the
development of specific, measureable performance indicators. A clear plan should then be
developed for the ongoing collection of performance measurement data, and mechanisms
should be established in all participating departments to maintain the accuracy and
usefulness of these data for each of the program components under this theme.
For example, performance indicators should be developed and a mechanism should be
implemented to collect and maintain these performance measurement data for APP-funded
projects. While it is too early to assess project success, there should be a system in place
that will capture outcomes for funded projects so that these outcomes can be assessed
against established APP objectives in the future. Because of the dynamic nature of the
policy directions and context of some activities under the International Actions Theme, it is
challenging to develop accurate and useful ongoing performance indicators. Flexible
strategies for program-component-level performance measurement and reporting may be
considered to account for changing policy directions, but a clear plan should be developed
that describes how these data will be recorded and managed.
Management Response
The Assistant Deputy Ministers (ADMs) of the International Affairs Branch at Environment
Canada, the Global Issues Branch at the Department of Foreign Affairs and International
Trade (DFAIT), the Energy Sector at Natural Resources Canada (NRCan), the Industry
Sector at Industry Canada and the Associate Assistant Deputy Minister (AADM) of the
Environmental Stewardship Branch at Environment Canada, accept the evaluation and all of
its recommendations, and have provided a plan to implement the following management
actions in response to the evaluation recommendations.
Recommendation 1: Environment Canada, in consultation with other participating
departments, should review and implement options for enhancing coordination and
management of program components to improve the strategic direction and
effectiveness for the theme overall.
The ADM of International Affairs Branch at Environment Canada (EC) agrees with this
recommendation.
Environment Canada will work with the Department of Foreign Affairs and International
Trade (DFAIT), Natural Resources Canada (NRCan) and Industry Canada to review and
implement options for enhancing coordination and management of program components to
x
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
improve the strategic direction and effectiveness for the theme overall. Meetings of all
International Actions Theme Program Director Generals will be held every four months to
facilitate communication across departments at the management level, provide strategic
direction, discuss interconnections between programs, provide a forum for thematic issues
and to review level of achievement towards the International Actions Theme goals
Timeline
Deliverable(s)
Responsible Party
March 31, 2011
 Quarterly meetings of all
Director General, Climate
International Actions Theme
Change International
Program Directors General (first
Directorate, EC
meeting to be held in summer of
2010).
Recommendation 2: Environment Canada should consider the costs and benefits of
integrating PM Annex program activities with Clean Air Regulatory Agenda (CARA)
activities.
The ADM of the International Affairs Branch and the AADM of the Environmental
Stewardship Branch agree with this recommendation
Particulate Matter (PM) Annex activities are more closely aligned to the domestic objectives
of the Clean Air Regulations Agenda (CARA) than to international climate change activities.
Both the CARA and PM Annex programs’ existing funding will expire March 31, 2011.
Senior management has considered the efficiencies gained from single-point reporting
instead of reporting to two funding programs as well as the strong alignment with CARArelated domestic goals and concluded that work related to the PM Annex would be more
effective and efficient if included in the CARA program. Initial consideration for future
funding of CARA is underway and includes the PM Annex. The PM Annex will begin to
officially integrate into the CARA program this fiscal year, while continuing to report through
the International Actions Program.
Timeline
Deliverable(s)
Responsible Party
March 31, 2011
 Integrate PM Annex
Director General, Strategic
activities into CARA
Priorities Directorate,
activities this year.
Environmental Stewardship
Branch, EC
Recommendation 3: Environment Canada should improve the effectiveness of their
engagement with provinces, territories and other domestic stakeholders for the PM
Annex by considering the most appropriate formal mechanisms for consultation that
will provide strategic direction for the future.
The AADM of Environmental Stewardship Branch agrees with this recommendation
The content of the PM Annex will be based on the respective domestic policies to address
air pollution in Canada and the U.S.
Provinces and territories were consulted formally on Canada’s proposed domestic approach
in 2007 through the Environmental Planning and Protection Committee (EPPC) of the
Canadian Council of Ministers of the Environment.
These consultations led to the formation of a multi-stakeholder process that included
members of provincial and territorial governments, industry and environmental groups. A
xi
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Canadian domestic approach to address air pollutants has been the subject of the work of
this group over the last two years.
We will be working to further elaborate our approach (as will the U.S) and when both
countries are in a position to engage in negotiations, an appropriate process for
engagement, using existing modalities where possible, will be set up.
There is also provincial representation on the subcommittees set up under the Canada-US
Air Quality Agreement that allows for direct participation in the process.
Timeline
Deliverable(s)
Responsible Party
TBD
 Domestic consultations
Director General, Strategic
take place prior to further
Priorities Directorate,
negotiations on the PM
Environmental Stewardship
Annex under an
Branch, EC
appropriate process.
Recommendation 4: Environment Canada, Foreign Affairs and International Trade
Canada, and Natural Resources Canada should develop an overarching strategic
framework to guide the selection of voluntary contributions to international bodies
and programs.
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT) and
Energy Sector of Natural Resources Canada (NRCan) agree with this recommendation
EC will, in close consultation with DFAIT and NRCan, work to create an overarching
strategic framework to guide the selection of voluntary (non-assessed) contributions to
international bodies and programs.
A strategic framework was developed to ensure that projects funded under the Asia Pacific
Partnership program met defined objectives. The experience of developing an APP strategic
framework will serve as a useful guide to development of the overarching framework.
The implementation of the framework will guide the allocation of contribution funds to
recipients that will best contribute to achieving the program objectives.
The framework will be developed in this final funded year to ensure the balance of funds is
distributed in the most effective manner.
Should a new program of international actions be funded beyond the current financial year
that includes similar voluntary contributions, this framework could be used as the basis for
managing the new funds.
Timeline
Deliverable(s)
Responsible Party
September 30, 2010.
 Develop a strategic framework Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director General, Energy
xii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
and Sustainable
Development Directorate,
DFAIT
Director General, Energy
Policy Branch, NRCan
March 31, 2011
 Test the framework using
contribution funds for fiscal
year 2010-11.
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director General, Energy
and Sustainable
Development Directorate,
DFAIT
Director General, Energy
Policy Branch, NRCan
Recommendation 5: Environment Canada, NRCan, DFAIT and Industry Canada
should consider how program activities and outputs can be used to provide trade
opportunities for the development, diffusion and deployment of clean technologies
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy
Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry
Canada (IC) agree with this recommendation
The clean energy technology program has worked with Canadian private industry, many of
which are Small and Medium Sized Enterprises (SMEs), on baseline-setting and research
projects with some deployment of clean technology projects. SMEs account for 45% of
gross domestic product (GDP), much of the economy’s growth, 60% of all jobs in the
economy, and 75% of net employment growth1 As we become aware, through the
climate change negotiation sessions and other relevant international processes covered by
the program, of emerging trade opportunities related to climate change, we agree that we
need to increase SMEs awareness of these opportunities in order to increase deployment of
clean technology.
The Departments will create a plan for better communication of clean technology trade
opportunities to SMEs and will implement some activities prior to funding termination on
March 31, 2011. The plan can be used as designed if the program is continued or as a
foundation if the program is continued with changes.
In addition to ensuring information sharing with the Canadian SMEs, we can better inform
the international community of Canada’s developing clean technology products to increase
trade opportunities. For example, participation in the APP has resulted in sharing of
Public Works and Government Services Canada, “Importance of SMEs.” Accessed online on 21 May 2009 at
http://www.tpsgc-pwgsc.gc.ca/app-acq/pme-sme/importance-eng.html.
1
xiii
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
technology information and products among the APP countries thereby increasing world
awareness of Canada’s clean technology.
Established in 2009, and not included in the scope of this Evaluation, EC leads the CanadaU.S. Clean Energy Dialogue (CED), with support from NRCan and DFAIT. The CED aims to
enhance bilateral collaboration in advancing clean energy technology solutions to reduce
greenhouse gas emissions and address climate change. The focus areas of the CED
include: expanding clean energy research and development; advancing the development
and deployment of clean energy technologies, with a focus on carbon capture and storage;
and building a more efficient electricity grid based on clean and renewable generation. EC,
NRCan and DFAIT will continue to work together under the CED, and where appropriate,
engage with other relevant departments, such as IC, to advance the development and
deployment of clean energy technologies. A result of this program could be an increase in
Canadian business trade opportunities in clean technologies, although this is not a specific
goal of the program.
Beyond the International Actions Theme, NRCan leads a number of key partnerships with
organizations such as the International Energy Agency, the Climate Technology Initiative
and the Global Carbon Capture and Storage Institute (GCCSI), which contribute significantly
to advancing private sector engagement and should be further leveraged in order to
maximize support for Canadian business opportunities on clean technologies.
Also beyond the Theme, Industry Canada has been participating actively in the International
Partnership for Hydrogen & Fuel Cell in the Economy (IPHE) and working with Canadian
companies to build hydrogen and fuel cell industry worldwide. IC is also engaged in the
Steel Committee of the Organization for Economic Cooperation and Development (OECD)
and the North American Steel Trade Committee (NASTC). While neither the Steel
Committee of the OECD nor the NASTC is a forum that is meant to deal with climate change
issues, they are often raised by industry representatives and IC officers as being important.
These issues will no doubt affect the sector's competitiveness going forward because of
different environmental obligations for developed and developing countries
Timeline
October 31, 2010
Deliverable(s)
 NRCan, IC, DFAIT and
EC representatives will
develop an action plan to
increase the Canadian
SMEs’ awareness of
increased opportunities in
clean technology
Responsible Party
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director, Resource
Manufacturing and ValueAdded, Industry Sector, IC
Director, Partnerships,
Climate Change International
Directorate, EC
Director, International Affairs,
NRCan
Director, Climate Change
xiv
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
and Energy Division, DFAIT
March 31, 2011
 Implement activities
identified in the action
plan.
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director, Resource
Manufacturing and ValueAdded, Industry Sector, IC
Director, Partnerships,
Climate Change International
Directorate, EC
Director, International Affairs,
NRCan
Director, Climate Change
and Energy Division, DFAIT
Recommendation 6: Environment Canada, DFAIT, NRCan and Industry Canada
should develop and implement mechanisms that will ensure the ongoing collection of
meaningful and accurate performance measurement data.
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy
Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry
Canada (IC) agree with this recommendation
As mentioned in the Evaluation Report, gathering meaningful performance measurement
data for policy-related outcomes is extremely difficult and must be qualitative in nature rather
then quantitative. We recognize that we need better measurement of outcomes for the
policy-related components of the program and EC will, in close cooperation with DFAIT and
NRCan, enhance the performance measurement framework to be more flexible regarding
the policy-related components of the International Actions Theme and establish a method of
tracking performance for use in decision making.
The APP and Methane to Markets components of the program provide an opportunity to
track performance data against program goals (reductions in greenhouse gases, economic
benefits for Canadians and accelerating development of clean technology). On project
completion, the companies will be asked to report on the achievements of their projects
against the three priorities.
Timeline
September, 2010
Deliverable(s)
 Establish a system to
track performance of
policy-related activities
Responsible Party
Lead:
Director General, Climate
Change International
xv
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
under the program
against qualitative
performance
measurement indicators.
Directorate, EC
Support:
Director General,
Environment, Energy and
Sustainable Development
Bureau, DFAIT;
Director General, Energy
Policy Branch, NRCan
February, 2011
 Establish a system to
report level of
achievement of meeting
the three priorities of the
APP program.
Director General, Climate
Change International
Directorate, EC
xvi
Audit and Evaluation Branch
1
Evaluation of the CAA International Actions Theme
Introduction
This evaluation was undertaken from December 2009 to April 2010 in order to evaluate the
relevance and performance of the Clean Air Agenda’s (CAA’s) International Actions Theme.
The intention of this evaluation is to meet the information requirements, related to
international initiatives under the CAA, of senior management within Environment Canada
(EC), Foreign Affairs and International Trade (DFAIT), Natural Resources Canada (NRCan),
and Industry Canada (IC)—and to meet the information needs of the Treasury Board
Secretariat. As well, international commitments were identified as a risk area in the 2008–09
Audit and Evaluation Plan for EC, which was approved by the Departmental Evaluation
Committee on April 4, 2008.
For the International Actions Theme, EC’s Evaluation Division led the evaluation and
represented IC’s interests. The CAA International Actions Theme Evaluation Committee
performed an oversight role throughout the evaluation process, providing input and
feedback to the consultant hired to conduct the evaluation, approving all deliverables
outlined in the contract for evaluation, and coordinating the management response to
evaluation recommendations in the evaluation report.
This report summarizes the evaluation process, findings, conclusions and
recommendations, and comprises the following sections:






Background describes the CAA’s International Actions Theme and related program
components, including the governance structure, allocated resources and the
expected outcomes;
Evaluation Design describes the evaluation objectives, scope, issues, and
approach taken;
Findings by Evaluation Question documents the evaluation findings across the
four program components;
Conclusions;
Recommendations;
Management Response.
1
Audit and Evaluation Branch
2
Evaluation of the CAA International Actions Theme
Background
The International Actions Theme is one of eight themes under the CAA architecture, which
includes 43 programs and eight federal departments and agencies. The other seven themes
within the CAA are: Clean Air Regulations, Clean Energy, Clean Transportation, Indoor Air
Quality, Adaptation, Partnerships, and Management and Accountability. Each lead
department is responsible for conducting evaluations at the thematic level under the CAA.
The evaluation results for all themes will be aggregated into a horizontal evaluation of the
entire CAA. EC is the lead department on the overall CAA and is responsible for
implementation of the horizontal evaluation.
This evaluation focuses only on the International Actions Theme. EC has the lead in
coordinating program activities and the evaluation under this theme with the other involved
departments: DFAIT, NRCan and IC. Program components under this theme involve
activities intended to advance Canada’s international action, improve Canadian air quality
and reduce greenhouse gas (GHG) emissions. Specific program outcomes of each of the
four components captured under this theme collectively contribute to achieving the Theme’s
broad results. In turn, the international theme contributes to achieving the ultimate objectives
of the CAA by addressing air quality–related emissions whose sources originate from
outside Canada and by ensuring that international actions on climate change and air
pollutants are in line with Canada’s interests. The activities within the International Actions
Theme have received funding for the period 2007–08 through 2010–11.
The purposes of the activities are to address the impacts of poor air quality and climate
change, including respiratory-related hospital admissions and premature mortality due to
particulate matter (PM); and climate change impacts on energy security, sustainable
development, health, trade and industrial growth. These activities respond to the Speech
from the Throne commitments in April 2006 related to “measures to achieve tangible
improvements in our environment, including reductions in pollution and greenhouse gas
emissions.” These international activities reflect the fact that Canada’s domestic air quality
goals are influenced by international activities and that, to achieve progress, Canada needs
to be aligned with international actions.
Profile – International Actions Theme Program Components
2.1
The CAA International Actions Theme comprises four program components:
1.
2.
3.
4.
Particulate Matter Annex
International Partnerships and Negotiations
International Financial and Other Obligations
Asia–Pacific Partnership on Clean Development and Climate
Activities within International Partnerships and Negotiations2 and International Financial and
Other Obligations are related to the United Nations Framework Convention on Climate
Change (UNFCCC) and the Kyoto Protocol,3 while the PM Annex and Asia–Pacific
2
Some activities under International Partnerships and Negotiations are not United Nations (UN)–related (e.g., participation at
international fora such as the G8 and G12).
3
Activities within the International Actions Theme that are related to Canada’s obligations under the Kyoto Protocol involve the
development of climate change policy, participation in negotiations at international fora, and meeting Canada’s financial
2
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Partnership on Clean Development and Climate (APP) are not. Specific program outcomes
for each of these components collectively contribute to achieving the broad results of the
International Actions Theme. The participating departments within this theme have varying
levels of responsibility across these four components, as described below.
2.1.1 Particulate Matter Annex
PM comprises air-bound, microscopic pollutants that have negative impacts on air quality,
the environment and human health. Led by EC, this component of the CAA International
Actions Theme strives to establish and maintain negotiated commitments between Canada
and the United States to reduce PM emissions and their flow between the two countries.
Historically, Canada and the United States have had a solid history of cooperation on air
quality, dating from the signature of the bilateral Air Quality Agreement in 1991. An Ozone
Annex to address transboundary ozone formation was negotiated and signed in 2000. Since
then, further technical and scientific analysis culminated in a joint Canada-U.S. science
assessment of transboundary PM in 2004. This assessment demonstrated that further
reductions in transboundary pollution, including PM, would help both countries make
continued progress in protecting human health and the environment. Canada’s Clean Air
Regulatory Agenda (CARA) and, more particularly, the proposal to have a national
regulatory framework in Canada to address various pollutants, including PM and its
precursors, was welcomed by the United States.
Canada and the United States are interested in pursuing negotiations for a PM Annex.
Stemming from the 1991 Air Quality Agreement, such an annex would establish
commitments by the United States and Canada to reduce emissions of PM and precursors,
thereby helping Canada to secure substantial improvements in air quality and associated
health and environmental benefits. From a Canadian perspective, a PM Annex should
provide for:





commitments by the United States to reduce sulphur dioxide (SO2) and nitrogen
oxide (NOX) emissions, and to reduce emissions of primary PM, which will decrease
transboundary flows of SO2 and NOx;
U.S. recognition, within the terms of the Agreement, that it is a source of
transboundary PM pollution;
joint monitoring of PM levels in air and a review procedure to assess progress on
these commitments against achieving the goals for air quality and acidification;
definition of a transboundary Pollution Emission Management Area (PEMA); and
cooperative scientific and technical activities to further the understanding of PM, its
formation and transport, and its impacts on human health.
In addition, it is anticipated that the PM Annex could provide an opportunity to obtain a U.S.
commitment to pursue the development of a cross-border emissions trading regime for SO2
and NOX.
EC’s Air Emissions Priorities Division4 (under whose auspices the Air Quality Agreement is
managed) provides day-to-day administrative and coordination support related to the PM
Annex.
obligations as a member of the Conference of the Parties of the United Nations Framework Convention on Climate Change.
Other activities related to the Kyoto Protocol are funded under other sources and are not part of this evaluation.
4
Formerly the North American Smog Program.
3
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
2.1.2 International Partnerships and Negotiations
This component focuses on Canada’s international climate change activities completed in
international fora. Given the growing awareness in Canada of the increasing impacts of
climate change and the need for a strategic, global approach to effectively address this
issue, Canadians are looking to the Government of Canada to advance international action
on climate change. Canada’s engagement in international climate change discussions and
negotiations is supported by substantive policy development, research and analysis (using
in-house resources), and bilateral outreach and advocacy. These activities are imperative
for developing and preparing Canada’s negotiating positions, and for consultations with
relevant stakeholders, in advance of Canada’s participation in multiple fora where climate
change issues are addressed both within and outside the UN.
In order to deliver on expected results, EC, DFAIT and NRCan work together to provide
analysis and policy input for the development and articulation of Canada’s positions and
policies on international climate change. EC coordinates the Government of Canada’s
participation within and outside the UN negotiation process, in close collaboration with
DFAIT, NRCan and other interested departments, and leads and coordinates Canada’s
efforts to shape and develop a future global approach to international climate change (post2012). The Minister of the Environment is the Cabinet Minister responsible for the
international climate change file, a role carried out in close collaboration with the Minister of
Foreign Affairs.
A number of activities aim to ensure that Canadian environmental and economic interests
are advanced through international negotiations and discussions. These include:
a) negotiation and development of policies, and participation in fora, relating to a wide
range of climate change issues, including global mitigation actions,
developing-country issues, impacts and adaptation aspects of climate change,
economic diversification, market-based mechanisms, financing and investment flows,
technology development, deployment, diffusion and transfer, forestry,
capacity-building, research and systematic observation, and legal and administrative
matters;
b) acting as Canada’s Designated National Authority for the Clean Development
Mechanism;
c) development of a strategic approach for Canada’s role in a more effective and
inclusive post-2012 agreement on climate change under the UNFCCC;
d) strategic bilateral engagement with other countries on international climate change in
order to inform Canada’s position, identify areas of mutual cooperation, and influence
the negotiating positions of key countries;
e) gathering input from and engaging domestic stakeholders—such as industry,
environmental non-governmental organizations (NGOs), Aboriginal groups, and
provinces/territories—on international climate change issues, to inform them and to
garner support for Canada’s international negotiating positions; and
f) participation in non-UN processes (e.g., Major Economies Forum on Energy and
Climate [MEF]) that support the negotiations of a future agreement on international
climate change.
Additionally, DFAIT is responsible for the disbursement of additional grants and
contributions (G&C) funds under the International Actions Theme. Projects funded under
4
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
this program help Canada fulfill its international climate change commitments and further its
international negotiating position. The contributions made under this program consist of
those that are supplementary both to assessed contributions5 and non-assessed
contributions.6
2.1.3 International Financial and Other Obligations
This component of the CAA International Actions Theme ensures that Canada meets its
financial obligations under the UNFCCC7 and its Kyoto Protocol,8 and funds its share of the
UN Climate Change Secretariat’s Program Budget. The UNFCCC and Kyoto Protocol are
legally binding international treaties that serve as an overall framework for
intergovernmental efforts to tackle the challenge posed by international climate change.
Under these agreements, Canada is required to contribute its share of an assessed
contribution to the UN Climate Change Secretariat’s Program Budget, which consists of a
core operating budget (assessed) and supplementary budget for both the UNFCCC and
Kyoto Protocol. Most recently, the assessed funding for the UNFCCC and Kyoto Protocol for
2008–09 was approximately $954,000, of which CAA funding covers half.9
This component also includes Canada’s commitments to establish a National Registry for
Kyoto-recognized emission credits, a condition for Canada’s participation in the flexibility
mechanisms of the Protocol (i.e., the Clean Development Mechanism, Joint Implementation,
and the ability to trade national emissions allowances). National registries are required of all
Annex 1–country signatories to the Kyoto Protocol, and are linked to an international,
United Nations–based interactive database (the International Transaction Log [ITL]) that is
used to monitor each country’s holdings and transfers of Kyoto-recognized emissions
credits.
Within this component, Canada makes voluntary contributions to support initiatives in which
Canada has a specific interest. These include the ITL, and funding to enable Canada to
participate in and contribute to international technology initiatives outside the UNFCCC,
such as Methane to Markets (M2M) and the Renewable Energy and Energy Efficiency
Partnership (REEEP). The M2M partnership is an international initiative that advances
cost-effective, near-term methane recovery and use as a clean energy source. REEEP is a
capacity-building initiative that has completed over 130 renewable energy projects since
2004.
DFAIT acts as the focal point for overseeing financial contributions to the UNFCCC in order
to ensure that assessed funding requirements are met, through providing administrative
support and management coordination for the provision of assessed funding on behalf of
5
Assessed contributions refer to Canada’s assessed share of the costs of operating an international organization of which
Canada is a member as a result of an act of Parliament, a Cabinet decision, an Order in Council or an international treaty.
6
Non-assessed contributions refer to Canada’s voluntary contributions.
7
The UNFCCC is an international treaty that sets an overall framework for intergovernmental efforts to address the challenges
posed by climate change. The Convention, which entered into force on March 21, 1994, has near-universal membership with
191 countries and one regional organization (the European Economic Community) having ratified. Under the Convention,
governments gather and share information on GHG emissions, national policies and best practices; launch national strategies
for addressing GHG emissions and adapting to expected impacts, including the provision of financial and technological support
to developing countries; and cooperate in preparing for adaptation to the impacts of climate change.
8
The Kyoto Protocol is an international agreement linked to the UNFCCC. The defining feature of the Protocol is that it sets
binding targets for 37 industrialized countries and the European community for reducing GHG emissions. The Protocol was
adopted on December 11, 1997, and entered into force on February 16, 2005. A total of 184 parties to the Convention have
ratified the Protocol.
9
The other half of assessed funding for the UNFCCC and the Kyoto Protocol is covered by DFAIT through other funding
sources, and is not part of this evaluation.
5
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Canada. EC serves as technical lead on developing the National Registry for emission
credits, managing the connection to the interactive ITL, participating on behalf of Canada in
technology partnerships (e.g., REEEP), and leading Canada’s technology trade initiatives
through M2M.10
2.1.4 Asia–Pacific Partnership on Clean Development and Climate
Led by EC, with input from NRCan and IC,11 the APP is a public-private partnership
designed to accelerate the development, deployment and diffusion of clean energy
technologies to address climate change. This partnership brings together seven countries
(Australia, Canada, China, India, Japan, the Republic of Korea and the United States) in a
voluntary initiative to address the issues of sustainable development, clean energy and
climate change. These countries collectively represent approximately 50% of the world’s
economy, population and energy use, and over 50% of the world’s GHG emissions. From a
strategic perspective, the APP provides a collaborative forum for the engagement of major
emitting sectors and countries that will need to realize significant reductions in business-asusual emissions in order to stabilize global GHG concentrations at a level that will not
dangerously interfere with the global climate system.
The APP is organized around eight task forces that address five energy-intensive sectors
(aluminum, buildings and appliances, cement, coal mining, and steel) as well as three
energy supply sectors (renewable energy and distributed generation, power generation and
transmission, and cleaner fossil energy). Each task force is chaired and co-chaired by
participating countries12 and oversees public sector–private sector collaborations for
developing and implementing projects on climate-friendly technologies related to its
particular sector. Project proposals are intended to be selected through a transparent,
merit-based process.
The Policy and Implementation Committee (PIC) is an Assistant Deputy Minister
(ADM)–level decision-making body within the APP that, in overseeing task force activities,
has mandated responsibilities for policies, frameworks, technical advice, financing,
public-private partnerships and procedures. PIC meetings are hosted annually by one of the
participating countries. Canada hosted the October 2008 meeting in Vancouver,
British Columbia. An Administrative Support Group currently hosted by the United States
supports the PIC and APP more broadly.
Membership in the APP allows Canada to engage in joint public sector–private sector and
bilateral or multilateral partnerships, and in projects related to climate-friendly technologies.
Canadian delegations consisting of public and private sector representatives have been
participants in the APP’s international activities through their involvement on various task
forces and leadership and subject-matter expertise in the development of various clean
technology projects.
Canada became the seventh member of the APP on October 15, 2007 at the 2nd Ministerial
meeting in New Delhi. In June 2008, Canada made an initial commitment of C$20 million to
fund Partnership activities and initiatives. $15.48 million of this contribution, managed by
10
Theme goals to advance the uptake of technologies and reduce GHG emissions are also pursued outside of formal UN
negotiations through other fora such as the Expert Group on Technology Transfer, Carbon Sequestration Leadership Forum,
and International Energy Agency.
11
Industry Canada’s participation in the APP will be addressed through Environment Canada’s program evaluation activities.
12
Canada was selected to the Co-Chair position on the Cement Task Force in May 2008.
6
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
EC, allows Canada to partner with the Canadian private sector on strategic international
clean technology projects in APP member countries. Given the amount of money available
for these grants and contributions (approximately $3.8 million per year), it is anticipated that
Canada’s funding may be focused on deployment and commercialization projects. It is
expected that the shorter time-frame and tangible results of these types of projects should
create the greatest and most immediate benefit for Canadian developers and exporters of
clean technologies.
2.2
Governance Structure for the International Actions Theme
The governance of the International Actions Theme relies on existing structures within the
CAA Horizontal Management and Accountability Reporting Framework (HMARF) structure
that has been established to govern the CAA process. Four interdepartmental committees
under the CAA HMARF accountability structure are responsible for providing strategic
direction, coordination and managerial oversight of the CAA:




Deputy Minister (DM) Committee
ADM Steering Committee
Director General Theme Leads Coordinating Committee (DGTLCC)
Director General Management Committees (DGMCs)
A DGMC governing body was established for the International Actions Theme. It includes
the DGs from the Climate Change International Directorate at EC, the Environment, Energy
and Sustainable Development Bureau at DFAIT, the Energy Policy Branch at NRCan, and
the Service Industries and Consumer Products Branch at IC. The Committee convenes on
an as-needed basis to discuss issues and activities of concern to the International Actions
Theme.
The chair of the International Actions DGMC, the DG of the Climate Change International
Directorate at EC, also sits on the DGTLCC of the CAA HMARF to represent the
International Actions Theme. The DGTLCC is the primary operational committee responsible
for implementing the elements of the CAA across all themes and for ensuring that
participating departments provide a systematic review of the relevance and funding of
programs—a review that is aligned with Canadians’ priorities and that effectively and
efficiently delivers on the Government’s CAA responsibilities.
To support governance of the CAA, the Clean Air Agenda–Results Management Secretariat
(CAA–RMS) was established to ensure that information is available to the DGTLCC, the
ADM Steering Committee and the DM Committee within the CAA HMARF accountability
structure. The CAA–RMS supports the oversight and monitoring of results, performance,
resources and risks across the CAA themes, initiatives and programs.
2.3
Resource Allocation for Program Components
Funding of $50 million13 was allocated across four years from 2007–08 to 2010–11 for the
International Actions Theme under the CAA. Table 1 describes the resource allocations for
each program component by department by fiscal year.
13
Resource allocation figures presented in this report may differ from those published on the CAA horizontal initiative in
Environment Canada’s Departmental Performance Reports due to differences in reporting periods.
7
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Table 1: Resource Allocations by Department and Program Componenta
FISCAL
YEAR
PM ANNEX
INTERNATIONAL
FINANCIAL AND OTHER
OBLIGATIONS
$2,892,000
$1,023,000
$623,000
$623,000
$623,000
APP
TOTAL
$2,200,000
$550,000
$550,000
$550,000
$550,000
INTERNATIONAL
PARTNERSHIPS AND
NEGOTIATIONS
$11,400,000
$3,000,000
$2,900,000
$2,800,000
$2,700,000
EC
2007–08
2008–09
2009–10
2010–11
$18,882,700
$4,628,200
$4,738,200
$4,738,200
$4,778,200
$35,374,700
$9,201,200
$8,811,200
$8,711,200
$8,651,200
DFAIT
2007–08
2008–09
2009–10
2010–11
$0
-----
$6,400,000
$1,750,000
$1,650,000
$1,550,000
$1,450,000
$1,908,000
$477,000
$477,000
$477,000
$477,000
$0
-----
$8,308,000
$2,227,000
$2,127,000
$2,027,000
$1,927,000
NRCan
2007–08
2008–09
2009–10
2010–11
$0
-----
$5,200,000
$1,400,000
$1,300,000
$1,300,000
$1,200,000
$0
-----
$877,300b
$271,800
$211,800
$211,800
$181,800
$6,077,300
$1,671,800
$1,511,800
$1,511,800
$1,381,800
IC
$0
$0
$0
$240,000b
$240,000
2007–08
---$100,000
$100,000
2008–09
---$50,000
$50,000
2009–10
---$50,000
$50,000
2010–11
---$40,000
$40,000
TOTAL
$2,200,000
$23,000,000
$4,800,000
$20,000,000 $50,000,000
a
Includes G&C expenditures. Refer to Table 2 for breakdown of G&C allocations.
b
Transfer of funds from EC to NRCan and IC, for operating expenditures related to Canada’s participation in the APP’s task
forces. This includes Public Works and Government Services Canada accommodation costs.
EC’s G&C budget in the amount of $16.872 million is managed under the Department’s
Terms and Conditions for the Class Contributions to Support Environmental Research and
Development, Environmental and Sustainable Development Initiatives and Canada’s
International Commitments. These Class Terms and Conditions address the range of
funding activities, including research related to transboundary emissions sources (under the
PM Annex), participation in environmental initiatives (under International Financial and Other
Obligations), and supporting clean technology development and deployment (for the APP).
As these were valid to March 31, 2010, new Terms and Conditions have been developed.
DFAIT’s $840,000 G&C program supports organizations in staging meetings and
conferences, conducting studies, and undertaking activities related to climate change (under
International Partnerships and Negotiations). This contribution is managed through DFAIT’s
Terms and Conditions for a Class of Contributions in Support of International Environmental
Programs, which was submitted to Treasury Board for approval in the fall of 2007.
Evaluation of this contribution program has been incorporated into the evaluation of the
International Actions Theme.
Grants and contributions of $1.908 million are realigned from EC to DFAIT to cover EC’s
share of Canada’s assessed contribution to the UNFCCC and Kyoto Protocol (under
International Financial and Other Obligations). This contribution is managed under DFAIT’s
Terms and Conditions for Assessed Contributions to the United Nations Framework
8
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Convention on Climate Change and the Kyoto Protocol. International agreements such as
the UNFCCC and Kyoto Protocol have governance mechanisms that include oversight
services. DFAIT relies on oversight regimes to assist in monitoring the international
organization’s accountability for achieving those objectives agreed to by its members for
stewardship of its funds, contributed by member states including Canada.
NRCan’s $425,000 class contribution agreement supports key activities to help advance
Canada’s objectives within the international climate change negotiations. Activities include
support to the Climate Technology Initiative, the multilateral International Energy Agency
(IEA), and the Expert Group on Technology Transfer (EGTT) under the UNFCCC.
Table 2 describes the G&C funds allocated for the International Actions Theme.
Table 2: Grants and Contributions Allocations by Department and Program
Componenta
FISCAL
YEAR
PM ANNEX
INTERNATIONAL
FINANCIAL AND
OTHER OBLIGATIONS
$1,292,000
$323,000
$323,000
$323,000
$323,000
APP
TOTAL
$100,000
$25,000
$25,000
$25,000
$25,000
INTERNATIONAL
PARTNERSHIPS AND
NEGOTIATIONS
$0
-----
EC
2007–08
2008–09
2009–10
2010–11
$15,480,000
$3,770,000
$3,870,000
$3,870,000
$3,970,000
$16,872,000
$4,118,000
$4,218,000
$4,218,000
$4,318,000
DFAIT
2007–08
2008–09
2009–10
2010–11
$0
-----
$840,000
$280,000
$230,000
$180,000
$150,000
$1,908,000 b
$477,000
$477,000
$477,000
$477,000
$0
-----
$2,748,000
$757,000
$707,000
$657,000
$627,000
NRCan
$0
$425,000
$0
$0
$425,000
2007–08
-$125,000
--$125,000
2008–09
-$100,000
--$100,000
2009–10
-$100,000
--$100,000
2010–11
-$100,000
--$100,000
TOTAL
$100,000
$1,265,000
$3,200,000
$15,480,000
$20,045,000
a
No G&C payments were allocated to IC for the CAA International Actions Theme.
b
Transfer of $1.908 million ($0.477 million per year over four years) from EC to DFAIT to cover EC’s share of Canada’s
assessed contribution to the UNFCCC and Kyoto Protocol.
2.4
Program Logic Model
The following Logic Model provides a visual representation of the CAA International Actions
Theme,14 identifying the linkages between the activities and the achievement of its
outcomes. Consistency has been maintained at the level of the intermediate and long-term
outcomes of the International Actions Theme Logic Model and the higher-level outcomes of
the CAA’s overarching Logic Model.
14
The key milestones in the development of this International Actions Theme Logic Model is presented under separate cover
in a technical appendix.
9
Audit and Evaluation Branch
Evaluation of the CAA International
Actions Theme
Figure 1: International Actions Theme Logic Model
10
Audit and Evaluation Branch
3
Evaluation of the CAA International Actions Theme
Evaluation Design
3.1
Purpose, Objectives and Scope of the Evaluation
This evaluation was conducted as a part of the Treasury Board Secretariat’s requirement to
evaluate funding for the program components of the CAA International Actions Theme. The
objectives of this evaluation were to:



determine the ongoing relevance of the CAA International Actions Theme;
assess the degree of implementation of the Theme; and
assess the Theme programs’ performance.
This evaluation examined all planned activities for the International Actions Theme of the
CAA, covering the four-year time frame from 2007–08 through 2010–11. The evaluation
examined evidence provided on the program components from the start of the initiative in
2007–08 up to December 31, 2009. Given the timing of the implementation phase for this
evaluation (the evaluation was conducted at this point to contribute to the overall evaluation
of the entire CAA expected in October 2010), performance data for intermediate, final and
even some immediate outcomes were not available. The focus of this evaluation, therefore,
was on outputs and movement toward achieving early outcomes. Where processes are
ongoing, the evaluation examined the extent to which the outcomes have been achieved
and/or the likelihood that planned activities will achieve expected results. In addition, the
evaluation examined how external factors have affected, or are likely to affect, the Theme’s
ability to achieve its expected outcomes.
Of note, the International Actions Theme is linked to the UNFCCC and Kyoto Protocol
through the International Financial and Other Obligations and the International Partnerships
and Negotiations program components (refer to section 2.1 for the program profile), which
are both within the scope of this evaluation. Activities directly related to Canada meeting
Kyoto targets for emissions are not part of the International Actions Theme and are outside
the scope of this evaluation.
Other initiatives that were considered in scoping this evaluation included the following:

DFAIT’s Class of Contributions in Support of International Environmental Programs,
which was included in this evaluation. Originally, a formative program evaluation was
planned for the end of fiscal year 2009–10 by DFAIT as part of the renewal of the
Terms and Conditions for payment of contribution programs. However, that
evaluation has been incorporated into this evaluation of the International Actions
Theme. Indicators have been included in the evaluation matrix of the CAA
International Actions Theme evaluation to capture its relevance and performance,
and results will be included in the evaluation report.

The CED, which was not included in this evaluation. The CED was added to the CAA
International Actions Theme in the fall of 2009, and is at the end of its first year of
implementation. Therefore, even though it is part of the Theme, the CED is not part
of the evaluation with the other program components. A separate evaluation is
planned for the CED in 2010–2011, and results will be rolled up with results from the
11
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
International Actions Theme evaluation in time to be aggregated to the CAA level in
the fall of 2010.
Thirteen evaluation questions, grouped according to issue, were addressed in this
evaluation to assess relevance and performance. (Evaluation Framework is provided in
Annex 1.)
Relevance:
Issue 1: Continued Need for the Program
1. Are program components within the International Actions Theme connected with key
international climate change and air quality needs?
Issue 2: Alignment with Government Priorities
2. Are program components within the International Actions Theme aligned with federal
government priorities?
3. Are program components within the International Actions Theme aligned with the
priorities of participating departments?
Issue 3: Alignment with Federal Roles and Responsibilities
4. Is there a federal role and responsibility for the Government of Canada to deliver
program components within the International Actions Theme?
Performance:
Issue 4: Achievement of Expected Outcomes
5. To what extent have intended outputs been achieved as a result of programs within
the International Actions Theme?
6. To what extent have immediate outcomes been achieved as a result of programs
within the International Actions Theme?
7. Have there been any unintended (positive or negative) outcomes that can be
attributed to the program areas within the International Actions Theme? Were any
actions taken as a result?
8. Are there any external factors outside the International Actions Theme that influence
the success of its programs?
Issue 5: Demonstrated Efficiency and Economy
9. Are there more economic and efficient means of achieving objectives under the
International Actions Theme?
10. How could the efficiency of programs under the International Actions Theme be
improved?
11. To what extent have each of the International Actions Theme program areas been
implemented, or are they on track to being implemented as planned and on time?
12. Is the management and accountability structure for the International Actions Theme
in place and functioning adequately to achieve expected results?
13. Is appropriate performance data being collected, captured and safeguarded? If so, is
this information used to inform senior management / decision makers?
12
Audit and Evaluation Branch
3.2
Evaluation of the CAA International Actions Theme
Evaluation Approach and Methodology
The three methods used to collect data for this evaluation are described below.15
1. Document Review:
An in-depth review of all relevant documents provided by the departments engaged in
the International Actions Theme was conducted, and further documents were requested
where gaps were identified or where additional information was needed to supplement
the evidence base. The project team reviewed each document to understand the
mandated requirements, goals and objectives, governance structure, authorities,
activities, outputs and outcomes for each component. The overall adequacy of the
document evidence base (e.g., availability of performance information) was also
assessed. All evidence from the documentation was recorded against the defined
evaluation issues. This data collection method addressed evaluation questions #1–13.
2. Key-Informant Interviews:
Interviews were conducted with key informants identified by the departments involved in
the CAA International Actions Theme, with suggested additions by the project team
(e.g., external contacts, international experts). Interviews were either conducted in
person (e.g., senior management, executives in the private sector) or by phone.
Interviews provided information on the program components’ relevance and
performance. Additional interviews were conducted as the evaluation progressed based
on feedback from interviewees and information learned through the process. A total of
43 interviews were conducted. This data collection method addressed evaluation
questions #1–13.
3. Focus Group of Experts:
A two-hour focus group session was held with five Canadian experts on climate change
and air quality issues.16 The purpose of the focus group was to:


share and test preliminary findings and key messages emerging from the evaluation;
and
collect additional information and expert advice on Canada’s international
performance related primarily to climate change, and also to air quality.
This data collection method addressed evaluation questions #5–8.
Preliminary findings were validated by the appropriate program officers from each of the four
components of the International Actions Theme in order to correct for omissions and errors
before completion of the evaluation report.
3.3
Limitations
The evaluation focused on the completion of program activities using allocated resources
from the fiscal years 2007–08 to 2009–10. As such, the evaluation focused on the
achievement of outputs and intended immediate outcomes only, since data collection took
15
The key methodological instruments used for data collection are presented under separate cover in a technical appendix.
16
External experts did not fulfill an advisory or oversight function to the evaluation.
13
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
place approximately one year before the end of the funding (presented in the program Logic
Model in Figure 1). The achievement of these higher-level outcomes may be assessed in
future evaluations, depending on future funding for activities under this theme.
The confidentiality of the UNFCCC negotiations, and the direction provided by Cabinet to
the Canadian delegation, constrained the evaluators’ ability to investigate the extent to
which success in securing Canada’s stated interests in the Convention can be attributed
directly or indirectly to efforts by the Government. In particular, interviews with other
international negotiators were limited as were interviews with representatives from industry17
and the non-governmental sector. These interviews would have provided an external
perspective on the Theme’s performance and relative impact on advancing Canada’s
positions in the UNFCCC as well as in other multilateral negotiation processes in which
Canada seeks to advance its positions. As a result, the evaluation was limited in providing
an independent perspective on the Theme’s performance, particularly how its negotiation
activities influence the international negotiation process within and outside of the UNFCCC.
Ongoing performance data were not available for all program components, particularly
financial information, which limited the demonstration of achievement of expected outcomes.
4
Findings by Evaluation Question
This section provides evaluation findings for each of the defined evaluation issues and
questions. Findings are presented by evaluation issue, under which there is a summary of
integrated findings across all four program components, followed by detailed findings for
each program component.
4.1
Rating of Findings
Ratings have been provided to indicate the degree to which this theme and each component
have addressed the key evaluation criteria. The rating is assessed according to the chart
below, provided by EC’s Evaluation Division. A summary table of these ratings/findings is
provided in Annex 2.
Table 3: Ratings Legend
Rating
Significance
Achieved
Progress Made;
Attention Needed
Little Progress;
Priority for Attention
N/A
~
The intended outcomes or goals have been achieved or met
Considerable progress has been made to meet the intended outcomes or
goals, but attention is still needed
Little progress has been made to meet the intended outcomes or goals
and attention is needed on a priority basis
A rating is not applicable
Outcomes achievement ratings are based solely on subjective evidence
17
Industry representatives beyond those related to the APP.
14
Audit and Evaluation Branch
4.2
Evaluation of the CAA International Actions Theme
Evaluation Issue: Relevance – Continued Need
Issue 1: Continued Need for the
Program
1. Are program components within
the International Actions Theme
connected with key international
climate change and air quality
needs?
Indicators



Overall Rating
Evidence that International Actions
Theme programs adapt with the
emergence of relevant scientific
evidence on air quality and climate
change
Evidence that International Actions
Theme programs adapt to changes in
government policy and priorities
Achieved
Evidence that the contribution program
supports international objectives
SUMMARY:
The evaluation found that, across all program components, there remains a continued need for the
programs that are part of the International Actions Theme in order to continue meeting key
international climate change and air quality needs. There is evidence that the programs have adapted
to changes in government policy and priorities and that contribution programs support international
objectives. However, program components need to continue adapting, given the emergence of
relevant scientific evidence on air quality and climate change. Due to the continued need for the
programs, the overall rating has been assessed as “achieved.”
1. Are program components within the International Actions Theme connected with
key international climate change and air quality needs?
1a Particulate Matter Annex
The PM Annex component of the International Actions Theme clearly responds to and
connects with key air quality needs in Canada.
The understanding of environmental and human health impacts of PM has grown
significantly in recent years. Many studies now link PM, particularly in its finer fractions
(PM10 and PM2.5), to a host of health impacts—including aggravated respiratory diseases
such as bronchitis and asthma. PM can also damage vegetation and negatively affect
visibility.18
In recognition of the transboundary
flow of PM across the Canada-U.S.
border, the Canada–U.S.
Subcommittee on Scientific
Cooperation completed the Canada–
U.S. Particulate Matter Science
Assessment in 2004. The Committee
was directed to better understand,
through a scientific assessment, the
current knowledge of transboundary
transport of PM and PM precursors
between Canada and the United
Canada US Air Quality Agreement Progress Report 2008
Exposure to PM has been associated with premature mortality as
well as indices of morbidity, including respiratory hospital
admissions and emergency department visits, school absences,
lost work days, restricted activity days, effects on lung function and
symptoms, morphological changes, and altered host defence
mechanisms.
Recent epidemiologic studies have continued to report associations
between short-term exposures to fine particles and effects such as
premature mortality, hospital admissions or emergency department
visits for cardiopulmonary diseases, increased respiratory
symptoms, decreased lung function, and physiological changes or
biomarkers for cardiac changes.
18
Canada–-U.S Subcommittee on Scientific Co-operation (2004). Canada–-United States Transboundary Particulate Matter
Science Assessment. Available at: http://www.ec.gc.ca/Publications/default.asp?lang=En&xml=3BC8288E-E427-4E0E-A624F67334D31BB0. This report was completed in support of the Canada–-U.S. Air Quality Agreement.
15
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
States. The results of this assessment noted the following:





PM is recognized as an important health concern;
high ambient levels of PM and its precursors are observed in North America, and in
Canada the areas with high levels include southwestern Ontario and the Georgia
Basin – Puget Sound Region;
precursors of PM generally contribute to the acidification of ecosystems and are a
significant cause of visibility impairment;
PM and its precursors can be transported long distances and are transported
between Canada and the United States; and
reductions in SO2 are likely to result in reductions in PM, acid deposition, and
visibility impairment.
Based on the evidence provided by this assessment, Canada and the United States
committed to negotiating a PM Annex to the Canada–U.S. Air Quality Agreement. As the
Agreement is aimed at reducing the cross-border flow of air pollution and its impact on the
health and ecosystems of Canadians and Americans, both countries are committed to
develop and implement emission reduction programs to reduce fine particle concentrations.
Such an Annex will establish commitments by the United States to reduce its emissions of
PM and its precursors, and to help Canada secure improvements in air quality and
associated health and environmental benefits. A PM Annex would aim to ensure that both
Canada and the United States develop commitments to reduce emissions of SO2 and NOx
as well as PM, leading to a substantive decrease in transboundary flows (including mobile
sources of PM19) of these pollutants. A PM Annex would also aim to ensure that there is a
shared responsibility to monitor PM levels, conduct science, and assess progress toward
fulfilling PM commitments. It is further anticipated that a PM Annex could provide a starting
point for developing a cross-border SO2 and NOx trading regime.
Negotiations for a PM Annex began in 2007, but have not progressed due to evolving policy
context in both countries. There is therefore a continued need to work toward and finalize
the PM Annex.
Rating: Achieved
1b International Partnerships and Negotiations
The International Partnerships and Negotiations component of the International Actions
Theme clearly responds and is connected to international climate change needs.
In particular, this component reflects the growing prominence of climate change in a range
of multilateral and bilateral international fora. For example, climate change has been a
fixture on the agenda of G8 Leaders since 2005. This focus has prompted several new
climate change–related initiatives, such as the Gleneagles Plan of Action and the U.S.-led
MEF. Bilateral fora such as the Canada–European Union Partnership and the
CED provide strategic opportunities for Canada to define and advance its climate change
activities. In addition to ad hoc working groups of the UNFCCC, multiple UN bodies have
addressed climate change in recent years, including the UN General Assembly and the UN
19
In this context, mobile sources are air pollution generated by vehicles, engines, equipment that move between Canada and
the United States.
16
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Commission on Sustainable Development. International deliberations in each of these fora
have highlighted the importance of sustained international action on climate change.
The program component also reflects scientific evidence on the need to address climate
change, as evidenced most recently by the 4th Assessment Report published by the
Intergovernmental Panel on Climate Change (IPCC) in 2007. Among its findings, this report
noted that warming of the climate system is unequivocal and that global atmospheric
concentrations of GHGs have increased markedly since the Industrial Revolution, now far
exceeding pre-industrial values.
During the summer of 2009, and in recognition of the IPCC’s 4th Assessment Report,
leaders from the G8 and MEF recognized the scientific view that the increase in global
average temperature above pre-industrial levels should not exceed 2 degrees Celsius. G8
leaders also affirmed their commitment to reduce global emissions by at least 50% by 2050.
Canada’s long-term targets are consistent with this vision.20
This component of the International Actions Theme is clearly relevant and responsive to
international climate change.21
Rating: Achieved
1c International Financial and Other Obligations
Evidence from this evaluation indicates that Canada is meeting international climate change
and air quality needs through its ongoing payment of international financial obligations.
Canada is a party to the UNFCCC, the legal agreement that provides the international
response to global climate change. The UNFCCC Secretariat supports all institutions
involved in the climate change process, particularly the UNFCCC Conferences of the Parties
(COP), the subsidiary bodies and the Bureau. As a Party to the UNFCCC, Canada is
obligated to contribute its share of dedicated assessed funding and, in this way, contributes
financially to the UNFCCC Secretariat in order to address international climate change and
air quality needs. Evidence from this evaluation demonstrates that Canada is meeting this
obligation.
Beyond the assessed funding, other important obligations include Canada’s commitments to
establish a National Registry for Kyoto-recognized emissions credits, a condition for
Canada’s participation in the flexibility mechanisms of the Protocol (i.e., the Clean
Development Mechanism, Joint Implementation, and the ability to trade national emissions
allowances). National registries are required of all Annex 1 country signatories to the Kyoto
Protocol and are linked to an international database (the ITL) that is used to monitor each
country’s holdings and transfers of Kyoto-recognized emissions credits. In addition to
establishing the Registry and the ITL connection, evaluation evidence indicates that Canada
makes supplementary contributions to develop user guidance for the ITL. These obligations
directly support international efforts to facilitate monitoring and accountability of emissions
20
http://www.climatechange.gc.ca/default.asp?lang=En&n=5497F282-1
It should be noted that, while Canada’s current performance and stated medium-term goals are not fully consistent with
international climate change needs as articulated by the IPCC, setting Canada’s reduction targets is not under the purview of
this theme and is outside the scope of this evaluation.
21
17
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
trading, and contribute to establishing a level playing field through consistent technical-user
guidance for the ITL.
Within this component, Canada also makes voluntary contributions to support initiatives in
which Canada has a specific interest. These include funding to enable Canada to participate
in and contribute to international technology initiatives outside the UNFCCC, such as M2M
and REEEP.
Supplementary G&C funds are also provided to the Organisation for Economic Cooperation
and Development (OECD) Climate Change Expert Working Group, which provides a forum
to work out technical details related to implementation of the UNFCCC and Kyoto Protocol.
Rating: Achieved
1d Asia–Pacific Partnership
The APP supports key international climate change and air quality needs, as the main
objectives of the program are to:
 promote the development, deployment and diffusion of existing and emerging
cleaner, more efficient technologies and practices;
 complement the UN climate processes to address development, energy and climate
issues; and
 build on existing bilateral and multilateral initiatives to enhance cooperation in order
to meet increasing energy needs and the associated challenges of air pollution,
energy security, and GHG intensities.
The APP brings together key major-emitting and major-economy countries to collectively
address the issues of sustainable development, clean energy and climate change. As
climate change is a global issue, cooperation among these countries is essential for
achieving the long-term GHG reductions needed for a sustainable future. The APP partners
(Australia, Canada, China, India, Japan, Korea and the United States) focus on expanding
investment and trade in cleaner-energy technologies, goods and services in key market
sectors. The seven partner countries collectively account for more than half of the world’s
economy, population and energy use, and they produce about 65% of the world’s coal, 62%
of the world’s cement, 52% of the world’s aluminum, and more than 60% of the world’s
steel.22
The clean-technology projects developed and supported by the APP help reduce GHGs,
and improve air quality in the partner countries. Information sharing between countries is
also an objective. This exchange of technologies and expertise between countries provides
for a more level playing field for the industries involved, helping to address the issue of
competitiveness resulting from the disparate climate change / air quality regulations of the
partner countries.
The APP is seen by the partner countries as a complementary process to the UN climate
change process, supporting but not replacing the Kyoto Protocol.
Rating: Achieved
22
Vision Statement of the APP, found at http://www.asiapacificpartnership.org/pdf/resources/vision.pdf .
18
Audit and Evaluation Branch
4.3
Evaluation of the CAA International Actions Theme
Evaluation Issue: Relevance – Alignment with Government Priorities
Issue 2: Alignment with
Government Priorities
Indicators
Overall Rating
2. Are program components within
the International Actions Theme
aligned with federal government
priorities?

Achieved
Extent to which strategic directions
within International Actions Theme
match federal government priorities
3. Are program components within
the International Actions Theme
aligned with the priorities of
participating departments?
SUMMARY:
The Theme’s program components are clearly aligned with federal government priorities and the
priorities of participating departments. Due to this strong alignment, the overall rating has been
assessed as “achieved.”
2. Are program components within the International Actions Theme aligned with
federal government priorities?
2a Particulate Matter Annex
The PM Annex is clearly aligned with federal government
priorities.
2007 Speech from the Throne
“Our government will also bring
forward the first ever national air
pollution regulations. In so doing,
our government will put Canada
at the forefront of clean
technologies to reduce air
pollution and greenhouse gas
emissions.”
The 2006 and 2007 Speeches from the Throne highlight that
Canada (through domestic action) will take action to reduce
pollution by regulating GHG and air pollutant emissions.
Similarly, the federal Turning the Corner plan to reduce GHGs
and air pollution as well as the Regulatory Framework for Air Emissions laid out the broad
design of the regulations for industrial emissions of GHGs and air pollutants. These
initiatives focused on reducing emissions from industry, and outline regulatory measures to
reduce emissions from the transportation sector as well as consumer and commercial
products, and actions to improve air quality.
Furthermore, the Government of Canada has made a concentrated effort to work with the
United States to develop a North American position on a number of environmental issues in
key fora, such as the negotiations under the UNFCCC, the MEF, the APP and the CED.
The PM Annex is aligned with the long-standing history between Canada and the United
States on addressing transboundary air quality issues, dating to the signature of the Air
Quality Agreement23 in 1991. The Agreement was originally established to address shared
concerns regarding transboundary air pollution (SO2 and NOx) that leads to acid rain.
23
Available at 23 Available at http://www.ijc.org/rel/agree/air.html..
19
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
This program component is consistent with an agreement signed in April 1997 between the
two countries signalling their intent to develop a Joint Plan of Action to more specifically
address the problem of transboundary air pollution and smog. Ground-level ozone was
identified as the first priority under this plan, and in December 2000 the Ozone Annex was
added to the Air Quality Agreement. The Joint Plan of Action called for the development of a
PM science assessment, to then serve as the basis for a decision on whether to proceed
with an additional annex on PM.
As such, negotiating the PM Annex is aligned with the Government of Canada’s position to
address air quality issues in Canada generally, and more specifically to reduce cross-border
air pollution (including PM and its precursors) affecting human health and the environment in
Canada and the United States.
Rating: Achieved
2b International Partnerships and Negotiations
This program component is strongly aligned with federal government priorities.
For example, the October 2007 Speech from the Throne noted that climate change is a
global issue requiring a global solution, and reiterated Canada’s commitment to “press for a
new international agreement that cuts global emissions in half by 2050.” The November
2008 Speech from the Throne reiterated the Government’s intention to “continue its realistic,
responsible approach to addressing the challenge of climate change,” by working with its
international partners and by reducing Canada’s total GHG emissions by 20% by 2020.24
This program component is also consistent with a September 2007 speech by Canada’s
Prime Minister at the UN Secretary General’s High Level Event on Climate Change,25
outlining the federal government’s priorities related to climate change. The principles
identified in this speech provided the basis for Canada’s priorities and negotiating positions,
which are to pursue a new global consensus on a legally binding international climate
change agreement that is environmentally effective, comprehensive, and inclusive of all
major emitters. These priorities and positions include:



Canada’s commitment to reduce the country’s total emissions by 17% from 2005
levels and by 60–70% from 2006 levels by 2050;
the need for a long-term approach that provides the time needed to develop and
deploy transformative technologies to achieve these goals; and
a series of principles26 underlying Canada’s approach that could assist in forging
consensus around a new international framework for long-term action. These
principles emphasize the following:
o Balancing environmental protection with economic growth: taking sustained
action to reduce emissions while achieving low-carbon growth and
sustainable development and not unduly burdening the growth of any single
country.
24
This commitment was subsequently revised to reducing Canada’s total GHG emissions by 17% from 2005 levels by 2020
(see http://www.climatechange.gc.ca/default.asp?lang=En&n=D43918F1-1) so as to align with the target of the United States.
25
The Prime Minister was an invited speaker at this international meeting, where he presented Canada’s commitments on
climate change and clean technology (http://pm.gc.ca/eng/media.asp?id=1827).
26
The principles can be found at www.climatechange.gc.ca/default.asp?lang=En&n=5497F282-1.
20
Audit and Evaluation Branch
o
o
o
o
Evaluation of the CAA International Actions Theme
Having a long-term focus: putting international policies in place to achieve
deep emissions reductions over time, aiming to cut global emissions in half
by 2050.
Supporting the development and deployment of clean technologies:
stimulating innovation and rapid and widespread deployment of low-carbon
technologies through measures such as strategic technology initiatives and
investments.
Engaging, and seeking commitments from, all major economies: working
toward emission reduction obligations for all countries (particularly the
largest-emitting economies) that suit their unique circumstances.
Supporting constructive and ambitious global actions: working with North
American partners and all UNFCCC parties through the negotiation process
to reach a fair, effective and comprehensive global climate agreement.
These principles also reflect Canada’s revised approach to: address climate change within
the country as Canada seeks to modify and evolve the position initially reflected in Turning
the Corner; adapt to the economic downturn; and align its domestic targets and work with
the current U.S. Administration on a new long-term agreement to address international
climate change.
Rating: Achieved
2c International Financial and Other Obligations
This program component is consistent with direction provided in Speeches from the Throne
and Turning the Corner.
Specifically, obligations to establish the National Registry for emissions and support the ITL
contribute to the Government of Canada’s national commitment to address climate change
by providing an official UN-sanctioned forum where companies can trade Kyoto-supported
emissions.
The technology partnership activities pursued within this program component (M2M and
REEEP) demonstrate alignment with Government of Canada priorities outlined in Canada’s
Innovation Strategy, which coordinates science and technology trade activities across
federal departments. As a result of the Strategy, the Government of Canada increased focus
and placed more emphasis on innovation, science and technology, as key elements of
increasing Canada’s prosperity. The subsequent strategy, Mobilizing Science and
Technology to Canada’s Advantage, set out a comprehensive, multi-year science and
technology agenda for the nation. In 2008, the Government of Canada released Seizing
Global Advantage: A Global Commerce Strategy for Securing Canada’s Growth and
Prosperity, which outlines the Government’s international priorities for strengthening
Canadian engagement in the global economy. These broad Government of Canada
priorities correspond with the program activities undertaken within this program component,
including: fostering technology partnerships abroad through M2M using Canadian climate
change mitigation technologies; and participating and contributing to international capacity
development programs related to climate change mitigation science and technology through
REEEP. Other technology partnerships, including the APP, the Global Carbon Capture and
Storage Institute, the Carbon Sequestration Leadership Forum (CSLF) and the IEA, are
noted in other program components where relevant.
21
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Rating: Achieved
2d Asia–Pacific Partnership
The APP is well-aligned with federal government priorities and the federal mission of
balancing environmental protection with economic development. The APP supports
Canada’s domestic and international air quality, climate change and clean technology
objectives, and is consistent with direction provided in multiple federal Throne Speeches
(2008–10) and the Turning the Corner initiative. Several Canadian projects funded through
the APP also support the objectives of reducing GHG emissions and ensuring clean air,
water, land and energy while ensuring a strong economy.
The following are the priorities of the federal government (as outlined in the 2008–10
Speeches from the Throne) that align with the objectives and goals of the APP:






addressing climate change and preserving Canada’s environment by reducing GHG
emissions and ensuring that 90% of our electricity needs are met by non-emitting
sources;
working on global cooperation to advance the climate change agenda, which
includes an agreement that includes all the world’s major GHG emitters, and to
provide funding to help developing economies reduce their emissions and adapt to
climate change;
enhancing energy security, which includes pursuing new cleaner-energy supplies
and enhancing energy efficiency;
expanding investment and trade and diversifying opportunities for Canadian
business, through bilateral and multilateral trade agreements;
stimulating the economy and supporting Canadian industries; and
investing in clean energy technologies.
The federal government recognizes that reducing global emissions over the long term will
require the development and deployment of clean, low-carbon technology by the world’s
major emitters. Canadian participation in the APP supports Canadian priorities in clean
energy research and development, such as large-scale carbon capture storage (CCS),
renewable energy and energy efficiency. Activities in the APP are also aligned with the
broader government priority to strengthen bilateral relationships (e.g., with the United States
and China).
Rating: Achieved
3. Are program components within the International Actions Theme aligned with the
priorities of participating departments?
3a Particulate Matter Annex
This program component is clearly aligned with the priorities of EC.
22
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
EC leads all aspects of work completed under this component, which is consistent with the
Department’s mandate, strategic outcomes and program activities. A PM Annex with
commitments from Canada and the United States to improve air quality by reducing SO2 and
NOx emissions and PM can lead to a substantive decrease in transboundary flows of these
pollutants and an increase in health and environmental benefits for Canadians.
As described in subsection 4(1) of the Department of the Environment Act,27 and as
presented on EC’s website, the Department’s mandate is to “preserve and enhance the
quality of the natural environment, including water, air and soil quality; conserve Canada’s
renewable resources, including migratory birds and other non-domestic flora and fauna;
conserve and protect Canada’s water resources; carry out meteorology; enforce the rules
made by the Canada - United States International Joint Commission relating to boundary
waters; and coordinate environmental policies and programs for the federal government.”28
Furthermore, work under this component is directly aligned with EC’s third Strategic
Outcome29: “Threats to Canadians and their environment from pollution and waste.are
minimized” This Strategic Outcome is supported by the “threats to Canadians, their health
and their environment from air pollutants and greenhouse gas emissions are minimized”
program activity. As such, work to complete a PM Annex that includes commitments to
minimize air pollutants and improve air quality is aligned with this outcome as it aims to
reduce the potential health risks to Canadians caused by poor air quality.
Rating: Achieved
3b International Partnerships and Negotiations
This program component is strongly aligned with the mandates and priorities of all
participating departments. In particular, advancing Canadian positions within the climate
change negotiations assists EC in delivering its environmental protection mandate, while
also enabling DFAIT to fulfill its role in advancing Canadian interests internationally.
Ongoing participation in international climate change fora also supports NRCan in its efforts
to promote Canadian interests internationally and to position Canada as a world-leading
steward of our natural resources.
In addition, this program component is consistent with the departmental strategic outcomes,
key program activities and operational priorities of each participating department (see
Table 4 below).30
Table 4: Departmental Priorities
Participating
Department
EC
Relevant Strategic
Outcome
Canadians and their
environment are protected
from the effects of pollution
and waste
Relevant Program
Activity
Clean air program
Operational Priorities
Reducing GHG
emissions domestically
and through
international
agreements
27
Available at http://laws.justice.gc.ca/eng/E-10/FramesView.html.
Available at http://www.ec.gc.ca/introec/mandate.htm.
29
As reported in Part III of the 2008–2009 Departmental Performance Report, available at www.tbs-sct.gc.ca/dpr-rmr/20082009/inst/doe/doe-eng.pdf.
30
Although IC is not a formal member of the Canadian delegation, that departmentey participates in interdepartmental working
groups and provides information on an as- required basis.
28
23
Audit and Evaluation Branch
NRCan
DFAIT
Canada is a world leader in
the environmentally
responsible development
and use of natural
resources
Canada’s international
agenda
Evaluation of the CAA International Actions Theme
Clean energy
Address climate
change and air quality
through clean energy
International policy advice
and integration
United States and the
western hemisphere
(air agreements,
climate change)
Rating: Achieved
3c International Financial and Other Obligations
The activities under International Financial and Other Obligations are aligned with DFAIT
and EC departmental priorities.31
Within this component, DFAIT serves as the focal point for overseeing financial contributions
to the UNFCCC in order to meet the assessed funding requirements, providing
administrative support and management coordination for the provision of assessed funding
on behalf of Canada. EC serves as technical lead on developing the National Registry for
emissions credits, managing the connection to the UN-based interactive ITL, participating on
behalf of Canada in technology partnerships (e.g., REEEP), and leading Canada’s
technology trade initiatives through M2M.
DFAIT is responsible for Canada’s Innovation Strategy noted above (see question 3).
DFAIT’s priorities include strengthening Canada’s missions abroad, leading a
government-wide approach to formulating and implementing policies on foreign affairs and
international trade as well as related programs, and promoting international trade and
commerce.
The technology partnerships programming (e.g., M2M), led by EC, correspond with the
Department’s mandate of being a science-based department with a priority to address risks
to health and environment from air pollution, and to reduce GHG emissions through
international agreements, as stated in the 2009–2010 Environment Canada Report on Plans
and Priorities. The M2M program also draws on technology research undertaken by EC and
NRCan through the Program for Energy Research and Development, which also
demonstrates coordination across participating departments. However, data from interviews
suggest that the renewable energy / energy efficiency capacity-development-programming
membership (e.g., REEEP) might more closely align with the priorities of NRCan, given the
clear departmental mandate in its Energy Sector and, in particular, NRCan’s specific
research mandate with respect to renewable energy technology deployment. There is
evidence of NRCan’s participation in energy technology partnerships through other activities
within other program components of this theme, such as the EGTT, APP, CSLF and IEA.
Rating: Achieved
31
Neither NRCan nor IC participate in this program component.
24
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
3d Asia–Pacific Partnership
The evaluation found that the APP and its activities are appropriately aligned with the
mandates and priorities of the participating federal departments (EC, NRCan, IC and DFAIT)
as articulated in their Reports on Plans and Priorities and as evidenced by their efforts and
cooperation to promote Canada’s participation in the APP.
Below is a summary of the relevant priorities by department:32
EC: There is a direct link to EC’s priorities for GHG and air pollutant priorities and to the
“climate change and clean air” program activity. APP project development corresponds with
EC’s mandate to address risks to health and environment from air pollution, and the goal of
reducing GHG emissions through international agreements.
DFAIT: There is a link to DFAIT’s priorities to enhance energy security, help Canadian
business with opportunities in growing and emerging markets (such as China and India),
and strengthen bilateral relations with the United States. DFAIT’s priorities include
strengthening Canada’s international platform, bolstering Canada’s presence abroad, and
ensuring that missions overseas are in places that matter most to Canadian interests.
NRCan: There is a direct link to NRCan’s priorities of addressing climate change and air
quality through science, technology and adaptation. NRCan participation in the APP
supports priorities in clean energy research and development, such as large-scale CCS,
renewable energy and energy efficiency.
IC: There is a link to IC’s priorities of integrating the economic, environmental and social
interests of Canadians, and helping Canadian industry become more competitive in the
global economy.
Rating: Achieved
32
Priorities are referenced in Reports on Plans and Priorities (2008–2010).
25
Audit and Evaluation Branch
4.4
Evaluation of the CAA International Actions Theme
Evaluation Issue: Relevance – Alignment with Federal Roles and
Responsibilities
Issue 3: Alignment with Federal
Roles and Responsibilities
4. Is there a federal role and
responsibility for the Government of
Canada to deliver program
components within the International
Actions Theme?
Indicators



Demonstrated evidence of federal role
and responsibility in the area of
international actions for climate change
and air quality
Absence/presence of similar programs
within the federal and/or provincial
governments
Extent to which similar programs
complement/duplicate one another
Overall Rating
Achieved
SUMMARY:
There is a clear federal role and responsibility for the Government of Canada to deliver this
International Actions Theme, as the federal government has the responsibility to address
environmental issues across international boundaries. Therefore, the overall rating has been
assessed as “achieved.”
4. Is there a federal role and responsibility for the Government of Canada to deliver
program components within the International Actions Theme?
4a Particulate Matter Annex
There is a clear federal role and responsibility to address environmental issues across
international boundaries, recognizing that provinces and territories play an active role in
managing PM across Canada.
Key federal areas of responsibility related to PM and air pollution generally include: working
with foreign governments, including the United States, to reduce transboundary and
international sources of air pollution; regulating substances declared toxic under the
Canadian Environmental Protection Act, 1999; regulating contaminants in fuels and enacting
motor vehicle emission standards; and maintaining the National Pollutant Release Inventory.
In managing PM across Canada, provinces and territories typically serve as permitting
authorities (e.g., by issuing and monitoring Certificates of Approval) for emissions from point
sources such as smelters, thermal power plants, and oil refineries. They also regulate
municipal and hazardous waste incinerators and control provincial transportation, including
inspection and maintenance of on-road vehicles.
Given the shared responsibilities for air quality management in Canada,33 federal, provincial
and territorial governments coordinate through the Canadian Council of Ministers of the
Environment (CCME) and its Air Management Committee. Through the CCME, federal,
provincial and territorial governments except Quebec developed the 2000 Canada-wide
33
The federal government has constitutional power to regulate interprovincial and international air pollution, and can also
regulate air pollution under its exclusive jurisdiction over the criminal law. It also has the power to negotiate and enter into
treaties on Canada’s behalf (e.g., the 1991 Canada–U.S. Air Quality Agreement). Provinces have a very broad power to
regulate pollution that occurs within provincial boundaries. As such, they are the primary regulators of industrial air pollutants.
They also have the power to delegate, to municipal and regional governments, the responsibility for dealing with local pollution
problems. www.cleanair.ca/law_jurisdiction.html
26
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Standards for PM and Ozone,34 which commit all governments to significantly reduce PM
and ground-level ozone by 2010.
The Government of Canada also participated in a multi-stakeholder process that included
representatives of provincial governments, industry and non-governmental organizations to
develop a proposal for a comprehensive air management system to address air pollution in
Canada.35
Rating: Achieved
4b International Partnerships and Negotiations
There is clear federal jurisdiction for this program component. In its 2006 overview report on
climate change, the Commissioner of the Environment and Sustainable Development
(CESD) noted that “The federal government has jurisdiction over environmental issues that
cross international and provincial boundaries. It addresses national concerns about the
environment and negotiates, signs and ratifies international treaties on behalf of Canada.”36
At the same time, provinces and territories have a critical role to play in implementing
international commitments related to climate change and other environmental issues. For
example, they have jurisdiction over natural resources within their boundaries, issue permits
specifying allowable emission levels by a variety of emission sources, and have
responsibility for provincial transportation and municipalities.
As such, significant cooperation is required between federal, provincial and territorial levels
of governments to ensure coordinated and effective strategies to address climate change
both domestically and internationally.
Rating: Achieved
4c International Financial and Other Obligations
There is a clear federal role and responsibility to address environmental issues across
international boundaries, in particular with respect to air quality and climate change. Key
federal areas of responsibility supporting climate change include working with and
supporting the UN to develop international climate change protocols and agreements, and
supporting UNFCCC activities since 1992, when Canada ratified the Convention.
Under the Convention, Canada is legally required to:


gather and share information on GHG emissions, national policies and best
practices;
launch national strategies for addressing GHG emissions and adapt to expected
impacts, including the provision of financial and technological support to developing
countries;
34
Document can be found at 34 Document can be found at http://www.ccme.ca/assets/pdf/pmozone_standard_e.pdf. .
The work of the group was completed in April 2010.
36
Page 38, 2006 Report of the Commissioner of the Environment and Sustainable Development. to the House of Commons
35
27
Audit and Evaluation Branch


Evaluation of the CAA International Actions Theme
cooperate in preparing for adaptation to the impacts of climate change; and
fund its annual share of UNFCCC activities (each country’s “share” is “assessed” by
the Conference of the Parties [COP] of the UNFCCC).
The federal government has jurisdiction to implement program activities within this
component. As a party to the UNFCCC, Canada is legally obligated to provide financial
support through assessed funding. This support is coordinated through EC and DFAIT.
DFAIT has a mandate to represent Canada on all foreign policy missions by serving as the
key point of contact for the UNFCCC. DFAIT carries out its primary role, i.e., as Canada’s
point of contact for fulfilling international obligations, through processing all payments
related to Canada’s assessed contributions to the UNFCCC. With respect to climate change,
DFAIT is responsible for engaging in multilateral discussions on climate change in various
international fora, including the UNFCCC and its subsidiary bodies. DFAIT also participates
in ongoing dialogues on climate change policies with key partner countries. In addition, that
department serves as the Government of Canada’s Designated National Authority for the
Clean Development Mechanism and as Focal Point for Joint Implementation projects.
EC has long been a contributor to the Government of Canada’s international environmental
agenda by advancing and sharing science expertise, as well as through negotiations and
policy dialogue in international fora. In this role, EC leads some of Canada’s activities
related to international clean technology partnerships intended to reduce GHG emissions
and improve air quality globally. Provincial participation in the technology partnerships
activities was not identified, but there is a potential role for provincial involvement in these
programs as a participant, not necessarily as a funding contributor (e.g., M2M).
Rating: Achieved
4d Asia–Pacific Partnership
The federal government has the lead role in the APP, as the Partnership’s activities
comprise international actions to address transboundary/global environmental issues (which
is clear federal jurisdiction). There is cooperation among federal departments to coordinate
technical and policy work related to Canada’s participation in the APP's Task Forces and to
develop a process to evaluate and fund projects that promote the development, diffusion
and deployment of clean technologies.
A number of similar programs (e.g., the CED, ecoEnergy Initiatives [NRCan], M2M, activities
of the IEA, and activities supported by various industry associations) support the
development of clean technologies. These programs are seen by stakeholders as
complementary to APP activities, as opposed to being duplicative.
Rating: Achieved
28
Audit and Evaluation Branch
4.5
Evaluation of the CAA International Actions Theme
Evaluation Issue: Performance – Achievement of Expected Outcomes
Issue 4: Achievement of
Expected Outcomes
Indicators
Overall Rating
5. To what extent have intended
outputs been achieved as a result of
programs within the International
Actions Theme?

Evidence of achievement of outputs as
noted in the Logic Model and specific
indicators noted in section 4.1
Progress Made;
Attention
Needed
6. To what extent have immediate
outcomes been achieved as a result
of programs within the International
Actions Theme?

Evidence of achievement of outcomes
as noted in the Logic Model and specific
indicators noted in section 4.1

Presence/absence of unintended
outcomes
Where appropriate, documented
management actions and/or lessons
learned from unintended outcomes
7. Have there been any unintended
(positive or negative) outcomes that
can be attributed to the program
areas within the International
Actions Theme? Were any actions
taken as a result?
8. Are there any external factors
outside the International Actions
Theme that influence the success of
its programs?



Evidence of factors outside the
International Actions Theme that have
influenced the achievement of outputs
and early outcomes
Where appropriate, documented
management actions to address the
influence of external factors
SUMMARY:
While the majority of the intended outputs have been achieved across component areas, there have
been some significant delays and the PM Annex work has not proceeded as planned. Although it is
still early in the implementation phase, some progress has been made in achieving the immediate
outcomes and no significant unintended outcomes have been identified. However, external factors
have influenced the success of programs in this theme, particularly the increasing prominence of the
climate change file, changes in policy direction and positions, and staff and Ministerial changes.
Due to the delays experienced and lack of progress in some component areas, the overall rating has
been assessed as “Progress Made; Attention Needed.”
5. To what extent have intended outputs been achieved as a result of programs within
the International Actions Theme?
5a Particulate Matter Annex
There are two expected outputs for the PM Annex: Partnerships and processes, and formal
and informal consultations with domestic stakeholders. Overall, some progress was made
toward achieving the expected outputs, with evidence indicating that activities completed to
date have focused more on developing partnerships and processes with the United States
to establish the PM Annex, and less on conducting formal or informal consultations with
domestic stakeholders. Below is a summary of how this program component has achieved
these expected outputs.
29
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
A) Partnerships and processes outside the UNFCCC
The evaluation found evidence that a number of partnership and process activities (outside
the UNFCCC) have been completed to design and develop the PM Annex, primarily in 2007
and 2008. During that time, Canada–U.S. negotiations toward the PM Annex were launched
pursuant to a Statement of Intent announced in April 2007. The Statement marked the
beginning of negotiations to develop the PM Annex, and EC officials emphasized the
importance of this milestone as it underlines the need to address transboundary PM flowing
from the United States into Canada. Three working groups were established to address
mobile sources, monitoring and reporting, and the PM Annex Purpose Statement, and to
complete work in preparation for the negotiation sessions.
Two negotiating sessions were held: one in November 2007 and one in May 2008. These
provided an opportunity for both countries to collectively draft several elements of the
Annex, including its:




Purpose Statement;
pollutants of concern (i.e., the Annex is to address primary PM2.5 and two secondary
precursors, SO2 and NOx);
definitions of the PEMAs for both countries (Canada indicated that its PEMA would
consist of Canada as a whole, while the U.S. PEMA would comprise 28 states
across the northern part of the country); and
mobile-source commitments that could be included as part of the emission reduction
commitments, given that transportation is a major contributor to PM.
The evaluation also found evidence highlighting that the work completed to date has been
largely internal and has focused on preparing materials for the Canadian delegation in
advance of the negotiating sessions. As such, the evaluation found evidence to confirm that
work was completed to develop Canada’s negotiating mandate (e.g., discussion document
identifying what elements should be included under the PM Annex, multiple conference calls
between EC headquarters and regional offices / other government departments). Ongoing
contact was also maintained between EC and the U.S. Environmental Protection Agency
(EPA).
Currently, program activities are limited to sharing monitoring and reporting information
about stationary sources and other condensable PM emissions, and ongoing collaboration
on visibility issues.
B) Formal and informal consultations with domestic stakeholders
Outside of the activities to prepare Canada during the negotiation sessions, the evaluation
found that the program components within the International Actions Theme have made
contributions to further the understanding of air quality issues at the regional level. For
example, from 2007 to 2009 the British Columbia Lung Association received an annual
contribution to support activities to further understand visibility issues in British Columbia.
These activities included workshops in 2008 and 2010 (with representatives from EC, the
EPA, the B.C. Ministry of Environment, Metro Vancouver and the Fraser Valley Regional
District) to discuss historical and current visibility work in Canada and the U.S. Similarly,
program resources have been allocated to the EC regional offices to complete a series of
30
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
pilot studies on visibility in western Canada and, more broadly, ongoing work to collect
information about ambient air emissions and their associated health effects.37
The evaluation did not find evidence that the provinces, territories or other domestic
stakeholders were consulted directly on the process to design the PM Annex. Program staff
indicated that although specific consultations were not held, other processes, such as the
CCME Canada-wide Standards for PM and Ozone, were used to keep Canadians apprised
of the PM Annex negotiations.
Rating: Little Progress; Priority for Attention
5b International Partnerships and Negotiations
Intended outputs (discussed below) associated with the International Partnerships and
Negotiations component have been largely achieved.
A) Contribution to international bodies
There is strong demonstrated evidence of federal government participation in all major
negotiating sessions conducted pursuant to the UNFCCC, including the three most recent
meetings of the Conferences of the Parties to the UNFCCC (COP 13 in 2007 in Bali,
COP 14 in 2008 in Poznan, and COP 15 in 2009 in Copenhagen). The Government of
Canada also participated in the Ad Hoc Working Group on Further Commitments for Annex I
Parties under the Kyoto Protocol (AWG-KP) and the Ad Hoc Working Group on Long-term
Cooperative Action under the Convention (AWG-LCA), as well as meetings of other
subsidiary bodies to the UNFCCC.38
There is also demonstrated evidence of ongoing participation by Canada in a number of
expert groups established in support of the UNFCCC and its parties. For example:




Canada (represented by NRCan) is the elected representative for the developed
countries and one of eight Annex I Party members of the 19-member globally
represented EGTT, which supports and advances technology transfer activities.
Canada is perceived to be an active and valuable contributor to these efforts.
Canada (represented by NRCan) most often leads the Umbrella Group’s Technology
Negotiating Working Group39 during negotiations and between sessions.
Canada (represented by DFAIT) is one of three developed-country members of the
Least Developed Countries Expert Group (LEG), which provides advice to
least-developed countries on the preparation and implementation of national
adaptation plans.
Canada (through DFAIT) has a seat within the UNFCCC’s Consultative Group of
Experts on National Communications, which has a mandate to help improve the
preparation of national communications required from developing countries under the
Convention.
37
Documentation to confirm the completion of scientific activities funded through the International Theme was not submitted
despite several requests from the evaluators. Evidence presented in this report represents the views of program staff and
includes references found in summary documents and briefing materials. Scientific reports were not submitted.
38
Two permanent subsidiary bodies have been created by the Convention: the Subsidiary Body for Implementation and the
Subsidiary Body for Scientific and Technological Advice. Additional subsidiary bodies may be established as needed.
39
The Umbrella Group can be defined as: a loose coalition of non–European Union-EU developed countries, typically including
Australia, Iceland, Japan, New Zealand, Norway, the Russian Federation, the Ukraine, and the United States.
31
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
EC also participates in the OECD Climate Change Expert Group (formerly the OECD/IEA
Annex I Expert Group), which is, globally, the key intergovernmental forum for technical
discussion of long-term climate change issues. It is an ad hoc group of government officials
from environment, energy, and foreign affairs ministries in countries that are listed in Annex I
to the UNFCCC and countries that have acceded to Annex I commitments.40 Canada’s
involvement in the Group includes participation in meetings and seminars, reviews of
analytic papers relevant to the ongoing UNFCCC negotiations, input into the Group’s work,
and presentations on relevant issues.
In addition, there is evidence of Canadian participation in a range of formal and informal
groups within the negotiations, such as contact groups and expert meetings hosted by the
UNFCCC. Canadian officials have also participated in a range of workshops and events
relevant to the negotiations, such as:



a March 2007 UNFCCC workshop on reducing emissions from deforestation in
developing countries;
a September 2007 UNFCCC workshop on adaptation planning and practices under
the Nairobi work program on impacts, vulnerability and adaptation to climate change;
and
a June 2008 workshop on methodological issues relating to reducing emissions from
deforestation and forest degradation in developing countries.
NRCan has also built strong partnerships with organizations such as the IEA, the World
Bank and the World Business Council for Sustainable Development, in an effort to advance
thinking and discussions in areas such as technology transfer, innovative financing and
intellectual property. As well, NRCan has provided expertise to the IPCC mitigation and
adaptation working groups, contributing to the development of the IPCC’s 4th Assessment
Report; and NRCan continues to work with the IPCC on the development of the 5th
Assessment Report.
In some instances, Canada has either chaired or hosted discussions. For example, DFAIT
convened a Chatham House Rule workshop on adaptation on behalf of the Government of
Canada in 2009. DFAIT also participates in the Clean Development Mechanism Executive
Board and Joint Implementation Steering Committee established pursuant to the Kyoto
Protocol. Additionally, NRCan has advanced Canadian objectives and supported strategic
direction by drafting text that resulted in all-party agreement on the Poznan Strategic
Program on Technology Transfer, a key outcome of COP 14 and a key technology policy
objective for Canada.
The evaluators also found evidence that Canada submitted required country-submissions to
the UNFCCC, including national communications (submitted in March 2007 and February
2010) and national inventory reports (submitted in 2008 and 2009).
B) Policy and research papers
The development of briefing notes, backgrounders, and proposed position papers is a core
activity undertaken under this program component. As such, there is extensive evidence of
40
Annex I countries include most OECD member states, and some countries from central and eastern Europe and the
Commonwealth of Independent States that are undergoing the transition to a market economy. These are the primary
countries participating in UNFCCC negotiations and national climate change policy development.
32
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
participating departments providing the Canadian delegation with input and advice on all key
elements of the negotiations for relevant meetings.
DFAIT provided the delegation with input and advice on adaptation, developing country
issues, non–Annex I national communications, and analytical work on the broader
implications of climate change regarding human rights. DFAIT provided a table listing 40
briefing notes and backgrounders prepared on adaptation issues, and a listing of almost 200
briefing notes for senior officials. Similarly, NRCan provided over 100 documents as
evidence of activity related to this program function, including: briefing notes and reports to
inform Canada’s position on key topics such as technology and trade, CCS and clean
technology; reports and analysis of key countries’ activities and negotiating positions; and
summary reports to brief NRCan senior management and executives on Canada’s
negotiating position and activities at UNFCCC meetings and other international fora.
Additionally, EC provided over 80 documents indicating that instructions and scenario notes
are prepared for the Canadian delegation in advance of UNFCCC and other international
meetings, and that summary reports are completed to document and report to EC senior
management on the delegation’s ability to achieve its objectives as well as highlight the key
results and outcomes from the meetings.
DFAIT, NRCan and EC have also provided ongoing advice on bilateral engagement efforts,
including how climate change affects bilateral foreign relations and foreign policy priorities,
particularly with G8+5 countries.
In addition, NRCan provided a submission to the UNFCCC review of technology
commitments under the Convention (Articles 4.2 (c) and 4.5) and two submissions on the
use of CCS as a Clean Development Mechanism project activity. These submissions explain
Canada’s experience with developing, deploying and transferring climate-friendly
technologies, providing recommendations and best practices that could be adapted to suit
the national circumstances of other countries and position CCS as a measure to reduce
global GHGs. NRCan also provides substantive advice on international climate change
actions, particularly on issues relating to clean energy technology (including technology
financing and intellectual property issues), adaptation practices, and forestry issues (i.e.,
deforestation in developing countries).
DFAIT, EC and NRCan have provided inputs and briefing notes for leader-level discussions
on climate change, such as those conducted under the G8, as well as other multilateral fora
such as the MEF. EC and DFAIT provide briefings materials for bilateral meetings
conducted by the departments’ ministers and senior officials.
C) Negotiations under the UNFCCC
There is strong evidence of instruction documents, speaking points and associated briefing
materials for major and inter-sessional international negotiations. In addition, the
Government of Canada has established clear negotiating positions on all key topics under
the UNFCCC (e.g., adaptation, deforestation, clean technology, etc.) as well as clear
principles and positions related to the overall shape and content of a new long-term
agreement on climate change.
Interdepartmental working groups have been established to share knowledge and analyze
information to inform Canada’s negotiation position at the UNFCCC and in other multilateral
fora. For example, as the lead for the interdepartmental Technology Policy and Analysis
Working Group, NRCan provided leadership on energy- and technology-related issues and
33
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
undertook policy analysis and research exercises, which directly supported the Chief
Negotiator and Ambassador for Climate Change, and the Government of Canada’s policy
development and positioning on international climate change. Similarly, DFAIT coordinates
the interdepartmental Working Group on Adaptation to facilitate analysis and develop
positions on adaptation issues.
In addition, there is evidence that negotiators prepared summary reports for all major
negotiations for use within the federal system, as well as analysis and associated work
plans related to subsequent activities.
D) Partnerships and processes outside the UNFCCC
The evaluators found evidence of active Canadian participation in a number of multilateral
meetings and partnerships outside of the UNFCCC. Canadian participation has focused on
advancing the key principles and elements of its position related to a new long-term
international agreement on climate change, and building support for these positions with
potential allies.
Examples of this participation include evidence of:




participation in the Major Economies Process (2007, 2008) aimed at advancing
emission reduction commitments by developing-country emitters;
sustained, high-level participation in the MEF (established in 2009), including
ministerial and ambassadorial participation in key meetings and consistent, engaged
participation by technical staff;
participation in G8 deliberations related to climate change (2007–2009), including
sustained efforts to ensure relevant statements reflect Canadian interests and
priorities; and
participation in a wide range of multilateral fora, including the Commonwealth Heads
of Government, Asia–Pacific Economic Cooperation (APEC), Canada–European
Union Summit, and Gleneagles Dialogue.
Additional evidence demonstrates that Canada has worked bilaterally with key emitting
countries such as China and Mexico to formalize working relationships and identify
opportunities for implementing new and clean technologies, in part through the Canada–
China Working Group for Climate Change and the Canada–Mexico Partnership.
Canada and the United States established the CED in February 2009 to enhance joint
collaboration on the development of clean energy science and technologies to reduce GHGs
and combat climate change. Three joint working groups under the initiative are working on
key issues, including: developing and deploying clean energy technologies, with a focus on
CCS; expanding clean energy research and development; and building a more efficient
electricity grid based on clean and renewable generation.41
As well, NRCan has played a leadership role in promoting the benefits of CCS. Canada
leverages knowledge gained through research and development as well as domestic
demonstration projects in order to internationally disseminate lessons learned, primarily via
the IEA, CSLF and Global Institute on Carbon Capture and Storage. NRCan policy and
technical experts engage in these multilateral fora to advance clean energy technology
objectives, through such means as chairing the CSLF Strategic Planning Committee and
41
The CED is being evaluated in the summer of 2010, and is outside the scope of this International Actions Theme evaluation.
34
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
representing Canada on the CSLF Capacity Building Governing Council. Canada’s
proportionately large share of demonstration projects also contributes to the G8 goal of
launching 20 CCS demonstration projects by 2010. Additionally, Canada (represented by
NRCan) is a founding member of the Global Institute on Carbon Capture and Storage.
NRCan has used its CCS technical and policy expertise to advance Canada’s objectives on
the treatment of CCS under the Clean Development Mechanism in the UNFCCC
negotiations. Moreover, participating in CCS-related fora has contributed to information
sharing and learning, advancing Canada’s interests in the development and deployment of
CCS.
E) Formal and informal consultations with domestic stakeholders
The evaluators found evidence of regular and sustained engagement with provinces and
territories, particularly in the year leading up to COP 15 in Copenhagen. For example, three
meetings (chaired by Canada’s Chief Negotiator and Ambassador for Climate Change) with
ADM- and DG-level officials from the provinces and territories occurred prior to COP15 as
were 11 conference calls and more than 30 electronic exchanges of documents and
relevant information.
In addition, one ministerial meeting was held prior to COP 15, with an additional session for
participating ministers held in Copenhagen. These meetings were supplemented with
bilateral ministerial meetings with Ministers of the Environment, and Premiers in some
cases, from each of the provinces and territories.
Provinces and territories were also invited to participate in the Canadian delegation to the
Copenhagen meeting, and were provided with instruction telexes and relevant briefing
materials. As well, the Canadian delegation offered two daily meetings for provinces and
territories in Copenhagen in addition to broader, daily stakeholder briefings.
Evidence was found of regular engagement with Aboriginal groups (e.g., representatives
from the Assembly of First Nations, Métis National Council, Inuit Circumpolar Council),
including three meetings, four conference calls and 10 electronic exchanges of documents
and relevant information in the lead-up to Copenhagen.
Evidence of a two-tiered approach to civil society engagement was found. Evaluators were
provided with a record of 15 engagements with environmental NGOs (e.g., the Pembina
Institute), industry associations, the Federation of Canadian Municipalities, and NGOs (e.g.,
UNICEF) between March and December 2009. Multiple methods were used, including
in-person interviews, teleconferences and video conferences.
In addition, the International Institute for Sustainable Development, with Government of
Canada funding, facilitated three public policy dialogue sessions on issues relevant to the
negotiations prior to COP 15. These sessions included information-sharing on key issues,
as well as dialogue with the Chief Negotiator and Ambassador for Climate Change on
issues of concern.
Rating: Achieved
35
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
5c International Financial and Other Obligations
The intended outputs have been achieved as a result of programs within the International
Actions Theme. However, there is some room for improvement with respect to the timeliness
of achievement (see B below).
The following outputs were expected as a result of International Financial and Other
Obligations:
A) Contributions to international bodies
EC and DFAIT fulfilled their obligations to provide assessed funding to the UNFCCC in the
four-year period of implementation of the International Actions Theme ($477,000 per year).
EC’s non-assessed G&C funds ($323,000 per year) were allocated to REEEP and M2M. In
addition, EC accessed other departmental funds to make payments to the UNFCCC ITL and
OECD Climate Change Expert Group. This demonstrates a modest level of support, and
moderate potential for non-assessed contributions to enhance Canada’s position abroad.
Although the funding level is modest, M2M demonstrated potential to leverage funds from
the United States and from private sectors in Canada and abroad. There was one project
implemented in the M2M program during this time period (in China), and two others are in
the planning stages (in Mexico).
There has not been an increase in the number of non-assessed EC G&C recipients.
However, the design of the funding program for M2M and REEEP did not plan for increased
recipients. The target recipients for EC G&C funds were strategically selected at the start of
the four-year period, with annual contributions to support Canada’s strategic interests. EC’s
contribution program was not designed to solicit new proposals from new recipients.
B) National Registry for carbon dioxide (CO2) emission units
EC did fulfill international obligations to establish a National Registry for GHGs. The Registry
was initialized with the ITL in June 2008. Canada was approximately one year late in
fulfilling this obligation, due to the procurement contracting process with Public Works and
Government Services Canada and the Treasury Board’s Common Look and Feel
requirements for Government of Canada websites. The Registry was connected “live” to the
ITL in February 2010. Canada’s obligations under the Kyoto Protocol do not include a
requirement for public accounts in the Registry. As such, this was funded under a separate
request.
Rating: Progress Made; Attention Needed
5d Asia–Pacific Partnership
The evaluation has found evidence that all the intended outputs for the APP as outlined in
the CAA Logic Model have been achieved, although there were significant delays in
achieving some outputs in the first two years (such as clean technology agreements under
the APP not being completed).42
The following outputs were expected as a result of the APP:
42
The reasons for the delay in achieving outputs are identified under Question 11.
36
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
A) Consultations with domestic stakeholders
Consultations with domestic stakeholders regarding Canada’s involvement in the APP were
held prior to Canada joining the APP in October 2007. Consultations were held with industry
groups such as the Industry Committee on Climate Change, Canadian Association of
Petroleum Producers, Canadian Gas Association, Ford Canada, Imperial Oil, Quebec
Business Council on the Environment, Cement Association of Canada, Aluminum
Association of Canada, Canadian Steel Producers Association, and associations
representing chemicals and electricity. Industry groups were largely in favour of Canada
joining the APP but sought to better understand the links between APP and industry
compliance considering obligations under CARA (i.e., whether the APP would be a type of
credit mechanism that would help companies meet their domestic obligations).
B) Partnerships and processes outside the UNFCCC
Creation of governance and organizational structures: Canada established APP operations,
including creation of an APP Secretariat to coordinate Canada’s APP activities at the
national and international levels, and a DG-level Interdepartmental Steering Committee (EC,
NRCan, IC and DFAIT) to oversee Canada’s involvement in the APP. The Minister of the
Environment was given overall responsibility for the APP, with the APP Secretariat housed
within EC’s International Affairs Branch.
Creation of working groups: The APP Secretariat completed a consultation process with
stakeholders, which led to the establishment of public/private domestic working groups for
all eight APP Task Forces. Task Force Working Groups are chaired by departmental task
force leads. Canada’s APP Secretariat engaged all Canadian industrial sectors covered by
the APP Task Forces. This engagement covered business and professional associations,
their members, and individual companies that expressed an interest in the APP, including,
for example, the Cement Association of Canada, the Aluminum Association of Canada, the
Canadian Steel Producers Association, the Net-Zero Energy Housing Coalition, and industry
associations representing renewable energy sectors (wind, solar, etc.). Private sector
participants in Canada’s APP activities have generally funded their own participation,
including overseas travel.
Evidence of clean technology: Canada did not fund any APP projects during the first two
years (2007–08 and 2008–09). However, Canada participated in and contributed to a
number of existing projects, including the Management of Perfluorocarbon Emissions
Project through the Aluminum Task Force, and the Indicators for Energy Saving project and
State-of-the-Art Clean Technology Handbook project through the Steel Task Force. Canada
announced the first round of approved projects in the summer of 2009, and the second
round of approved projects in December 2009. Canada has now provided approximately
$12 million for 28 “clean tech projects” that are currently being developed. Approximately
$870,000 remains to be allocated to the last round of projects in 2010–11.
Rating: Achieved
6. To what extent have immediate outcomes been achieved as a result of programs
within the International Actions Theme?
6a Particulate Matter Annex
37
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
The PM Annex is expected to achieve one immediate outcome: increased consideration of
domestic stakeholder interests in the development of Canada’s positions. As part of their
processes, the program components within the International Actions Theme consider
domestic interests and concerns in order to inform Canada’s positions for negotiations with
other countries. There is limited evidence to confirm that the PM Annex has achieved this
expected immediate outcome.
A) Increased consideration of domestic stakeholder interests
The evaluation was unable to confirm that domestic stakeholder interests have been
reflected in Canada’s positions at the negotiation sessions completed in 2007 and 2008. As
noted for the output on consultations with domestic stakeholders, the evaluation did not find
any evidence to indicate that public engagements or consultations about the PM Annex
were held with Canadians. Although program staff stated that CAA themes with a mandate
to address domestic activities (e.g., clean air regulations and clean energy) and other
national processes (e.g., CCME Canada-wide Standards for PM and Ozone) were used to
consult with and inform Canadians about the PM Annex, no evidence was found concerning
how these fora were used to collect information and develop Canada’s position.
Rating: Little Progress; Priority for Attention
6b International Partnerships and Negotiations
While some progress has been made in this area, the immediate outcomes associated with
this program component, discussed below, have not been fully achieved to date.
A) Increased consideration of domestic stakeholder interests
Interviewed officials stated that participating departments were generally aware of
stakeholder interests, with knowledge of their positions and main concerns regarding key
issues in the negotiations.
Evaluators heard from interviewees that there was good technical engagement with
provinces on issues related to Land Use, Land Use Change and Forestry (LULUCF), and
that perspectives provided by provinces were incorporated into the ultimate position taken
by Canada on this issue. In addition, there is evidence that, in Copenhagen, provinces and
territories were granted greater access at more senior levels to negotiating instructions and
the negotiations themselves than at previous Conferences of the Parties.43 For example, the
federal Minister of the Environment invited all premiers to attend in Copenhagen, and
provincial and territorial officials were invited to participate fully in the Canadian delegation.
Ongoing consultations were also held with provincial and territorial representatives in the
lead-up to Copenhagen.
While these efforts were clearly appreciated by those involved, some provincial partners and
external observers felt that these activities represented opportunities for information sharing
rather than providing direct input into the development of Canadian negotiating positions. As
well, in interviews and focus group discussions, external observers suggested that
consultations with NGOs regarding UNFCCC negotiations were inadequate in the lead-up to
COP 15. These consultation mechanisms were perceived as focusing on informing the NGO
community about Canada’s negotiation position as described by the Government of
43
Provinces were also included as members of the delegation in Poznan.
38
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Canada, and did not provide a forum for exchanging information in support of developing a
position that reflects the issues, concerns and opinions of the NGO community. This
evidence suggests that this expected outcome was not fully achieved.
B) Canada’s positions are advanced
With respect to the advancement of Canadian interests within the climate change
negotiations and other relevant international and multilateral fora, the evaluators found
evidence of submissions made to the UNFCCC and key subsidiary bodies (including the
AWG-KP and AWG-LCA) on issues of strategic concern to Canada. For example,
submissions to the AWG-LCA included:




inputs to the negotiating text for consideration at the 6th Session of the AWG-LCA
(submitted April 2009);
ideas and proposals on paragraph 1 of the Bali Action Plan (December 2008);
views regarding the work program for the AWG-LCA (March 2008); and
several joint submissions with Australia, Belarus, the European Community and its
member states, Iceland, Japan, New Zealand, Norway, the Russian Federation,
Switzerland and Ukraine on information relating to possible quantified emissions
limitation and reduction objectives.
Submissions to the AWG-KP included:


information and data on the mitigation potential of policies, measures and
technologies (August 2007) and CCS; and
a series of 2008 and 2009 submissions on the treatment of LULUCF, including
information on options for LULUCF accounting, and Reducing Emissions from
Deforestation and Forest Degradation in Developing Countries.
In many cases, these submissions were provided directly by Canada. In other cases, they
were submitted jointly with some of Canada’s allies, such as Australia and Japan. This
provides evidence of Canadian coordination to advance common positions with various
other parties to the negotiations, primarily through the Umbrella Group. Bilateral exchanges
were also used for this purpose.
The evaluators also found evidence of Canada promoting its interests directly through
concerted involvement in UNFCCC discussions related to the post-2012 international
framework for action. This included direct interventions, as well as active participation in
relevant contact groups and other informal discussions on key issues such as technology
transfer, CCS, and more broadly on the scope and breadth of a new long-term agreement to
address climate change. For example, Canada outlined its strategic interests through its
National Statement at COP 15. Minister Prentice stated, in part:
We have come to Copenhagen to secure a fair, effective and comprehensive climate
change agreement. We need an agreement that will put us on a path toward
ambitious reductions in greenhouse gases and sustainable, low-carbon economic
growth. An agreement that will ensure a growing supply of clean, affordable energy
for all countries. An agreement that brings countries together to address our shared
global economic and environmental challenges.…
39
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Canada’s broad based actions to address climate change take into account our large
diverse land mass, our growing population and the importance of our energy sector
for meeting global demand. Our approach also reflects the strong economic ties
between Canada and our neighbour, the United States, in the need to ensure our
actions are aligned with our continental partners…
A new global agreement should consist of a single, comprehensive undertaking that
includes measurable, reportable and verifiable commitments and actions covering
the vast majority of greenhouse gas emissions in developed and developing
countries. To be as fair and effective as possible, a new global agreement should
support mutual confidence, and encourage countries to assume increased ambition
over time. It needs to speed the development and deployment of clean, low-emitting
technology. And it needs to support enhanced global action to help the poorest and
most vulnerable countries.…44
According to interviewees, Canada has made a particular contribution (both in terms of
financial support and with respect to issues of substance) to several of the UNFCCC expert
groups, namely the EGTT and LEG. For example, NRCan has played a leadership role in
facilitating developing country participation, advancing innovative financing objectives, and
encouraging the role of the private sector to meet long-term financial needs to address
global climate change. For instance, as a result of the partnerships that NRCan has built
with key organizations such the World Bank and the World Business Council for Sustainable
Development, representatives of these organizations are now meeting with the EGTT and
negotiators, including Canada, to speak with private sector interests about advancing action
on climate change, particularly with respect to the development and deployment of clean
technologies.
The evaluators also found strong evidence of work to promote Canada’s interests at a range
of international meetings addressing the post-2012 climate change framework. For example,
federal officials participated in efforts to craft leaders’ statements for meetings of the G8,
Commonwealth Heads of Government, and MEF. The statements emerging from these fora
are consistent with Canadian interests as defined in federal negotiating instructions, and
specific Canadian positions are reflected in a number of these statements.
For example, the July 2008 G8 Statement on Climate Change and the Environment
confirms the significance of the IPCC’s 4th Assessment Report, and outlines a goal of
achieving “at least a 50% reduction of global emissions by 2050, recognizing that this global
challenge can only be met by a global response, in particular, by the contributions from all
major economies, consistent with the principle of common but differentiated responsibilities
and respective capabilities.” The statement also underscores that, “in order to ensure an
effective and ambitious global post-2012 climate regime, all major economies will need to
commit to meaningful mitigation actions to be bound in the international agreement to be
negotiated by the end of 2009.”45
Canadian officials were active in advancing positions through strategic bilateral
engagements at the leader, Ministerial, and officials levels. Canadian positions were
advanced by the Prime Minister in bilateral meetings with priority countries such as China
and Mexico, and were incorporated into or strengthened through key bilateral partnerships
44
45
From Canada’s National Statement at COP 15, December 18, 2009.
From G-8 Statement on Climate Change and Environment, paragraphs 23 and 24, July 2008.
40
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
such as the Canada–China Climate Change Working Group and the Canada–Mexico
Partnership (Environment and Forestry Working Group). Both of these bilateral partnerships
have resulted in agreed-upon work plans, and projects initiated through operations and
maintenance funding, APP partnerships, REEEP and/or M2M. Additionally, program staff
highlighted that these initiatives increase awareness of trade opportunities in climate-friendly
technologies between Canada and other participating countries.
The Government also signed a Memorandum of Understanding on climate change with
China in December 2009 to strengthen Canada–China cooperation in areas such as energy
conservation and efficiency, renewable energy, CCS, methane recovery and utilization, and
sustainable land management.
The evaluators found evidence of bilateral discussions at different points in the negotiations
with a range of developed and developing countries, such as Germany, the United Kingdom,
Indonesia, South Africa, and Poland. Furthermore, through DFAIT, Canada’s network of
missions and embassies was used to inform and support these meetings.
With respect to the outcomes of COP 15, the Copenhagen Accord does reflect some of the
main themes of Canada’s position: it calls for a global solution toward significant long-term
emission reductions, including commitments by both developed and developing countries;
there was progress toward the concept of measurable, reportable and verifiable actions by
developing countries; and it reflects a commitment to financing, particularly for least
developed countries, in both the short- and long-term.
At the same time, not all of Canada’s stated interests are reflected within the Accord. For
example, there are still two tracks of discussion under way (i.e., under the Kyoto Protocol
and the UNFCCC), rather than the “single comprehensive undertaking” promoted by
Canadian officials.
The Accord was not adopted by the COP as a whole. Rather, the COP “took note” of the
Accord in a COP decision, which was developed by approximately 25 countries who served
as “friends of the COP President.” Without adoption by the COP, the Accord has no clear
formal or legally binding status, and the precise path forward toward refining and
operationalizing it remains unclear. It also opens the door to multiple interpretations on key
elements of the agreement by various countries and/or factions. Canada was not a member
of this group, although several of its allies and partners through the Umbrella Group
(including the United States) participated in its development.
It was difficult for the evaluators to ascribe the level of influence exerted by Canada in
securing the objectives of either COP 15 or preceding COPs. There was no consensus
among those interviewed on Canada’s role in securing the outcomes described, with
program staff and senior management describing the delegation as having a much greater
level of influence than that perceived by outside observers (including those interviewed and
those participating in the focus group).
A number of external interviewees noted that Canada’s domestic performance in reducing
emissions, and the fact that Canada has distanced itself from the Kyoto Protocol, have
affected our country’s credibility to date, and will make it difficult for Canada to ultimately
secure an agreement consistent with its interests. In particular, these observers believe that
Canada may not be well placed to build sufficient bridges with key developing countries in
order to secure their meaningful participation in and support for a post-2012 climate change
41
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
agreement. Some interviewees did suggest that Canada’s ability to engage positively with
other parties to the negotiations (particularly with developing countries) has improved during
the period being evaluated. They noted that Canada was criticized in the lead-up to and
during the Bali and Poznan COPs, due both to Canada’s domestic performance on climate
change and its position related to the Kyoto Protocol. However, they indicated that with the
creation of the Chief Negotiator and Ambassador position and the enhanced clarity that this
brought to the structure of and messaging from the delegation, Canada’s ability to engage
constructively on key issues was enhanced.
Rating: Progress Made; Attention Needed
6c International Financial and Other Obligations
The intended outcomes, outlined below, have been achieved as a result of International
Financial and Other Obligations.
A) Canada has complied with financial and other obligations
There is evidence in departmental reporting documents that indicate all assessed
contributions have been made, and, due to the strict UNFCCC oversight processes, there is
no evidence to indicate that these payments were not made in a timely fashion. Regarding
non-assessed contributions for additional UNFCCC initiatives, EC made annual membership
contributions to REEEP, and three project investments in M2M. In addition, EC made
contributions to the OECD Climate Change Expert Group.
Regarding technology partnerships evaluated within this program component, there is
evidence that Canada made all contributions in a timely manner to both technology
programs, i.e., M2M and REEEP. The evaluators found evidence of Canada’s participation
in the M2M steering committee, oil and gas subcommittee meetings, and partnership expo.
These activities have contributed to enhancing Canada’s profile internationally with respect
to clean technology. The M2M program has expanded Canada’s reach into new alliances
with the U.S. EPA, China, and Mexico (government and private sectors). REEEP
membership provides limited ability to influence how annual contributions are spent.
However, since M2M is project-based, Canada can strongly influence how M2M
contributions are spent, focusing spending on projects that have the greatest potential to
decrease environmental risks by reducing methane emissions. Both REEEP and M2M are
long-term programs, and their anticipated return on investment could not be determined at
the time of the evaluation.
B) Canada’s positions are advanced
Canada’s contribution to the UNFCCC assessed funding allows Canada to continue
participation in the negotiations (see Partnerships and Negotiations for details on advancing
Canada’s position in this respect).
Canada’s position of promoting clean technology partnerships to address air pollution issues
was advanced through M2M. Clean technologies are being used in partnership with
Mexican and Chinese industry sectors to address GHG emissions, and the M2M project
investments have established new partnerships with the U.S. EPA, which offers Canada a
broad platform to continue advancing its strategic technology interests.
Rating: Achieved
42
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
6d Asia–Pacific Partnership
The following outcomes were expected as a result of the APP:
A) Increased awareness of trade opportunities in climate-friendly technologies
Evidence indicates that Canadian APP Task Force–related industries have an overall
increased awareness of trade opportunities relating to climate-friendly technologies.
However, the initial portfolio of APP projects as described in the Executive Summary of Task
Force Action Plans46 is weighted toward activities such as sectoral assessments, capacity
building, identifying best practices, and technology research and demonstration, as opposed
to actual technology deployment.
To promote the activities of the APP and increase stakeholder awareness of trade
opportunities, APP representatives presented at a number of conferences/expos. Industry
representatives in some task force groups (such as cement) communicate and work closely
with individual industry stakeholders throughout Canada on APP activities.
Although there has been a small increase in awareness of trade-related opportunities with
respect to climate-friendly technologies, it is still too early to assess project success let alone
any climate change and air pollutant benefits because projects are in the process of being
implemented.
Rating: Progress Made; Attention Needed
7. Have there been any unintended (positive or negative) outcomes that can be
attributed to the program areas within the International Actions Theme? Were any
actions taken as a result?
7a Particulate Matter Annex
The evaluation could not confirm any positive or negative unintended outcomes.
Rating: N/A
7b International Partnerships and Negotiations
There have been no significant unintended outcomes associated with this program
component, but an opportunity exists to foster Canadian industry’s strengths in clean
technology export.
Program documents demonstrate that there are positive opportunities arising from the
international climate change negotiations held at COP 15. The approach anticipated under
the Copenhagen Accord, wherein large developing countries will be articulating their climate
change actions and technology requirements, will provide opportunities for Canadian
business to develop and promote clean technology internationally (e.g., opportunities
46
Asia–Pacific Partnership on Clean Development and Climate Executive Summary of Task Force Action Plans, October 31,
2006. Available at www.asiapacificpartnership.org/english/task_forces.aspx
43
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
beyond CCS, including technology deployment in areas such as enabling technologies for
smart-grid development in countries looking to improve energy efficiency, sophisticated
monitoring and control systems, etc.). Moreover, the negotiations are laying groundwork for
private sector involvement and investment, particularly in developing countries, by improving
the integrity of project development, addressing the needs for technologies for adaptation,
and updating guidance on issues such as innovative financing, technology needs
assessments and project development.
However, interviewees noted that declining resources can affect a department’s ability to
maintain an appropriate level of human resources to develop the policy required to advance
international commitments. There was a marked increase in international attention on
climate change following COP 13 in Bali in 2007. Due to an associated increase in
international meetings and events (within and outside the realm of the UNFCCC) to which
delegation members had to travel, EC has had to supplement travel to these events through
A-Base dollars, and other departments such as NRCan have had to cancel participation in
events. The declining level of resources associated with this component (both for
departments and for related G&C funds—see tables 1 and 2) is not proportional to the
increased need for engagement in this area. For example, NRCan management noted the
particular difficulty with the dichotomy between declining funding levels over the program
period and the escalating demands on the international climate change file through UN
processes and other fora such as the MEF, G8, IEA and APEC. Interviewees noted that
NRCan funding expires in March 2011, yet the demands will continue over the long term
given the necessity to conclude and subsequently implement a comprehensive international
climate change agreement.
Rating: N/A
7c International Financial and Other Obligations
There was one unintended negative outcome of this program area. The lateness of
developing the national GHG registry and connecting it to the ITL had implications for at
least one company that purchased credits but was not able to trade. In addition, this delay
resulted in an investigation by the UNFCCC enforcement group into Canada’s compliance
with UN obligations. EC program management responsible for the ITL connection considers
this a very minor flaw in Canada’s otherwise solid reputation in fulfilling UNFCCC obligations
over the past decade.
Rating: N/A
7d Asia–Pacific Partnership
The evaluation could not confirm any positive or negative unintended outcomes as a result
of the APP.
Rating: N/A
44
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
8. Are there any external factors outside the International Actions Theme that
influence the success of its programs?
8a Particulate Matter Annex
The evaluation found evidence that external factors, both within and outside the program
component, have affected the component’s ability to design, negotiate and complete the PM
Annex.
Negotiations for the PM Annex were put on hold toward the end of 2008 as a result of
shifting domestic policy contexts in both countries. In the United States, a July 2008 court
decision to vacate (annul) the Clean Air Interstate Rule (CAIR), followed by a December
2008 Court of Appeals decision to remand the CAIR without vacatur,47 created significant
uncertainty for the U.S. EPA regarding specific emission reduction commitments that could
be included under the PM Annex. According to the evidence reviewed, the CAIR program
(which was designed to reduce millions of tons of smog- and particulate-forming pollution
from coal-fired power plants in 28 eastern states) was central to what the EPA would likely
have proposed as targets under the Annex. Around this time, the change in U.S.
administration48 led to a review of all existing environmental programs and their regulations.
Since the United States does not enter into international agreements without first having
specific regulations or legislation in place to ensure compliance, the United States made the
decision to not proceed with the PM Annex negotiations until the status of CAIR (or any
replacement to it) is resolved.
The evidence also suggests that at the same time, Canada began to consider alternatives to
the Turning the Corner plan, which, once implemented, would have provided similar clarity
regarding the types of emission reduction commitments and specific programs that could be
referenced under the PM Annex. Ongoing coordination with the United States, however, is
continuing under the broader bilateral Air Quality Agreement, and work to address
transboundary PM is currently occurring under other CAA themes and international fora
such as the Convention on Long-Range Transboundary Air Pollution and the Gothenburg
Protocol to Abate Acidification, Eutrophication and Ground-level Ozone.
As a result of the changing policy context in Canada and the United States, and the
significant turnover experienced in the Air Emissions Priorities Directorate, the evaluation
found that the program was not able to meet its commitments fully, and under-spent in 2007
and 2008 for the PM Annex.
Rating: N/A
8b International Partnerships and Negotiations
External factors influence Canada’s ability to achieve its objectives related to the post-2012
international framework for action on climate change. These include decision-making
processes under the UNFCCC and an associated lack of procedural rules. With respect to
decision-making processes, substantive decisions under the Convention must be made by
consensus that can enable a relatively small group of countries to block progress agreed to
47
Remand without vacatur is a mechanism by which a court remands back to an agency a decision in circumstances in which
the court believes the agency rationale is flawed, yet the court declines to vacate the agency decision.
48
U.S. elections occurred on November 4, 2008, and Barack Obama assumed the presidency on January 20, 2009.
45
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
by the majority of participants. This issue, and the aforementioned lack of procedural rules,
will require attention in order for Canada to achieve the ultimate outcomes for this theme.
Departments have indicated that they are working with international partners, through formal
and informal processes, to move forward.
Perhaps of most significance, the new U.S. Administration and efforts to strengthen ties
within North America have created an opportunity for Canada and the United States to
explore regional initiatives to address climate change. The creation of the CED in 2009 is an
example of these potential partnerships.
Rating: N/A
8c International Financial and Other Obligations
There was no evidence of external factors outside the Theme that influenced the success of
this component.
Rating: N/A
8d Asia–Pacific Partnership
A number of external factors have influenced the success of the APP program in achieving
its expected outcomes in a timely manner. The economic downturn starting in 2008 has led
to tightening of budgets for many stakeholders, affecting the attendance of representatives
at international task force meetings. Participation in project development for some industries
has also been reduced due to budget constraints. The high overhead for participation in the
APP, and the limited financial support from government, exacerbated the financial burden
on industry stakeholders.
Another external factor that affected the success of the APP was the uncertainty around the
domestic climate change agenda and associated GHG regulations. Industry indicated some
hesitation to become involved in international APP projects, because they looked to focus
resources on compliance with domestic GHG and air pollutant regulations first and foremost.
Staff turnover and ministerial changes (between 2008 and 2009) also led to delays and a
lack of continuity in the APP, particularly with respect to final project approvals and the
announcement of funding commitments. The first round of Canadian APP projects were not
approved for several months (despite their successful reviews), delaying their progress.
Rating: N/A
46
Audit and Evaluation Branch
4.6
Evaluation of the CAA International Actions Theme
Evaluation Issue: Performance – Demonstrated Efficiency and Economy
Issue 5: Demonstrated Efficiency
and Economy
9. Are there more economic and
efficient means of achieving
objectives under the International
Actions Theme?
10. How could the efficiency of
programs under the International
Actions Theme be improved?
11. To what extent have each of the
International Actions Theme
program areas been implemented,
or are they on track to being
implemented as planned and on
time?
Indicators

Opinions of program managers,
deliverers, partners and stakeholders on
the ability of program elements to
achieve intended results, compared to
alternative design/delivery approaches

Opinions of program managers,
deliverers and partners on whether
investments under the International
Actions Theme are a good use of public
funds and whether the cost of producing
outputs is as low as possible

Opinions of program managers and
deliverers on how the efficiency of
programs under the International
Actions Theme could be improved

Evidence that the International Actions
Theme is being implemented as
designed
Evidence that program objectives have /
have not been met within the
anticipated time frame and/or budget
Barriers to program implementation are
identified
Barriers to program implementation are
resolved
Evidence that the contribution program
implements funding agreements in a
timely and well-managed manner
Evidence that applications to the
contribution program are reviewed in a
timely manner
Opinions of program managers,
deliverers and partners
Evidence of decisions being made
Evidence that information, results and
decisions are being communicated to
the appropriate parties
Presence of populated performance
data systems, where appropriate
Evidence of collection of performance
data, as appropriate
Evidence of decisions being made
based on performance information





12. Is the management and
accountability structure for the
International Actions Theme in
place and functioning adequately to
achieve expected results?
13. Are appropriate performance
data being collected, captured and
safeguarded? If so, is this
information used to inform senior
management / decision makers?






Overall Rating
Progress Made;
Attention
Needed
47
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
SUMMARY:
The evaluation found that the international climate change and clean air agenda is driven by the
federal government’s positions and prescribed international requirements, which leave little flexibility
in implementing alternative approaches to enhance economy and efficiency. However, options exist
in some sub-areas (e.g., strategic allocation of G&C funds, location of the APP Secretariat). Options
have been implemented to improve effectiveness, notably the establishment of the Chief Negotiator
and Ambassador for Climate Change.
While there has been progress overall in implementing the Theme as planned, there have been
significant delays in implementing the APP over the first two years, and the PM Annex has not
proceeded as expected, due to delays in negotiations.
Integrated findings (applicable to all program components):
 An HMARF was developed for the entire Theme and is used for performance and financial
reporting at the corporate level. However, it has not achieved its other objectives of
supporting decision making in the International Actions Theme or providing useful
performance information to managers. In fact, the evaluation found that the International
Actions Theme is not managed or implemented as a “theme” as originally articulated.
Instead, individual program components are managed separately (with overall coordination of
climate change areas). The inclusion of the PM Annex in this theme is considered a misfit
with the other climate change program areas, and is more closely aligned with the domestic
CARA than this theme. Further illustrations that the “theme” is not an integrated entity
include: the intended DGMC is not operational (with program managers meeting only as
required to manage individual programs); and there is departmental (not Theme)
management of funds.

Performance data on the outputs generated from the International Actions Theme have been
reported in Horizontal Performance Reports for the first two years. Compilation of this
information is coordinated by the CAA–RMS with input from managers. Departments
indicated that: instructions to fill out templates are often unclear and with short deadlines;
there are challenges and risks associated with this type of performance reporting; and there
also seems to be a lack of understanding around activities/outputs/outcomes for these
policy-related programs. An assessment of this performance data49 resulted in a ranking for
the International Actions Theme of “basic to fair,” with the quality weakening by the second
year. The International Actions Theme performance data were assessed as failing to
consistently report against expectations, excessively focused on activities as opposed to
outputs and outcomes, and only recently reporting on progress or challenges.
Due to the delays in program implementation in some areas, and the fact that the HMARF and
performance data are not used by or useful to managers, the overall rating has been assessed as
“Progress Made; Attention Needed.”
9. Are there more economic and efficient means of achieving objectives under the
International Actions Theme?
9a Particulate Matter Annex
As previously stated in section 4.4, the evaluation confirms that EC is the most appropriate
federal department to complete activities to finalize the PM Annex. However, under the
existing CAA structure, the majority of Canada–U.S. Air Quality Agreement work is funded
by other sources (within CARA) to which the International Actions Theme makes a limited
contribution.
49
An assessment of performance data for the International Actions Theme is available in the Review of CAA Reporting
completed by Marbek in October 2009.
48
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Rating: Progress Made; Attention Needed
9b International Partnerships and Negotiations
The international climate change agenda is driven by federal government positions and
policy which, along with requirements in international frameworks and procedures, provide
little flexibility in the approach and alternatives that Canada can take in implementing its
related activities. As there are no clear alternatives for delivering the objectives of this
program component, the evaluators consider the current approach to be the most economic
and efficient means to achieve the program’s objectives. However, the evaluation noted that
there is growing concern both internally and externally that funding is not adequate to
resource international climate change activities. Interviewees from all participating
departments note that dedicated funds are required to maintain the momentum of
international negotiations, but, in some cases, negotiators were not able to attend
international meetings due to declining available funds (e.g., EGTT meetings for NRCan).
Rating: Achieved
9c International Financial and Other Obligations
There are no alternatives identified for assessed contributions. There are no alternatives
identified for other obligations such as the National Registry or ITL contributions. There were
two alternatives identified for REEEP. There were no alternatives identified for M2M.
For non-assessed contributions, internal and external interviewees commented that M2M is
seen as valuable and considered to be economic with efficient projects planned (i.e.,
projects are highly replicable with high GHG-reduction potential). REEEP, however, is
viewed as being slightly less economic, though it is difficult to determine the program value
for dollars spent in terms of improvements to air quality. The International Renewable
Energy Agency (IRENA)50 and IEA51 were identified by program managers as potential
alternatives to REEEP, but there is no evidence to suggest that either IRENA or IEA would
be more cost-effective means of achieving the objectives than REEEP. No decision had
been made at the time of this evaluation on contributing to these other initiatives instead.
Rating: Achieved
9d Asia–Pacific Partnership
The evaluation found that there could be more economic and efficient means of achieving
objectives under the APP, but the APP is still a good use of public funds overall.
50
IRENA was established in January 2009, with 142 states and the European Union as members to date. IRENA promotes the
adoption and sustainable use of all forms of renewable energy. Acting as the global voice for renewable energies, IRENA
facilitates access to all relevant renewable energy information, including technical data, economic data and renewable
resource potential data. IRENA aims to share experiences on best practices and lessons learned regarding policy frameworks,
capacity-building projects, available finance mechanisms, and renewable energy–related energy efficiency measures.
51
The IEA is an intergovernmental organization that acts as energy policy advisor to 28 member countries in their effort to
ensure reliable, affordable and clean energy for their citizens. The mandate of the IEA incorporates the “three Es” of balanced
energy policy–making: energy security, economic development and environmental protection. Current work focuses on climate
change policies, market reform, energy technology collaboration, and outreach to the rest of the world, especially major
consumers and producers of energy.
49
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
The APP provides a number of benefits as expressed by participating stakeholders, such as
allowing industry to collaborate with government on international climate change activities,
providing opportunities for the development of projects with international partners,
developing a common-level playing field between international competitors such as the
Chinese in the aluminum and cement sectors, and exchanging valuable clean-technology
expertise between member countries. These benefits suggest that the APP is a good use of
public funds.
The evaluation found that the funding allocated to the APP has been largely used to foster
bilateral and multilateral relationships, and is not necessarily driven by the most efficient
means of achieving other objectives such as GHG reductions or trade promotion. In order to
achieve any real reductions in GHGs or air pollutants (an objective of the federal
government), stakeholders suggest that Canada should focus on fewer task forces (or fewer
projects) and avoid spreading limited resources over too many smaller projects that are
perceived to have fewer benefits for GHG and air pollutant reduction.
The evaluation found that the APP Secretariat located at EC may not be working well in all
respects. Interviews with Task Force members support alternative approaches to managing
the APP. For example, some interviewees believed that relocating the Secretariat to NRCan
or developing a joint secretariat between NRCan and EC would be more efficient because
the APP’s focus on the development and deployment of clean technologies is more aligned
with the mandate and expertise of NRCan.
Rating: Progress Made; Attention Needed
10. How could the efficiency of programs under the International Actions Theme be
improved?
10a Particulate Matter Annex
Program staff commented that efficiencies could be gained if domestic and international air
quality activities were merged into a single funding source. It is believed that a single
funding structure would provide opportunities for streamlining air quality activities, sharing
information sources and developing partnerships, thus leading to a more coordinated and
comprehensive approach to addressing PM. It is also believed that this approach would
minimize planning and reporting requirements, and maximize the resources available for air
quality activities.
Rating: Progress Made; Attention Needed
10b International Partnerships and Negotiations
Progress has already been made in enhancing the efficiency of programs associated with
this theme component, and no specific additional measures have been identified.
The 2008 establishment of a Chief Negotiator and Ambassador for Climate Change, as well
as a streamlined delegation housed together at EC, reflects a more efficient, coherent and
coordinated approach to managing international partnerships and negotiations. This shift
50
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
has also enabled the delegation to provide a clear Canada-wide voice on international
climate change issues. NRCan’s International Environmental Policy Division increases
efficiency by acting as a focal point for horizontal coordination on behalf of that department.
This coordination relates to the overall policy framework on international climate change and
its components (mitigation, technology, financing, adaptation and forestry), providing a
strategic and holistic perspective on policy development and acting as a key contact with
other departments. In addition to maximizing efficiency, this approach has ensured an
integrated NRCan voice on international climate change issues.
Rating: Achieved
10c International Financial and Other Obligations
There were no specific programs identified as better suited to support Government of
Canada objectives for this component, nor were any specific instruments to improve
program efficiency identified. Canada’s participation in the international technology
partnerships within this component appear to be operating efficiently, although the
technology partnership programs funded within this component do not have specific
program accountability or management structures available. Rather, they are
contribution-based and beyond the control of EC. Other technology partnerships (e.g., on
CCS) that also advance Canadian objectives on international climate change are not
included within this program component.
Rating: Achieved
10d Asia–Pacific Partnership
The evaluation found that the efficiency of the APP program could be improved.
Suggestions include the following:




Information regarding project funding should be clear and provided in a timely
fashion so that industry can make well-informed decisions.
The project approval process could be improved—the call for project proposals could
be done in a timely fashion, and adequate time could be given for project reviews.
Also, more project-specific expertise and technical input should be provided in the
review of projects.
Increased communication and clarity should be provided regarding the mandate of
the APP and the direction of the APP post-2011 in order to provide reassurance to
stakeholders and investors.
Due to the high costs of international travel for Canadian representatives, the
number of international task force meetings should be reduced. Meetings should be
better organized in order to focus on the exchange of valuable expertise and
enhance collaboration between member countries. Project updates and other
less-interactive aspects could be covered in conference calls or web-based
meetings.
While progress has been made, there have been delays in implementing plans as
expected—particularly over the first two years of the APP program, during which time there
was significant under-spending: the full $4 million (G&C funds) allocated for 2007–08 was
51
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
re-profiled ($2 million to 2008–09, $1 million to 2009–10 and $1 million to 2010–11), while in
2008–09 only $818,629 was spent with $2.4 million lapsed. In 2009, $12 million was
committed to projects, with approximately $870,000 left for a third round of projects in
2010–11.
Rating: Progress Made; Attention Needed
11. To what extent have each of the International Actions Theme program areas been
implemented, or are they on track to being implemented as planned and on time?
11a Particulate Matter Annex
The evaluation found that the changing policy context in relation to addressing air pollution
in both Canada and the United States to advance PM Annex–related work has delayed
completion of the PM Annex and its required components.
As noted in the response to questions 5 and 6, some progress in achievement of outputs
and early outcomes has been made in developing a PM Annex. However, the majority of
these activities were completed between 2007 and 2008, with fewer activities in 2009 and
2010. Of the $550,000 annual allocation to develop the PM Annex, approximately 25% of
the resources were spent in 2007–08, with a significant increase of 72% spent in 2008–09.
Table 5: PM Annex Planned Spending vs. Actual Spending, 2007–08 to 2008–09
2007–08
2008-09
Planned Spending
Actual Spending
Planned Spending
Actual Spending
$550,000
$136,699
$550,000
$397,238
Although the evaluation could confirm that work plans for the period 2007–08 to 2009–10
exist, there was no systematic approach to reporting on which activities have or have not
been completed. Similarly, the evaluation did not find an overarching timeline or critical path
for completing the PM Annex.
Rating: Little Progress; Priority for Attention
11b International Partnerships and Negotiations
Negotiation and partnership activities have been consistently carried out pursuant to this
component of the International Actions Theme. However, overall progress toward a
post-2012 international climate change framework has been slow due to a variety of external
factors largely outside departmental control (see above), and it is unlikely that such a
framework will be in place within the time frame of this program component (i.e., 2011).
Interviews with members of the delegation highlight that an increase in resources will be
required to ensure that Canada can play an active role in finalizing a global long-term
agreement on climate change. These interviewees underlined that the existing resource
allocations are limited and do not meet the level of demand dictated by the international
climate change agenda.
Rating: Progress Made; Attention Needed
52
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
11c International Financial and Other Obligations
For assessed contributions, evidence indicated contributions were paid on time. For G&C
funds, there is no evidence to indicate whether EC’s G&C program implements funding
agreements in a timely and well-managed manner. Contributions are not application-based;
there are no strict requirements to meet specific timelines. However, the M2M contribution
program does appear to be moving slowly, with only one project implemented and two
projects planned in a four-year time frame. This was observed in the absence of having
accountability measures to evaluate given that the M2M is an international program
operating outside of EC’s control. A strategic framework for selecting organizations to
receive G&C contributions within the M2M partnership (e.g., project selection, timelines,
accountability, etc.) was not found.
Other obligations were completed, such as fees to the ITL and development of the National
Registry. Fee payments to the ITL were on time but development of the Registry was six
months late and it took some time after that to connect to the ITL.
Rating: Progress Made; Attention Needed
11d Asia–Pacific Partnership
Evidence indicates that the APP has been implemented as initially designed. However,
substantive evidence has been collected which indicates that program objectives have not
been met within the anticipated time frame. Delays in project approvals and funding
announcements appear to be the result of changes in senior levels of government, and
senior levels of government focused on higher priority issues. The delays have led to project
development being delayed, as well as:



a loss of interest by the private sector to engage in task force meetings and projects;
a timeline crunch for projects (projects must spend funding within the year); and
industry representatives placed in an awkward position with project partners.
No domestic barriers to program implementation were identified. Membership in the APP
was approved by senior levels of government, and $20 million was allocated to the APP for
implementation of the EC Secretariat ($4.6 million for operations and maintenance) and for
the funding of projects ($15.4 million G&C funds).
Rating: Progress Made; Attention Needed
12. Is the management and accountability structure for the International Actions
Theme in place and functioning adequately to achieve expected results?
12a Particulate Matter Annex
Program staff commented that the current reporting structure prescribed by the HMARF is
useful for financial tracking. However, this structure is limited in terms of its ability to
accurately reflect program performance and progress against planned activities and desired
53
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
outcomes and is not appropriate for resolving barriers (e.g., policy shifts and approaches to
address air emissions) or identifying alternative activities.
Furthermore, PM-related work in EC is tied to multiple funding requests, which program staff
argue has led to multiple planning and reporting requirements that affect the Department’s
ability to communicate a coherent and coordinated approach on how to plan, complete and
report on the results of PM activities.
Rating: Progress Made; Attention Needed
12b International Partnerships and Negotiations
Program staff consistently highlighted that the HMARF is being used primarily for financial
tracking rather than overall management of activities under the component. It was also
noted that expectations, deliverables, roles and responsibilities are not always clear.
Program staff interviewees commented that communication between the CAA–RMS and
other federal partners on roles, responsibilities and expectations could be improved to
ensure that connections between issues and departmental mandates are made and that
shared objectives are met.
Rating: Progress Made; Attention Needed
12c International Financial and Other Obligations
The management and accountability structure for assessed contributions is clear and
functioning adequately to achieve expected results.
Information was not identified to verify whether there is a management and accountability
structure for EC’s G&C funds for this program component (within this program component,
the G&C funds relate primarily to REEEP and M2M).
The management and accountability structure for other obligations, such as ITL fees, is
clear and functioning adequately to achieve expected results.
Within this component, DFAIT serves as the focal point for overseeing financial contributions
to the UNFCCC in order to meet the assessed funding requirements, and EC serves as
technical lead on the National Registry for emissions credits, managing the connection to
the ITL, participating on behalf of Canada in technology partnerships (e.g., REEEP), and
leading Canada’s technology trade initiatives through M2M.
The management and accountability structure for assessed contributions is clear and
primarily administrative: the UNFCCC Secretariat has oversight functions for this
accountability check. For other UN-related obligations, such as the National Registry and
associated ITL connection, the UN has a technical review committee that reviews technical
aspects of the registry design and an enforcement branch to follow up on anything that is
non-compliant with the provisions of the UNFCCC and Kyoto Protocol.
54
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
With respect to supplementary funding provided through EC’s G&C funds (M2M, REEEP), a
management and accountability structure was not identified. This may be because these
programs are international-contribution-based programs, and largely operate outside the UN
processes and beyond EC’s control. This is not seen as problematic in any significant way,
since it is a contribution program that operates similarly to the UNFCCC contributions of
assessed funding, which does not have a separate management and accountability
structure either. Both the G&C funds and assessed funding amounts within this component
are small, with project information publicly available on program websites.
Rating: Progress Made; Attention Needed
12d Asia–Pacific Partnership
Notwithstanding the suggestion of a potential relocation to NRCan (see Evaluation
question 9 for the APP), the APP Secretariat at EC is well-organized and functioning
effectively. Federal departments coordinate technical and policy work related to Canada’s
participation in the APP’s task forces, and have developed a process to evaluate and fund
projects that promote the development, diffusion and deployment of clean technologies.
However, expected outputs/outcomes from the APP have not been consistently understood
by all stakeholders. More clarity on priorities among the desired outcomes and the resulting
direction of the APP from EC’s APP Secretariat and senior levels of government would be
advantageous.
As well, higher-level debriefs from Policy Implementation Committee meetings are not
always provided, or are not provided in a timely manner to all Canadian stakeholders. This
information would be very useful for their successful participation in the APP.
Rating: Progress Made; Attention Needed
13. Is appropriate performance data being collected, captured and safeguarded? If so,
is this information used to inform senior management / decision makers?
13a Particulate Matter Annex
Performance data associated with this program component is reported in CAA Horizontal
Performance Reports and departmental Reports on Plans and Priorities, as well as in
reporting through the Canada–U.S. Air Quality Agreement. However, the source of this
performance data is not clear. The evaluation found neither evidence of specific
performance-measurement metrics for this component nor documentation (e.g., internal
quarterly reporting, annual reports, etc.) to verify that program staff regularly report on the
achievement of program activities in accordance with their work plans.
Rating: Little Progress; Priority for Attention
55
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
13b International Partnerships and Negotiations
Performance data (outputs and spending levels) associated with this program component
are reported in CAA Horizontal Performance Reports and departmental Reports on Plans
and Priorities.
However, performance measures are aggregated for reporting purposes, and are not used,
or seen as useful, by program managers. Managers commented that the performance
framework and Logic Model were too rigid, too time-consuming to complete, and not able to
adapt to the evolving policy/directional changes that have affected the program.
In addition, the existing HMARF structure is not reflective of all international climate change
work under way throughout the federal government. A full understanding of progress
associated with this component is impeded by the exclusion of NRCan’s Canadian Forestry
Service, which is engaged in several key areas under the UNFCCC, and the Canadian
International Development Agency, which can play an important role in bilateral partnerships
and understanding the needs of least-developing countries.
Rating: Progress Made; Attention Needed
13c International Financial and Other Obligations
Performance data for assessed contributions is collected by the UNFCCC and publicly
available. Senior managers are aware of its existence.
Performance data for supplementary funding through EC’s G&C funds to the M2M and
REEEP programs was not available, and it is uncertain whether it is being collected.
The performance data collected for assessed contributions is clear and primarily
administrative, overseen by the UNFCCC Secretariat. For other UN-related obligations (e.g.,
the National Registry and associated ITL connection), the UN has a technical review
committee to review registry design, and an enforcement branch to address noncompliance. All of these cross-checks, review reports and investigative documents are
publicly available on the UNFCCC website. This evaluation did not identify any evidence to
show that EC senior managers access and use this publicly available information to support
decision making within their obligations and technology programs in this theme.
With respect to supplementary funding provided through EC’s G&C funds (M2M, REEEP), a
management and accountability structure was not identified, and no program performance
data were made available upon request.
Rating: Progress Made; Attention Needed
13d Asia–Pacific Partnership
There is no evidence to suggest that performance data is being collected, captured and
safeguarded at this time.
56
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
The need for performance data to assess the success of the APP has been discussed at
APP meetings, but such data have not yet been developed. Performance indicators such as
the amount of GHG reductions per project have been suggested by APP stakeholders. The
absence of any performance indicators symptomizes the lack of an overall results
framework (noted in questions 2 and 9).
In 2006, an initial set of action plans were developed for each of the Task Forces, containing
initial priority activities for implementation by each of the Task Forces. Objectives, immediate
and medium-term specific actions, milestones, and potential projects were also included in
the action plans. However, no evidence was found to suggest that progress of the Task
Forces was compared against the goals and milestones outlined in the action plans.
Rating: Little Progress; Priority for Attention
4.7
Findings for DFAIT’s Class of Contributions in Support of International
Environmental Programs
As previously described in section 3, the evaluation team was tasked to examine DFAIT’s
Class of Contributions in Support of International Environmental Programs, and more
specifically how that department complements the Government of Canada’s approach to
achieving the outcomes intended for the International Partnerships and Negotiations
component. As this was a distinct request of the evaluation, the findings unique to this
contributions program are presented separately in this section.
The information presented below has been aligned with a select number of evaluation
indicators that were included in the evaluation framework (see Annex 1 for details).
4.7.1 Overview
DFAIT is responsible for the disbursement of additional G&C funds under the International
Actions Theme. Under this theme, DFAIT received $840,000, of which $280,000 was
allocated in 2007–08, $230,000 in 2008–09, $180,000 in 2009–10, and an expected
$150,000 in 2010–11.
Developed in April and September of 2007, the Terms and Conditions for a Class of
Contributions in Support of International Environmental Programs and the Accountability,
Risk and Audit Framework (ARAF) for a Class of Contributions in Support of International
Environmental Programs provide the architecture and broad guidance for identifying and
selecting recipients in accordance with Canada’s international environmental agenda.
The ARAF states that the key objective for payments under the Class of Contributions is:
“To promote and enhance Canada’s role in international environmental efforts by supporting
international environmental programs, activities and projects in support of Canadian foreign
policy objectives.”
Achievement of the key objective is anticipated to lead to the following intermediate to
long-term outcomes:

position Canada as an international leader in environmental protection;
57
Audit and Evaluation Branch


Evaluation of the CAA International Actions Theme
advance Canada’s foreign policy objectives in relation to sustainable development
and environmental protection; and
facilitate the advancement of governance on global environmental issues through
Canada’s support for multilateral environmental agreements and organizations as
well as institutions and NGOs.
58
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
4.7.2 Findings
Evaluation Issues (and related
questions):
Indicators
Overall Rating
Issue 1: Continued Need for the
Program

Evidence that the contribution
program supports international
objectives
Achieved

Evidence of the level of support and
impact of non-assessed contributions
on enhancing Canada’s position
abroad on its objectives
Achieved

Identification of opportunities for
collaboration (enhanced networking at
international meetings/conferences)
through the approval of non-assessed
contributions

Increase in the number of nonassessed contribution recipients

Evidence that the impact of nonassessed contributions is targeted
strategically and is timely to identify
potential stakeholders and
beneficiaries

Evidence that the reach has
expanded, and new networks have
been created and strategic alliances
formed, through the non-assessed
contributions

Evidence that non-assessed
contributions are for activities to
reduce harmful environmental risks

Evidence that the contribution
program shows a return on
investment to support international
clean air objectives

Evidence that the contribution
program implements funding
agreements in a timely and
well-managed manner

Evidence that applications to the
contribution program are reviewed in
a timely manner
1. Are program components within the
International Actions Theme
connected with key international
climate change and air quality needs?
Issue 4: Achievement of Expected
Outcomes
5. To what extent have intended
outputs been achieved as a result of
programs within the International
Actions Theme?
6. To what extent have immediate
outcomes been achieved as a result
of programs within the International
Actions Theme?
Issue 5: Demonstrated Efficiency
and Economy
11. To what extent have each of the
International Actions Theme program
areas been implemented, or are they
on track to being implemented as
planned and on time?
Progress Made;
Attention
Needed
Achieved
SUMMARY:
The evaluation confirms that DFAIT’s G&C program is: relevant and in high demand by members of
the international climate change community; executed efficiently; used to better understand
59
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Evaluation Issues (and related
Indicators
Overall Rating
questions):
international climate change issues and communicate Canada’s position to organizations within and
outside of the UNFCCC; and appropriate for achieving desired outcomes.
The section below indicates the more detailed findings by the evaluation questions and
indicators selected as applicable in the evaluation framework (see Annex 1).
1. Are program components within the International Actions Theme connected with
key international climate change and air quality needs?
Indicator: Evidence that the contribution program supports international objectives
The evaluation confirmed that DFAIT has provided $690,000 in funding to ten projects from
five organizations between 2007–08 and 2009–10. These contributions were found to
provide funding to initiatives that align with the class contribution’s Terms and Conditions
and the key objective and anticipated (short to intermediate to long-term) outcomes
referenced in the ARAF. Program staff commented that these contributions have been
particularly valuable in advancing Canada’s understanding of adaptation and mitigation
issues in Least Developed Countries (LDCs) and Small Island Developing States (SIDS).
Table 6: DFAIT Grants and Contributions, 2007–08 to 2009–10
Voluntary Contributions: 2007–08
Support to the Central America and Caribbean
Global Climate Observing
component of the Technical Support Project for the
$50,000
System
Americas
Contribution to support the activities of the Center for
Center for Clean Air Policy
$50,000
Clean Air Policy’s Future Actions Dialogue
To support a workshop in which developing countries
UNFCCC–Consultative Group
that are part of the UNFCCC are assisted in
$50,000
of Experts Meeting
completing their reporting obligations
UNFCCC/Kyoto Protocol:
Contribution to support additional UNFCCC negotiating
$130,000
Supplementary Fund
sessions in 2008
Voluntary Contributions: 2008–09
Support for the Central American and Caribbean
Global Climate Observing
component of the Technical Support Project for the
$35,000
System
Americas
Contribution to support the activities of the Center for
Center for Clean Air Policy
$40,000
Clean Air Policy’s Future Actions Dialogue.
UNFCCC/ Kyoto Protocol
Support for ongoing activities and to the LEG
$125,000
Supplementary Fund
Organization for American
Assessing and Mitigating the Impacts of Climate
$30,000
Change on Coastal Infrastructure in the Caribbean
States–Caribbean Support
Voluntary Contributions: 2009–10
IPCC Secretariat
Support to the IPCC to assess the scientific, technical
and socio-economic information relevant for the
$50,000
understanding of the risk of climate change and
options for addressing it
UNFCCC LEG
To provide support to the program of work for the LEG,
progress adaptation, and the development of GHG
$130,000
inventories in developing countries
Total from 2007–10
$690,000
60
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Rating: Achieved
5. To what extent have intended outputs been achieved as a result of programs within
the International Actions Theme?
Indicator: Evidence of the level of support for and impact of non-assessed contributions on
enhancing Canada’s position abroad on its objectives
Program staff highlighted that Canada has contributed to international organizations (e.g.,
IPCC, Global Climate Observing System), with a mandate to conduct science in order to
help clarify and develop a shared understanding of the issues and challenges associated
with climate change that need to be discussed at the international negotiations. Thus,
contributions to these international organizations are seen to be valuable because they help
provide a sound base of evidence for developing Canada’s international positions.
Indicator: Identification of opportunities for collaboration (enhanced networking at
international meetings/conferences) through the approval of non-assessed contributions
DFAIT’s voluntary contributions consist of those that are supplementary to both assessed
and non-assessed contributions. Contributions flowing to the UNFCCC have been
particularly useful in contributing to the work completed by the LEG Expert Working Group,
of which Canada is one of three participating developed countries. As a result of providing
supplementary funding to this group, Canada is well-placed to maintain an ongoing working
relationship with developing countries and demonstrate leadership in the areas of adaptation
and mitigation. Moreover, DFAIT’s non-assessed contributions have allowed it to establish
ongoing partnerships and dialogue with organizations outside of the UNFCCC (e.g., the
Organization of American States, the Global Climate Observing System, etc). Program staff
indicated that this funding is particularly helpful for communicating Canada’s positions to a
broader audience as well as maintaining a presence in the international community in order
to address international climate change.
Indicator: Increase in the number of non-assessed contribution recipients
DFAIT has provided 10 contributions to different recipients: four in 2007–08, four in
2008–09, and two as of December 31, 2009 (fiscal year 2009–10). The evaluation was
unable to confirm that department’s planned recipients for fiscal year 2010–11.
Rating: Achieved
6. To what extent have immediate outcomes been achieved as a result of programs
within the International Actions Theme?
Indicator: Evidence that the impact of non-assessed contributions is targeted strategically
and is timely, in order to identify potential stakeholders and beneficiaries
The Terms and Conditions provide criteria for selecting eligible recipients, and indicate that
a selection committee within DFAIT’s Environment, Energy and Sustainable Development
Bureau will select eligible recipients that support the achievement of the class of
contributions’ anticipated results. The evaluation confirmed that although a formal selection
committee has yet to be established, the Bureau’s Climate Change and Energy Division
ensures that funds are awarded to recipients in accordance with recipient criteria outlined in
61
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
the Terms and Conditions. The evaluation could not identify a formal plan that demonstrates
alignment between Canada’s objectives or negotiating position and the projects funded
under the Class of Contributions in Support of International Environmental Programs.
However, the evaluation confirmed that the funding concentrates on enhancing Canada’s
understanding of global climate change issues such as mitigation and adaptation, and that
this enhanced understanding is used to advance Canada’s leadership in the LEG and
international negotiations.
Indicator: Evidence that the reach has expanded, and new networks have been created and
strategic alliances formed, through the non-assessed contributions
The evaluation notes that program staff believe the G&C program is valuable for
communicating Canada’s position to developing countries and international organizations,
and addressing specific topics (e.g., effects of climate change in SIDS).
Interviews with recipients revealed that the overall level of awareness of DFAIT’s program is
low in the international community, with several interviewees indicating they were surprised
that DFAIT was providing funding to advance global thinking on international climate
change. In some cases, interviewees commented that they would have assumed that
departments with a natural resource or environment mandate (e.g., EC, NRCan) would be
offering this type of funding as opposed to DFAIT.
Indicator: Evidence that non-assessed contributions are for activities to reduce harmful
environmental risks
Although the evaluation could not confirm that the results of a specific grant or contribution
have directly reduced harmful environmental risks, there is strong evidence to indicate that
the non-assessed contributions assist with developing a more fulsome understanding of the
adverse effects caused by climate change in key topic areas such as adaptation and GHG
emission reduction.
Indicator: Evidence that the contribution program shows a return on investment to support
international clean air objectives
The evaluation was unable to determine a link between the contribution program and
DFAIT’s international climate change objectives. Performance reporting that demonstrates
how projects funded by the contribution program are contributing to objectives was not
found.
However, interviews with program staff and recipients underlined that the program is
well-run and supports a global effort to further the level of understanding and thinking on
international climate change. Additionally, interviews with recipients confirm that program
funds are valuable for pooling resources to fund large-scale or ongoing programs dedicated
to addressing international climate change.
Rating: Progress Made; Attention Needed
11. To what extent have each of the International Actions Theme program areas been
implemented, or are they on track to being implemented as planned and on time?
62
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Indicator: Evidence that the contribution program implements funding agreements in a timely
and well-managed manner
Contribution recipients indicated that funding is distributed to organizations quickly.
Furthermore, recipients stated that DFAIT’s application and review process is neither rigid
nor time-consuming and that the rules, restrictions and oversight mechanisms are minimal
compared to processes and procedures required by other donors.
Indicator: Evidence that applications to the contribution program are reviewed in a timely
manner
Interviews with contribution recipients confirmed that the application and selection process is
efficient, and that when accepted, payments flow to the organizations quickly.
Rating: Achieved
63
Audit and Evaluation Branch
5
Evaluation of the CAA International Actions Theme
Conclusions
The conclusions for the CAA International Actions Theme by the five key evaluation issues
are presented below.
5.1
Relevance
Issue 1: Continued Need for the Program
1.
The program components and activities under the CAA International Actions
Theme are relevant and continue to be needed to achieve expected outcomes.
Findings from each program area indicate that the implemented activities reflect the
need for action on climate change and air quality based on scientific evidence. Program
components are designed to address climate change and air quality needs and continue
to be relevant. Canada’s long-term goals are in line with commitments by G8 leaders to
reduce global emissions by at least 50% by 2050.
Findings from each program component also indicate that the activities have adapted
appropriately to changes in federal government policy and priorities. However, federal
priorities and work in this area do not necessarily reflect provincial/territorial views on the
actions needed to address climate change and provincial/territorial perspectives are
critical in order for federal positions on climate change to be truly national in scope.
Thus, further work in engaging the provinces and territories is necessary to achieve
increased consideration of domestic stakeholders’ interests, particularly for the PM
Annex.
In addition, the evaluation has shown that the International Actions Theme’s contribution
programs have been implemented in support of the intended international objectives,
and continue to be needed to work toward the expected outcomes.
Issue 2: Alignment with Government Priorities
2.
Activities are in direct alignment with federal government priorities.
Each program component has demonstrated that the implemented activities have been
as a direct result of priorities articulated in federal Throne Speeches from 2006 to 2009,
as well as responding to the 1997 Canada–U.S. Joint Plan of Action on air quality, the
2007 Turning the Corner Plan, and Canada’s commitment to develop and deploy clean
technologies and provide economic benefits to Canadian industry. Actions are also
aligned with the broader Government priority to strengthen bilateral relationships
(especially with major developing-country emitters such as China) and promote closer
cooperation in North America (e.g., joint action on transboundary PM and alignment of
Canada–U.S. positions on climate change). Thus, activities are in direct alignment with
federal government priorities.
In addition, there remain opportunities to more closely align Canada’s industrial
strategies and activities with the approach emerging from the Copenhagen Accord
(which anticipates that large developing countries will articulate their climate change
64
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
actions and technology requirements), and thus take advantage of opportunities for
Canadian business. This would better align program activities with Canada’s federal
priority to develop and deploy clean technologies and provide economic benefits to
Canadian industry.
3.
Activities are aligned with the priorities of participating departments.
The evaluation found that the program components and activities are appropriately
aligned with the priorities of the participating federal departments, as articulated in their
Program Activity Architectures and Reports on Plans and Priorities. These include:
•
•
•
•
EC: direct linkage to the “climate change and clean air” program activity and “GHG
and air pollution reduction” priorities;
DFAIT: direct linkage to the “international policy and advice and integration” program
activity and the “U.S. and hemisphere” priority;
NRCan: direct linkage to “clean energy” program activity and priority;
IC: direct linkage to “economic development” program activity and priority to “foster
internationally competitive businesses and industries” priority.
Issue 3: Alignment with Federal Roles and Responsibilities
4.
Activities are aligned with federal roles and responsibilities.
The federal government has the appropriate lead role in this theme, as activities relate to
international actions to address transboundary/global environmental issues, which is an
area of federal jurisdiction. The evaluation also found that the federal roles are clear and
well-coordinated among the federal partners, and no duplication was found with other
similar program areas.
However, given that provinces and territories have a crucial role in implementing the
targets and timetables that Canada agrees to internationally, there is also a role for
them. While it was found that there had been more structured and integrated efforts to
involve provinces/territories after the Chief Negotiator position was established (i.e., in
2008 compared to 2006), particularly in climate change negotiations on the Copenhagen
Accord, they were not engaged in other components such as negotiating the PM Annex.
5.2
Performance
Issue 4: Achievement of Expected Outcomes
5.
With some delays, progress has been made in the achievement of outputs in
component areas.
The evaluation found the achievement of the following expected outputs:
•
•
•
•
assessed contributions have been delivered to international bodies;
the National Registry is in place;
negotiations on a new Accord have been carried out under the UNFCCC;
policies, negotiating positions and advice have been produced for climate change,
although only initial research and early negotiating positions have been completed
for air quality components;
65
Audit and Evaluation Branch
•
•
Evaluation of the CAA International Actions Theme
there has been participation in a variety of international bodies and initiatives outside
of the UNFCCC, and support has been provided to foster partnerships (e.g., a range
of initiatives, such as the EGTT, LEG and contribution programs, that have built
capacity, increased understanding in areas such as adaptation, advanced
discussions on key issues in negotiations, and advanced clean technologies); and
formal and informal information sessions have taken place with other levels of
government, Aboriginal groups and domestic stakeholders, to support climate
change activities.
However, the evaluators found significant delays in achieving certain activities and
outputs:
•
•
•
6.
the National Registry was approximately six months late;
many activities under the APP were not initiated in the first two years; and
the PM Annex work has not proceeded as planned.
Some progress has been made in the achievement of immediate outcomes.
The evaluation found the achievement of the following expected immediate outcomes:
•
•
•
•
•
Canada has complied with its financial and other obligations under international
climate change treaties and agreements.
Canada’s positions with respect to the post-2012 international climate change
framework have been clearly and consistently communicated within the context of
the UNFCCC, in a range of other multilateral and bilateral fora, and with strategic
allies such as members of the Umbrella Group. In addition, a number of these
Canadian positions are now reflected in the Copenhagen Accord, although key gaps
remain. However, it is difficult to attribute the advancement of these interests directly
to Canadian international activities under this theme, particularly given the multiple
factors involved in international negotiations.
There has been improved information sharing and more formal engagement
mechanisms with provinces and territories, but enhanced consultation activities have
not necessarily resulted in increased consideration of stakeholder interests in
Canada’s positions. For the PM Annex, this evaluation found no evidence that
stakeholders were consulted directly, and it was not clear how Canada’s position
was developed or how the views of domestic partners and stakeholders have had an
influence or have been meaningfully reflected.
There has been a small increase in the awareness of trade-related opportunities
related to climate-friendly technologies.
While bilateral and multilateral relationships have been advanced through voluntary
(i.e., non-assessed) contributions, and contributions have contributed to a shared
understanding of issues and advancing Canada’s position abroad, there is no
evidence yet of a return on investment in terms of GHG reductions.
It is too early to ascertain the degree to which the intermediate or final outcomes can be
achieved.
7.
No significant unintended outcomes were identified, but opportunities are
emerging from the UNFCCC process to enhance Canada’s export
opportunities and partnerships for clean technology.
66
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
No significant unintended outcomes (positive or negative) were found due to program
activities. However, significant opportunities are emerging from the UNFCCC process
that Canada can build upon to enhance its export opportunities and partnerships for
clean technology, although these linkages seem not yet to be explicitly made with
Canada’s industrial strategies.
8.
External factors have influenced success and delayed the achievement of
outcomes.
A number of significant external factors have influenced (and continue to influence) the
success of the program in achieving its outcomes fully and in a timely manner, including
the following:
•
•
•
•
•
The increasing prominence of climate change in the broad international arena, and
the associated proliferation of meetings within and outside the realm of the UNFCCC
(beyond expectations), is placing larger-than-planned demands on program
resources, particularly around the need for more and more frequent international
meetings and related program support.
Staff turnover and ministerial changes have led to delays and a lack of continuity on
some program files, such as the PM Annex and the APP.
Changes in Canadian and U.S. policy direction and positions on key issues have
affected the ability of Canada to exercise influence and achieve expected goals (e.g.,
the PM Annex not yet being achieved). The emergence of a new U.S. position and
engagement on climate change, and evolving positions of large developing country
emitters, has largely determined the outcome of international climate change
negotiations.
Domestic performance on climate change (and Canada’s distancing from the Kyoto
Protocol) has affected Canada’s credibility in international negotiations, particularly in
2006–2008.
At Copenhagen, the UNFCCC decision-making process (e.g., consensus required)
and associated lack of procedural rules created challenges and barriers to achieving
international climate change outcomes in line with federal interests.
Issue 5: Demonstrated Efficiency and Economy
9.
More economic and efficient means to achieve objectives are limited and have
not been the focus of program implementation.
The international climate change and clean air agenda is driven by federal government
positions and policy, which, along with prescribed requirements of international
frameworks and procedures, provide little flexibility in the approach and alternatives that
federal government departments can take in implementing its program component
activities.
That said, choices exist for Canada to use its voluntary contributions to international
bodies in the most economic and efficient manner. The evaluation found that these
contributions have been largely used to foster bilateral and multilateral relationships, and
that the choices made are not necessarily driven by the most efficient means of
achieving other objectives such as GHG reductions or trade promotion. There is no
67
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
overarching strategic framework in place to guide the economic allocation of
contributions, and the multiple goals of programs are not prioritized.
In addition, the efficiency of the APP program could be improved in terms of
communication with stakeholders and investors, and the project approval process. The
evaluation also found that the APP Secretariat may not be best-placed to take efficient
advantage of the federal expertise in clean energy (i.e., its location in EC as opposed to
leveraging NRCan’s expertise).
As well, the PM Annex is better-placed under CARA, because the majority of
Canada–U.S. Air Quality Agreement work is funded under this initiative and a single
funding source will streamline activities and provide a more streamlined and coordinated
approach to addressing PM.
10.
Some options to improve efficiency have been implemented.
The evaluation found two key initiatives that have been implemented to improve
efficiencies. First, the establishment of a Chief Negotiator and Ambassador for Climate
Change in the federal government was widely seen as highly beneficial and a key factor
in improving coherence and coordination on the climate change file. Second, the
establishment of the HMARF, which aims to enhance efficiencies in management across
the entire CAA, is useful for corporate and financial reporting on the Theme but has not
been applied in a meaningful way at the program level by managers.
11.
There has been progress in program implementation as planned, but there
have been significant delays in some areas.
While progress has been made, as noted in conclusions 5 and 6, there have been
delays in implementing plans as expected—particularly during the first two years, during
which time there was significant under-spending on the International Actions Theme.
Much of the under-spending was due to delays in allocating APP funds (some funds
were re-profiled), with some under-spending on the PM Annex as well, due to delays in
negotiations.
Conversely, there has been slight over-spending on the International Partnerships and
Negotiations component, with a higher than expected workload demand (including
increased meetings and travel) due to heightened global action on climate change and
the need to secure a post-2012 agreement. If international actions continue on the same
track, supplemental resources will be required in future years for this component area.
The evaluation found that the DFAIT contribution program was implemented in a timely
and strategic manner, with full allocation of funds each year to date.
12.
The HMARF is in place and facilitating Government-wide reporting of financial
performance, but is not yet supporting decision making at the program level.
The HMARF, and the supporting infrastructure of committees and the CAA–RMS, were
established to provide clarity on management, reporting and accountability for the CAA.
The International Actions Theme has a distinct Logic Model and performance
measurement strategy under this HMARF. It has provided performance reports for the
68
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
first two years (2007–08 and 2008–09) that are included in Canada’s Performance
Report.
The HMARF has achieved its objective of facilitating Government-wide reporting of
financial performance for the International Actions Theme. However, in the first few
years, it has not yet achieved its other objectives of supporting decision making in the
International Actions Theme or providing useful performance information (only the
financial component is used by managers).
In addition, the evaluation found that the International Actions Theme is not managed or
implemented as a “theme” as originally articulated. Instead, individual program
components are managed separately (with overall coordination of climate change
areas), with an operational DGMC that oversees activities beyond the management of
individual programs. The inclusion of the PM Annex in this theme is considered a misfit
with the other climate change program areas, and is more closely aligned with the
domestic CARA than this theme.
13.
Performance data are collected and reported, but not fully across all program
components, and are not used by managers.
While the evaluation found that performance data were used for reporting to senior
management, Treasury Board and Parliament, there was no evidence of decisions made
in the program areas based on this performance data. Managers emphasized that the
performance framework and Logic Model for the International Actions Theme were too
rigid, too time-consuming to complete, and not able to adapt to the evolving
policy/directional changes that have affected the programs. In addition, the performance
framework does not include all international climate change and air quality activities
under way in the Government of Canada (e.g., work by the Canadian Forestry Service
and Canadian International Development Agency, Agriculture and Agri-Food Canada),
providing challenges in fully assessing Canada’s performance.
6
Recommendations
The following recommendations were developed based on evaluation findings and
conclusions. These recommendations are directed to the responsible ADMs of the
International Affairs Branch at EC, the Global Issues Branch at DFAIT, the Energy Sector at
NRCan, the Industry Sector at IC and the Associate ADM of the Environmental Stewardship
Branch at Environment Canada. They were developed to continue improving and advancing
the CAA International Actions Theme.
Recommendation 1: EC, in consultation with other participating departments, should
review and implement options for enhancing coordination and management of
program components in order to improve the strategic direction and effectiveness
for the Theme overall.
Although there is some overall coordination across departments, each program component
within the International Actions Theme is essentially implemented and managed separately
by the department(s) responsible. This theme is not managed or implemented as a “theme”
as originally articulated. Therefore, areas for improving possible synergies and connections
69
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
between the goals of program components and departmental mandates should be explored,
and formal mechanisms should be put in place (e.g., regular senior-level meetings across
participating departments) to ensure that shared objectives for this theme and the CAA
overall are met.
Recommendation 2: EC should consider the costs and benefits of integrating PM
Annex program activities with CARA activities.
Streamlining all air quality activities together within the Department can lead to a more
coordinated and coherent approach to planning, completing and reporting on the results for
initiatives related to PM. Under the existing CAA structure, the majority of air quality
activities are funded by sources within CARA. Since PM Annex activities are more closely
aligned with the domestic objectives for CARA than to international climate change
activities, EC should explore the appropriateness of integrating PM Annex resources and
activities with those of CARA. This integration should facilitate the coordination of all work
related to PM at EC.
Recommendation 3: EC should improve the effectiveness of its engagement with
provinces, territories and other domestic stakeholders for the PM Annex, by
considering the most appropriate formal mechanisms for consultation that will
provide strategic direction for the future.
There was little evidence that the provinces, territories and other domestic stakeholders
were engaged as part of the PM Annex’s negotiations process. Given the role that these
groups play in implementing clean air and clean energy programs and technologies,
regional partners and other levels of governments should be engaged to enhance
opportunities and articulate region-specific issues and context to inform priorities and
negotiating positions for the Annex. Because the Annex experienced some delays in
implementation, it is imperative to develop effective processes that will enable the Annex to
move forward efficiently toward achieving its expected outcomes in the future. EC should
therefore review existing processes for consultations and consider formal mechanisms for
implementation of consultations that will provide strategic direction for the Annex’s
negotiating positions.
Recommendation 4: EC, DFAIT and NRCan should develop an overarching strategic
framework to guide the selection of international climate change voluntary
contributions to international bodies and programs.
Within the International Actions Theme, there are multiple and sometimes conflicting goals
assigned to Canada’s voluntary (non-assessed) contributions to international bodies and
programs, which makes it difficult to determine the cohesiveness of these contributions in
meeting broader thematic objectives. While some program components allocate funds using
specific selection criteria based on their specific objectives, some components allocate
funding based on opportunity rather than proactively. Therefore, an overarching framework
should be developed to guide the selection of the International Actions Theme’s voluntary
contribution projects. This framework should articulate the overarching strategic priorities,
criteria for selection, appropriate terms and conditions, and broad processes to review,
select, approve and monitor/oversee contribution projects.
This recommendation’s aim is to ensure that all funds are allocated to those initiatives that
can best achieve the Theme’s broad expected outcomes. This overarching strategic
70
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
framework should apply to all voluntary contributions in the Theme, to enable a more
integrated process for soliciting, reviewing and approving proposals based on common
expected outcomes and priorities (e.g., GHG reduction potential, trade-development
potential, or value in developing partnerships, or some combination of these or other
objectives). The intention is to have a Theme-level framework and a set of priorities that
govern all voluntary grants and contributions programs under the Theme. However, this
framework is intended to guide, not replace, the component-specific processes that are
already in place for program components.
To support international partnerships and negotiations, this contributions strategy should
also consider the goal of developing strategic alliances with developing countries,
particularly with LDCs and SIDS, to better understand the potential adverse effects
associated with climate change (e.g., adaptation in SIDS and LDCs), thereby building on
commitments to support the poorest and those most vulnerable to climate change.
Engagement with other federal departments not included in the International Actions Theme
(e.g., the Canadian International Development Agency) may be beneficial.
Recommendation 5: EC, NRCan, DFAIT and IC should consider how APP program
activities and outputs can be used to provide trade opportunities for the
development, diffusion and deployment of clean technologies.
The initial portfolio of APP projects focused less on technology deployment and more on
capacity building and technology research. However, opportunities for Canadian businesses
to develop and promote clean technologies internationally are emerging as large developing
countries articulate their climate change actions and technology requirements to reduce
GHG emissions. In order to maximize these opportunities (e.g., enabling technologies for
smart-grid development in countries looking to improve energy efficiency), federal
departments should explore how the International Actions Theme’s voluntary contributions
can be used to increase support for these emerging trade opportunities for Canadian
businesses.
Recommendation 6: EC, DFAIT, NRCan and IC should develop and implement
mechanisms that will ensure the ongoing collection of meaningful and accurate
performance measurement data.
Given that this evaluation occurred early in program implementation, the achievement of
immediate outcomes was assessed at this time. Ongoing performance measurement data
are required to assess progress toward achievement of intermediate and final outcomes in
subsequent evaluations. However, some program components under this theme do not
have a clear strategy for collecting ongoing performance information. As well, the
performance measurement data collected as part of the HMARF are not used by, or seen as
useful for, program managers. Therefore, EC, in consultation with participating departments
and working with CAA–RMS, should enhance the performance measurement strategy for
this theme, including the development of specific, measureable performance indicators. A
clear plan should then be developed for the ongoing collection of performance measurement
data, and mechanisms should be established in all participating departments to maintain the
accuracy and usefulness of these data for each of the program components under this
theme.
For example, performance indicators should be developed and a mechanism should be
implemented to collect and maintain these performance measurement data for APP-funded
projects. While it is too early to assess project success, there should be a system in place
71
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
that will capture outcomes for funded projects so that these project outcomes can be
assessed against established APP objectives in the future. Because of the dynamic nature
of the policy directions and context of some activities under the International Actions Theme,
it is challenging to develop accurate and useful ongoing performance indicators. Flexible
strategies for program-component-level performance measurement and reporting may be
considered to account for changing policy directions, but a clear plan should be developed
that describes how these data will be recorded and managed.
7
Management Response
This section outlines the management response to the evaluation recommendations. The
Assistant Deputy Ministers (ADMs) of the International Affairs Branch at Environment
Canada, the Global Issues Branch at the Department of Foreign Affairs and International
Trade (DFAIT), the Energy Sector at Natural Resources Canada (NRCan), the Industry
Sector at Industry Canada and the Associate Assistant Deputy Minister (AADM) of the
Environmental Stewardship Branch at Environment Canada, accept the evaluation and all of
its recommendations, and have provided a plan to implement the following management
actions in response to the evaluation recommendations.
Recommendation 1: Environment Canada, in consultation with other participating
departments, should review and implement options for enhancing coordination and
management of program components to improve the strategic direction and
effectiveness for the theme overall.
The ADM of International Affairs Branch at Environment Canada (EC) agrees with this
recommendation.
Environment Canada will work with the Department of Foreign Affairs and International
Trade (DFAIT), Natural Resources Canada (NRCan) and Industry Canada to review and
implement options for enhancing coordination and management of program components to
improve the strategic direction and effectiveness for the theme overall. Meetings of all
International Actions Theme Program Director Generals will be held every four months to
facilitate communication across departments at the management level, provide strategic
direction, discuss interconnections between programs, provide a forum for thematic issues
and to review level of achievement towards the International Actions Theme goals
Timeline
Deliverable(s)
Responsible Party
March 31, 2011
 Quarterly meetings of all
Director General, Climate
International Actions Theme
Change International
Program Directors General (first
Directorate, EC
meeting to be held in summer of
2010).
Recommendation 2: Environment Canada should consider the costs and benefits of
integrating PM Annex program activities with Clean Air Regulatory Agenda (CARA)
activities.
The ADM of the International Affairs Branch and the AADM of the Environmental
72
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Stewardship Branch agree with this recommendation
Particulate Matter (PM) Annex activities are more closely aligned to the domestic objectives
of the Clean Air Regulations Agenda (CARA) than to international climate change activities.
Both the CARA and PM Annex programs’ existing funding will expire March 31, 2011.
Senior management has considered the efficiencies gained from single-point reporting
instead of reporting to two funding programs as well as the strong alignment with CARArelated domestic goals and concluded that work related to the PM Annex would be more
effective and efficient if included in the CARA program. Initial consideration for future
funding of CARA is underway and includes the PM Annex. The PM Annex will begin to
officially integrate into the CARA program this fiscal year, while continuing to report through
the International Actions Program.
Timeline
Deliverable(s)
Responsible Party
March 31, 2011
 Integrate PM Annex
Director General, Strategic
activities into CARA
Priorities Directorate,
activities this year.
Environmental Stewardship
Branch, EC
Recommendation 3: Environment Canada should improve the effectiveness of their
engagement with provinces, territories and other domestic stakeholders for the PM
Annex by considering the most appropriate formal mechanisms for consultation that
will provide strategic direction for the future.
The AADM of Environmental Stewardship Branch agrees with this recommendation
The content of the PM Annex will be based on the respective domestic policies to address
air pollution in Canada and the U.S.
Provinces and territories were consulted formally on Canada’s proposed domestic approach
in 2007 through the Environmental Planning and Protection Committee (EPPC) of the
Canadian Council of Ministers of the Environment.
These consultations led to the formation of a multi-stakeholder process that included
members of provincial and territorial governments, industry and environmental groups. A
Canadian domestic approach to address air pollutants has been the subject of the work of
this group over the last two years.
We will be working to further elaborate our approach (as will the U.S) and when both
countries are in a position to engage in negotiations, an appropriate process for
engagement, using existing modalities where possible, will be set up.
There is also provincial representation on the subcommittees set up under the Canada-US
Air Quality Agreement that allows for direct participation in the process.
Timeline
Deliverable(s)
Responsible Party
TBD
 Domestic consultations
Director General, Strategic
take place prior to further
Priorities Directorate,
negotiations on the PM
Environmental Stewardship
Annex under an
Branch, EC
appropriate process.
Recommendation 4: Environment Canada, Foreign Affairs and International Trade
Canada, and Natural Resources Canada should develop an overarching strategic
73
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
framework to guide the selection of voluntary contributions to international bodies
and programs.
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT) and
Energy Sector of Natural Resources Canada (NRCan) agree with this recommendation
EC will, in close consultation with DFAIT and NRCan, work to create an overarching
strategic framework to guide the selection of voluntary (non-assessed) contributions to
international bodies and programs.
A strategic framework was developed to ensure that projects funded under the Asia Pacific
Partnership program met defined objectives. The experience of developing an APP strategic
framework will serve as a useful guide to development of the overarching framework.
The implementation of the framework will guide the allocation of contribution funds to
recipients that will best contribute to achieving the program objectives.
The framework will be developed in this final funded year to ensure the balance of funds is
distributed in the most effective manner.
Should a new program of international actions be funded beyond the current financial year
that includes similar voluntary contributions, this framework could be used as the basis for
managing the new funds.
Timeline
Deliverable(s)
Responsible Party
September 30, 2010.
 Develop a strategic framework Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director General, Energy
and Sustainable
Development Directorate,
DFAIT
Director General, Energy
Policy Branch, NRCan
March 31, 2011
 Test the framework using
contribution funds for fiscal
year 2010-11.
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director General, Energy
and Sustainable
Development Directorate,
DFAIT
Director General, Energy
74
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Policy Branch, NRCan
Recommendation 5: Environment Canada, NRCan, DFAIT and Industry Canada
should consider how program activities and outputs can be used to provide trade
opportunities for the development, diffusion and deployment of clean technologies
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy
Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry
Canada (IC) agree with this recommendation
The clean energy technology program has worked with Canadian private industry, many of
which are Small and Medium Sized Enterprises (SMEs), on baseline-setting and research
projects with some deployment of clean technology projects. SMEs account for 45% of
gross domestic product (GDP), much of the economy’s growth, 60% of all jobs in the
economy, and 75% of net employment growth52 As we become aware, through the
climate change negotiation sessions and other relevant international processes covered by
the program, of emerging trade opportunities related to climate change, we agree that we
need to increase SMEs awareness of these opportunities in order to increase deployment of
clean technology.
The Departments will create a plan for better communication of clean technology trade
opportunities to SMEs and will implement some activities prior to funding termination on
March 31, 2011. The plan can be used as designed if the program is continued or as a
foundation if the program is continued with changes.
In addition to ensuring information sharing with the Canadian SMEs, we can better inform
the international community of Canada’s developing clean technology products to increase
trade opportunities. For example, participation in the APP has resulted in sharing of
technology information and products among the APP countries thereby increasing world
awareness of Canada’s clean technology.
Established in 2009, and not included in the scope of this Evaluation, EC leads the CanadaU.S. Clean Energy Dialogue (CED), with support from NRCan and DFAIT. The CED aims to
enhance bilateral collaboration in advancing clean energy technology solutions to reduce
greenhouse gas emissions and address climate change. The focus areas of the CED
include: expanding clean energy research and development; advancing the development
and deployment of clean energy technologies, with a focus on carbon capture and storage;
and building a more efficient electricity grid based on clean and renewable generation. EC,
NRCan and DFAIT will continue to work together under the CED, and where appropriate,
engage with other relevant departments, such as IC, to advance the development and
deployment of clean energy technologies. A result of this program could be an increase in
Canadian business trade opportunities in clean technologies, although this is not a specific
goal of the program.
Beyond the International Actions Theme, NRCan leads a number of key partnerships with
organizations such as the International Energy Agency, the Climate Technology Initiative
and the Global Carbon Capture and Storage Institute (GCCSI), which contribute significantly
Public Works and Government Services Canada, “Importance of SMEs.” Accessed online on 21 May 2009 at
http://www.tpsgc-pwgsc.gc.ca/app-acq/pme-sme/importance-eng.html.
52
75
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
to advancing private sector engagement and should be further leveraged in order to
maximize support for Canadian business opportunities on clean technologies.
Also beyond the Theme, Industry Canada has been participating actively in the International
Partnership for Hydrogen & Fuel Cell in the Economy (IPHE) and working with Canadian
companies to build hydrogen and fuel cell industry worldwide. IC is also engaged in the
Steel Committee of the Organization for Economic Cooperation and Development (OECD)
and the North American Steel Trade Committee (NASTC). While neither the Steel
Committee of the OECD nor the NASTC is a forum that is meant to deal with climate change
issues, they are often raised by industry representatives and IC officers as being important.
These issues will no doubt affect the sector's competitiveness going forward because of
different environmental obligations for developed and developing countries
Timeline
October 31, 2010
Deliverable(s)
 NRCan, IC, DFAIT and
EC representatives will
develop an action plan to
increase the Canadian
SMEs’ awareness of
increased opportunities in
clean technology
Responsible Party
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director, Resource
Manufacturing and ValueAdded, Industry Sector, IC
Director, Partnerships,
Climate Change International
Directorate, EC
Director, International Affairs,
NRCan
Director, Climate Change
and Energy Division, DFAIT
March 31, 2011
 Implement activities
identified in the action
plan.
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director, Resource
Manufacturing and ValueAdded, Industry Sector, IC
Director, Partnerships,
Climate Change International
Directorate, EC
Director, International Affairs,
NRCan
76
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Director, Climate Change
and Energy Division, DFAIT
Recommendation 6: Environment Canada, DFAIT, NRCan and Industry Canada
should develop and implement mechanisms that will ensure the ongoing collection of
meaningful and accurate performance measurement data.
The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues
Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy
Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry
Canada (IC) agree with this recommendation
As mentioned in the Evaluation Report, gathering meaningful performance measurement
data for policy-related outcomes is extremely difficult and must be qualitative in nature rather
then quantitative. We recognize that we need better measurement of outcomes for the
policy-related components of the program and EC will, in close cooperation with DFAIT and
NRCan, enhance the performance measurement framework to be more flexible regarding
the policy-related components of the International Actions Theme and establish a method of
tracking performance for use in decision making.
The APP and Methane to Markets components of the program provide an opportunity to
track performance data against program goals (reductions in greenhouse gases, economic
benefits for Canadians and accelerating development of clean technology). On project
completion, the companies will be asked to report on the achievements of their projects
against the three priorities.
Timeline
September, 2010
Deliverable(s)
 Establish a system to
track performance of
policy-related activities
under the program
against qualitative
performance
measurement indicators.
Responsible Party
Lead:
Director General, Climate
Change International
Directorate, EC
Support:
Director General,
Environment, Energy and
Sustainable Development
Bureau, DFAIT;
Director General, Energy
Policy Branch, NRCan
February, 2011
 Establish a system to
report level of
achievement of meeting
the three priorities of the
APP program.
Director General, Climate
Change International
Directorate, EC
77
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
ANNEX 1 – Evaluation Framework
Question
What should be observed
Indicators
Sources/Methods
Evidence that International Actions
Theme programs adapt with the
emergence of relevant scientific
evidence on air quality and climate
change
Evidence that International Actions
Theme programs adapt to changes in
government policy and priorities
Evidence that the contribution program
supports international objectives*
Document or File
Review
Key Informant
Interviews
Extent to which strategic directions
within International Actions Theme
match federal government priorities
Document or File
Review
Key Informant
Interviews
Document or File
Review
Key Informant
Interviews
RELEVANCE
Issue 1: Continued Need for Program
1. Are program components
Activities under the International
within the International Actions
Actions Theme are connected with
Theme connected with key
key international climate change
international climate change
and air quality needs
and air quality needs?
Issue 2: Alignment with Government Priorities
2. Are program components
Alignment with federal government
within the International Actions
priorities
Theme aligned with federal
government priorities?
3. Are program components
Alignment with the departmental
within the International Actions
priorities of EC, DFAIT, NRCan,
Theme aligned with the
and IC
priorities of participating
departments?
Issue 3: Alignment with Federal Roles and Responsibilities
4. Is there a federal role and
Activities under the International
responsibility for the
Actions Theme are connected with
government of Canada to
demonstrated federal role and
deliver program components
responsibilities
within the International Actions
Theme?
Extent to which strategic directions
within International Actions Theme
match priorities of each of the
participating departments
Demonstrated evidence of federal role
and responsibility in the area of
international actions for climate
change and air quality
Absence/presence of similar programs
within the federal and/or provincial
governments
Extent to which similar programs
complement/duplicate one another
Document or File
Review
Key Informant
Interviews
78
Audit and Evaluation Branch
Question
Evaluation of the CAA International Actions Theme
What should be observed
Indicators
Sources/Methods
PERFORMANCE
Issue 4: Achievement of Expected Outcomes
5. To what extent have intended
Demonstrated achievement of the
outputs been achieved as a
following outputs:
result of programs within the
a. Contributions to international
International Actions Theme?
bodies
b. National Registry for CO2 emission
units
c. Formal and informal consultations
with domestic stakeholders
Evidence of payment of Canada’s
assessed contributions to UNFCCC
Evidence of participation in UNFCCC
expert groups, workshops and other
events
Evidence of the level of support and
impact of non-assessed contributions
on enhancing Canada’s position
abroad on its objectives*
Extent to which National Registry is
established and maintained
Number of consultations with
Environment NGOs prior to UNFCCC
sessional meetings
Number of consultations with industry
prior to UNFCCC sessional meetings
Number of consultations with federal,
provincial and Aboriginal
organizations prior to UNFCCC
sessional meetings
Evidence of domestic consultations
held prior to entry into APP
Identification of opportunities for
collaboration (enhanced networking
at international
meetings/conferences) through the
approval of non-assessed
contributions.*
Increase in the number of potential
non-assessed contribution
recipients.*
Document or File
Review
Key Informant
Interviews
79
Audit and Evaluation Branch
Question
Evaluation of the CAA International Actions Theme
What should be observed
d. Policy and research papers,
negotiating positions and advice
e. Negotiations under UNFCCC
f. Partnerships and Processes
outside UNFCCC
Indicators
Research reports, advice and policy
support provided to key negotiators
Evidence of facilitating research,
studies and communication of
national and international clean
issues
Evidence of decisions, conclusions,
summaries, statements and
workplans from UNFCCC meetings
Evidence of creation of appropriate
governance and organizational
structures to oversee APP activities
and objectives
Evidence of creation of APP taskforce
working groups
Evidence of clean technology through
related agreements under APP
Evidence of decisions, conclusions,
workplans, summaries, and
statements from multilateral meetings
and partnerships which support
Canadian positions
Sources/Methods
80
Audit and Evaluation Branch
Question
6. To what extent have
immediate outcomes been
achieved as a result of
programs within the
International Actions Theme?
Evaluation of the CAA International Actions Theme
What should be observed
Evidence of progress towards the
achievement of the following
intended immediate outcomes
Indicators
a. Canada has complied with its
financial and other obligations
under international climate treaties
and agreements
Demonstrated evidence that Canada’s
total annual assessed contribution to
UNFCCC has been paid in timely
fashion
Evidence of payment of supplementary
contributions for additional UNFCCC
initiatives
Submission of Initial Report of National
Registry for CO2 emissions under
the Kyoto Protocol
Evidence that Canada paid annual
membership fees to REEEP
Evidence of clean technology transfer
related agreement under M2M
Increased awareness of domestic
stakeholder interests when
developing Canada’s negotiating
positions on transboundary air
pollution and international climate
change
Evidence that the impact of nonassessed contributions is targeted
strategically and is timely to identify
potential stakeholders and
beneficiaries*
Evidence of increased awareness
among Canadian APP Task Forcerelated industries
Evidence that reach has expanded and
new networks created and strategic
alliances formed through the nonassessed contributions*
b. Increased consideration of
domestic stakeholders’ interests in
the development of Canada’s
negotiating positions
c. Increased awareness of trade
opportunities in climate-friendly
technologies to reduce
transboundary flows of air
pollutants or to address
international climate change
Sources/Methods
Document or File
Review
Key Informant
Interviews
81
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Question
What should be observed
d. Canada’s positions for a future
international climate change
agreement are advanced
7. Have there been any
unintended (positive or
negative) outcomes that can
be attributed to the program
areas within the International
Actions Theme? Were any
actions taken as a result of
these?
8. Are there any external factors
outside the International
Actions Theme that influence
the success of its programs?
Observed unintended outcomes can
be attributed to activities within
International Actions Theme
Where appropriate, actions were
taken to address unintended
impacts
Factors external to the International
Actions Theme that have
influenced the achievement of
results
Issue 5: Demonstrated Efficiency and Economy
9. Are there more economic and
Alternative design/delivery methods
efficient means of achieving
that are less costly/effort-intensive,
objectives under the
enable quicker achievement of
International Actions Theme?
results, and /or enable achievement
Indicators
Submissions are made to UNFCCC
detailing Canada’s views on the post2012 Climate Change Agreement.
Active participation at international
workshops and meetings detailing
Canada’s views on the post-2012
Climate Change Agreement.
Canada’s views are reflected in
decisions, conclusions, workplans,
summaries, and statements
Evidence that non-assessed
contributions are for activities to
reduce harmful environmental risks*
Evidence that the contribution program
shows a return on investment to
support international clean air
objectives*
Presence/absence of unintended
outcomes
Where appropriate, documented
management actions and/or lessons
learned from unintended outcomes
Sources/Methods
Evidence of factors outside the
International Actions Theme which
have influenced the achievement of
outputs and early outcomes
Where appropriate, documented
management actions to address the
influence of external factors
Document or File
Review
Key Informant
Interviews
Opinions of program managers,
deliverers, partners and stakeholders
on the ability of program elements to
achieve intended results, compared
Document or File
Review
Key Informant
Interviews
Document or File
Review
Key Informant
Interviews
82
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
Question
What should be observed
of better results
10. How could the efficiency of
programs under the
International Actions Theme be
improved?
11. To what extent have each of
the International Actions
Theme program areas been
implemented, or are they on
track to being implemented, as
planned and on time?
International Actions Theme activities
show opportunities for increases in
efficiency
Program implementation is
consistent with defined approach
Indicators
to alternative design/delivery
approaches
Opinions of program managers,
deliverers and partners on whether
investments under International
Actions Theme are a good use of
public funds and whether the cost of
producing outputs is as low as
possible
Opinions of program managers and
deliverers on how the efficiency of
programs under the International
Action Theme could be improved
Evidence that the International Actions
Theme is being implemented as
designed
Evidence that program objectives
have/have not been met within
anticipated timeframe and/or budget
Barriers to program implementation
are identified
Barriers to program implementation
are resolved
Evidence that the contribution program
implements funding agreements in a
timely and well-managed manner*
Evidence that applications to the
contribution program are reviewed in
a timely manner*
Sources/Methods
Document or File
Review
Key Informant
Interviews
Document or File
Review
Key Informant
Interviews

83
Audit and Evaluation Branch
Question
12. Is the management and
accountability structure for the
International Actions Theme in
place and functioning
adequately to achieve
expected results?
13. Is appropriate performance
data being collected, captured
and safeguarded? If so, is this
information used to inform
senior management/decision
makers?
Evaluation of the CAA International Actions Theme
What should be observed
Management structure for activities is
defined, implemented and
functioning effectively, including
accountabilities, responsibilities
and roles of all participating
departments
Expected outputs and outcomes are
clear and understood internally and
externally
Overlap/duplication of effort is
minimized and no gaps in
programming exist
Required performance data collection
systems are in place, where
appropriate
Performance data are collected
against outputs and is used to
inform decision-making
Indicators
Opinions of program managers,
deliverers and partners
Evidence of decisions being made
Evidence that information, results and
decisions are being communicated to
the appropriate parties
Sources/Methods
Document or File
Review
Key Informant
Interviews
Presence of populated performance
data systems, where appropriate
Evidence of collection of performance
data, as appropriate
Evidence of decisions being made
based on performance information
Document or File
Review
Key Informant
Interviews
84
Audit and Evaluation Branch
Evaluation of the CAA International Actions Theme
ANNEX 2 – Summary of Findings
Evaluation
Issue
Achieved
Progress
Made;
Attention
Needed
Little
Progress;
Priority for
Attention
Not
Applicable
Relevance:
Issue 1: Continued need for the program
1. Are program components within the
International Actions Theme
connected with key international
climate change and air quality needs?

Issue 2: Aligned to federal government priorities
2. Are program components within the
International Actions Theme aligned
with federal government priorities?

3. Are program components within the
International Actions Theme aligned
with the priorities of participating
departments?

Issue 3: Program consistent with federal roles and responsibilities
4. Is there a federal role and
responsibility for the government of
Canada to deliver program
components within the International
Actions Theme?

Performance:
Issue 4: Achievement of intended outcomes
5. To what extent have intended outputs
been achieved as a result of
programs within the International
Actions Theme?
6. To what extent have immediate
outcomes been achieved as a result
of programs within the International
Actions Theme?
7. Have there been any unintended
(positive or negative) outcomes that
can be attributed to the program
areas within the International Actions
Theme? Were any actions taken as a
result of these?
8. Are there any external factors outside
the International Actions Theme that
influence the success of its
programs?




85
Audit and Evaluation Branch
Evaluation
Issue
Evaluation of the CAA International Actions Theme
Achieved
Progress
Made;
Attention
Needed
Little
Progress;
Priority for
Attention
Not
Applicable
Issue 5: Demonstrated Efficiency and Economy
9. Are there more economic and
efficient means of achieving
objectives under the International
Actions Theme?
10. How could the efficiency of programs
under the International Actions
Theme be improved?
11. To what extent have each of the
International Actions Theme program
areas been implemented, or are they
on track to being implemented, as
planned and on time?
12. Is the management and
accountability structure for the
International Actions Theme in place
and functioning adequately to achieve
expected results?
13. Is appropriate performance data
being collected, captured and
safeguarded? If so, is this information
used to inform senior
management/decision makers?





86