Professionalising Policy:
Cost Opportunity Benefit
Risk Analysis
The COBRA
policy guide
Version 2
May 2011
Foreword
The Ministry for the Environment is at the forefront of some of the most challenging and
invigorating issues and challenges facing New Zealand today. The policies that we advise on
directly influence the prosperity of New Zealand. They have far-reaching economic,
environmental, social and cultural implications and will influence all New Zealanders for many
decades to come.
Providing good policy advice to the Government of the day requires the Ministry to draw on a
wide range of skills and disciplines. Policy analysts must interact with a range of stakeholders, be
informed across a wide variety of areas, be adept at managing processes, exercise good
judgement, and be able to contextualise issues within international, national, regional and subregional contexts. The issues involved can be complex and multifaceted and will often require
many different ways of thinking. All of this makes for an exciting and challenging role.
In 2010 a policy review of the Ministry for the Environment was undertaken. At the time, it was
noted that we need to have the right people in the right roles, as well as the right processes,
systems, frameworks and culture to support this. This was a fundamental review of the policy
function of the Ministry, and we are in a stronger position as a result.
The COBRA guide provides an approach for considering policy analysis. It is a key element of our
Strategic Direction in providing support to the Ministry’s policy teams, as well as other teams
within the Ministry. It is designed to be accessible and practical to users.
We have also introduced clearer and more transparent ways of measuring and assessing the
quality of policy advice produced by the Ministry. Although there is no set formula for producing
high-quality policy advice, I am absolutely convinced that unless the steps in the policy approach
are all considered, such advice will not be forthcoming. I also see that this guide is an essential tool
for all policy issues within the Ministry, whether operational, corporate, strategic or directly
advising the Minister. It will help us to be better and different.
Advice can take on many forms. It can be formal (as in a briefing note to Ministers or a memo to
Directors), informal (as in email) or verbal. Part of the excitement of working in a policy Ministry is
that one never stops learning, either as an individual or as a team. Providing policy advice is a craft
to be learnt, and one that you will continue to learn through working at the Ministry.
I would like to extend my personal thanks to those who have been involved in preparing this
material. Many individuals have contributed, both from within and outside the Ministry. Without
exception, time and effort have been generously granted. I commend the COBRA guide to you all.
Dr Paul Reynolds
Chief Executive, Ministry for the Environment
Contents
Executive summary
Section 1 – The COBRA guide
Section 2 – Applying the policy case
Appendices
6
6
6
7
Section 1 – The COBRA approach
9
A Introduction
Welcome to COBRA
Purpose of the Ministry’s COBRA guide
What does good policy advice look like?
Ensuring the Ministry produces high-quality policy advice
How to use the COBRA guide
10
10
10
11
12
12
B How to apply the COBRA approach
Method and judgement both have a role to play
What does the policy cycle look like?
Cost Opportunity Benefit Risk Analysis (COBRA)
Other frameworks provide different ways of thinking
Economy and environment principles are one way of thinking
Evidence and engagement are key inputs
Involve stakeholders wisely
Telling the story
Formal communication tools
COBRA and the regulatory impact assessment and statement
13
13
13
14
15
16
17
18
18
19
20
C Context to policy
Ministry for the Environment priorities
Political, external and historical drivers
Cultural values, working with Māori and the Treaty of Waitangi
21
21
21
21
D Tricks of the trade
Giving hard advice
Ensuring advice is fit for purpose
Dealing with a pre-determined viewpoint or policy solution
Handling changes in priorities
Dealing with success and failure
Dealing with difficult departments
Shutting down a process
Making judgements when there is no evidence
Dealing with orders from on high
Working with your manager
Giving feedback to peers
Dealing with technical information
Approaching engagement
Coping when it’s all too hard
23
23
23
23
24
24
24
25
25
25
26
26
27
27
27
Section 2 – Applying the policy cycle
29
How to apply the COBRA policy guide
Policy cogs
Key elements
30
30
30
Getting started
Planning
Tools and templates
Ensuring shared expectations with your manager
Project management
Planning stakeholder engagement
31
31
31
31
32
33
1 Identify the opportunities and issues
Planning
Process
34
34
34
4
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Analysis and questioning
35
2 Is government consideration warranted?
Planning
Process
Analysis and questioning
39
39
39
39
3 Define outcome, objectives and criteria
Planning
Process
Analysis and questioning
41
42
42
43
4 Construct options
Planning
Process
Analysis and questioning
45
45
45
46
5 Assess options and play out outcomes
Planning
Process
Analysis and questioning
48
48
48
49
6 Confront the critical choices
Planning
Process
Analysis and questioning
53
53
53
54
7 Recommend and tell the story
Planning
Process
Analysis and questioning
55
56
56
56
8 Implement (legislation and/or programme)
Planning
Process
Analysis and questioning
58
58
59
59
9 Monitor and evaluate
Planning
Process
Analysis and questioning
60
60
61
62
Appendix 1 – Glossary of key concepts
Policy concepts
Environmental concepts
Economic concepts
Māori and Treaty of Waitangi concepts
Social and ethical concepts
Institutional concepts
Risk and uncertainty concepts
64
64
66
67
70
73
73
73
Appendix 2 – Quality of policy advice
1 Customer focus
2 Context
3 Problems and opportunities
4 Analysis and argument
5 Risks
6 Consultation and collaboration
7 Options
8 Conclusions and recommendations
9 Presentation
74
74
74
74
74
75
75
75
76
76
Appendix 3 – Checklist
Things to consider
Where do I go for help?
77
77
77
WL 13527
5
Executive summary
This guide is an attempt at capturing a policy approach for the Ministry.
Section 1 – The
COBRA guide
A Introduction
Gives the purpose of the guide, what good policy advice looks like, how the
Ministry will measure it, and explains how the guide is set out.
B How to apply the COBRA approach
Explains the roles of method and judgement, looks at the nine phases of the
policy life cycle and sets out that the Ministry’s organising policy approach is
based on the Cost Opportunity Benefit Risk Analysis (COBRA) approach. This
section also explains how other frameworks, including the economy and
environment principles, provide different ways of thinking to apply to your
policy analysis. It also discusses the importance of communicating policy
analysis effectively in different situations.
C Context to policy
Discusses the environment priorities for the Ministry, the political, external,
and historical drivers that provide the context for policy, and talks about
working with Māori and the Treaty of Waitangi.
D Quality of policy advice
Sets out how the Ministry measures the quality of policy advice produced.
E Tricks of the trade
Discusses how to give hard advice, make advice fit for purpose, deal with
predetermined viewpoints, deal with changes in priority, and deal with
success and failure. It also examines dealing with difficult departments,
shutting down a process, making judgements where there is no evidence,
dealing with orders from on high, giving feedback to peers, what to do if
you’re not a technical expert, how to approach engagement, and what to do
if it’s all just too hard!
Section 2 – Applying the
policy case
How to apply the COBRA policy guide
Explains the approach used.
Getting started
Examines the tools, templates and skills required.
1 Identify the opportunities and issues
Explains how identifying opportunities and issues gives you a reason for doing
the work and a sense of direction and focus for evidence-gathering and later
stages of the policy process.
2 Is government consideration warranted?
Looks at whether government consideration is really warranted in the
circumstances and/or whether others should be involved in addressing
the issues.
3 Define outcome, objectives and criteria
Talks about setting the parameters and ‘rules’ by which you’ll make the
judgement about your preferred approach.
6
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
4 Construct options
Gets you started in identifying and developing alternative courses of action,
or alternative strategies of intervention to solve or mitigate the problem or
risk or explore the opportunity.
5 Assess options and play out outcomes
Discusses the assessment and ranking of available options, based on the ideal
outcome embodied in the criteria, while understanding the costs,
opportunities, benefits and risks.
6 Confront the critical choices
Discusses the need to confront the critical choices that have to be made in
making a final recommendation and decision on a preferred alternative.
7 Recommend and tell the story
Talks about providing a clear preferred recommendation to the Minister and
telling the story as a critical part of good policy analysis.
8 Implement (legislation and/or programme)
Looks at the implementation processes related to legislation, developing and
implementing national instruments and regulations, and developing and
administering delivery programmes.
9 Monitor and evaluate
Examines the effectiveness and efficiency of the programmes and policies,
discusses learning from that consideration and applying that learning to
improve the policies.
Appendices
Appendix 1 – Glossary of key concepts
Appendix 2 – Quality of policy advice
Appendix 3 – Checklist
WL 13527
7
Section 1 – The COBRA approach
A
Introduction
Welcome to COBRA
Please help us to make it better!
This guide is an attempt to capture a policy approach for the Ministry.
We’re keen to get your feedback on what’s good about it, what bits you don’t
understand or don’t find useful and what else you’d like to see in here.
If you have any ideas on what you’d like, please let your manager know, or
email us (cobra@mfe.govt.nz).
Purpose of the Ministry’s
COBRA guide
Key approaches and processes at your fingertips
The Ministry’s COBRA guide gives you the key concepts, approaches and tools
to work through a policy process to deliver high-quality policy advice.
The guide supports a policy process that:
• is iterative, revisiting assumptions, preliminary conclusions and analysis as
new information comes to light
• incorporates the economy and environment principles, as well as other
concepts and approaches relevant to the issue
• is underpinned by evidence and engagement throughout
• encourages you to work openly and collaboratively.
Cost Opportunity Benefit Risk Analysis (COBRA) at the heart of policy
COBRA is the organising approach that you should apply to the policy issues
the Ministry advises on.
To apply it, you will need to approach each issue through a range of other
policy approaches. Using different lenses helps you approach issues in a new
way and identify alternative options, opportunities and risks. It also reflects
the complexity of the issues the Ministry deals with, which cut across a range
of policy agendas and interest groups.
Underpinned by the Strategic Direction behaviours
The Ministry’s COBRA guide is underpinned by the behaviours set out in the
Strategic Direction:
• analyse
• engage
• learn
• collaborate
• validate.
You will find yourself using all these behaviours in developing policy in
the Ministry.
10
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
What does good policy
advice look like?
Grounded in the bigger picture
• Forward looking. Defines policy outcomes and takes a long-term view.
• Outward looking. Takes account of the strategic context and the national
and international situation. Learns from the experience of other countries
but designs policies tailored to the local context.
• Looks at issues beyond institutional boundaries and sees them in their
wider context. Does not isolate issues in pre-existing ‘silos’.
Customer focus
• Fit for purpose.
• Length and depth of advice is tailored to the circumstances, including the
nature of the Minister’s request and the time available for analysis.
Strong analytical foundation
• Continually pushes to test and understand the costs, opportunities,
benefits and risks and questions underlying assumptions.
• Approaches the issue with a clear analytical framework, robust reasoning
and logic.
• Innovative, flexible. Questions established ways of dealing with things and
encourages new and creative ideas. Does not assume that government
intervention is the automatic response to an issue.
• Considers risks, (in)direct effects and impacts, and builds implementation
and evaluation considerations into policy advice.
Evidence-based
• Advice is based on the best available information and evidence from a
wide range of sources.
• Is clear about where the uncertainties lie, how substantial they are and
what the uncertainties mean for the resulting advice, including any risks
they create.
Values engagement
• Engages with those who have experience of the issues or are affected by
the policy.
• Sees engagement and consultation as an opportunity for learning, testing
ideas, design of options and understanding effects and risks.
• Engagement is not seen as a compliance exercise.
Monitoring and evaluation
• Builds systematic evaluation of the policy’s effectiveness into the policymaking process.
WL 13527
11
Introduction, continued
Ensuring the Ministry
produces high-quality policy
advice
The Ministry is improving the way we measure the quality of the Ministry’s
policy advice. Throughout the year, we assess samples of our policy advice
against our nine quality assessment criteria. This assesses both the quality of
the analysis and the effectiveness of how it is communicated.
These criteria for policy advice are consistent with the Ministry’s COBRA
policy guide. Using the COBRA approach will help you produce high-quality
policy advice.
Policy advice is expected to:
1 focus on the needs of the audience and the decisions needed
2 provide appropriate context to explain the big picture
3 provide a clear problem definition that indicates the size and scope of
the problem, how current policy settings contribute and how changes
can lead to better outcomes
4 display a robust approach to analysis, based on evidence and logic
5 identify the risks of the problem, risks of change options, delivery risks
and mitigation strategies
6 display evidence of appropriate consultation and collaboration across
government and with affected parties
7 identify and evaluate a range of practical options for meeting the policy
objectives, and select a preferred option
8 provide action-oriented recommendations with realistic commitments
9 be conveyed in a well-structured briefing, Cabinet paper, or other suitable
form, in a correct format and free of errors.
These assessments will help us:
• learn – support continuous improvement, by giving us feedback on what
we are doing well and where we need to improve
• validate – provide an evidence base we can use to account for our
performance to external stakeholders.
For more detail on assessing the quality of policy advice, see Appendix 2, page
74.
How to use the COBRA
guide
This guide
The guide has two sections:
• Section 1: The COBRA approach
• Section 2: Applying the policy cycle.
Professionalising Policy Handbook
You should use this guide alongside the Professionalising Policy Handbook to
understand the behaviours and skills you need to develop and to improve
your craft as a policy specialist.
12
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
B
How to apply the COBRA approach
Method and judgement
both have a role to play
Policy analysis is a craft, rather than a science. Method is important, but so too
are your own judgement, reflection and experience. Don’t be afraid of being
creative.
As with any craft, when you begin working in policy, you need to learn and
apply the basics before moving onto more elaborate and complex projects.
This is the same as the way a musician needs to learn and practise their basic
chords and harmonies before they can become a headline act.
The policy cycle sets out the phases and processes that will help you approach
a policy issue in a coherent way and support you in delivering high-quality
advice with a real chance of successful implementation. It isn’t a substitute for
you testing your judgement with your manager, but it will help you as a guide.
What does the policy cycle
look like?
Nine phases are identified in the policy cycle
Although the phases are set out as a cycle, developing policy is an iterative
process, where you will need to look both forward and backward and
continuously challenge your earlier assumptions or conclusions as you become
aware of new information.
WL 13527
13
B How to apply the COBRA approach, continued
Cost Opportunity Benefit
Risk Analysis (COBRA)
COBRA is the Ministry’s organising policy approach
The Ministry has chosen to use COBRA: Cost Opportunity Benefit Risks
Analysis as an organising policy approach. This is slightly broader than a
conventional narrow cost-benefit analysis and includes the concepts of
opportunity and risk as well.
Costs and benefits
A cost-benefit analysis (CBA) is an economic technique designed, in its
simplest form, to add up the costs and the benefits of a particular action.
There are a range of techniques implicit in CBA, such as discounting
techniques for valuing costs and benefits over longer timeframes.
It is not always possible to monetise (express in monetary terms) particular
costs and benefits, especially where environmental issues exist. For example,
although there are techniques to estimate the value of wetlands so that the
cost of destruction of a wetland can be balanced against the value of
increased agricultural production, these techniques are difficult, timeconsuming and controversial.
Notwithstanding this, you should identify where costs and benefits fall and
how significant they may be, even if it is not possible to quantify all of these
costs and benefits fully.
Opportunity
Opportunity implies looking at issues positively, rather than considering issues
simply through a ‘problem lens’. Building an ‘opportunity lens’ into the
Ministry’s overall approach allows us to consider chances to improve matters.
Does a chance for advancement or progress exist, or can it be made to exist?
Risk
Risk is different from cost. Risk takes into account different probabilities of
events occurring and the effects of those events if they occur.
Low-probability events can have near-catastrophic or very high-impact
consequences (for example, the global financial crisis). This can be very
important in an environmental and economic context, as BP and TransOcean
have discovered.
There is vast literature on risk, there are many forms of risk and there are
many ways to quantify risk. Essentially risk refers to a situation where there is
potential for difference between the expected outcome and possible
outcomes. Risk has both an upside and a downside element, and both
are important to be aware of.
This guide provides you with two key tools to help frame your policy
approaches:
• ‘Section 2 – Applying the policy cycle’ starting on page 29 poses questions
for each stage of the policy cycle to encourage you to approach issues
using a range of approaches, including incorporation of the economy and
environment principles set out in the next subsection
• ‘Appendix 1 – Glossary of key concepts’ on page 64 provides you with
definitions of key concepts across a spectrum of different policy
approaches.
14
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Other frameworks provide
different ways of thinking
COBRA is the organising approach you should apply in the policy issues the
Ministry advises on. To use it, you will need to approach an issue through a
range of dimensions and alternative policy approaches.
In policy circles, the term ‘framework’ is often used. As with many words, it
has multiple meanings and the definition of framework in a policy sense is, to
some extent at least, in the eye of the beholder. To some, the term
‘framework’ means the way in which thoughts and arguments are organised,
whereas for others, it means the viewpoint from which issues are considered.
This guide uses the term ‘ways of thinking’ to describe different ways of
looking at issues throughout the policy cycle. There are multiple ways of
thinking about an issue including:
• economic
• legal
• scientific
• environmental
• political
• Treaty of Waitangi
• sociological
• philosophical.
Even within the different ways of thinking identified above, many variants or
schools of thought exist. There are, for example, significant differences
between neo-liberal economics, behavioural economics and environmental
economics, even though all are relevant to the Ministry’s work. Similarly,
contract law and tort law focus on different aspects of the law, and again,
both are relevant to the Ministry’s work. Just to confuse matters even further,
law and economics is a discipline or way of thinking in itself.
It is not possible for an analyst to be across all of the arguments and
considerations implicit within each of the ways of thinking, but this is an area
where you can continue to expand your knowledge through exposing yourself
to different ideas. In carrying out your policy analysis, it is important that you
are aware that others may not approach an issue from the same viewpoint as
you. Further, it is important to keep an open mind as other ways of thinking
are valid. The very fact that so many disciplines have emerged verifies that
there are many ways of looking at issues.
If you find that someone you’re working with is using big words like ‘if one
took a constructivist approach’ or ‘neo-liberal endogenous growth theory
would provide you with a different answer’, ask them to explain what they
mean. It’s the best way to understand and to learn how to use such big words
yourself!
Always remember to be clear about your own assumptions and starting
points, and recognise that it is never possible to be completely objective
about arguments.
WL 13527
15
B How to apply the COBRA approach, continued
Economy and environment
principles are one way of
thinking
The Natural Resources Sector Network (NRSN) has developed a set of
principles that have been signed off by the chief executives of the
participating agencies. These highlight the priority placed by the NRSN on
understanding the interface between the economy and the environment
within policy advice.
The principles are set out below. They provide a way of thinking natural
resource policy development to take into account different value sets and the
interface between the economy and environment.
The principles highlight key ideas, issues or questions to be considered in
natural resource policy development. They apply throughout the policy cycle.
The policy cycle set out in ‘Section 2 – Applying the policy cycle’ starting on
page 29 will help you apply these in your work.
Principle 1: Intertwined
A healthy environment, based on healthy functioning ecosystems, is integral
to meeting economic needs and aspirations.
Principle 2: Government’s role
Government has an essential role to play in creating the framework in which
resource scarcity and competing interests are managed and environmental
bottom lines are protected.
Principle 3: Clear goals
Multiple policy goals create complexity – tensions between the achievements
of these goals are inevitable.
Principle 4: Supporting good decision-making
Base analysis and decision-making on a strong evidence base alongside broad,
transparent, and participative processes that recognise the legitimacy of
competing interests.
Principle 5: Adaptive management
Natural resource management must be adaptive, reflecting the dynamic
nature of both the resources and the knowledge we have about them.
Principle 6: Designing a solution
Effective policy will involve a mixture of regulation, economic instruments,
and other forms of intervention.
16
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Evidence and engagement
are key inputs
Evidence informs your policy advice
Evidence-informed or evidence-based policy can be defined as high-quality
information informing decision-making.
A number of types of evidence can be obtained from many different sources.
Other contextual elements include the government’s agenda and political risk
assessment, and the ‘saleability’ of proposals. It’s important to remember
that evidence is only one factor that feeds into decision-making.
Good evidence in ‘evidence-based policy’ is impartial, relevant
and persuasive
• Evidence is impartial when it is not selectively gathered to support a
particular conclusion.
• Evidence is relevant when it concerns the problem directly or describes a
situation similar to the one you are analysing.
• Evidence is persuasive when it is targeted at the audience and is
presented so it supports the policy story and its conclusions.
Different types of evidence
•
•
•
•
•
Economic and social data and analysis.
Statistical and data analysis.
Comparative international data.
Scientific data, including biophysical and environmental indicators.
Information on policy evaluation processes and tools.
Often some evidence will be contradictory, so it is important to collect from a
range of sources and to be ready to use your judgement, especially where the
evidence base is incomplete. When you make assumptions due to incomplete
evidence, it is important to document the assumptions so others are aware of
the limitations of your evidence.
You should also think about reasonable responses to the limitations of your
evidence. For example, you may need to gather more information, or use a
precautionary or adaptive approach – one that will allow for a policy
intervention to be assessed and adapted in the light of new evidence.
Within the Ministry, the Information Directorate is a good place to start when
collecting evidence. The Information Directorate has access to a huge range
of data and is able to work with you to find the right information and present
it in a way that is convincing. They can also help you understand the
limitations of the data you use.
Don’t just go looking for a graph or table to add to your briefing. Keep your
eyes open for evidence from all sources:
• anecdotes
• the media
• think tanks
• international sources such as the OECD Committee Information Service
(OLIS)
• empiric information from a range of sources and theoretical arguments.
WL 13527
17
B How to apply the COBRA approach, continued
Engage with the Information Directorate early to ensure you are building an
impartial evidence base. Talk to them about what information is out there
and how it might be relevant to your work, and challenge them to present the
information in a way that is accessible and persuasive to the audience.
See the Information Strategy pages on the intranet for more guidance.
Involve stakeholders wisely
Involving stakeholders when you are developing policy is vital
• It helps you understand the issues of greatest importance.
• It improves the quality of your evidence base and the advice you provide.
• It improves the likelihood that your policy will be successfully
implemented.
Engagement with people who are affected by the Ministry for the
Environment’s work is a key focus of our Strategic Direction.
During engagement, you will come across both supporters and detractors.
They each have a legitimate and valid role and you need to make sure you
speak to both.
Supporters can be useful allies in ‘selling’ a policy idea more widely and in
contributing to its successful implementation.
Detractors can be useful critics. If you listen early and respect the views of
detractors you may be able to both improve your policy advice and be able to
move them into a more supportive position or, at the least, a position of
respectful disagreement.
Telling the story
Policy is about telling a story, as well as gathering evidence to provide advice.
Telling the story often happens towards the end of your policy process and is
represented in the Ministry’s policy cycle as a phase (see page 55). It can
happen on any major policy issue at all levels, with chief executives, Ministers,
Cabinet committees and/or stakeholders.
You should be thinking about the story before you come to this point of the
policy process for a number of reasons:
• You may not be the one telling the story, so you will also need to prepare
others (including senior managers) to tell the story effectively.
• Your story needs to be able to move across domains at rapid speed:
– problem definition – ‘Is this really a problem?’
– option identification – ‘Why don’t we do this?’
– implementation – ‘Can this be done?’ ‘Is it too expensive?’
– historical – ‘We tried this three years ago and failed; what is
different now?’
As with any story, your policy advice will improve with practice and revision,
so you need to be scripting your story and trying it out throughout your
policy process.
18
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Essentially, it means rehearsing arguments – in safe environments at first
(even in your head while in the shower) – to describe:
• the problem
• possible ways forward
• alternative options and considerations.
The policy cycle is designed to help you to tell the story well. With a good
story, the recommendations should be self-evident, and good analysts should
arrive at a similar set of recommendations given the same story.
Watch and observe those around you, including your manager and director,
and principal analysts. See how they go about telling the story. If you listen
carefully, you’ll notice the story evolving as they try it out with different
audiences and changing as the policy process progresses.
Formal communication
tools
Policy analysts must familiarise themselves with a range of formal
communication tools. The most prominent of these are briefing notes,
Cabinet papers and regulatory impact statements. The Ministry also uses a
weekly status report to seek decisions from the Minister.
Each of these has a set format and clear protocols for development and
sign-off. Mastering these formats and protocols will give you a
significant advantage.
The Executive Relations team can provide you with more information about
formats and processes.
WL 13527
19
B How to apply the COBRA approach, continued
COBRA and the regulatory
impact assessment and
statement
If you’ve followed the COBRA approach, you’ve basically covered off the
requirements of the regulatory impact assessment process. The regulatory
impact statement is therefore a tool to help you ‘recommend and tell the
story’.
The table below sets out how the requirements of the regulatory impact
statement draw from the analysis you will have carried out as part of your
COBRA approach.
20
Regulatory Impact Assessment
Policy Framework (Treasury RIA
Handbook)
Relevant section of COBRA guide
Description of the status quo
(explain the current situation)
Context
Explain nature and scale of problem
• Nature and size of the problem
(scope and magnitude)
• Quantify to the extent possible
the costs and benefits of the
problem
• Methodology including any gaps
and limitations for qualitative
evidence
• Outcomes in the absence of
further government intervention?
• Identify who is affected
• Identify the root cause of the
problem
• If legislation – design or
implementation, or both?
• Identify the opportunities and
issues
• Is government consideration
warranted?
Set out the policy objectives
Define outcomes, objectives and
criteria
Identify the range of feasible options
Construct options
Analyse the options
• Identify the full range of impacts
• Quantify and analyse the impacts
(to the extent possible)
• Identify on whom the costs and
benefits fall
• Assess the risks and opportunities
• Assess the Pros and cons
• Assess the policy packages and
options against evaluation criteria
• Determine preferred policy
package
• Assess options and play out
outcomes
• Confront critical choices
• Implementation and monitoring
• Implement (legislation and/or
programme)
• Monitor and evaluate
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
C
Context to policy
Your role as a policy analyst sits within a broader context including external,
political and organisational priorities and drivers. A selection of these is set
out in the outermost ring of the Ministry’s policy wheel on page 9.
You should be aware of these in developing your policy advice but your
analysis of the issues and evidence should be the main driver for your
recommendations.
Ministry for the
Environment priorities
In most circumstances, the issues you work on are likely to be driven by the
priorities set out in the Ministry’s Strategic Direction, Statement of Intent
(SOI), Outcomes Framework and/or Output Plan. If you cannot see how they
fit, you should discuss this with your manager.
Political, external and
historical drivers
Policy development does not occur in isolation from other contextual
settings. These settings are likely to play a very important role in decisionmaking, but your advice should not be primarily driven or constrained by
them.
These contextual settings might include:
• government priorities as set out in a coalition agreement, political
manifestos or political statements
• economic and fiscal environments
• social context, including cultural concerns
• the environmental context
• the international economic and geopolitical environment.
The history of a policy area is also important to help you understand why it
looks the way it does today.
Cultural values, working
with Māori and the Treaty
of Waitangi
Much of our policy work at the Ministry for the Environment involves Māori.
For Māori, natural resources are central to identity and economic
development. It is important that you are aware of this relationship and these
values in the policy development process. Incorporating cultural values
throughout policy thinking not only ensures that we meet our obligations
under the Treaty of Waitangi, but can also provide valuable opportunities to
enhance current thinking and initiatives.
Legislation often refers to the ‘principles’ of the Treaty. These are the highlevel obligations ‘distilled’ from the text of the Treaty itself.
The Ministry has produced an internal Treaty Checklist for Advisers that lists a
version of the principles and explains what they mean in practice for the
Ministry, which is available on the COBRA home page on the intranet. This
list is based on the Principles for Crown Action on the Treaty of Waitangi
(1989). These principles are based partly on the pronouncements of the Court
of Appeal in 1987.
WL 13527
21
C Context to policy, continued
Views on what the principles are vary, and the law is still evolving. There are
also differing views on how to implement the principles, and what they mean
for the Crown and Māori. This difficulty is furthered by the differences in
interpretation between the English and Māori versions of the Treaty.
However, some of the key principles for the Ministry for the Environment are:
• kāwanatanga (government)
the Crown’s right to govern
• rangatiratanga (self-determination)
the right for iwi to organise as iwi and control the resources they own
• reasonable cooperation
the requirement for government and iwi to accord each other reasonable
cooperation on issues of shared concern
• redress
the responsibility of government for the resolution of grievances and
reconciliation
22
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
D
Tricks of the trade
Developing policy is an art or a craft, not a science. Being aware of and
following the process is important, but will take you only so far. There are
inevitably some tricks to the trade. Listed below are some useful tips.
Giving hard advice
Hit me with your best shot
There is a tension between giving free and frank advice, and being politically
aware. This can be a difficult balance – and in some cases, there are no easy
answers. At times, being a policy adviser is a thankless task.
You do have a responsibility to provide the hard advice, and you should.
However, be very careful about the way it is presented. Do not labour the
point, Ministers will sometimes make decisions that policy analysts find
uncomfortable, so don’t burn your relationship capital unnecessarily.
If an issue has been raised clearly, and Ministers have not accepted the
advice, think very carefully before you raise the issue again. Part of your job is
also to explain government’s decisions and implement enthusiastically. If you
want to be a decision-maker, run for Parliament.
Ensuring advice is fit for
purpose
Greased lightning
Timeframes are often short – far too short to run a ‘normal’ policy process.
Alternatively, you may be asked for advice on a specific part of a broader
issue.
Use your judgement, and the judgement of others around you. Sometimes
time is genuinely short and a quick phone call, email or aide-memoire with
just one key message is just what the doctor has ordered.
Be responsive. Be aware of the 80/20 rule (aka the law of diminishing
returns). Your stock can rise significantly, along with your job satisfaction.
Dealing with a
pre-determined viewpoint
or policy solution
Sunday, bloody Sunday
Heard of a solution desperately in search of a problem? A nice quip, and in
some cases, very true. If, however, the Minister has prescribed a particular
solution, it is your job to consider it seriously, and to make the best of what
may appear to be a pretty poor approach.
Remember a few things.
• First, Ministers are Ministers (that is, they are the boss) and your job is to
serve them.
• Secondly, Ministers have different information from you, talk to different
people than you, and are subject to different pressures from you. Respect
this.
• Thirdly, good policy is policy that works – so what on the surface may
appear to be pragmatic and crude can in fact be highly effective.
• Finally, Ministers are human; sometimes you or others may question their
judgement. That’s what democracy is about.
WL 13527
23
D Tricks of the trade, continued
Seek to frame suggested approaches in a broader context, seek to offer
alternatives, discuss your concerns with other agencies, try to understand
what is driving your Minister, but don’t sweat the small stuff.
Ministers make decisions all the time, most of them good, some of them not
so good. Not so good ones often get reversed (or stopped through the
process). Get over it.
Handling changes in
priorities
Blowing in the wind
You are heading beautifully along a wonderful policy lane, smelling the spring
flowers when suddenly, from behind a rock, comes a big giant ugly policy
ogre who turns it into winter again. What to do?
By all means check in with others that the change in direction is real. But,
there may not be much you can do about it. Priorities change. Roll with the
punches. A rapid reaction to a change in direction is one of the features of a
good analyst.
Sometimes these changes can be quite subtle within a project. These can be
harder to discern but again, recognising small changes in direction (and even
being a part of moving the overall direction) is part of the game.
Dealing with success
and failure
Hit me, baby, one more time
Policy can be exciting. Coming up with an idea, discussing it with Ministers,
and seeing it manifest into positive outcomes can be truly exhilarating. It can
also be disheartening or frustrating, seeing good policy fail at the last hurdle
– or even the first.
Remember Kipling’s twin imposters Triumph and Disaster; treat them the
same.1 You should not seek (or hope) for your advice to be either accepted or
rejected in full. You should aspire to be in a situation where your advice is
respected – and to be able to tell high-quality policy stories.
Dealing with difficult
departments
You’re so vain
You are the policy adviser, and the Ministry is the lead agency. It may be
critical to listen and to consult – but consultation does not mean that you
stop thinking and consultation does not mean you have to agree to every
comment made in relation to the work.
Think about what people are telling you and keep an open mind, but back
your judgement. Think about why people are saying what they are saying.
Discuss issues with mentors, managers, and principal analysts. Be prepared
to explain why you are not accepting others’ views. Get some satisfaction.
1
Extract from If by Rudyard Kipling (1895), If: ‘If you can meet with Triumph and Disaster and treat those two imposters just the
same’.
24
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Shutting down a process
Stop right now
Someone has come up with an idea: you have considered it (with an open
mind) and concluded that it should get no further than the nearest Silent One
folder. What can you do – other than to tell them just to ‘Stop right now!’?
Well, ignoring it and hoping it goes away might work but probably isn’t super
sustainable. Test your thinking with others. Is the idea in question really such
a dog? Can something positive be drawn from the concept?
Are there other ways in which the idea can be formulated that have some
merit? Is there any underlying problem that could or should be considered,
even if the idea in question barks quite a lot, goes to the toilet beside a lamp
post, and chases uncomfortably fast-moving cars?
Sometimes it may not be possible to close down processes, but you can have
a go at it, normally by analysing the issue openly, and then talking it through
with those concerned, including your manager, in the first instance.
Making judgements when
there is no evidence
I still haven’t found what I’m looking for
You have looked for evidence and not found much. Sometimes that is just the
way it is. Don’t give up, don’t fret. There are always things that can be said
and judgements can always be made. Talk to colleagues and mentors by all
means, but look for parallels – here and offshore – and seek to make
judgements based on first principles. Think hard about what you know and
how to use that information.
Acknowledge, if this is the case, that the evidence base is weak and that there
may be some uncertainties. Consider taking a precautionary or adaptive
approach that will allow for a policy intervention to be assessed and adapted
in light of new evidence. Ultimately, your job is about helping others to make
the judgements and informing them of those critical choices.
Dealing with orders from
on high
Paint it, Black
In cruel and difficult times, analysts can be a couple of steps down the chain
from the high and mighty level that ideas originate from. Instructions may be
sparse, or – if the world is completely cruel – contradictory. What to do?
Well, you can paint it any colour you like – but black may not be best. You
need to be clear on what to do – and what the orders are. Find out
timeframes, context, urgency, and whether any actions are required. Try to
track down people who were at the relevant meeting. Triangulate. Ask others
what they might do with that incomplete information.
WL 13527
25
D Tricks of the trade, continued
More generally, there is an art to managing upwards. Think about an
organisation as being a series of overlapping layers. Managers and directors
are paid to help and typically have useful expertise and experience. They also
face pressures you cannot fully understand. Own your problems and issues,
but think about how they can help you – and how you can help the
organisation.
Working with your manager
Beast of burden
Well... Mmmmmmm. Critically important. Mmmmmmm... Does the term
‘providing support’ give a sufficient answer?
Managers come in all shapes and sizes, have different backgrounds, and bring
different attributes to the table. As always, search for and work with the
positives and provide constructive feedback on the things that aren’t working
for you. While you will have one manager and you should not seek to
undermine him or her, there is a management team and you should not
expect one single person to supply you with all the assistance you need. Work
with your manager, not against them.
There are some fundamentals, however. An analyst can reasonably expect a
manager to be clear on expectations, to assist with problems, to be available
to discuss issues with, to be consistent, to be as fair as possible and to back
up the team. However, the ultimate responsibility for your performance, and
your contribution to the organisation, rests with yourself. Good performers
will do well under any management, and the opposite is also true.
In return, your manager can expect you to behave like an adult, use your skills
and knowledge wisely, and communicate. Managers aren’t mind readers –
use one-to-one sessions to your advantage to provide feedback on what’s
working for you and what’s not. Treating your manager well will generally
result in you being treated well by them.
Giving feedback to peers
With a little help from my friends
So you have been asked to comment on a draft paper – but how do you go
about giving a little help to your friends?
Probably the first thing to do – if your friend sings out of key – is to
understand where they are up to and what part they are playing. Is it the start
of the process – with a little bit of ‘blue-sky thinking’ going on – or are they
right ‘up against the wire’, with very few degrees of freedom in terms of
movement?
This affects how quickly you should provide comment, but also gives a little
guidance as to the nature of the comments you should provide. If there is
little room to move, provide comments quickly and seek to work within their
scope for movement.
By all means, point out any broader questions that you consider are worthy of
consideration – but don’t harp on about them, especially if previous advice
has been provided (and rejected). Make the relevant manager aware if you
have significant concerns with implicit directions.
26
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Never rewrite someone’s document for them; you are helping the process,
not driving it!
Dealing with technical
information
Mamma mia
You can never be a technical expert in all areas, and will have to work in areas
outside your expertise. It’s so easy to get lost – and mystified – by the socalled experts.
You are, however, an expert in public policy. Much of the art of dealing with
this situation is in asking the right questions and thinking about the COBRA
approach and relevant principles.
Are there experiences from elsewhere you can draw on? Listen hard and talk
to ‘tame’ people who understand the subject matter. While you need to
know something of the relevant policy area to be effective, with care and
thought you can pick up enough information along the way. Consider it a
challenge!
Approaching engagement
You took the words right out of my mouth
You are happily drifting through the policy process (on a hot summer night)
when some nasty gremlin asks you for a copy of your engagement plan.
Naturally, you note that it is all well in hand, before retreating into a quiet
corner in a mild panic. What to do?
Well, the Ministry has significant expertise in running engagement exercises,
so consult with those who have ‘been there and done that’. Talk to the
Communications Directorate, your manager, and anyone else handy with
some experience in the matter.
The key thing to consider is the purpose of the engagement exercise.
• Is it primarily to raise awareness of a particular initiative, to solicit
feedback on a set of proposals, or to fulfil a statutory obligation?
• Is it open in nature or limited in scope?
As well as thinking about the major relevant stakeholders, think also about
how the communication will be received. The person receiving the message is
more important than the sender. Think about what risks are implicit, the
difficult issues that may arise (and how they can be handled), and the ways of
managing more challenging stakeholders.
Most of all, enjoy the process and learn from it. Engagement processes are
great opportunities. Much can be gained if the processes are run well. Attack
it with a positive mind set.
Coping when it’s all
too hard
WL 13527
Stuck in the middle with you
You have ended up with a real doozy of a policy conundrum, worked your
way through the policy process, had a chai latte with your friends, and it is all
just too hard. Everywhere you look is a jumbled mass of contradictions,
options, factors to consider, missing evidence and mixed objectives.
27
D Tricks of the trade, continued
There are three things to do.
• First, have a real coffee – or your preferred equivalent. (This is very
important for the emotional rescue.)
• Second, try to categorise your thoughts. Locate the issues in a broader
frame. Seek to narrow and group options rather than widen them.
• Third, and most important, ask for assistance from respected, experienced
people.
Limit yourself to maybe one or two respected bods, not lots of them – and try
to get as much guidance as possible from them. Asking lots of people is likely
to confuse you, and asking your peers will probably further confuse the
muddle.
If necessary, ask your manager to locate you a suitable mentor for the project.
28
Ministry for the Environment May 2011
Section 2 – Applying the policy cycle
How to apply the COBRA policy guide
Policy cogs
For good policy development, you need to take full advantage of the
expertise, processes and tools the Ministry for the Environment and external
stakeholders have to offer.
These have particular application in a policy context. At each stage of the
policy cycle, this guide provides you with prompts and useful contacts to
think about the full range of things you need to be considering.
Key elements
Planning, process, and analysis and questioning
The policy cogs integrate the key elements of planning, process, and analysis
and questioning that together will support you in delivering high-quality
policy advice. These processes are more powerful when used together. When
used effectively they can improve the quality and timeliness of your policy
advice, including helping you to better identify and manage risks through
the process.
Planning
It is important to plan and manage your work carefully to ensure others know
what you’re planning on doing; how you intend to approach the issues; the
milestones you’re working to; and to support you in monitoring progress
carefully and identifying risks early. For help on how you could go about doing
this well, speak to the Project Management Office (PMO).
Process
The Ministry has processes designed to ensure that quality and key
relationships internally and externally are well managed. These processes
also provide checks on quality and consistency across the Ministry. Look at
these sections of this guide for ideas of who to engage (including other parts
of the Ministry), when to engage, and how to improve your advice.
Analysis and questioning
In developing your policy advice, you will need to undertake analysis and
questioning. This will include continuing to revisit what you’ve learned as you
move from stage to stage to check whether you should challenge any of the
assumptions or conclusions you formed earlier in the cycle.
You don’t have to answer every question. Use your judgement to work out
which questions are most relevant.
30
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Getting started
Planning
Any piece of policy work could be considered a project, in that it generally has
a defined set of objectives and scope, and a defined end date.
However, policy projects are by their nature quite dynamic and non-linear.
This creates particular challenges when planning your work, as the scope of
the work and its timeframes can and often do change significantly due to
factors outside of your control (such as a Minister’s decisions).
Tools and templates
In a policy context, good use of project management tools can help you:
• manage resources including your time and the time of others and help
you work out when you might need external advice so you can get onto it
early
• identify and mitigate risks, including helping you work out when you need
to get someone else to help you manage a risk
• track and communicate progress, key milestones and risks to the project
team, your manager, senior managers, the Ministry and externally, where
appropriate
• adapt to changing circumstances such as a change in direction from the
Minister or a fast-forwarding of the timetable.
In your project planning, focus on answering the following six questions:
• Why are you doing this?
• What are you going to do?
• How are you going to do it?
• Who is going to do it?
• When does it need to be done?
• What risks exist for the project?
You and any advisors from the Project Management Office (PMO) who you
may approach for help will need to consider the best way to plan and manage
the project to enable sufficient visibility and control of the project without
creating excessive process overheads. You will need to consider your
approach to planning and managing the work and how you plan to go about
the policy process.
Templates are available at the PMO’s intranet pages. Use these wisely to
make sure your use of them is fit for purpose and proportionate to what
you’re working on. PMO can help you with ideas of what could work well in
your circumstances. Remember that what might be appropriate for a large,
complex, high-profile project is likely to be overkill for most policy work.
Ensuring shared
expectations with your
manager
Before you kick off the policy process or get too stuck into your project
management templates, talk with your manager to make sure you both
understand each other’s expectations of how you’ll keep them updated on
progress, new developments and risks.
It’s particularly useful to discuss what each of you think is in or out of scope of
the project. This will help avoid nasty surprises down the track.
WL 13527
31
Getting started, continued
Project management
The PMO has designed and implemented the Ministry’s project methodology.
Methodology
The methodology provides guidance under four processes.
• Approval to start
Decision to proceed with planning the project.
• Plan the project
Process to lay down the foundations for the project – scope, schedule
(timing), costs, resources, establishing project controls (managing
risks, issues, quality, change), and how the project will be monitored
and reported.
• Manage the work
Doing what you planned to do – completing the work, reviewing the
project plan and revising when required, managing risks, issues, quality
and changes, monitoring progress and reporting.
• Close the project
Formally closing the project and handing over the deliverables to the
business.
Templates
The templates you’ll probably find most useful are these.
• Policy work package
Use the most relevant sections for planning your work.
• Timeline
Focus this on the key milestones and phases of work you’ll be carrying
out. Identify any interdependencies – remember to build in sufficient
time for engagement and for drafting the final products of your policy
advice, especially any Cabinet paper or regulatory impact statement.
• Risk register
Keep it short and focused. Be clear about what you can and can’t
control. Include how you might mitigate the risks. Identify where you’ll
need someone else’s help to mitigate a risk.
Guidance on scoping and policy work is available on the COBRA home page
on the intranet. Ask a COBRA expert to help you with applying this guidance
to your issue.
The templates are intended to help you, so make them work for you, rather
than the other way around. You can use them selectively and adapt them to
fit your needs better. Do check with the PMO early if you want to try
something different. They’ll have some good ideas.
Help
• You can contact the Project Management Office (PMO) if you need any
help. The PMO also offers a coaching service for anyone managing or
involved in projects. It’s best to engage with them early.
• COBRA experts can help you scope your policy work as part of the
planning process.
32
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Planning stakeholder
engagement
Who do you need to engage with, what about, why, and when?
(EEP Principle 4)
Preparing for your policy work should include stakeholder mapping and
analysis – both internal stakeholders and external stakeholders. You should
consider how Māori and other stakeholders and affected parties view the
problem.
Your plans and preparation for stakeholder engagement should be scaled to
the size of the issues you are working on and the timeframes you have
available.
The analysis should include thoughts on:
• why we need to engage with this group or groups
• what the Ministry seeks to achieve through that engagement
• what the benefits and risks of the engagement are
• what the stakeholder would expect or want from the engagement.
This must occur before an engagement plan is written to inform the scope
and channels used for engagement, for instance are public meetings, email or
one-on-one meetings the most appropriate form of engagement?
The Senior Engagement Advisor in the Communications Directorate should be
involved at this stage of the process if the project is a large one. They are also
available as a resource to offer guidance to smaller projects.
WL 13527
33
1
Identify the opportunities and issues
Identifying the opportunities and issues gives you a reason for doing the work
and a sense of direction and focus for evidence-gathering and later stages of
the policy process.
This stage is often known as the ‘problem definition’. Thinking about it as
‘opportunities and issues’ will help you think more widely around the
situation. You’ll keep coming back to this phase of your analysis – regularly
testing with yourself and others whether you’ve got it right or whether you
need to adapt it in light of new information you’ve discovered.
Identifying the key opportunities and issues involves three key things:
• thinking – by yourself, out loud, and discussing and testing ideas
with others
• assembling evidence – this can take many forms from researching and
assessing data, statistics, previous studies, theoretical work, and so on to
talking to people who are experienced in the area (the Information
Directorate, academics, think tanks, businesses, local government
administrators, and so on)
• regularly challenging yourself on whether you’ve identified them well.
Planning
Finding help
Contact the Project Management Office if you need help.
Things to check
• How is the work progressing? Refer to your original scoping and planning
tools. Does anything need raising?
• Have any new issues or risks been identified? If yes, how are you
managing these?
Process
Internal and government department engagement
Are there others in your team or the Policy Group you could discuss this issue
with?
Are there others in the Ministry who you could discuss this with?
• The Information Directorate can help define the problem and identify the
opportunities from many perspectives, including whether it’s a national or
localised issue. If they don’t have the information in-house they can help
you to source it elsewhere.
• Contact the Statistics and Geospatial, or Science teams for more
information.
• Are there Treaty implications? Do you need to speak to the Treaty team?
34
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Consider getting a small group together from across the Natural Resources
Sector Network departments to work through some of the analysis questions.
Are there existing advisory groups or government-mandated processes that
could:
• add value to what you are doing?
• be undermined or understandably unhappy if you do not engage with
them or inform them of the work?
External engagement
Were there any stakeholders identified in your stakeholder mapping who
could help you with evidence or ideas at this stage? Don’t be shy about
approaching them, but let your manager know before you contact them.
• Who are the stakeholders that could shed light on the problem?
• Why might the stakeholder group have raised this issue? What vested
interest might they have?
• What information might you need about the communities and groups that
are affected by the problem or by your policy proposals? (EEP Principle 4).
Sources of information could include (but are not limited to) Census data,
observations and experiences of stakeholders and iwi, or a cultural or social
impact assessment. Speak to the Information Directorate.
Research needs
Depending on the size of the project you may wish to consider whether some
form of research might be needed to define the opportunities and issues of
the policy and to understand the perspectives of your stakeholders – either by
accessing data already gathered by the Ministry or other agencies or by
commissioning statistically robust quantitative or qualitative research. Speak
to the Communications Directorate or the Information Directorate if you want
help with this.
Analysis and questioning
How could thinking about the issue through a COBRA lens help you
identify the opportunities or issues?
• What other policy frameworks and ways of thinking could help you in this
situation? The glossary can provide you with some concepts which might
apply. See also page 15.
Economy and environment principles
What are the key economic values associated with this issue? (EEP Principle 1)
How are natural resources used to provide for economic needs and
aspirations in this case?
Do the economic benefits change over time? Can the resource use in question
continue to provide economic benefits indefinitely, or is there a foreseeable
endpoint to the economic activity (such as the consumption of non-renewable
resources)? How does considering timeframes change your assessment of
economic benefits?
WL 13527
35
1 Identify the opportunities and issues, continued
What are the key environmental values associated with this issue? (EEP
Principle 1)
• Can you describe the important values associated with this natural
resource or ecosystem (for example, ecological, biodiversity, recreational,
aesthetic, intrinsic, cultural, or social value)? You should consider both use
and non-use values.
• What ecosystem services are provided by this natural resource?
For your particular natural resource issue, how do economic values affect the
environment, both in positive and negative ways? (EEP Principle 1)
• What are the effects on the environment arising from economic activities?
Do the effects or risks change over time?
• Are there any positive or negative externalities arising from these
economic activities? What are they?
What economic activities rely on a healthy environment and in what ways?
(For example, river tourism may rely on a river being clean and safe for
contact recreation.)
How does providing for environmental values affect economic activities? (For
example, clean air laws may require an industry to invest in an upgrade of
their smoke stack. How will this industry, its employees and supporting
businesses be affected by this regulation?)
Issues and opportunities for Māori
Are there any aspects of the issue or of your proposed options that will be of
particular interest to, or impact upon, Māori? (EEP Principle 2)
What are the Treaty of Waitangi implications of the problem?
Is the issue you are trying to understand one in which Māori have a strong
cultural or economic relationship? Is there a resource involved with which
Māori have a strong cultural or economic relationship? Don’t just assume
not… If the answer is not sure or yes, consider getting Kaahui Taiao to work
with you from the problem definition stage, avoiding issues further along.
Are there statutory obligations with any iwi surrounding the issue that you
need to address/be aware of?
Are there opportunities to involve Māori and iwi that will improve Crown/iwi
relationships?
36
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Evidence
What evidence might you need about the economic activities involved? (EEP
Principle 4)
• This could include (but is not restricted to) economic analysis (cost-benefit,
cost-effectiveness analysis), quantitative and qualitative data from primary
and secondary sources, industry statistics, forecasting and modelling, and
expert knowledge of individuals, groups and networks. The Information
Directorate has economic capability in-house that can help you with this.
Contact the Manager of the Statistics and Geospatial team for more
information.
What evidence might you need about the environmental and social activities
involved? (EEP Principle 4)
What are the limitations, assumptions, and gaps in your evidence base? (EEP
Principle 4)
• Do you need to signal anywhere where there is a lack of evidence, or
where evidence may be biased or uncertain, or where opinion differs from
the evidence?
• How might others respond to the limitations of your evidence?
Do you need to gather more information, or should you use a precautionary
or adaptive approach that will allow for a policy intervention to be
assessed and adapted in light of new evidence? Consider mentioning this
evidence gap to the Information Directorate. They may be able to fill it.
Theory
What does theory tell you about the issue?
What are the competing theories of what the problem is and how might we
distinguish between them using evidence?
How will the symptoms of the problem be separated from its essence?
Have you identified the key underpinning problem?
Can you consider the status quo as a process? How might this change your
approach to the issue?
Problem tree / cause and effect analysis
Developing a problem tree can help broaden thinking about possible causes
for a problem. It can be used to develop a range of hypothesis for the causes
of the situation (some of which will likely be incorrect), but it will help capture
ideas.
See the guidance available on the COBRA homepage of the intranet.
WL 13527
37
1 Identify the opportunities and issues, continued
History lessons
What is your or others’ experience with similar problems and how could you
draw on that experience?
How did the status quo get to look like it does? What future external trends
can you see and will the existing approach still be fit for purpose?
Has this issue been looked at previously? By the Ministry? Elsewhere? How
did the process play out previously? What lessons can be learnt?
International perspective
When you are developing a piece of policy advice, picture the Minister asking
you ‘what are other countries doing about this issue?’ You need to have the
answer.
The Ministry has a range of cooperative relationships with counterpart
agencies in other countries. These stem from involvement in multilateral
processes, formal environmental cooperation agreements, international
bilateral partnerships, and a history of positive cooperation on a range of
environmental issues.
Ask your manager about your team’s relationships with international policy
counterparts (for example, many teams have exchanged work plans with
counterparts in the UK’s Department of Environment, Food and Rural Affairs
(DEFRA) to identify shared work areas) or how policy advice can usefully be
informed by international experience. Contact the International Climate and
Environment team to see if they can help you make connections with
overseas policy makers.
There are also useful online communities of policy makers through OLIS (the
OECD online information service).
See the intranet for some suggestions.
38
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
2
Is government consideration warranted?
Identifying the issues and opportunities is one step. But you also need to
consider whether government consideration is really warranted in the
circumstances and/or whether others should be involved in addressing the
issues. Government is not the only actor in the economy or society.
Businesses, communities and individuals may be able to more effectively deal
with the situation without government assistance. In some circumstances, the
situation is too complex to identify how government intervention could make
a difference. In a bad case, intervention can also make things worse
(‘government failure’).
Planning
Finding help
Contact the Project Management Office if you need help.
Things to check
• How is your work progressing against your timeline and original scope?
• Does it need updating to reflect what you’ve learnt?
• If things have changed, does this have resource implications (people or
time)?
Process
Engagement
Do you need to review who your key stakeholders are? If there have been any
changes, your key stakeholders may have changed. Is there anyone else who
now needs to be involved in the next stage of the process or consulted?
On reflection, do you need to reconsider any earlier analysis that you’ve
carried out. That is, are you confident that what you’ve learnt supports your
original identification of issues and opportunities?
Analysis and questioning
Economy and environment principles
Why is government intervention being considered at this time? (EEP
Principle 2)
• What has changed to bring this issue to the Government’s attention now?
Is there a case for government intervention? (EEP Principle 2)
• Are there other groups or organisations that hold the responsibility for
addressing the issue, or are better placed to deliver a solution?
• What would happen if government didn’t act?
• What risks of government failure could there be in this circumstance?
What are the possible ways government could respond to this issue? (EEP
Principle 6)
What are the benefits, costs, opportunities and risks of government
intervention (including the risk of government failure)? (EEP Principle 2)
WL 13527
39
2 Is government consideration warranted?, continued
Nature of the problem
Is this a national or localised problem or opportunity?
How was this situation been dealt with in the past?
Treaty issues
Have you identified anything that could lead to a contemporary Treaty breach
or claim? Are Article Two rights being affected? For example, taonga, property
rights, right to govern, and so on.
40
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
3
Define outcome, objectives and criteria
Developing policy is about analysis but it’s also about judgement. Defining the
outcome, objectives and criteria that you will apply is about setting the
parameters and ‘rules’ by which you’ll make the judgement about your
preferred approach(es). Part of this judgement is also likely to include issues
about practicality and feasibility. Be careful you don’t stray into defining
options in your objectives or criteria. Remember that there should be a clear
‘line of sight’ between outcomes, objectives and criteria.
Ideal outcome
At this stage you should try to define the ideal outcome that would solve the
policy problem. In reality, you will probably need to settle for solving it to an
acceptable degree. You will use the criteria to assess how close the options
you develop are likely to come to achieving your desired outcome.
What is an outcome?
An outcome is the ultimate state that you’re working towards. Asking yourself
‘why’ questions will help you identify this. Outcomes typically have a long
term focus. The Ministry’s Outcomes Framework (in the Statement of Intent)
has some examples of inputs, outputs, impacts, and intermediate or end
outcomes of the Ministry’s work programmes.
What is an objective?
Objectives sit beneath outcomes. They can be seen as intermediate steps
towards achieving an outcome – often they’ll be the initial answer to your
‘why’ question. Objectives are more concrete statements about desired end
states. Be careful not to slip into defining options or interventions at this
point.
Objectives describe at a summary level what the project must do or deliver.
What could criteria look like?
• Some criteria will be absolute. They will be high priority and nonnegotiable – for example, the policy must be fiscally neutral.
• Other criteria may be relative, meaning they will have different settings
across options and decision-makers will make a judgement about their
relative importance – for example, a policy that is easiest to implement
might be preferred.
• Criteria may be associated with the inputs into the policy – for example,
cost. Or they might be associated with outcomes – for example,
effectiveness.
• As a general rule, criteria should be independent of one another.
What is the difference between objectives and criteria?
Criteria will generally be those features of an initiative that
achieve the objective. Criteria can also be thought of as measurable
dimensions of objectives. In practice, you will be assessing them both
qualitatively and quantitatively.
WL 13527
41
3 Define outcome, objectives and criteria, continued
Planning
Finding help
Contact the Project Management Office if you need help.
Things to check
• How is the work progressing? Refer to your original timetable and policy
work package. Do you need to review any elements of these and
communicate these changes to your manager?
• Do you need to provide any progress reports?
• Have any new issues or risks been identified? If yes, how are these being
managed?
Process
Who in the Ministry can help you at this point?
• There are a number of teams in the Ministry with whom you might want
to test your thoughts or outcomes, objectives and criteria including:
– the Regulatory Impact Assessment Panel
– the Monitoring Compliance and Review team who can provide advice
on whether or how your outcomes can be monitored, measured or
evaluated2 (worth noting is that some outcomes may not be able to be
measured).
– the Science team who can help you think about what information you
might need to assess options against your criteria. If time is on your
side, this early engagement may result in key information gaps being
filled.
Engagement
To identify and mediate strongly competing values, you will need to consult
and engage with the relevant parties (EEP Principle 3)
• It may be helpful to consider stakeholder processes that allow the
different parties to come up with proposals for managing tensions.
Do you need to review who your key stakeholders are? If there have been any
changes as your work has progressed, your key stakeholders may have
changed. Is there anyone else who now needs to be involved in the next stage
of the process or consulted?
Lessons learnt
On reflection, do you need to reconsider any earlier analysis that you’ve
carried out. That is, are you confident that what you’ve learnt supports your
original identification of issues and opportunities?
2
It is worth noting that not all outcomes and objectives will be able to be measured quantitatively. This does not mean they are
‘wrong’, nor does it mean that evaluation is impossible. Sometimes you have to monitor and evaluate initiatives as best as possible.
42
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Analysis and questioning
Drawing on your earlier application of the COBRA approach and other policy
frameworks, how can these help you define your outcomes or objectives?
• Should you consider this issue from a perspective of maximising
opportunities or minimising risks?
Outcomes
Drawing on your earlier problem definition, what outcome(s) are you trying
to achieve?
• What does an ideal outcome look like?
• What other possible outcomes are relevant?
• Is it appropriate to have an outcome around strengthening the Treaty
relationship?
• Is your outcome focused on reducing risk or maximising opportunities?
Objectives
What are the Government’s policy goals (objectives) for this particular issue?
(EEP Principle 3)
• Are they relevant for this issue?
What other government policy goals (objectives) are relevant to your issue
(that is, social, cultural, economic, environmental)? (EEP Principle 3)
• Are there tensions between these goals and yours? What are they?
• Are there synergies between your goals and other government policy
goals that could be built upon? What are they?
What are the interests and aspirations of different groups in society (for
example, Māori, businesses, recreational users, environmental groups) in
relation to the issue? (EEP Principle 3)
• What tensions exist between the goals (objectives) and aspirations of
different parties?
• How will each party’s pursuit of their goals (objectives) impact on the
other?
• Are there synergies between the goals (objectives) of different parties
that could be built upon? What are they?
• Are there ‘bottom line’ values or interests that must be met? What are
they?
Are there environmental bottom lines that need to be protected? (EEP
Principle 2)
• Are there any well-established environmental bottom lines relating to the
key environmental values you have established in your problem
definition? These may be in the form of standards or guidelines, or a ‘cap’
on resource use, such as a fishing quota or greenhouse gas reduction
target.
• If the limits of resource use or environmental bottom lines are lacking or
not well established for key environmental values relating to your policy
issue, this gap may need to be addressed.
Are your objectives realistic? (EEP Principle 5)
WL 13527
43
3 Define outcome, objectives and criteria, continued
Criteria and evidence
How will you measure the performance of the options against the objectives
and the desired outcome?
What are the ‘must haves’? What are the ‘nice to haves’?
What are the parameters any solution needs to work within?
How would you weight the criteria relative to each other?
Do you need to consider social justice implications (such as equity issues)?
How will we know when we’ve achieved the outcome?
How have you factored ease of implementation into your criteria?
What data will be needed to assess options against criteria?
What information will you need at a later stage to determine how the success
of this policy will be monitored or evaluated?
Who will collect the data needed to monitor and evaluate the policy?
Intervention logic diagram / clarify cause and effect
Consider developing an intervention logic diagram to help articulate clear
objectives and outcomes.
44
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
4
Construct options
This is where you start to identify and develop alternative courses of action,
or alternative strategies of intervention to solve or mitigate the problem or
risk or explore the opportunity.
At this stage, you should lean towards a comprehensive list of options, which
you’ll narrow down as you continue your policy process. You should always
include the option of ‘do nothing’ or ‘let things continue as they are’; as well
as considering regulatory and non-regulatory options. Try to group variant
options together. This will help with focusing your assessment at the next
phase on the full options.
You need to be aware of whether your options are mutually exclusive, or
could be combined to tackle the problem.
You also need to have some ‘flesh’ on your options; this might create variants
of your option. It’s helpful to have a one-line summary statement that gives
the strategic thrust of your option, but you’ll also need to have some idea of
how the option will be designed and implemented and so on. There needs to
be enough flesh to allow an informed reader to understand the implications
of the option, to allow effective analysis of the risks and benefits, and for the
option to be able to be differentiated against other options (or variants of the
option).
Planning
Finding help
Contact the Project Management Office if you need help.
Things to check
• How is the work progressing? Refer to your original timetable and policy
work package and communicate these changes to your manager?
• You might want to re-scope your policy at this stage, using the tools
available on the COBRA site. This can be useful for constructing your
options to a suitable level of detail.
• Have any new issues or risks been identified? If yes, how are these being
managed?
Process
Who in the Ministry might be able to help?
The Operations Directorate can help you test the practicality of your options
and identify possible implementation risks.
The Resource Management Tools team develops and implements national
environmental standards and is able to identify pros and cons of a number of
different intervention tools.
Engagement
Do you need to review who your key stakeholders are? If there have been any
changes, your key stakeholders may have changed. Is there anyone else who
now needs to be involved in the next stage of the process or consulted?
WL 13527
45
4 Construct outcomes, continued
Lessons learnt
On reflection, do you need to reconsider any earlier analysis that you’ve
carried out. That is, are you confident that what you’ve learnt supports your
original identification of issues and opportunities? And that you’ve identified
the appropriate outcomes and objectives?
Analysis and questioning
Application of policy frameworks
Drawing on your earlier application of the COBRA approach and other policy
frameworks, how can these help you construct options?
• What issues and opportunities did you identify that your options should
be targeting?
• How can design of your option(s) maximise benefits and opportunities and
reduce costs and risks?
• What issues did you consider government should be considering?
• Did you identify any market failures that needed to be targeted?
How do you want people’s behaviour to change and how will the proposed
interventions create this change? (EEP Principle 6)
How can different policy frameworks help construct options to tackle
behaviour? Do you need to consider institutional issues; market failure or
government failure?
Does the problem involve serious or irreversible impacts on natural
resources? Should you consider a precautionary approach; or consider an
adaptive management solution? (EEP Principle 5)
• What level of risk is acceptable in this situation? Is there agreement about
this across the Natural Resources Sector Network departments? Across
stakeholders more widely?
• What could a precautionary approach look like for this policy problem?
Could there be different options for this?
• If there are differences of view about the level of acceptable risk, or what
a precautionary approach could look like, what is driving these different
views?
Have you thought about the status quo as a system? What implications does
this have for the options you should consider?
What are the competing theoretical solutions to the problem and how do
these translate into practical options? For example, pollution tax as a
response to an un-priced negative externality.
What does experience overseas tell us about possible solutions and how do
these translate into the New Zealand context?
Are there opportunities to involve Māori and iwi that will improve Crown/iwi
relationships?
Scope of options
Have you included ‘do nothing’ or ‘let things continue as they are’ as an
option, and if not, why not?
46
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
How can these options be categorised or organised in a way that helps make
sense of them as a set? For example, increasing cost, or increasing
effectiveness.
What are the interdependencies between the options?
What are the interdependencies with other policies?
Is there complementary work being done elsewhere in the Ministry?
Is there complementary work being done elsewhere in the Natural Resources
Sector Network?
Quality of evidence
What are the limitations, assumptions, and gaps in your evidence base? (EEP
Principle 4)
• Do you need to signal anywhere where there is a lack of evidence, or
where evidence may be biased or uncertain, or where opinion differs from
the evidence?
• How might others respond to the limitations of your evidence?
– Do you need to gather more information, or should you use a
precautionary or adaptive approach that will allow for a policy
intervention to be assessed and adapted in light of new evidence?
Risks of options
Are there any transitional issues or risks that you need to consider? How can
you mitigate these risks?
WL 13527
47
5
Assess options and play out outcomes
Assessing options against criteria
To assess an option against a criterion that is associated with policy
outcomes, we need to project or predict these outcomes.
Once this is done the options need to be assessed against the criteria and the
costs and benefits of the proposal.
This is effectively trying to deduce a ranking for each option based on the
ideal outcome embodied in the criteria, while understanding the costs,
opportunities, benefits and risks.
Play out outcomes – thought experiments
This stage is really a thought experiment to challenge how your options might
play out in the future. Imagine your policy option as a pebble that you throw
into a pool of water – what outcomes happen as the ripples from the pebble
move their way out?
This is your time to be realistic, rather than optimistic about your policy
options. It should lead to the identification of a preferred option. Try to
identify any unintended consequences down the line.
Planning
Finding help
Contact the Project Management Office if you need help. Remember to
discuss with your manager if anything significant has changed.
Things to check
• How is the work progressing? Refer to your original timetable and policy
work package. Do you need to review any elements of these and
communicate these changes to your manager?
• Do you need to provide any progress reports?
• Have any new issues or risks been identified? If yes, how are these being
managed?
Process
48
How can others in the Ministry help you with this assessment?
• How could the Regulatory Impact Assessment Panel be used to refine the
assessment?
• Could the Information Directorate help with your assessment? You may
want to approach the Information Directorate about appropriate
assessment methods/tools and data to assess options, including economic
and social science expertise. Contact the Statistics and Geospatial or the
Science teams for more information.
• You may want to get the Operations Directorate involved to help assess
options, especially around implementation risk.
• See ‘8 Implement (legislation and/or programme)’ on page 58 for further
information on implementation. The Resource Management Tools team
develops and implements national environmental standards, and is able
to identify pros and cons of a number of differing national intervention
tools.
• Are there any statutory requirements that the project needs to go
through, for example, Section 32 of the Resource Management Act? Talk
to the Legal team if you are unsure.
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Lessons learnt
On reflection, do you need to reconsider any earlier analysis that you’ve
carried out. That is, are you confident that what you’ve learnt supports your
assessment of the role for government consideration? Do you need to revisit
the options you constructed?
Analysis and questioning
Tools and evidence
What analytical tools would be appropriate for you to use while considering
options for this problem?
How will you assess costs, opportunities, benefits and risks in this situation?
What information/data do you need to assess the options? What do you
have? What is the impact of the gaps in your evidence base?
Is a pilot or more empirical work necessary to determine performance against
criteria?
If a criterion is subjective, who is best placed to make a judgement?
Are there any impacts that are difficult to quantify that may need to use nonmarket valuation? How will you go about this?
Assessment of options
How does each option measure up:
• to the outcomes and objectives you identified?
• against the criteria?
What might be the social and economic impacts of taking an adaptive
approach? (EEP Principle 5)
• The tension between flexibility and certainty for resource users will be
different for each case.
How do the options address the problem?
What are the costs, opportunities, benefits and risks of the different options?
How could you mitigate or reduce the costs or risks?
• What are the costs of the change/transition? Are they one-off or ongoing?
• What risks to delivery can you identify? How can you mitigate these?
• Who will implement the policy and can they realistically and effectively
achieve it?
• See the COBRA page on the intranet for more information on cost benefit
analysis.
Who will be affected by the policy? How will they be affected? Does this raise
any equity issues?
How can you demonstrate clearly and logically how the proposed policy will
contribute to the desired objectives and outcomes? This is a crucial step
towards effective monitoring and evaluation.
WL 13527
49
5 Assess options and play out outcomes, continued
What do you need to ‘break even’ on this policy?
• What minimum level of effectiveness would the policy have to achieve to
justify the costs?
• What changes in the status quo process or situation could conceivably
produce this level of effectiveness?
• How likely is it that the changes will deliver this level of effectiveness?
• What’s your estimate of the probability of failure – would it be tolerable if
it occurred?
What is the institutional environment within which the solutions will operate,
and how does this affect them?
What other actors need to play a part and what is their willingness to be
involved?
How does each of the options fit with other government programmes? What
are the complementarities and synergies that might occur?
Reality check of options
What level of change is implied by the options you have identified? Is this
realistic?
What assumptions have you made in your assessment? How big a mistake
can you afford in each assumption before the analysis is in big trouble?
Māori issues
Does it help address an issue that is the topic of Treaty claims?
Does it close off any opportunities for iwi development?
Are there any particular conflicts/clashes with iwi that have ‘settled’ their
Treaty claims (that is, what's the impact on statutory acknowledgements)?
Are Article Two rights being affected? For example, taonga, property rights,
right to govern and so on?
What are the potential costs to iwi, and what are the anticipated benefits?
Will iwi have the capacity and capability to play the role envisioned?
Monitoring and evaluation
What aspects of the preferred option will help evaluation?
50
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Play out the options – focus on your preferred option(s)
If implemented, what is the projected outcome from each option?
How might the drivers of the problem (including resource use), social context,
or knowledge about the problem change over time? (EEP Principle 5)
• Try to identify key trends that might impact on the issue in 10 or 20 years’
time.
• What might happen if the effects of resource use are greater than
anticipated or if new information comes to light?
• Have you built in processes to review your problem definition, objectives,
and policy and adapt them over time?
Could any of the options potentially lead to a contemporary Treaty
breach/claim?
What kinds of unintended effects might be created by your proposal(s)? (EEP
Principle 6)
• How might people respond to the policy in ways that could thwart the
intention?
• How might the incentives you are looking to create play out in practice?
Are there any undesirable behaviours or effects that might flow?
• What would happen if the policy intervention did not achieve its
objectives?
• What would happen if ‘Murphy’s law’ were to apply?
• What would the impact be if it did go wrong?
What are the uncertainties that are involved with each option? What is the
likelihood of these arising?
How might individuals, firms, decision-makers, and so on respond to the new
world created by the option over time?
What scenarios might cause the proposal to fail to produce the desired
outcomes? Consider the impact of a low-probability event on the success of
your option.
What dangers could be posed by the implementation process, including the
political process? What impact could they have on the outcomes?
What undesirable side effects might come into play? For example, moral
hazard; rent seeking.
If your advice were adopted, what might be the costs of you having been
wrong, and who would have to bear those costs?
If you were the affected party / stakeholder / key actor, how might you react?
WL 13527
51
5 Assess options and play out outcomes, continued
Perverse outcomes and unintended consequences
One thing that playing out the outcomes does is help you identify perverse
outcomes. A perverse outcome is a type of unintended consequence that
works against the intended outcomes of the policy instrument, with an
outcome contrary to the intended, making the problem worse.
Example: Waste levies
In 2006, the Ministry was considering the introduction of a levy on solid waste
disposed of to landfill. In assessing options and playing out the outcomes, the
Ministry analysed the potential unintended consequences of a national waste
levy. It also published a report, which is available on the Ministry’s website
(http://mfe.govt.nz/publications/waste/waste-levy-discussion-nov06/).
Examples of the possible unintended consequences of a waste disposal levy
include an increase in illegal dumping to avoid disposal costs, waste that
should go to landfill being disposed of to cleanfills (which are not subject to
the levy, or an increased use of on-farm dumps.
Cobra effect
The term ‘cobra effect’ stems from an anecdote set at the time of British rule
in colonial India. The British government was concerned about the number of
venomous cobra snakes. The Government therefore offered a reward for
every dead snake.
Initially, this was a successful strategy, as large numbers of snakes were killed
for the reward.
Eventually, however, the Indians began to breed cobras for the income. When
this was realised, the reward was cancelled, but the cobra breeders set their
snakes free, and the wild cobras consequently increased in number.
The planned solution for the problem had actually made the situation worse.
A similar incident occurred in Hanoi, under French colonial rule, where a
programme paying people a bounty for each rat pelt handed in was intended
to exterminate rats. Instead, it led to rat farming.
Hit and runs
The stiffening of penalties for driving while intoxicated in the United States in
the 1980s led, at first, to an increase in hit-and-run accidents, most of which
were believed to have been drunken drivers trying to escape the law.
Legislators later stiffened penalties for leaving the scene of an accident.
52
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
6
Confront the critical choices
Looking at strengths and weaknesses
Occasionally, you will have one policy option which is expected to produce a
clearly better outcome than any of the other alternatives, for every single
evaluative criterion.
However, it’s more likely that you‘ll be in a situation where each option will
have strengths and weaknesses across the criteria. In this situation, you need
to confront the critical choices that need to be made in making a final
recommendation and decision on a preferred alternative.
Some options may be eliminated by not possessing the ‘must haves’.
Planning
Finding help
Contact the Project Management Office if you need help. Remember to
discuss with your manager if anything significant has changed.
Things to check
• How is the work progressing? Refer to your timetable. Do you need to
review any elements of this and communicate these changes to
your manager?
• Have any new issues or risks been identified? If yes, how are these being
managed?
Process
How can others in the Ministry help with you this assessment?
How can you effectively present the critical choices in your advice to the
Minister?
The Regulatory Impact Assessment Panel may have some ideas and be able to
share some best-practice examples with you.
Lessons learnt
On reflection, do you need to reconsider any earlier analysis that you’ve
carried out? That is, are you confident that what you’ve learnt supports your
assessment of the role for government consideration? Do you need to revisit
the options you constructed? Are there further conversations you should be
having with the Operations Directorate to understand implementation risks?
WL 13527
53
6 Confront the critical choices, continued
Analysis and questioning
Is there a clear preferred option, against your decision criteria? If not, are
there clear choices to be made? For example:
• Option A is less costly, but option B is more likely to make a bigger impact.
• Social benefits are delivered (for example, improved environmental
outcomes for all) but costs are borne by different identifiable groups (for
example, by polluting industries or consumers of particular services).
Are the assumptions behind each of the leading options equally believable,
including sensitivities of your analysis?
From playing out the option outcomes, is one of the alternatives more
credible or likely to deliver?
Are there greater risks (for example, implementation) from any of the
proposals?
If your preferred policy alternative is such a great idea, why isn’t it happening
already? Have you missed anything in your analysis?
54
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
7
Recommend and tell the story
Free and frank advice
As a policy analyst, your role is to provide free and frank advice to the
Minister. Although you’re not the final decision-maker, you should be seeking
to provide a clear preferred recommendation to him/her. This
recommendation could be ‘do nothing’, if that is the best option.
Your recommendation should be driven by the analysis you’ve done
throughout this process and should highlight clearly the critical choices that
need to be made in reaching a final decision.
Clear messages
Telling the story is a critical part of good policy analysis. It shows how the
policy package hangs together and it is an important part of explaining how
‘technical’ changes are valuable in the real world. It’s also a good test of
whether you’ve understood your own policy process well.
Identify the key short messages that any communication needs to get across.
Think carefully about your audience(s) for different communications products
and adapt your messaging accordingly. You should keep the problem
definition at the heart of your story.
Your Cabinet papers and recommendations are not the end of the story;
someone will have to implement your policy. At the minimum, you’ll need to
hand over to them.
Rehearsing the story
Recommendations don’t just magically appear. They are a key part of the
policy picture and are often the result of a great deal of thinking. It is possible
that the only part of a paper that a Minister will read is the
recommendations. These are the ‘doing’ part of the paper so it is critical they
are clear, concise, as unambiguous as possible and action-oriented.
There are a range of ways to develop recommendations. If you have any
doubts, talk to experienced practitioners about how they go about developing
recommendations. Some people rehearse the story in their head for weeks.
Alternatively, some staff members rehearse their recommendations and the
story aloud, changing the story slightly every time. This may be that person’s
process for testing their ideas and recommendations.
Challenges to your thinking may manifest themselves around the
recommendations. While this can seem extremely frustrating, the contesting
of ideas and advice is an integral part of the provision of public policy advice.
You can also seek advice on ways to manage the views of others if there are
difficulties in agreeing a set of recommendations, and the right advice and
assistance can be very useful.
WL 13527
55
7 Recommend and tell the story, continued
Planning
Finding help
Contact the Project Management Office if you need help.
Things to check
• Update your timetable with the latest milestones. Include briefing notes,
status items, and Cabinet paper, and regulatory impact statement drafts,
as well as final versions.
Process
Identify key messages and prepare a short (no more than one page) bullet
point narrative. Use this as the basis for briefing notes, any Cabinet papers,
and any regulatory impact statements.
Have someone who has not been involved in the work peer review your
narrative.
Review your stakeholder engagement plan, in light of the story you’re
planning on telling. Do you need to review your key stakeholders? Do you
need to reassess their likely reaction to your proposals? What does this mean
for your story?
Speak to the Executive Relations team if you’ve got any questions about
Cabinet processes.
Engage with the Regulatory Impact Analysis Panel early to understand
expectations around what the regulatory impact statement needs to include.
They can give you best-practice examples of how to communicate your
analysis effectively.
Analysis and questioning
Recommend
What are the critical choices that the Minister or decision-maker needs to be
aware of?
What are the assumptions that have driven your conclusions?
What are the risks around your preferred option and is there anything that
can be done to mitigate them?
Who will collect the data needed to monitor and evaluate the policy?
Telling the story
What are your critical messages?
Who is your audience? What implication does that have for how you tell the
policy story?
Are you telling the story, or are you preparing someone else to tell it?
Is this a phased telling of the story, or have you only got one chance to get
the message across?
If you got in the lift in the Beehive with the Minister and had seven floors to
tell him/her the story and recommendation, what would you say?
56
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
What challenges are you likely to receive to the analysis and/or
recommendations? Are there any defensives you need to provide in your
story?
How can you best present any complex data? Do you need to include it?
Would a map, chart, table or diagram make it simpler for the reader? The
Information Directorate has capability in this area – ask them for help.
WL 13527
57
8
Implement (legislation and/or programme)
If the Minister has made the decision on the policy and you’ve only just
started thinking about implementation, you’ve got a problem. Turn back to
the earlier sections of this guide to see how you could address
implementation issues earlier.
Implementation of ministerial decisions can take a number of forms at
different stages including:
• legislation
• developing and implementing national instruments and regulations
• developing and administering delivery programmes.
These processes can also be supported by guidance explaining further details
of implementation and monitoring
Implementation can be thought of as a further iteration of the policy cycle
itself. Implementation is critical to the success and effectiveness of your
policy intervention. It is about working up a clear approach to how the policy
will be delivered and realised and how it will be subsequently monitored and
evaluated. Developing your implementation strategy may identify additional
policy tools that may need developing to aid the effective implementation of
your policy, for example, a national environmental standard to complement
your national policy statement.
Environmental policy is implemented by a range of different players, including
local and regional councils. The Ministry also plays an implementation role,
including through the Environmental Protection Authority. You are fortunate
as a policy analyst to be sitting in the same building as people with this
implementation experience. They are an invaluable source of expertise for
you to tap into throughout the policy process to learn from the effectiveness
and efficiency of existing programmes, and to apply their experience to
review and improve existing interventions.
Staff in the Programmes Group often have long-standing relationships with
local government and industry, and may be able to give you hints on who the
key players are, and their likely areas of interest.
It is crucial for you to engage with the implementation teams ahead of
preparing your final advice to get their expertise on the workability of the
proposals and options you’re developing. A key part of your implementation
is thinking about monitoring and evaluation. Take a look ahead to page 60 for
more information.
Planning
Who in the Ministry can help you at this point?
The Resource Management Tools team has developed a structured project
management approach to implementation that aims to provide a smooth
transition from policy to implementation. See the COBRA intranet page for
more information.
Other members of the Programmes Group may be able to help. Both the
Science team and the Monitoring, Compliance and Review team are able to
advise on the development of plans under the Resource Management Act.
58
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Process
Engagement
Now could be the ideal opportunity to revitalise your stakeholder
engagement strategy!
(Refer to page 27 for more information on stakeholder engagement
strategies.)
Who are the key people in the Programmes Group and elsewhere outside of
the Ministry (such as local government) who will be involved in implementing
the policy? What will they need from it?
Who are the key stakeholders for implementation? Have they changed since
the policy was first developed? What are their needs?
Which key stakeholders need to be involved in developing implementation
plans?
Analysis and questioning
WL 13527
Programmes teams can help with the following questions:
• Who are your key implementers or delivery agents?
• How can you access them?
• What part do they play in a successful implementation and policy
outcome?
• What are the range of options and tools available to implement and
monitor a policy and its outcomes?
59
9
Monitor and evaluate
Develop plans for monitoring and evaluation
Before you reach this step, it is really important that you have clearly
described the policy, its intended results and objectives, and the criteria and
performance measures by which the policy's success would be assessed. You
should already have thought a bit about how the policy will be implemented
and monitored.
Evaluation enables organisations to consider the effectiveness and efficiency
of their programmes and policies, to learn from that consideration, and to
apply their learning to improve their work. Good policy development is an
evaluation process and requires a clear focus on desired outcomes. It is
crucial when developing policy to consider what success would look like (see
Define outcomes, objectives and criteria) and how this would be evaluated.
It is also important that you have developed monitoring and evaluation plans
before implementing the policy (see Implement (legislation and/or
programme)) and that monitoring activity is well underway early.
A formal Ministry-wide evaluation function sits in the Information
Directorate. This evaluation role prioritises areas needing formal evaluation
of core projects, develops tools and processes, and works with teams to build
up skills and knowledge.
Like the Project Management Office, the evaluation role is focused on
building evaluative capability within teams, rather than undertaking formal
evaluation of these projects/policies.
Planning
It is important that you start thinking about monitoring and evaluation early
on, so you can put in place ways to help smooth processes when it comes
time to actually monitor and evaluate.
Finding help
The Information Directorate is able to help with all stages of the evaluation
process, from development of plans to undertaking the evaluation. The
Monitoring, Compliance and Review team is able to advise on monitoring
considerations. The Resource Management Practice and Resource
Management Tools teams should also be engaged when the policy under
consideration includes Resource Management Act mechanisms.
60
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Links to further information on approaches for monitoring and evaluation are
available on the intranet.
At this stage in the policy development cycle you need to consider whether
you are well prepared to monitor and evaluate the policy. What you do now
depends on what has come before.
Consider these two extreme situations.
1 A monitoring and evaluation plan was approved before the policy was
implemented. The plan includes a clear description of how the policy will
achieve the intended outcomes. It even includes suitable performance
criteria and measures. Key stakeholders were involved in designing the
measures. Monitoring information is being collected. Key stakeholders
have received regular updates on how the policy is being implemented.
The policy and intended outcomes have not changed since the plan was
developed. All is well! You just need to follow through with that plan.
2 There are no monitoring and evaluation plans. The policy outcomes or the
link between the policy and its intended outcomes are not described.
There are no specified performance criteria or performance measures.
The people who developed the policy have left the building and you
cannot decipher their sketchy documentation. The policy has been
implemented for several years with no systematic plan to collect
monitoring data. Someone has just noticed that the Ministry needs to
report evaluation findings to Cabinet by next month. This situation is
pretty dire, but unfortunately not unheard of. You will need to start from
scratch.
Process
Engagement
Which key stakeholders need to be involved in developing monitoring and/or
evaluation plans?
Who will be involved in monitoring the policy and who will collect data for the
evaluation? What will they get out of it?
Is this evaluation sufficiently complex/long term/politically sensitive that it
needs an evaluation steering group? Do you need a communication strategy?
Do you need to review your key stakeholders? Have the stakeholders
changed since the monitoring and evaluation plans were developed?
Is there anyone else who now needs to be involved in the evaluation?
WL 13527
61
9 Monitor and evaluate, continued
Analysis and questioning
In designing monitoring and evaluation plans, think about the following steps.
Monitoring considerations
What data will be useful for monitoring and evaluating the policy? Who will
collect this information? Have they been involved in designing the measures?
Is existing data available or will they need to collect new data and put new
systems in place? When will they be able to provide monitoring data?
Are you actively tracking the implementation of a policy throughout the
process and not just collecting data at the end?
Evaluation purpose and scope
Why are we doing this evaluation – who needs it? For example, the
evaluation results might feed into a major review of legislation or perhaps
they will be used to tweak operational policy or to inform the effective use of
supporting measures such as guidance.
When do we need to evaluate? Do we need early feedback on the
implementation of a new policy or do we wish to establish whether a policy
has delivered the intended outcomes?
What are the timelines for this evaluation? Do you need to provide an
evaluation report by a certain date, for example, to report back to Cabinet?
What are the resources for this evaluation? What is the budget? Will you be
contracting an external provider or undertaking the evaluation in-house?
Have you included monitoring and evaluation costs in your regulatory impact
statements?
Evaluation questions and performance measures
Is there a clear and logical description of the relationship between the
proposed policy, the intended results of the policy and how these might
advance desired outcomes, goals or objectives?
Is there an intervention logic diagram? If not, consider developing one to help
design evaluation questions and performance measures.
Are we able to make use of data being collected for other purposes? Ideally
you will design the monitoring information to also be useful for evaluation
purposes, reducing the need to go back to people with repetitive information
requests.
• Have there been any previous evaluations? Can you build on these?
• Is there a monitoring and evaluation plan? Or has the policy changed so
much that these need to be revised? You will need to develop or revise
the plans. Go back to Implement (legislation and/or programme) for
guidance.
62
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
What type of evaluation methodology will you use to address the evaluation
questions? Many evaluations use a mixed method approach that combines
face to face interviews with a few key stakeholders along with broader
coverage surveys. What is the best approach given your resources, timelines
and information needs? – be realistic!
WL 13527
63
Appendix 1 – Glossary of key concepts
Policy concepts
Consultation
A structured process for seeking and responding to opinions and feedback
about a policy issue from specific interest groups or interested individuals,
or the community generally.
Contestable advice
Advice that may be delivered or challenged by other individuals or
organisations; specifically in the public policy arena, advice which may be
delivered by various agencies within the public service or by agencies
outside of it.
Cost benefit analysis (or CBA)
A method of assessing proposals according to the monetary value of their
benefits and costs over time.
Cost effectiveness analysis
A method of comparing alternative policies in terms of the specific
benefits to be delivered.
COBRA
(Cost opportunity benefit risk analysis) – a method that builds on a
traditional CBA analysis, but includes opportunities and risks.
Criteria
The specific measures of value and of impacts used to assess
policy options.
Evaluation
Evaluation uses monitoring information and other data to help assess
whether a policy achieves its intended results, and whether the results
contribute to the intended outcomes and objectives. An effective
evaluation can improve the quality of decisions by informing current and
future policy advice with lessons learned from previous experience.
Evidence-based policy
Policy analysis and advice underpinned by empirical evidence.
Hypothesis
A suggested explanation for a group of facts or phenomena made on the
basis of limited evidence as a starting point for further investigation. If you
do not have evidence to support it, you should be explicit about this.
Ideology
A theory or set of beliefs or principles, as applied to public matters.
Implementation
The process or act of putting a policy decision into practice.
Intervention / programme logic
An approach to determining the need for and appropriate point of
government intervention, on the basis of assumptions about the
relationship between outputs and outcomes and associated risks.
64
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Lobbying
Activities by which interest groups try to persuade policy makers to adopt
their position on specific policy issues.
Machinery of government
The structure of executive government departments and ministries as
decided by the Prime Minister (or Premier), which forms the basis for
allocating portfolios to Ministers.
Monitoring
Monitoring is a continuous or periodic process of collecting information on
how a policy is working and what its effects are. That may mean collecting
performance information directly related to the policy or collating
information on economic, environmental and social outcomes in areas
where the policy is operating.
Objectives
Objectives sit beneath outcomes. They can be seen as intermediate steps
towards achieving an outcome – often they’ll be the initial answer to your
‘why’ question.
Outcomes
Outcomes are the effects or consequences of policies, programmes and
services developed by government. In a policy context, they are often
intended (what effect are you aiming for?).
Outputs
The particular, discrete services or products produced by an agency.
Policy analysis
Analysis of a policy problem (or opportunity) which includes assessment of
alternative responses, for decision-makers to choose between.
Policy capability
The knowledge, skills and competencies (individual or collective) required
to provide sound policy analysis and advice.
Policy capacity
The extent to which an organisation or government has the skills and
resources to carry out its policy intentions.
Policy design/formulation
The activity of developing a position on a policy issue.
Policy implementation
The process of converting a policy decision into action.
Stakeholder
A person or group with a stake or interest in a given policy; anyone who
might be affected by a pending or implemented policy.
Strategic policy
Policy that sets out medium- to long-term settings and charts a broad
direction. (It is contrasted with operational policy).
WL 13527
65
Appendix 1 – Glossary of key concepts, continued
Strategic thinking
Thinking which has regard for medium- to long-term effects and
consequences and/or is very broad in scope.
Systems approaches to policy
Approaches that portray individuals and organisations as part of a larger
complex and interconnected system which affects policy inputs,
processes, outputs and outcomes.
Environmental concepts
Adaptive management
Adaptive management is a structured, iterative process of optimal
decision-making in the face of uncertainty with an aim to reducing
uncertainty over time by collecting more information. Adaptive
management is often characterised as ‘learning by doing’.
Ecosystem services
The benefits supplied, directly or indirectly, to humans by natural
ecosystems are often referred to as ecosystem services. For example, a
river provides human drinking water, aquatic habitat for birds and
animals, and a source of nutrients for the coastal marine environment. A
forest might provide erosion control and help mitigate flooding.
Environmental bottom line
‘Environmental bottom line’ refers to limits (which may be a standard,
guideline, qualitative state or quantitative measure) that denote an
acceptable level of impact on ecosystems or environmental values.
Irreversible impacts
When thinking about irreversibility, it may be useful to consider not just
whether an impact on the environment can be reversed, but the costs and
time involved in reversing it. If an impact is irreversible it does not mean
that we should therefore avoid that impact, but that the opportunity cost
of lost choices must be taken into account.
Precautionary approach
The precautionary approach is based on the concept of taking anticipatory
action to prevent possible harm under circumstances where there is a
level of scientific uncertainty.
Sustainable development (Brundtland definition)
Meets the needs of the present without compromising the ability of future
generations to meet their own needs.
Triple/quadruple bottom line
An approach to measuring performance in terms of the achievement of
economic, social and environmental/cultural outcomes.
66
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Economic concepts
Agent
A decision-maker in a model. Typically, every agent makes decisions by
solving a well or ill-defined optimization/choice problem. The term agent
can also be seen as equivalent to a player in game theory.
Average cost
The total cost of production divided by the number of units produced (cf.
marginal cost).
Demand
The amount of product that is desired by customers (cf. supply). At a
lower price for a product, demand would – for most goods – be higher,
and vice versa. The point at which demand equals supply will be at the
market clearing (or equilibrium) price.
Efficiency
Can be divided into allocative efficiency (the concept of assigning
resources to their best use), production (or technical) efficiency (the
concept of producing outputs of a given quality at least cost), and dynamic
(ensuring that resources can move over time to their best use as
technologies and markets change).
Elasticity
The ratio of the per cent change in one variable to the per cent change in
another variable (for example, if petrol prices rose by 10 per cent and
demand for petrol dropped by 2 per cent then the elasticity for petrol
would be –0.2).
Externalities
Consumption or production costs (monetary or otherwise) borne by
people other than the producers or consumers because the market price
of the good does not capture all of its costs. For example, a factory that
pollutes a stream imposes costs (loss of clean water, loss of biodiversity,
loss of enjoyment) on other users of the stream.
Government failure (narrow definition)
When government intervention allocates goods and resources less
efficiently than they would have been allocated otherwise. (See market
failure for the private sector version of this problem.) Even if particular
markets may not meet the standard conditions of perfect competition
required to ensure social optimality, government intervention may make
matters worse rather than better.
Government failure (general definition)
Failure to find an efficient and equitable government solution for a
policy problem.
Information asymmetry
When one party to a transaction (such as a buyer or seller) possesses less
information about the goods being exchanged than the other, limiting
their scope for rational choice.
WL 13527
67
Appendix 1 – Glossary of key concepts, continued
Marginal cost
The increase (or decrease) in cost of producing one more (or one less) unit
of production. Many economists will suggest that firms operate at the
point where the marginal cost of an extra unit of production equals the
marginal revenue received.
Marginal revenue
The increase (or decrease) in revenue of producing one more (or one less)
unit of production. Many economists will suggest that firms operate at the
point where the marginal cost of an extra unit of production equals the
marginal revenue received. For example, the average cost per passenger
on a plane trip might be $200 per passenger, but the marginal cost $20
per passenger. So long as marginal revenues exceed marginal costs then
airlines should continue to accept passengers – even though it may be less
than the average cost.
Market failure
Failure of the market to efficiently or equitably supply a good or a service.
Common examples include public goods, externalities, monopolies,
information asymmetries and equity. Market failures are a common
rationale for government intervention.
Moral hazard
The situation when individuals or firms are cushioned from the
consequences of risky behaviour, which may in turn cause them to take
more risks. For example, insuring your car may lead you to be less careful
about the way you drive.
Opportunity cost
The cost of an opportunity forgone (and the loss of the benefits that could
be received from that opportunity); the most valuable forgone alternative
(for example, the opportunity cost of investing money in a bank may be
that the money cannot be invested in the share market).
Public goods
Goods that are non-rival in consumption (one person using the good does
not reduce another person's access) and non-excludable (it is impossible,
impractical, or illegal to stop someone from consuming the good).
Competitive markets will not produce such goods because it is difficult to
determine the users and charge them for services. An example is public
street lighting.
Public value
The correlate of shareholder value in the private sector and used to
denote the value that arises from collective action.
Rent seeking
An individual, organisation or firm seeks to earn income by capturing
economic rent through manipulation or exploitation of the economic or
political environment, rather than by earning profits through economic
transactions and the production of added wealth.
68
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Social capital
The value that individuals and groups derive from the various relationships
and networks connecting them to individuals and families; related to civil
society.
Sunk cost
Costs that have been spent on a project. Often these cannot be recovered;
it is false economics to include sunk costs in a cost-benefit analysis.
Supply
The amount of some product which is available to customers. At a lower
price for a product, supply would typically be lower (and vice versa).
Transaction costs
Costs associated with searching for information, bargaining for goods, and
developing, policing and enforcing contracts.
Welfare economics
The use of microeconomic techniques to determine allocative efficiency
within an economy and the aggregation of individual income distributions
(welfare) associated with it. Pareto efficiency defines optimal welfare as
the point when no individual can be made better off without making
someone else worse off.
Westminster system
A democratic parliamentary system of government modelled after that in
the UK, which includes a professional and apolitical public service.
Whole of government / integrated government
Working horizontally between agencies and vertically within agencies and
between levels of government to improve coordination and alignment.
Wicked issues / problems
Intractable issues for which there is little agreement about the problem
and/or the solution, and on which the government often lacks the
mandate from citizens to take action.
WL 13527
69
Appendix 1 – Glossary of key concepts, continued
Māori and Treaty of
Waitangi concepts
Ahi-kā-roa
Rights of occupation or use of resources in an area.
Articles of the Treaty
The articles of the Treaty are given here in both te reo Māori and English.
Ko te Tuatahi
Ko nga Rangatira o te wakaminenga me nga Rangatira katoa hoki ki hai i uru ki
taua wakaminenga ka tuku rawa atu ki te Kuini o Ingarani ake tonu atu – te
Kawanatanga katoa o o ratou wenua.
Article the First
The Chiefs of the Confederation of the United Tribes of New Zealand and the
separate and independent Chiefs who have not become members of the
Confederation cede to Her Majesty the Queen of England absolutely and
without reservation all the rights and powers of Sovereignty which the said
Confederation or Individual Chiefs respectively exercise or possess, or may be
supposed to exercise or to possess over their respective Territories as the sole
sovereigns thereof.
Ko te Tuarua
Ko te Kuini o Ingarani ka wakarite ka wakaae ki nga Rangitira ki nga hapu – ki
nga tangata katoa o Nu Tirani te tino rangatiratanga o o ratou wenua o ratou
kainga me o ratou taonga katoa. Otiia ko nga Rangatira o te wakaminenga me
nga Rangatira katoa atu ka tuku ki te Kuini te hokonga o era wahi wenua e pai
ai te tangata nona te Wenua – ki te ritenga o te utu e wakaritea ai e ratou ko
te kai hoko e meatia nei e te Kuini hei kai hoko mona.
Article the Second
Her Majesty the Queen of England confirms and guarantees to the Chiefs and
Tribes of New Zealand and to the respective families and individuals thereof
the full exclusive and undisturbed possession of their Lands and Estates
Forests Fisheries and other properties which they may collectively or
individually possess so long as it is their wish and desire to retain the same in
their possession; but the Chiefs of the United Tribes and the individual Chiefs
yield to Her Majesty the exclusive right of Preemption over such lands as the
proprietors thereof may be disposed to alienate at such prices as may be
agreed upon between the respective Proprietors and persons appointed by
Her Majesty to treat with them in that behalf.
Ko te Tuatoru
Hei wakaritenga mai hoki tenei mo te wakaaetanga ki te Kawanatanga o te
Kuini – Ka tiakina e te Kuini o Ingarani nga tangata maori katoa o Nu Tirani ka
tukua ki a ratou nga tikanga katoa rite tahi ki ana mea ki nga tangata o
Ingarani.
Article the Third
In consideration thereof Her Majesty the Queen of England extends to the
Natives of New Zealand Her royal protection and imparts to them all the
Rights and Privileges of British Subjects.
70
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Co-governance
(No agreed definition.) Government governing (decision-making) a
resource with the involvement of a party(ies).
Co-management
(No agreed definition.) Government managing (sometimes seen as more
involvement in the day-to-day management compared to co-governance)
a resource with the involvement of a party(ies).
Deed of recognition
An acknowledgement of the special association a group has with a
particular place and obligation on the Crown to consult with the postsettlement governance entity and have regard to their views regarding the
special association they have with a site. They also specify the nature of
the input by iwi into management of those areas by the Department of
Conservation and/or the Commissioner of Crown Lands.
Environment accord
An agreement between the Ministry and an iwi group, currently
specifically through the Waikato River Settlement, which sets out
relationship expectations and obligations between parties; also other
Accords for other government departments. Also known as protocols,
relationship agreements.
Hapū
Subtribe.
Iwi
Tribe.
Iwi management plan
(In the Resource Management Act as ‘iwi planning document’) provide
Māori aspirations and values for their rohe. Note that there is no set
standard for what these should be.
Kaitiaki/kaitiakitanga
Guardian/guardianship; intergenerational responsibility inherited through
whakapapa and whanaungatanga at birth to care for the environment.
Kaumātua
Respected elder with knowledge and/or authority.
Kaupapa
Subject, topic, issue, plan scheme or proposal.
Kāwanatanga
Government; dominion; rule; authority; right to govern and to make laws.
Mana
Authority; control; influence; prestige; power.
Mana whenua
Power associated with possession of lands.
WL 13527
71
Appendix 1 – Glossary of key concepts, continued
Mātauranga Māori
Māori world views which are based on the values, traditions and
experience of Māori.
Mauri
Life force, or essence of living things.
Papatūānuku
Mother Earth.
Ranginui
Sky father.
Rangatira
Chief; well born, noble; can be either male or female.
Rangatiratanga
Political sovereignty; chieftainship; leadership; self-determination; selfmanagement.
Rohe
Boundary/tribal boundary.
Rūnanga
Assembly or council.
Statutory acknowledgement
An acknowledgement of the special association a group has with a
particular place and requires that consent authorities provide the post
settlement governance entity with summaries of all applications for
resource consent under the Resource Management Act that may affect
the areas named in the acknowledgements.
Takiwā
District; space; interval of time.
Tāngata whenua
People of the land.
Taonga
Treasures or property which is highly prized.
Tapu
Sacred; subject to restriction.
Tikanga Māori
Māori custom, rule or method; the right way of doing something.
Tūrangawaewae
Home or place where a person comes from.
Wāhi tapu
Sacred place.
72
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Wairua
Spirit of a living thing.
Waka
Canoe; descendants of a particular canoe.
Whakapapa
Genealogy of all things.
Whānau
Extended family; birth.
Whenua
Land/country.
Social and ethical concepts
Equity
Fairness, or equality or equivalence; a value that may be defined against
various specific criteria, including equality of opportunity, outcome
or process.
Intergenerational equity
An ethical principle of fairness regarding policy outcomes affecting
successive generations.
Institutional concepts
Collaboration
Two or more agents or agencies working together purposefully.
Institutions (of policy)
Organisations and structures of the state, society and the international
system.
Institutionalism
A theory that emphasises the influence of the formal and legal structures
of government on public policy development.
Separation of powers
The doctrine that the executive, legislative and judicial branches of
government should be separate.
Risk and uncertainty
concepts
Risk
Risk refers to the effect of uncertainty on objectives 3. Uncertainty can
have a number of qualities including the extent of knowledge about the
likelihood of outcomes and the extent of knowledge about the range of
future outcomes.
With thanks for contributions from Claudia Scott (Professor, School of Government, Victoria University of Wellington).
3
This is the definition from AS/NZS ISO 31000:2009 Risk Management – Principles and guidelines.
WL 13527
73
Appendix 2 – Quality of policy advice
These are the criteria used to assess the quality of policy advice.
1 Customer focus
The purpose of the paper is clear. The paper is pitched to the needs of the
audience – this will usually be a Cabinet committee or the Minister.
It specifies the decisions being sought and when they are required, including
the risks around any delays. The paper makes clear where the advice sits in
relation to time and process. What has happened up to this point, and what
will happen next?
The paper presents advice with a premium on succinctness. The paper is free
of jargon and unexplained acronyms, and uses simple sentence construction
and short paragraphs. Key messages are readily apparent to the reader.
2 Context
The paper appropriately presents the big picture while being focused on the
task at hand. This means that it is both forward looking – what happens next?
and outward looking – the horizons of the paper are not restricted to a
narrowly scoped set of solutions. The Minister and his colleagues can both
see the big picture and take an informed decision about how our
recommended course will help them move towards a medium-term objective.
Done well, the author is not just thinking about the Minister making this
decision, but also about the decisions required over the next 6–12 months.
3 Problems and
opportunities
The paper has a clear problem definition (that is, it identifies what is
problematic with the status quo). The paper indicates the size of the problem,
who is affected and how. It explains how current policy settings contribute to
the problem and identifies why changes to existing policy settings will create
opportunities for better outcomes.
The paper draws a distinction between sectoral and national interests. Where
there are losers from some approaches, the paper sets out how these losers
might be dealt with.
4 Analysis and argument
The paper displays a clear approach to the analysis of options. Sometimes this
will mean spelling the options out, but the approach is at least transparent to
the reader. The framework is fit for purpose. Where there are novel aspects
to a framework, the limitations are made clear. The paper is grounded in the
role of the Government in stewardship of the environment and allowing finite
resources to be allocated amongst competing interests. The logic of the paper
is clear and based on evidence. The analytical frameworks are applied
logically and in a way that makes sense to the client.
The paper is complete in its analysis – it is not selective and does not skew the
data or the analysis to make a particular course of action seem more or
less attractive.
The paper is accurate in its analysis. It does not contain material errors.
74
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
5 Risks
The paper identifies the risks (economic and environmental) of the presenting
problem. (For example, if we ‘wait and watch’, is there a critical point after
which environmental damage becomes acute?)
The paper identifies risks of change options. (What is the probability that a
solution may not work as intended?)
The paper identifies delivery risks.
For all of these risks, mitigation strategies will usually be possible, and these
are identified.
Also, the paper identifies any different views amongst stakeholders about the
probability of both adverse and favorable events, and the level of risk that
is acceptable.
6 Consultation and
collaboration
Internal focus
The paper displays evidence of appropriate collaboration within the Ministry.
Relevant subject experts have been consulted (for example, the Regulatory
Impact Assessment Panel or the Treaty team).
External focus
The paper displays evidence of appropriate consultation and collaboration
with other government agencies and stakeholders. This needs to be fit for
purpose and realistic. If timeframes are short the paper might simply set out
how subsequent consultation should happen. If imposed timeframes are
really short we should tell the Minister about the risks inherent in not
consulting an optimum range of stakeholders.
7 Options
The paper includes a range of practical options for meeting the policy
objectives. The reasoning behind the selection of options is transparent.
Typically different options will reflect different weighting on objectives and
other factors like appetite for risk and implementation feasibility.
If the range of options presented has been constrained, for example, through
an imperative for brevity because of lack of time, this should be made clear.
There is usually a way through the competing demands for complete advice
and for succinct advice.
Options are evaluated against clear criteria. The potential impact of
alternative options is made explicit. The paper estimates the relative size of
any impacts. Where appropriate, issues of implementation, practicality and
timing have been considered.
The paper identifies a preferred option – the Ministry’s first, best advice. The
selection of a preferred option is transparent. The author is clear about how
judgements have been weighted to come to the preferred option.
WL 13527
75
Appendix 2 – Quality of policy advice, continued
8 Conclusions and
recommendations
The paper provides action-orientated recommendations. The
recommendations should be short, unambiguous, complete and have
realistic commitments.
The paper comes to a conclusion and suggests a clear way forward. The
advice is at the same time free and frank, and astute.
9 Presentation
76
The paper is in the correct format and free from errors in grammar,
punctuation and spelling. The paper is well structured, and makes appropriate
use of charts and pictures, if these help. The paper avoids jargon and
unexplained acronyms. Where relevant, the paper meets Cabinet Office
requirements.
Ministry for the Environment May 2011
Professionalising Policy: Cost Opportunity Benefit Risk Analysis
Appendix 3 – Checklist
Things to consider
Analyse
• Have all options, objectives and issues been identified? What
are the outcomes you’re seeking?
• Has business as usual been considered: Is government
intervention necessary?
• Is the policy both forward and outward looking – does it
consider both the bigger and future picture?
• Does it consider a variety of frameworks and theories to
support the analysis such as the environment and economy
principles, market failure or cost benefit analysis?
39, 43
39
51
15, 49
Engage
• Have relevant stakeholders been identified and their views
considered? Have other government departments been
consulted?
• Are there issues of particular interest to / impact on Māori?
Are there Treaty implications from these?
• Is the advice customer focused and tailored to meet the needs
of its audience?
21, 27,
34
36
55
Learn
• Have contextual settings such as social and cultural context,
the economic and fiscal environment and political statements
and government priorities been identified and considered?
• Is the history of a policy area/history of the issue understood
and acknowledged?
• Are there international examples of the policy such as best
practice examples from other countries to draw from? What
are other countries doing/thinking in this space
21
21
38
Validate
• Have all major risks been assessed and mitigation strategies
identified?
• Does the advice deliver a range of practical options and action
oriented recommendations?
• Is the advice of a high quality – is it free and frank, robust and
well presented? Is it consistent with the Ministry’s Quality of
Policy Advice?
Collaborate
• Has expertise/assistance within the Ministry been utilised –
Science team, Information Directorate, Treaty team and so
on?
• Have ideas been debated, discussed and refined with peers
and team members to ensure the advice is balanced and
considers a variety of perspectives?
Where do I go for help?
WL 13527
49–51
54
55
34
27
Talk to your manager or a COBRA expert.
77