PATHFINDER
Mission: Results for New Zealanders
Agenda for WG4
1. Introduction (Chair)
- includes short website update (Greg)
2. WG / WS Process (Chair)
- building effective relationships
- ideas for improvements (human & output)
- feedback on Q&A sessions
3. New WG Products (Roger / Greg)
- 10 early step performance cycle (StW 1-3)
- Intervention Logic (one-off presentation)
- Building Block / Learning Point documents
4. Other matters (Chair)
Focus on Outcomes
Strategic Plan;
Statement
of Intent
STRATEGY,
CAPABILITY,
PERFORMANCE
Value-forMoney
PATHFINDER
TOOLBOX; DECISION
MAKING SYSTEMS
TRADE-OFFS,
TRANSFERS
(Within Vote)
(Next FY)
Pathfinder ‘Building Blocks’
1. Define & measure ‘mission critical’ outcomes (using
‘state’ or ‘situation’ outcome indicators)
2.
3.
4.
5.
6.
7.
8.
Map causal logic linking outcomes to outputs
Assess impact of interventions / targeting
Assess cost-effectiveness (vs. outcomes achieved)
Management to maximise outcomes (e.g. core/pilots)
Benchmarking with outcomes (business units / nations)
Focus strategic / annual plans on improving o/c
Redesign planning & operations to maximise o/c
(incl. feedback & continuous improvement)
9. Improving outcomes across agencies / sectors
Steps are not sequential – If feasible, do steps #6-9 early
Observation vs. Action
Outcome Type
Measure
Applications
State /
Environment
Indicators
Situation Report
Benchmarking
(client /service groups) Crude targeting
Assessment /
Allocation
Decisions
Risk, Need & Priority setting
Likely Change Demand analysis
(individuals / cases) Risk managemnt
Impact
Assessment
Status quo
outcome
Continue?
Improvement
in outcome Modify? Grow?
(‘treated’ groups)
Kill?
Improving Results –
10 Step Performance Cycle
Identify Core
Outcomes
Define in
Measurable
Terms
Measure
as State
Indicators
Identify Areas
for Change
in State
Define / Refine
Intervention
Logic
Identify
Intervention
Options
Prioritise
Intervention
Options
Design &
Deliver
Interventions
Impact (o/c)
Evaluation
Framework
Cost
Effectiveness
Analysis
Intervention Logic
Decision: Sufficient interest to arrange half-day session?
1. Aligning activities / outputs to outcomes
2. Defining a robust Intervention Logic (IL)
3. Using IL 'backbones' as a top-down
(strategic) tool to scope intervention options
4. Testing intervention logic is robust, and
produces best approach (ex ante, ex post)
5. Strengthening intervention logic using
evidence-based approaches (R&D, eval)
6. Continuous evolution and testing of
intervention logic / frameworks
Documenting Learning Points
Decision: Approval of principles / generic process;
First Priciple … Keep it short(ish), keep it pithy;
1.
2.
3.
4.
5.
6.
7.
What’s best for NZ as generic guidance
Want strong support of documents (-> SG)
(Recognise there is no ‘one-size-fits-all’)
Seek consensus on any disputed points
Initial comments via e-mail
‘Living document’ for discussion at WG
Send to SG (discussion / approval / -> web site)
Documenting Learning
(Decision)
Decision: Approval of format below as start point for writing;
History: WG2 agreed focus on StW 1-3; no feedback on format;
Proposal: ‘Trial by fire’ of the following format …
1.
2.
3.
4.
5.
6.
7.
8.
Executive summary
Purpose statement
Generic framework (10 step process)
Attributes of good outcome frameworks
Methods / steps / issues
Management and ownership
Best practice / good practice (integrated)
Worked examples (refer to supplementary docs?)
Documenting Learning (Contd.)
Attributes of good outcome frameworks, e.g. …
1. Keep it tight (one or few outcomes, clear priorities for activities)
2. Researched (outcomes derived from institutional knowledge and,
where feasible, comparison with ‘similar / like’ agencies)
3. Clear causal links (outcome-output-input links clear; other means of
achieving outcome(s) visible; clear fit to your services / clients; covers
significant activities of your / relevant agencies)
4. Measures (objective / quantitative as possible; non-measurables
identified; built on robust data; confidence intervals known; directly
attributable to agency activities; can ‘drill down’ for underlying trends;
‘impact’ and ‘state’ outcomes; decision-making / quality outcomes)
5. Clear fit w. prioritisation / management decisions (e.g.
measures inform key mgmt decisions, fit resource information)
6. Clear fit with vision / purpose (e.g. strategy / legislation)
7. Known relation to outcomes of other agencies
Documenting Learning (Contd.)
Generic processes, e.g. Building Block #1…
1. Identify Core Outcome(s) (e.g. from mission / purpose identify key
outcomes; compare to o/c / missions used by similar agencies overseas;
ensure good fit w. results you expect from services you deliver)
2. Define in Measurable Terms (translate verbal statement of o/c into
precise definition that matches data available at reasonable cost; ID key
demographic / service / etc groups for which o/c are required)
3. Measure State / Environment Indicators (document system;
gather / process data; measure o/c for all groups w. confidence intervals)
4. Identify Desired Change in State (analyse state indicators for
statistically significant ‘highs and lows’; identify improvements sought;
identify where results are good / some resources could be shifted)
5. Build Results into Business (e.g. report internally and externally;
input into strategic plan, SoI, broad priority setting, KPI, benchmarking, etc)
6. Confirm Fit w. Outcome Management Framework (e.g.
how can results be used to benchmark, strategise, prioritise, etc; how do
measures have to change or be analysed to perform different functions)
Documenting Learning
(Decision)
Decision: Approval of format below as start point for writing;
History: WG2 agreed focus on StW 1-3; no feedback on format;
Proposal: ‘Trial by fire’ of the following format …
1.
2.
3.
4.
5.
6.
7.
8.
Executive summary
Purpose statement
Generic framework (10 step process)
Attributes of good outcome frameworks
Methods / steps / issues
Management and ownership
Best practice / good practice (integrated)
Worked examples (refer to supplementary docs?)
Agenda for WG4
1. Introduction (Chair)
- includes short website update (Greg)
2. WG / WS Process (Chair)
- building effective relationships
- ideas for improvements (human & output)
- feedback on Q&A sessions
3. New WG Products (Roger / Greg)
- 10 early step performance cycle (StW 1-3)
- Intervention Logic (one-off presentation)
- Building Block / Learning Point documents
4. Other matters (Chair)