RBA
Results-Based Accountability
The Fiscal Policy Studies Institute
www.raguide.org
www.resultsaccountability.com
Book - DVD Orders
amazon.com
resultsleadership.org
TM
SIMPLE
COMMON SENSE
PLAIN LANGUAGE
MINIMUM PAPER
USEFUL
Results Based Accountability
is made up of two parts:
Population Accountability
about the well-being of
WHOLE POPULATIONS
For Communities – Cities – Counties – States - Nations
Performance Accountability
about the well-being of
CUSTOMER POPULATIONS
For Programs – Agencies – and Service Systems
Results Based Accountability
COMMON LANGUAGE
COMMON SENSE
COMMON GROUND
THE LANGUAGE TRAP
Too many terms. Too few definitions. Too little discipline
Benchmark
Outcome
Result
Modifiers
Indicator
Measurable Core
Urgent
Qualitative
Priority
Programmatic
Targeted
Performance
Incremental Strategic
Systemic
Measure
Goal
Objective
Target
Measurable urgent systemic indicators
Lewis Carroll Center for Language Disorders
DEFINITIONS
RESULT or OUTCOME
1. A condition of well-being for
Population
children, adults, families or communities.
Children born healthy
healthy, Children ready for school,
school
Safe communities,
Environment, Prosperous Economy
communities Clean Environment
INDICATOR or BENCHMARK
2. A measure which helps quantify the achievement
of a result.
Rate of low-birthweight babies
babies, Percent ready at K entry AEDI
rate air quality index,
index unemployment rate
crime rate,
Performance
PERFORMANCE MEASURE
3. A measure of how well a service, agency or service
system is working.
1. How much did we do?
Three types: 2. How well did we do it?
3. Is anyone better off? = Customer Results
POPULATION
ACCOUNTABILITY
For Whole Populations
in a Geographic Area
Community Outcomes
for Christchurch, NZ
● A Safe City
● A City of Inclusive and Diverse Communities
● A City of People who Value and Protect the Natural
Environment
● A Well-Governed City
● A Prosperous City
● A Healthy City
● A City for Recreation, Fun and Creativity
● A City of Lifelong Learning
● An Attractive and Well-Designed City
Every Child Matters – Children Act
Outcomes for Children and Young People
Being Healthy: enjoying good physical and mental health and living a
healthy lifestyle.
Staying Safe: being protected from harm and neglect and growing up able
to look after themselves.
Enjoying and Achieving: getting the most out of life and developing
broad skills for adulthood.
Making a Positive Contribution: to the community and to society
and not engaging in anti-social or offending behaviour.
Economic Well-being: overcoming socio-economic disadvantages to
achieve their full potential in life.
City of London
Ontario, Canada
Priority Results
● A Strong Economy
● A Vibrant Diverse Community
● A Green and Growing City
● A Reliable Infrastructure
● A Safe City
Source:
Draft Strategic Plan Sept 2011
Luxembourg 2020
Population result: Smart Sustainable Inclusive Growth
Headline Indicators (and targets):
1. % of GDP invested in R&D (interval of 2.3–2.6%)
2. Rate of school dropouts (sustainably llower than10%)
3. % of 30-34 year olds completing 3rd level of education (66%)
4.% reduction in greenhouse gas emissions (20%)
5. % of energy from renewables (11%)
6. % increase in energy efficiency (14,06% in 2016)
7. % of 20-64 year olds employed (73%)
8. Reduction in # of people in or at risk of poverty and social exclusion (6000)
Luxembourg 2020 Headline Indicators
Scorecard
New Zealand
Santa Cruz, CA
Coventry, UK
Kruidenbuurt
Tilburg, Netherlands
Leaking Roof
(Results thinking in everyday life)
Experience
Inches of Water
BASELINE
Measure
Not OK
? Fixed
Turning the Curve
Story behind the baseline (causes)
Partners
What Works
Action Plan
Alcohol-Related Traffic Fatalities
United States 1975 to 2005
30
75 people per
day
25
45 people per
day
Tho usand s
20
15
10
28 people
per day
5
Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)
Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.
2005
2000
1990
1982
1980
1975
0
Newcastle, UK
20.00%
18.00%
16.00%
14.00%
12.00%
10.00%
8.00%
6.00%
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Ncle 14.5 14.5 16.8 14.5
Revised 9 Nov 2007
17
15
11.9 10.6
9.5
9.3
Nov 08 – Jan
09
8.5
Source: Connexions Tyne and Wear, UK
Christchurch, New Zealand
Number of Graffiti Sites
FY 2002 to FY 2010
no. of Tagged Sites visited
2002/03
18000
16000
14000
12000
10000
8000
6000
4000
2000
0
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
20
01
/0
20 2
02
/0
20 3
03
/0
20 4
04
/0
20 5
05
/0
20 6
06
/0
20 7
07
/0
20 8
08
/0
20 9
09
/1
0
no. of sites
2001/02
year
2009/10
Hull, UK
HEALTH AND PUBLIC SAFETY:
WOUNDINGS
Performance
Accountability
For Services, Agencies and
Service Systems
“All Performance Measures
that have ever existed
for any service
in the history of the universe
involve answering two sets of
interlocking questions.”
Service Performance Measures
Quantity
Quality
How
Much
How
Well
did we do?
did we do it?
(#)
(%)
Service Performance Measures
Effort
How hard did we try?
Effect
Is anyone better off?
Service Performance Measures
Effort
How
Much
How
Well
Effect
Effect
Effort
Program Performance Measures
Quantity
Quality
How much
did we do?
How well
did we do it?
Has something improved?
Is anyone
better off?
#
%
Effort
Education
Quantity
Quality
How much did we do?
How well did we do it?
Number of
students
Student-teacher
ratio
Effect
Is anyone better off?
Number of 16 olds
with 5 A to C
GCSE’s
Percent of 16 yr olds
with 5 A to C
GCSE’s
Number with good
school attendance
Percent with good
school attendance
Drug/Alcohol Treatment Program
How much did we do?
How well did we do it?
Number of
persons
treated
Unit
cost of
treatment
Is anyone better off?
Number of clients Percent of clients
off of alcohol &
off of alcohol &
drugs
drugs
- at exit
- 12 months after exit
- at exit
- 12 months after exit
Effort
Waste Management Services
Quantity
Quality
How much did we do?
How well did we do it?
# tonnes of residential
waste
Unit cost
per tonne
collected
Effect
Is anyone better off?
#/amt to land fill
% to land fill
#/amt diverted from
landfill
% diverted from
landfill
Commerce Tourism
Quality
Quantity
How well did we do it?
Effort
How much did we do?
# of
information
packages sent
Unit cost per
package
Effect
Is anyone better off?
Number of
tourists
Tourism
market share
Effort
Fire Department
Quantity
Quality
How much did we do?
How well did we do it?
Number of
responses
Response
Time
Effect
Is anyone better off?
# of fires
kept to
room of origin
% of fires
kept to
room of origin
Effort
General Motors
Quantity
Quality
How much did we do?
How well did we do it?
# of production hrs
Employees per
vehicle
produced
# tons of steel
Effect
Is anyone better off?
# of cars sold
% Market share
$ Amount of Profit
Profit per share
$ Car value after
2 years
% Car value after
2 years
Source: USA Today 9/28/98
Effort
Not All Performance Measures Are Created Equal
Quantity
Quality
How much did we do?
How well did we do it?
Least
Also
Very Important
Important
Effect
Is anyone better off?
Most
Important
Effort
The Matter of Control
Quantity
Quality
How much did we do?
How well did we do it?
Most
Control
Effect
Is anyone better off?
Least
Control
PARTNERSHIPS
Performance Accountability
Types of Measures found in each Quadrant
How much did we do?
How well did we do it?
# Clients/customers
served
% Common measures
# Activities (by type
of activity)
% Activity-specific
measures
e.g. client staff ratio, workload ratio, staff
turnover rate, staff morale, % staff fully
trained, % clients seen in their own language,
worker safety, unit cost
e.g. % timely, % clients completing activity,
% correct and complete, % meeting standard
Is anyone better off?
% Skills / Knowledge
#
#
#
#
Point in Time
vs.
2 Point
Comparison
(e.g. parenting skills)
% Attitude / Opinion
(e.g. toward drugs)
% Behavior
(e.g.school attendance)
% Circumstance
(e.g. working, in stable housing)
School Hospital Job Training Fire Department
Service: __________________________________
How much did we do?
How well did we do it?
Primary customers
# Students/patients/
persons trained
Unit cost
Workload ratio
Primary activity
of instruction
# hours
diagnostic tests
% of ___x___ that
happen on time
job courses
alarms responded to
Is anyone better off?
If your service works really well,
how are your customer's better off?
#
%
students who graduate
patients who fully recover
persons who get jobs
fires kept to room of origin
Primary v.
Secondary
Direct v.
Indirect
Internal v.
External
LR
UR
Baseline & Story
Watermain Breaks per year
London Ontario
Watermain Breaks per Year
300
35
30
250
25
200
150
100
Start of
Replacement
Program
Trend without
renewal
15
Trend of
Financial Plan
10
50
0
1970
5
1980
1990
Main Breaks
Pipe age (right axis)
Trend (Main Breaks without Renewal)
2000
2010
2020
2030
"Main Breaks without renewal
Trend (Financial Plan forecast)
0
2040
pipe age
20
How
Population
&
Performance
Accountability
FIT TOGETHER
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births
Rate of low birth-weight babies
POPULATION
RESULTS
Stable Families
Rate of child abuse and neglect
Children Ready for School
Percent fully ready per K-entry assessment
Alignment
of measures
PERFORMANCE ACCOUNTABILITY
Child Protection Program
# of
investigations
completed
% initiated
within 24 hrs
of report
# repeat
Abuse/Neglect
% repeat
Abuse/Neglect
Contribution
relationship
Appropriate
responsibility
CUSTOMER
RESULTS
Population Accountability
to which you contribute to most directly.
Result:
Indicators:
Every time
you present
your program,
Use a
two-part
approach.
Story:
Partners:
What would it take?:
Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
Story:
Partners:
Action plan to get better:
Different Kinds of Progress
1. Data
a. Population indicators Actual turned curves:
movement for the better away from the baseline.
b. Service performance measures:
customer progress and better service:
How much did we do?
How well did we do it?
Is anyone better off?
2. Accomplishments: Positive activities, not included above.
3. Anecdotes: Stories behind the statistics that show how
individuals are better off.
Board of Directors Meeting
AGENDA
1. New data
2. New story behind the curves
3. New partners
4. New information on what works.
5. New information on financing
6. Changes to action plan and budget
7. Adjourn
IN CLOSING
RBA in a Nutshell
2–3-7
2 - kinds of accountability
Population accountability
Performance accountability
plus language discipline
Results & Indicators
Performance measures
3 - kinds of performance measures.
How much did we do?
How well did we do it?
Is anyone better off?
7 - questions from ends to means in less than
an hour. Baselines & Turning the Curve
Resources
www.raguide.org
www.resultsaccountability.com
RBA Facebook Group
Book - DVD Orders
amazon.com
resultsleadership.org
THANK YOU !
Websites
raguide.org
resultsaccountability.com
Book - DVD Orders
amazon.com
resultsleadership.org
Turn the Curve
Exercises
Unemployment
# Cars Registers in Luxembourg
# Patents Apps (LU) EU Patent Office
% Renewable Energy
Rate CO2 Emissions
Houseing Prices per Sq Meter
% Built Surface (LU)
Creating a Working Baseline
from Group Knowledge
% with 5 good GCSE's
60%
Now
Not OK?
Turn the Curve Exercise: Population
5 min: Starting Points
- timekeeper and reporter
- geographic area
- two hats (yours plus partner’s)
10 min: Baseline
- pick a population result, and an indicator curve to turn
- forecast to 2016 – OK or not OK?
15 min: Story behind the baseline
- causes/forces at work
- information & research agenda part 1 - causes
15 min: What works? (What would it take?)
- what could work to do better?
- each partners contribution
- no-cost / low-cost ideas
- information & research agenda part 2 – what works
10 min: Report convert notes to one page
Two
pointers
to action
ONE PAGE Turn the Curve Report: Population
Population Result: _______________
Indicator
Baseline
Indicator
(Lay Definition)
Story behind the baseline
-----------------------------------------------------
(List as many as needed)
Partners
-----------------------------------------------------
(List as many as needed)
Three Best Ideas – What Works
1. --------------------------2. --------------------------3. ---------No-cost / low-cost
4. --------- Off the Wall
Sharp
Edges
Turn the Curve Exercise: Service Performance
5 min: Starting Points
- timekeeper and reporter
- identify a program to work on
- two hats (yours plus partner’s)
10 min: Performance measure baseline
- choose 1 measure to work on – from the lower right quadrant
- forecast to 2016 – OK or not OK?
15 min: Story behind the baseline
- causes/forces at work
- information & research agenda part 1 - causes
15 min: What works? (What would it take?)
- what could work to do better?
- each partners contribution
- no-cost / low-cost ideas
- information & research agenda part 2 – what works
10 min: Report Convert notes to one page
Two
pointers
to action
ONE PAGE Turn the Curve Report: Performance
Service: _______________
Performance
Measure
Baseline
Performance Measure
(Lay definition)
Story behind the baseline
-----------------------------------------------------
(List as many as needed)
Partners
-----------------------------------------------------
(List as many as needed)
Three Best Ideas – What Works
1. --------------------------2. --------------------------3. ---------No-cost / low-cost
4. --------- Off the Wall
Sharp
Edges
Turn the Curve Exercise – Lessons
Talk to Action in an hour
1. How was this different from other processes?
What worked and what didn’t work?
2. Why did we ask for:
a. Results before indicators?
b. Forecast?
c. Story?
d. No cost / low cost?
e. Two hats?
f. Crazy idea?
g. Only 3 best ideas?
3. Do you think a lay audience could understand
the reports?
4. How many think you could lead this exercise
with a small group? (2+ curves at the same time)
Least Harm
Cutback Exercises
RBA/OBA Least Harm Cutback Exercise 1: POPULATION
Least harm to POPULATION quality of life (Population Accountability)
1. Identify a population result/outcome to which your organization most directly
contributes. (e.g. Healthy People, Sustainable Environment, Children Ready for
School)
2. What is your organization's role in contributing to this result/outcome?
3. What do you do that causes the greatest contribution?
4. What do you do that causes the least contribution? (Least harm cut candidates)
5. How could you change the way you do your work so as to make the same, or
close to the same, contribution with less resources? (Least harm cut candidates)
- Time: 5 minutes for each question
- Record answers for each question.
- Report 3 best ideas and reasoning
RBA/OBA Least Harm Cutback Exercise 2: PERFORMANCE
Least harm to CUSTOMERS' quality of life (Performance Accountability)
1. Identify one service.
2. Identify the primary customer group.
3. What are you doing that is most effective in improving the lives of your customers?
4. What is least effective? (Least harm cut candidates)
5. How could you change the way you do your work so as to make the same, or close
to the same, contribution with less resources? (Least harm cut candidates)
- Time: 5 minutes for each question
- Record answers for each question.
- Report 3 best ideas and reasoning
What’s Next?
A Basic Action Plan for Results Accountability
TRACK 1: POPULATION ACCOUNTABILITY
●
●
●
●
Establish results
Establish indicators, baselines and charts on the wall
Create an indicators report card
Set tables (action groups) to turn curves
TRACK 2: PERFORMANCE ACCOUNTABILITY
● Performance measures, and charts on the wall
for programs, agencies and service systems
● Use 7 Questions supervisor by supervisor and program
by
program in management, budgeting and strategic planning
Next Steps:
1. What’s one thing I could personally
do with what I learned today?
2. What’s one thing I would like to ask
someone else or some organization to
do to support this work?
THANK YOU !
Book - DVD Orders
amazon.com
resultsleadership.org