Outcomes-Based Accountability TM “Turning the Curve” The Fiscal Policy Studies Institute Websites raguide.org resultsaccountability.com Book - DVD Orders amazon.com resultsleadership.org SIMPLE COMMON SENSE PLAIN LANGUAGE MINIMUM PAPER USEFUL Outcomes-Based Accountability is made up of two parts: Population Accountability about the well-being of WHOLE POPULATIONS For Communities – Cities – Counties – States - Nations Performance Accountability about the well-being of CUSTOMER POPULATIONS For Services – Agencies – and Service Systems COMMON LANGUAGE THE LANGUAGE TRAP Too many terms. Too few definitions. Too little discipline Benchmark Outcome Result Modifiers Indicator Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic Measure Goal Objective Target CoreYour qualitative made upsystemic jargonobjectives here Measurable urgentstrategic indicators Lewis Carroll Center for Language Disorders DEFINITIONS OUTCOME or RESULT 1. A condition of well-being for Population children, adults, families or communities. Children born healthy healthy, Children ready for school, school Safe communities, Environment, Prosperous Economy communities Clean Environment INDICATOR or BENCHMARK 2. A measure which helps quantify the achievement of an outcome. Rate of low-birthweight babies babies, Percent ready at K entry entry, rate air quality index, index unemployment rate crime rate, Performance PERFORMANCE MEASURE 3. A measure of how well a program, agency or service system is working. 1. How much did we do? Three types: 2. How well did we do it? 3. Is anyone better off? = Customer Outcome IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE? RESULT 1. INDICATOR 2. PERF. MEASURE 3. RESULT 4. INDICATOR 5. RESULT 6. INDICATOR 7. Safe Communities Crime rate Average police response time People are physically & mentally healthy Rate of heart disease People have living wage jobs and income % of people with living wage jobs and income PERF. MEASURE 8. % of participants in job training who get living wage jobs POPULATION ACCOUNTABILITY Community Outcomes for Christchurch, NZ 1. A Safe City 2. A City of Inclusive and Diverse Communities 3. A City of People who Value and Protect the Natural Environment 4. A Well-Governed City 5. A Prosperous City 6. A Healthy City 7. A City for Recreation, Fun and Creativity 8. City of Lifelong Learning 9. An Attractive and Well-Designed City City of London Ontario, Canada Priority Results ● A Strong Economy ● A Vibrant Diverse Community ● A Green and Growing City ● A Reliable Infrastructure ● A Safe City Source: Draft Strategic Plan Sept 2011 CARDIFF, WALES Community Outcomes 1. People in Cardiff are healthy 2. Cardiff has a clean attractive and sustainable environment 3. People in Cardiff are safe and feel safe 4. Cardiff has a thriving and prosperous economy 5. People in Cardiff achieve their full potential. 6. Cardiff is a great place to live, work and play. 7. Cardiff has a fair, just and inclusive society. New Zealand Cardiff, Wales Santa Cruz County Kruidenbuurt Tilburg, Netherlands Leaking Roof (Outcomes thinking in everyday life) Experience Inches of Water BASELINE Measure Not OK ? Fixed Turning the Curve Story behind the baseline (causes) Partners What Works Action Plan #2 Outcomes Experience Indicators Baseline & Story Alcohol-Related Traffic Fatalities United States 1975 to 2005 30 75 people per day 25 45 people per day Tho usand s 20 15 MADD 10 28 people per day 5 2005 2010 Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS) Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater. 2000 1990 1982 1980 1975 0 Christchurch, New Zealand Number of Graffiti Sites FY 2002 to FY 2010 no. of Tagged Sites visited 2002/03 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 20 01 /0 20 2 02 /0 20 3 03 /0 20 4 04 /0 20 5 05 /0 20 6 06 /0 20 7 07 /0 20 8 08 /0 20 9 09 /1 0 no. of sites 2001/02 year 2009/10 Newcastle, UK 20.00% 18.00% 16.00% 14.00% 12.00% 10.00% 8.00% 6.00% 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Ncle 14.5 14.5 16.8 14.5 Revised 9 Nov 2007 17 15 11.9 10.6 9.5 9.3 Nov 08 – Jan 09 8.5 Source: Connexions Tyne and Wear, UK PERFORMANCE ACCOUNTABILITY “All Performance Measures that have ever existed for any service in the history of the universe involve answering two sets of interlocking questions.” Service Performance Measures Quantity Quality How Much How Well did we do? did we do it? (#) (%) Performance Measures Effort How hard did we try? Effect Is anyone better off? Service Performance Measures Effort How Much How Well Effect Quantity Quality Input Effort How much service did we deliver? How well did we deliver it? Output Effect Service Performance Measures How much change / effect did we produce? What quality of change / effect did we produce? Effect Effort Service Performance Measures Quantity Quality How much did we do? How well did we do it? Is anyone better off? # % Effort Education Quantity Quality How much did we do? How well did we do it? Number of students Student-teacher ratio Effect Is anyone better off? Number of 16 olds with 5 A to C GCSE’s Percent of 16 yr olds with 5 A to C GCSE’s Number with good school attendance Percent with good school attendance Effort Drug/Alcohol Treatment Program Quantity Quality How much did we do? How well did we do it? Number of persons treated Percent of staff with training/ certification Effect Is anyone better off? Number of clients Percent of clients off of alcohol & off of alcohol & drugs drugs - at exit - 12 months after exit - at exit - 12 months after exit Effort Waste Management Services Quantity Quality How much did we do? How well did we do it? # tonnes of residential waste Unit cost per tonne collected Effect Is anyone better off? #/amt to land fill % to land fill #/amt diverted from landfill % diverted from landfill Effort Fire Department Quantity Quality How much did we do? How well did we do it? Number of responses Response Time Effect Is anyone better off? # of fires kept to room of origin % of fires kept to room of origin Effort Not All Performance Measures Are Created Equal Quantity Quality Quality How much did we do? How well did we do it? Least Also Very Important Important Effect Is anyone better off? Most Important Effort The Matter of Control Quantity Quality How much did we do? How well did we do it? Most Control Effect Is anyone better off? Least Control PARTNERSHIPS Performance Accountability Types of Measures found in each Quadrant How much did we do? How well did we do it? # Clients/customers served % Common measures # Activities (by type of activity) % Activity-specific measures e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost e.g. % timely, % clients completing activity, % correct and complete, % meeting standard Is anyone better off? % Skills / Knowledge # # # # Point in Time vs. 2 Point Comparison # (e.g. parenting skills) % Attitude / Opinion % (e.g. toward drugs) % Behavior (e.g.school attendance) % Circumstance (e.g. working, in stable housing) School Hospital Job Training Fire Department Service: __________________________________ How much did we do? How well did we do it? Primary customers persons trained patients # students Unit cost Primary activity hours diagnostic alarms courses of tests # job instruction to responded Workload ratio % of ___x___ that happen on time Is anyone better off? If your service works really well, how are your customer's better off? # students who persons get patients fully keptwho to room % fires graduate jobs recover of origin Primary v. Secondary Direct v. Indirect Internal v. External LR UR Baseline & Story Watermain Breaks per year London Ontario Watermain Breaks per Year 300 35 30 250 25 200 150 100 Start of Replacement Program Trend without renewal 15 Trend of Financial Plan 10 50 0 1970 5 1980 1990 Main Breaks Pipe age (right axis) Trend (Main Breaks without Renewal) 2000 2010 2020 2030 "Main Breaks without renewal Trend (Financial Plan forecast) 0 2040 pipe age 20 Advanced Baseline Display Create targets only when they are: FAIR & USEFUL Aspiration NOT Punishment Goal (line) Target or Standard ● Avoid publicly declaring targets by year if possible. x Your Baseline Comparison Baseline Instead: Count anything better than baseline as progress. How Population & Performance Accountability FIT TOGETHER THE LINKAGE Between POPULATION and PERFORMANCE POPULATION ACCOUNTABILITY Healthy Births Rate of low birth-weight babies POPULATION OUTCOMES Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage PERFORMANCE ACCOUNTABILITY Job Training Program # persons receiving training Unit cost per person trained # who get living wage jobs % who get living wage jobs CUSTOMER OUTCOMES Contribution relationship Alignment of measures Appropriate responsibility Different Kinds of Progress 1. Data a. Population indicators Actual turned curves: movement for the better away from the baseline. b. Service performance measures: customer progress and better service: How much did we do? How well did we do it? Is anyone better off? 2. Accomplishments: Positive activities, not included above. 3. Anecdotes: Stories behind the statistics that show how individuals are better off. Board of Directors Meeting AGENDA 1. New data 2. New story behind the curves 3. New partners 4. New information on what works. 5. New information on financing 6. Changes to action plan and budget 7. Adjourn SUMMARY OBA in a Nutshell 2–3-7 2 - kinds of accountability plus language discipline Population accountability Performance accountability Outcomes & Indicators Performance measures 3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off? 7 - questions from ends to means in less than an hour. Baselines and Turning the Curve “If I include you, you will be my partner. If I exclude you, you will be my judge.” - Rosell Turn the Curve Exercise POPULATION Creating a Working Baseline from Group Knowledge Indicator or Performance Measure 65% Now Not OK? Turn the Curve Exercise: Population Well-being 5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s) 10 min: Baseline - pick an outcome, and an indicator curve to turn - forecast to 2017 – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes 15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works 10 min: Report convert notes to one page Two pointers to action 4. --------- Off the Wall ONE PAGE Turn the Curve Report: Population Outcome: _______________ Indicator Baseline Indicator (Lay Definition) Story behind the baseline ----------------------------------------------------- (List as many as needed) Partners ----------------------------------------------------- (List as many as needed) Three Best Ideas – What Works 1. --------------------------2. --------------------------3. ---------No-cost / low-cost 4. --------- Off the Wall Sharp Edges Turn the Curve Exercise – Lessons Talk to Action in an hour 1. How was this different from other processes? What worked and what didn’t work? 2. Why did we ask for: a. Results before indicators? b. Forecast? c. Story? d. No cost / low cost? e. Two hats? f. Crazy idea? g. Only 3 best ideas? 3. Do you think a lay audience could understand the reports? 4. How many think you could lead this exercise with a small group? (2+ curves at the same time) Resources www.raguide.org www.resultsaccountability.com RBA Facebook Group Book - DVD Orders amazon.com resultsleadership.org “If you do what you always did, you will get what you always got.” Kenneth W. Jenkins President, Yonkers NY NAACP THANK YOU ! Websites raguide.org resultsaccountability.com --------Book - DVD Orders-------amazon.com resultsleadership.org