Add the Public’s Voice to Your
Performance Measures, Reports and
Management Practices
Oregon Public Performance Measurement
Management Association
Annual Conference
Willamette University
Salem, Oregon
July 15, 2009
Barbara J. Cohn Berman
National Center for Civic Innovation
Center on Government Performance
Overview of this Presentation
•
Who We Are
•
Why We Have Been Listening to the Public
•
How We Have Been Listening to the Public
•
–
What We Found: Some Highlights
–
Applying some new measures
What Happens When Governments Listen to the Public: Government
Trailblazers
–
What they are finding
–
What they are doing
– Integrating strategic planning, performance measurement and performance management
-- Who they are
•
What More Needs to be Done
© National Center for Civic Innovation, 2009
2
Who we are:
Three affiliated organizations–nonprofit, nonpolitical, independent
Fund for the City of New York (1968)
Has a long history of introducing innovations in performance
measurement and technology to government (introduced
Scorecards of government performance in the 1970s)
Mission: To be responsive to the needs of the city and seek opportunities to
improve the performance of government and the quality of life of its citizens
National Center for Civic Innovation (2002)
International Institute for Community Solutions (2002)
© National Center for Civic Innovation, 2009
3
Center on Government Performance
•
Established in 1995
•
Purpose
 Determine:
 how the public assesses government performance and city conditions
 if the public’s measures are different from government’s
 Create and apply some new measures that reflect the public’s
perspective
 Engage the public in government’s performance measurement and
reporting process whenever possible
 Spread the word about the significance of our findings
© National Center for Civic Innovation, 2009
4
Beyond that, the Center aims to:
•
Interest government in aligning its performance with the public’s
point of view
•
Provide data that government and the public can trust
•
Encourage constructive communication between government and
the public
•
Improve government performance
© National Center for Civic Innovation, 2009
5
Why
•
A lack of trust and understanding between government and the public
– Few, if any, opportunities to have constructive, neutral and informative
conversations that could improve trust and understanding
•
Historically, government performance measures are created without
consultation with the public
© National Center for Civic Innovation, 2009
6
Systemic misalignments…
Employees’ job descriptions and performance
evaluations are not related to the
government’s workload, budget goals and
strategic plans, and they are different from the
government’s performance measures, and they
are different from the
public’s desires, needs and expectations,
Major disconnects result
© National Center for Civic Innovation, 2009
7
Confusion on the part of
employees;
Low opinion of government -perception of poor
performance;
Frustration and anger for both
the public and government….
Makes effective performance management difficult.
© National Center for Civic Innovation, 2009
8
And there is this…
Public Employee Disassociation Syndrome
(my phrase)
Some government employees give up their “member of the public”
perspective when they are in the public employee role.
This disassociation can lead to unresponsiveness and lack of
empathy.
© National Center for Civic Innovation, 2009
9
Our Approach: Citizen Driven Measurement of
Government Performance
(Ongoing consultation with government)
© National Center for Civic Innovation, 2009
10
11
How we started this work -- in NYC:
focus group research
• Purpose: To find out what indicators people use to judge local
government performance
• Early application of private sector techniques to public sector
• First round of 15 focus groups in 1995; second round in 2001;
just completed the third round in April
• A private non-political research firm was our partner in
conducting the groups
© National Center for Civic Innovation, 2009
12
Why we chose to start our work with
focus group research
• Used successfully in the private sector for decades to align services
and products with the needs of the public
– Several hundred thousand focus groups conducted annually in the U.S.
• Historically, market research has not been used to align government
services with the public’s point of view
• Rigorous focus group research provides a deeper understanding of
people’s judgments and points of view than can be obtained in public
opinion polling and most surveys
© National Center for Civic Innovation, 2009
13
Focus group research and satisfaction surveys can
complement one another
© National Center for Civic Innovation, 2009
14
© National Center for Civic Innovation, 2009
15
Riverdale
Residence of
Participants in
Focus Groups
Washington Heights
Todt Hill
Midland Beach
Tottenville
Eltingville
Oakwood
Great Kills
Canarsie
Sheepshead Bay
Old Town
New Dorp
Arden Heights
Parkchester Country Club
Concourse Village
Morrisania
Melrose
Edgewater Park
Soundview
Longwood
Throgs
Neck
Mott Haven Hunts Point
Harlem
Clason Point
Upper West Side
College Point
Carnegie Hill
Lincoln Square
UES/Yorkville
East Elmhurst
Bayside
Astoria
Flushing
Midtown/Clinton
Long Island City
Chelsea
Elmhurst
Rego Park
Fresh Meadows
West Village
Stuyvesant Town
Forrest Hills
Little Italy
Hollis
Williamsburg
Financial District
Kew Gardens
East Williamsburg
Brooklyn Heights
Jamaica
Clinton Hill
Woodhaven
Bedford Stuyvesant
Fort Greene
Ocean Hill
Crown Heights
Springfield Gardens
Park Slope
Brownsville
Sunset Park
East Flatbush
Port Richmond
Tompkinsville
Bay Ridge
Stapleton
West Brighton
Westerleigh
Rosebank
Rossville
Williamsbridge
Norwood
Baychester
Fordham
Pelham Parkway
High Bridge
Port Ivory
Mariner’s Harbor
New Springville
Wakefield
Eastchester
Belle Harbor
In the diverse groups
• People individually rated 34 city functions:
-
Familiarity
-
Satisfaction with current performance (1-10 scale, 10 = best)
-
Importance (to self and city)
• Then, discussions:
-
Reasoning and cues behind their satisfaction rating
-
Nature of their interaction with the agency/function
-
Aspirations: how they would like this function to be performed in the future
(a proxy for performance standard)
© National Center for Civic Innovation, 2009
17
© National Center for Civic Innovation, 2009
18
Our findings: some highlights
•
People do assess their government’s performance
•
People assess government differently from the way government assesses itself
–
–
–
–
The public is interested in outcomes and the quality of work performed
Governments reports workloads, costs, fte’s,…..
People do not care about which agency or level of government is responsible for what
Governments reports performance by agency
•
Personal experiences and first impressions often define the public’s perception
of a whole agency or a whole government
•
People care about government, understand the work is difficult and complex,
need and want information, recognize improvements
•
People want information from government and about government
•
People want even-handed treatment
•
People feel powerless
© National Center for Civic Innovation, 2009
19
Differences: governments’ typical
measures vs. the public’s perspectives
Government Measure
Public Measure
Public Libraries
# of feet of shelf space
# of reference queries
Staff helpfulness
Availability of materials needed
Accessible hours
Emergency Medical Services
Response time
"They came quickly, knew what to do, and
took my grandmother to the right hospital
right away."
Health
# of restaurants/food stores inspected
Cleanliness and food safety ratings
Roadways
# of work requests
# of roadway miles resurfaced
Smoothness Score
Jolt Score
Street Cleanliness
Tons of refuse collected
© National Center for Civic Innovation, 2009
Absence of litter
Reliable collection schedule
20
Listening to the Public book
© National Center for Civic Innovation, 2009
•
Describes our work and the
importance of citizen-based
performance measurement
•
Introduces/suggests over 120
specific new measures for 21
agency functions
•
Describes three examples of
applying new public-suggested
performance measures
•
Calls for others to join in this work
21
One Focus Group Finding:
People judge government performance by observable street level conditions that involve
many government agencies. People often don’t know or care which agencies are
responsible.
•
When people see litter, graffiti, potholes,
broken street lights, abandoned vehicles,
weeds, rodents, dead trees, trip hazards,
etc., they think that government and the city
as a whole is not working well
•
Multiple agencies or sub-agencies,
contractors and public utilities, BIDs and
individuals are responsible
© National Center for Civic Innovation, 2009
22
Our Challenge
How to capture all the street level problems in one place, accurately, so that
they can be addressed appropriately -- by different government agencies,
utility companies, BIDs, community groups, local businesses, and
individuals, and changing conditions can be tracked over time.
ComNET:
Computerized
Neighborhood
Environment Tracking
8 cities:
110 areas; 45 outside NYC
© National Center for Civic Innovation, 2009
23
© National Center for Civic Innovation, 2009
24
Examples of ComNET results
(varies by neighborhood)

An interagency task force was created to resolve multiple problems on one
block

A $6 million project to reconstruct a street was approved

$25,000 was earmarked to fix an entire city block with potholes and ponding
problems

Refuse collection and cleanup were more effectively targeted

Missing caps on local fire hydrants were replaced

Missing pedestrian ramps were installed…
© National Center for Civic Innovation, 2009
25
Vacant lots cleaned and fences repaired
© National Center for Civic Innovation, 2009
26
Graffiti removed from mailboxes
© National Center for Civic Innovation, 2009
27
Weeds removed from tree pits
© National Center for Civic Innovation, 2009
28
Litter removed
© National Center for Civic Innovation, 2009
29
Why Is ComNET In Such Demand?
•
Everyone sees/experiences the streets; ComNET relates to people of all
ages
•
Street conditions are a proxy for government effectiveness
•
Provides data communities and government can trust
•
Not emotional
•
People learn what is and is not government responsibility
•
People consider which problems they want to refer to government/what
they want to fix themselves/what they can live with
•
Government can respond more efficiently
•
It helps bridge those gaps of communication and understanding between
government and the public
•
Using new technology is appealing and efficient
© National Center for Civic Innovation, 2009
30
Another finding:
City roadway conditions are highly important and received poor ratings
in 1995, and 2001
Action: Developed and applied two new measures on 676 miles of city streets,
1997 and 1999, to track roadway conditions accurately and objectively:
SMOOTHNESS SCORE and JOLT SCORE
(Using profilometry, matching IRI ratings to public’s ratings)
© National Center for Civic Innovation, 2009
31
© National Center for Civic Innovation, 2009
32
Focus Group Finding:
The way people are treated by government determines how
they judge government performance. First impressions count.
People want from all city agencies and employees:
–
–
–
–
–
–
Accessibility
Courtesy and Respect
Knowledge
Timeliness
Responsiveness
Even-handedness
ACKTREsponsively
© National Center for Civic Innovation, 2009
33
People can report and rate – without recrimination
- their experiences with government (positive and
negative) on these six elements

Government and the public learn about city offices that are
providing responsive services

Outstanding public service can be acknowledged

Areas of public service that need improvement are identified

Information can be summarized and reported for all city agencies
© National Center for Civic Innovation, 2009
34
What Happens When Governments
Listen to the Public?
Government Trailblazers
© National Center for Civic Innovation, 2009
35
Our Trailblazer Program (since 2003)
•
Encourages governments to engage the public in their performance
measurement and reporting
•
47 governments agreed to:
–
Get feedback from the public about their performance measures and reports
–
Test Suggested Reporting Criteria developed by Governmental Accounting
Standards Board
•
–
Implement some/all suggestions from the public
–
Disseminate new, revised versions of reports
–
Institutionalize the process
Pre-requisites for grantees:
–
Have been collecting some performance data already
–
Support from the top of the government
© National Center for Civic Innovation, 2009
36
• 28 cities
• 10 counties
• 2 city/county
• 1 state
• 6 special entities
© National Center for Civic Innovation, 2009
37
© National Center for Civic Innovation, 2009
38
Demonstration Grants (2003-2006)
1.
Alpharetta, Georgia
14. Minneapolis, Minnesota
2.
Ankeny, Iowa
15. Oklahoma Health Care Authority
3.
Austin, Texas
16. Oregon Progress Board
4.
Bellevue, Washington
17. Saco, Maine
5.
Chattanooga, Tennessee
18. Salisbury, North Carolina
6.
Des Moines, Iowa
7.
Durham, North Carolina
19. San Diego Unified Port District,
California
8.
Eugene, Oregon
20. State of Iowa
9.
Irving, Texas
21. Tucson, Arizona
10. Lauderhill, Florida
22. Washington County, Minnesota
11. Little Traverse Bay Bands of Odawa
Indians (Michigan)
12. Maricopa County, Arizona
23. Washington State Department of
Social and Health Services Children’s
Administration
13. Miami-Dade County, Florida
24. West Hartford, Connecticut
© National Center for Civic Innovation, 2009
39
Trailblazer Grants (2006-2008)
1.
Alcohol, Drug and Mental Health
Board of Franklin County, Ohio
11. Newark, New Jersey
12. Newport, Rhode Island
2.
Cambria County, Pennsylvania
13. North Las Vegas, Nevada
3.
City and County of Denver, Colorado
14. City of Ottawa (Canada)
4.
Cumberland County, Pennsylvania
15. City of Toronto (Canada)
5.
Decatur, Georgia
16. City of London (Canada)
6.
Derby, Kansas
17. City of Sudbury (Canada)
7.
District of Maple Ridge, British
Columbia (Canada)
8.
Guilford County, North Carolina
9.
King County, Washington
10. Metropolitan Government of Nashville
and Davidson County, Tennessee
© National Center for Civic Innovation, 2009
18. Region of Waterloo (Canada)
19. Palm Bay, Florida
20. St. Louis County, Minnesota
21. Snohomish County, Washington
22. Somerville, Massachusetts
23. Vancouver, Washington
40
What Trailblazers hear when they listen
to the public
People want:
•
Reports and information presented clearly
•
Honest reports about how government programs are working
•
All the news, not just good news
•
To understand the challenges that their government and their community are
facing
•
To know how and where they can obtain additional information about services
and key issues
•
To be able to evaluate information for themselves, without “spin”
•
To know what other jurisdictions are doing and how they are doing in
comparison
© National Center for Civic Innovation, 2009
41
How Trailblazers are getting feedback from the
public and learning about the public’s
perspective
• Focus group research
• Citizen surveys
• Online surveys
• Real time voting in a public meetings
• Direct interviews
• Feedback forms on websites
• Neighborhood meetings
• Mystery shoppers
© National Center for Civic Innovation, 2009
42
What They Are Doing
•
Integrating their strategic plans and strategic goals with their
performance measures in their performance reports – moving
toward performance management
•
Disseminating new reports to the public
– Websites that enable the public to drill down for detailed data
– Traditional hard copies of reports
– New formats
• Scorecards
• Newspaper inserts
•
Reports on themes not departments
– Infrastructure
Some are forming and/or joining regional benchmarking initiatives
© National Center for Civic Innovation, 2009
43
Who are Trailblazers?
Other
(Tax Collector, Service
Im provem ent
Coordinator, Strategic
Planner)
16%
Program Directors
(Planning,
Management,
Benchmarking, Admin,
Stat Programs)
28%
Chief
Financial/Budget/Audit
Officers
22%
Analysts
16%
Assistant City/County
Managers or
Administrators
18%
© National Center for Civic Innovation, 2009
44
Where do Trailblazers work?
Other
(HR, Data, Stat)
11%
Special Offices
(Strategic Planning,
PM&R)
13%
Finance/Budget
28%
Communications
4%
Audit
7%
City/County/State
Executive,
Manager, or
Administrators'
Office
37%
© National Center for Civic Innovation, 2009
45
Why a Trailblazer ?
•
“Lone ranger” government innovator
•
“Wanted to go beyond CAFR reports and existing performance
measurements”
•
Pre-existing interest in reaching out to communities and the public
•
Prestige – national recognition
•
Funding helped overcome several obstacles
– “Grant funding would and did give us the political will to move ahead with the next
steps we wanted to do and knew we need to do. The grant award made the
commissioners more supportive of the process.”
– “Additional funding for this work. Very little in the city’s budget for non-salary
spending.”
•
Pride in continuous improvement
– “We consider ourselves leaders in the field and understand the need for continuous
improvement.”
© National Center for Civic Innovation, 2009
46
What happened: changes since 2003
From no performance reports or foot-high budget documents
To imaginative, creative reports
From reports for internal use only
To broad distribution using varying means
Accessible on practically every Trailblazer
government’s website
From antipathy, reluctance, skepticism, fear of engaging
with the public
To Trailblazers saying:
“It’s good to know that [the public] is interested in us.”
“They helped us recognize that we have been collecting
some data needlessly.”
“All encounters with the public do not have to be confrontational.”
© National Center for Civic Innovation, 2009
47
Other changes
•
“Elected officials use the language of our program when talking and they use the outcomes as
a filter for decision making.”
•
“The biggest change is that we’re thinking about the same topics we’ve talked about in the
past, but from the perspective of neighborhoods [or other recipients of government services.]”
•
“Changing our budgeting process to one based on needs and measurable results rather than
being primarily political [pressure] based is a huge step.”
•
“Greater ‘listening’ by our leadership to citizen survey data.
•
Greater emphasis on a ‘data driven’ organization and more attention to citizen priorities.”
•
“Slowly the managers are taking ownership of the data and are reacting, in a positive way, i.e.,
by looking for ways to improve. Awareness of the data is encouraging the public to question
management and public officials about what they do to improve service delivery.”
© National Center for Civic Innovation, 2009
48
More…
Culture has changed in response to the public
– Less resistance to making performance data publicly available
– An increase in transparency and accountability
Performance measures are used in management
– Citizen feedback is being used to develop measures
– Performance data is driving management and budget decisions
– Performance data incorporated into strategic plans and vice versa
© National Center for Civic Innovation, 2009
49
Government managers’ advice
Expect resistance:
•
“Some legislators were resistant to the idea of a city office running
neighborhood meetings.”
•
“…. ‘push back’ from some department heads.”
•
“….concerned that this not create more work for their staff.”
•
Loathe to change existing reports
•
Disinclined to report “bad news,” especially staff of elected officials
- continued
© National Center for Civic Innovation, 2008
50
Typical reactions:
•
Struggled at first
– Not accustomed to listening to the public without a defensive pose
•
Comments from the public were “eye-opening,” “interesting,” “useful”
•
Creating new, understandable reports is a welcome,
creative challenge
•
Glad to learn that people are interested in what they are doing
•
Getting interesting feedback from the public
•
Learning new communication skills
- continued
© National Center for Civic Innovation, 2009
51
What is needed for success:
•
Support from the top
– Navigating political seasons is challenging. Some new mayors, city/county
managers, governors encouraged further development; others discouraged it.
•
Expert market research assistance is highly recommended.
•
Time: We observed major shift in attitude and performance
reports in a year’s time.
© National Center for Civic Innovation, 2009
52
Why are citizen-informed performance measurement
and reporting and performance management
catching on?
•
An important role in government transparency and accountability
– “It is the right thing to do.”
•
Data are used in making management decisions as well as informing to
the public
•
Non-confrontational methods of listening and informing have improved
the public’s understanding of government and the level of their trust
•
The President Obama message
“The question we ask today is not whether our government is too big or
too small, but whether it works….” President Barack Obama, January 20, 2009
© National Center for Civic Innovation, 2009
53
“Engaging the Public in
Local Government
Performance Measurement
and Reporting”
- Spring 2008 Special Issue
© National Center for Civic Innovation, 2009
54
What’s next?
• Already mentioned transparency, accountability, data driven, responsive
to constituents, ongoing….
• Other aspects of performance management:
–
Aligning government’s internal processes with one another and with the
public’s perspectives
© National Center for Civic Innovation, 2009
55
Remember this?
If employees’ job
descriptions are different
from the organization’s
performance standards
and measures and they
are different from the
public’s desires and
expectations, a major
disconnect is the result
Low opinion of
government, perception of
poor performance,
frustration and anger on
the part of both the public
and government….
Costly results!
© National Center for Civic Innovation, 2009
56
What needs to be done?
Management Implications/ Professional
Development Imperatives
•
Broad training of government employees is necessary:
–
–
–
–
–
–
•
Seeing the larger picture
Understanding the strategic planning and goal setting process
Seeing the connection between their job and the government's strategies and plans
Learning why listening to the public is important
Being responsive in a broader sense
Incorporating the training principles into employee performance standards
Listening and collaborative skills need to be developed so that effective
communication takes place between government and:
–
–
–
–
–
The public
Colleagues in other sections of an agency
Other agencies that complement their work
Contractors
Other public bodies
•
More intra- and inter-agency operational units need to be institutionalized
•
Always ask: “As a member of the public, do I think that what we are doing is
making sense? If not, why are we doing it? What must we do differently?
© National Center for Civic Innovation, 2009
57
Why listen?
Listening can lead to:
•
Informed and respectful dialogue between government and the public
•
Better alignment between government programs and the public’s
needs
– Improved services; improved management
•
Greater trust in and understanding of government
•
Thanks and congratulations from the public to government for
listening and being responsive
What could be better or more necessary in a democracy?
© National Center for Civic Innovation, 2009
58
© National Center for Civic Innovation, 2009
59
Thank you!
www.civicinnovation.org
www.fcny.org
© National Center for Civic Innovation, 2009
60