Performance Management for
Justice Information Sharing
David J. Roberts
Global Justice Consulting
Bob Roper, CIO/Director of JBITS
Colorado Judicial Branch
Michael Dever
State Policy Advisory, BJA
2006 BJA/SEARCH
Regional Information Sharing Conference
November 28, 2006 San Diego, California
0
Managing Performance
“Performance measurement, in simplest terms, is the
comparison of actual levels of performance to preestablished target levels of performance. To be effective,
performance must be linked to the organizational strategic
plan. Performance-based management essentially uses
performance measurement information to manage and
improve performance and to demonstrate what has been
accomplished. In other words, performance measurement
is a critical component of performance-based
management.”
Source: Will Artley, D.J. Ellison and Bill Kennedy, The Performance-Based Management Handbook,
Volume 1: Establishing and Maintaining a Performance-Based Management Program (Washington, DC:
U.S. Department of Energy, 2001), p. 4.
1
Universal IJIS Elements
• Definition: The ability to access and share
critical information at key decision points
throughout the whole of the justice enterprise;
• Scope: Recognition that the boundaries of the
enterprise are increasingly elastic—engaging not
only justice, but also emergency & disaster
management, intelligence, homeland security,
first responders, health & social services, the
public, etc.;
• Goal: Get the right information, to the right
people, all of the time—underscores the need for
dynamic information exchange
2
Information Sharing Objectives
• 3 Universal Objectives:
– Improve Public Safety and Homeland Security;
– Enhance the Quality and Equality of Justice;
– Gain Operational Efficiencies, Effectiveness, and
Demonstrate Return on Investment (ROI).
3
WHY EVALUATE A CJIS PROJECT?
Are We Beyond the “trust me” Stage?
• INFORMATION IS CONTROL
• PROVIDES FEEDBACK TO IMPROVE PRGM.
• PROVIDES INFO FOR RESOURCE
ALLOCATION
• ENABLES EFFECTIVE PLANNING
• TESTS GENERALIZATIONS BASED ON
INDIVIDUAL EXPERIENCES AND ASSUMP.
• MARKET TO AND DEVELOP SUPPORT
AMONG FUNDING BODIES, CONSTITUENTS,
AND STAFF
4
…In order to reach the goals and
objectives, one needs to measure
and publish because, “People
behave based on how they are
measured.”
5
Public Safety Measures
• Increase the percentage of court dispositions that can be
matched to an arrest—this will improve the quality of the
computerized criminal history records
• Decrease the average response time to establish a
positive identification following an arrest
• Reduce the number of incidents of criminal records being
associated with the wrong person
• Reduce recidivism
• Decrease the amount of time it takes to serve a warrant
• Decrease the amount of time for law enforcement to have
details on protection orders.
• Reduce the fear of crime in target neighborhoods
• Reduce the amount of time it takes users of the integrated
justice system to respond to a request from the public
• Reduce the time it takes to complete a criminal history
background check
• Reduce the number of agencies that can’t communicate
with each other.
6
Quality of Justice Measures
• Reduce the number of civilian complaints against local
law enforcement
• Reduce the number of continuances per case that result
from scheduling conflicts between the courts, law
enforcement, and prosecution
• Reduce the number of cases without a next scheduled
event
• Reduce the average number of days or hours from arrest
to arraignment
• Reduce the average time a defendant is held while waiting
for a bond decision
• Reduce the time it takes for correctional facility intake
• Reduce the number of days it takes to process cases from
arrest to disposition
• Reduce the number of false arrests.
• Reduce the amount of missing data.
7
Efficiency/Effectiveness Measures
• Reduce the number of hours
that staff spends entering data
manually or electronically
• Reduce the costs of copying
documents for justice
organizations
• Reduce the number of hours
spent filing documents
manually
• Reduce the number of hours
spent searching other
governmental databases
• Increase the number of law
enforcement personnel
performing community
policing tasks, instead of
administrative tasks
• Increase the number of
electronic data transfers
between justice agencies
• Reduce the amount of missing
information in criminal justice
databases
• Reduce the number of
corrections needed in
databases maintained by CJIS
agencies
• Decrease the number of
warrants that never get
entered into the state registry
• Increase the number of query
hits on each agency database
• Reduce the number of hours it
takes to enter a court
disposition into the state
criminal history repository
8
Walking Thru an Example
9
Specific Example
10
Process vs. Impact Evaluations
• Process evaluations focus on how the
initiative was executed; the activities, efforts,
and workflow associated with the response.
Process evaluations ask whether the
response occurred as planned, and whether
all components worked as intended.
Fundamentally, a process evaluation posits
the question, “Are we doing the thing right?”
• Impact evaluations focus on the outcome
(the what) of the initiative; the output
(products and services) and outcome
(results, accomplishment, impact). Did the
problem decline or cease? And if so, was the
response the proximate cause of the decline?
Fundamentally, the impact evaluation posits
the question, “Are we doing the right
thing(s)?”
11
Establishing a Performance
Management Program
The Six Steps to Establishing a Performance-Based Management Program
Source: Will Artley, DJ Ellison and Bill Kennedy, The Performance-Based
Management Handbook, Volume 1: Establishing and Maintaining a
Performance-Based Management Program (Washington, DC: U.S.
Department of Energy, 2001)
12
Step 1: Define Mission and Strategic
Performance Objectives
1. Mission statements identify the overall purpose for which
the organization is organized.
2. Vision statements describe the future business
environment and the role of the organization within it.
3. Value statements reflect fundamental beliefs and values
guiding the agency, the nature of their responsibilities, and
the philosophy underlying their approach.
4. Assumptions are also frequently discussed in strategic
planning efforts, describing business environmental
conditions that are expected in the future.
5. Business strategies identify how objectives are to be
accomplished, e.g., community-oriented policing,
integration of justice information systems,
subscription/notification capabilities, etc.
13
Step 2: Establishing an Integrated
Performance Management Framework
Major Elements in Creating a Performance Management
Framework
1.
Define the Relationship of Performance Measurement
to the Strategic Planning Process
2.
Build the Performance Management Team
3.
Address Stakeholder/Customer Needs
4.
Understand Performance Measurement Terminology
5.
Manage Performance Measurement
6.
Accept Accountability for Measures
7.
Communicate
8.
Know How to Check/Test Your Measures
9.
Learn From Others
10. What Do You Measure Yourself Against
14
Performance Planning Template
•
•
•
•
•
•
•
•
•
•
•
•
Performance Plan Template
Mission/Vision/Values/Assumptions
Strategic Performance Objective 1
Operation/Activity
Purpose
Execution
Performance Target(s)
Performance Measures
Owner
Resources
Budget
FTE
•
•
•
•
•
•
•
•
•
•
Strategic Performance Objective 2
Operation/Activity
Purpose
Execution
Performance Target(s)
Performance Measures
Owner
Resources
Budget
FTE
Defined/Action taken
Relate project to Agency Mission/Vision, etc.
Defined
Title
Describe purpose(s) of initiative
Define how you’re going to do it
Identify target(s) (e.g., reduce violent crime by 10%)
How you’re going to measure it
Person responsible/accountable
Resources needed for this initiative
Funding dedicated to this initiative
Staffing dedicated to this initiative
Defined
Title
Describe purpose(s) of initiative
Define how you’re going to do it
Target(s) (e.g., reduce violent crime by 10%)
How you’re going to measure it
Person responsible/accountable
Resources needed for this initiative
Funding dedicated to this initiative
Staffing dedicated to this initiative
15
Step 3: Establish Accountability for
Performance
•
Building Accountability:
–
–
–
•
Authority refers to the power to influence or
command thought, opinion, or behavior.
Responsibility means that one is liable to be called to
account as the primary cause, motive, or agent.
Accountability, on the other hand, is an obligation or
willingness to accept responsibility and to account for
one's actions.
Internal vs. External Accountability
–
–
Internal Organizational Accountability – Internal
organizational accountability refers to the
establishment of the upward and downward flow of
accountabilities between management and
individuals and teams within the organization.
External Organizational Accountability – In external
organizational accountability, the organization
answers to/reports to its stakeholders on both its
organizational performance and organizational
behavior.
16
Step 4: Establish a Process/System for
Collecting Data to Assess Performance
Develop a Plan:
1.
2.
3.
4.
5.
6.
7.
8.
Information Requirements
Information Sources
Data Collection Processes
Data Collection and Reporting Frequencies
Data Collection Costs
Data Protection
Data Quality
Trial Run
17
Step 5: Establish a Process/System to
Analyze, Review, and Report Data
Data Analysis Strategies
1.
2.
3.
4.
5.
6.
7.
Assess the Quality of Data
Employ Analytic Methods
Calibrate Baseline Measures
Test Hypotheses
Know What to Measure
Data Presentation—How Will the Data be Used and
Reported?
Create an Executive Dashboard
18
Sample Executive Dashboard
19
Step 6: Establish a Process/System to
Use Information to Drive Improvement
1.
2.
3.
4.
Shape Organizational Culture
Make Information Broadly Available
Reengineer Business Processes
Build Performance Management into Everyday
Operations
a.
b.
c.
d.
e.
f.
Executive Support and Organizational Commitment
is Required
Comprehensive System of Accountability and
Responsibility is Required
Flattens the Organization
Identify Problems Early
Ensure Progress and Keep Projects on Target
Demonstrate Value
20
Resources
•
•
•
•
•
Will Artley, DJ Ellison and Bill Kennedy, The Performance-Based Management Handbook,
Volume 1: Establishing and Maintaining a Performance-Based Management Program
(Washington, DC: U.S. Department of Energy, 2001) at
http://www.orau.gov/pbm/pbmhandbook/pbmhandbook.html
John E. Eck, Assessing Responses to Problems: An Introductory Guide for Police ProblemSolvers (Washington, DC: Center for Problem-Oriented Policing, no date), at
http://www.popcenter.org/Tools/tool-assessing.htm
Michael Geerken, The Art of Performance Measurement for Criminal Justice Information
System Projects, (Washington, DC: U.S. Department of Justice, Bureau of Justice
Assistance, 2006 [forthcoming])
Robert H. Langworthy (ed.), Measuring What Matters: Proceedings from the Policing
Research Institute Meetings, (Washington, DC: NIJ/COPS, July 1999, NCJ 170610), pp. 3753.
David J. Roberts, Law Enforcement Tech Guide: Creating Performance Measures that
Work! A Guide for Law Enforcement Executives and Managers to Assess and Measure
Performance, (Washington, DC: U.S. Department of Justice, Office of Community Oriented
Policing Services, 2006 [forthcoming from SEARCH and COPS])
21
Performance Measurement Resources
• More information on GPRA
and PART can be found at:
–www.omb.gov/part
–www.results.gov
–www.expectmore.gov
22