Division of Adult Correction and Juvenile Justice
The Standardized Program Evaluation Protocol
~SPEP~
Understanding the Tool
and
Program Improvement Implications
1
Historical Context of North Carolina’s Juvenile
Justice Evidence–Based Practices
► First
State-wide community-based
alternatives (CBA) ~ late 1970s
► First “wraparound” model in the US via
Willie M. Program~ 1979 class action
lawsuit
► 1998 NC Juvenile Justice Reform Act ~
Second state to mandate evidence-based
services
2
Historical Context of North Carolina’s Juvenile
Justice Evidence–Based Practices
► First
state to fully adopt the OJJDP Comprehensive
Strategy for Serious, Violent, and Chronic Juvenile
Offenders
► First legislated statewide, county-level Juvenile
Crime Prevention Councils (also first statewide use
of a risk factor assessment beginning in the late
1990’s)
► Most effective statewide use of a juvenile offender
risk and needs assessment instrument along with
a disposition matrix to reduce confinement (Nov,
2001)
3
Historical Context of North Carolina’s Juvenile
Justice Evidence–Based Practices
► First
state to begin implementation of a
state-wide gang prevention and intervention
initiative (Community-Based Youth Gang
Violence Prevention Project) 2009 (Use of
Recovery Act funds) (THE first U. S.
experiment to fund statewide
Comprehensive Gang Model programming!)
4
Historical Context of SPEP in NC
► 1998
NC Juvenile Justice Reform Act
required a statewide evaluation of programs
funded through the JCPC
► The Standardized Program Evaluation
Protocol ~SPEP~ was developed specifically
for this purpose in NC. 2006
► First statewide Level II services based on
SPEP primary service types (2010)
5
Most (57%) JJ programs reduce recidivism:
Outcomes of 556 studies (Dr. Mark Lipsey, 2002)
6
The SPEP Process
Key Steps to SPEP Score
7
SPEP SCORING PROCESS
IDENTIFY
SERVICES
MATCH
AGAINST
RESEARCH-BASED
CATEGORIES
DATA
(DEMOGRAPHIC,RISK,
QUALITY, QUANTITY
FOR EACH SERVICE
SPEP
SCORE
EVALUATE/PROGRAM IMPROVEMENT
8
IDENTIFY
/ MATCH
SERVICES
DATA
(DEMOGRAPHIC,
RISK,QUALITY,
QUANTITY FOR
EACH SERVICE)
Getting Started…….
Classification of Program
Intervention
10
SPEP 101 Definitions
► Structure
or Format: Setting within which the
program services are delivered
 Example: Residential, YDC, Day Treatment
► Program:
Services that are delivered within the
context of a structure or format.
► Service: The treatment ingredient of the
program. Services target key criminogenic risk
factors and treatment needs and are direct
interventions with the juvenile or with others
(peers or family) in order to alter juvenile behavior
that leads to reduced recidivism.
11
Identification of “Philosophies”
toward Altering Juvenile Behavior
► Program
Services Fall into 3 Broad Categories:
 External Control Techniques
► Behavioral
change through instilling discipline, fear, or detection
of bad behavior in the absence of treatment
 Therapeutic Techniques
► Behavioral
change through improved skills, relationships,
insight.
► These are the only services for which SPEP ratings are
completed.
 Other :Services for which there is insufficient research
to estimate the effects on recidivism
12
Program Types Sorted by General
Approach: Average Recidivism Effect
Discipline
Therapeutic
approaches
Deterrence
Surveillance
Restorative
Control
Control
approaches
approaches
Skill building**
Counseling*
Multiple services
-10
-5
0
5
% Recidivism Reduction from .50 Baseline
10
15
13
Further Sorting by Intervention Type
within, e.g., Counseling Approaches
Individual
Mentoring
Family
Family crisis
Group
Peer
Mixed
Mixed w/referrals
0
5
10
15
20
25
% Recidivism Reduction from .50 Baseline
14
Further Sorting by Intervention Type
within, e.g., Skill-building Approaches
Behavioral
Cognitive-behavioral
Social skills
Challenge
Academic
Job related
0
5
10
15
20
25
30
% Recidivism Reduction from .50 Baseline
15
SPEP 101 Definitions
► Primary




Service:
Main focus of the program
Most time/effort spent on this
Recognized as the dominant theme
Other program elements support its
effectiveness
16
SPEP 101 Definitions
► Supplemental




Service:
Other distinct service
Less time/effort spent on this
Serves to reinforce or complement the primary service
Note: Supplemental service can be deemed as a
“qualifying” one in order to receive additional points on
the SPEP tool OR points may be automatic based on
lack of evidence in the research base to couple a
supplemental service with the identified primary service.
17
WHAT IS THE
PROGRAM’S
PRIMARY SERVICE?
IN WHICH OF THE 5
SERVICE GROUPS
DOES THE SERVICE
FALL?
IS THERE A
QUALIFYING
SUPPLEMENTAL
SERVICE ?
Things to Consider………
►
JCPCs will need to fund structures or formats.
(Teen Court, Structured Day Programs)
►
It is important that an analysis of key therapeutic
element (s) within the structures be explored.
JCPCs need to fund Therapeutic
Approaches/Interventions
► JCPCs need to understand the SPEP process in
order to understand SPEP scores and their
implications for program improvement.
►
19
DATA
Examining the Service Quality, Amount of
Service (Duration and Contact Hours), and
Risk Level
20
Data Used for SPEP Scores
► Quality
of Service:
 SPEP Quality of Services Checklist~ Completed
with the Area Consultant
 Examines the following:
►Written
program protocol ~ program manual; written
protocol that describes how the intended services is
delivered
►Staff training /staff retention
►Monitoring of Program effectiveness, protocol and
corrective action to correct deviations
21
SPEP
Quality of
Services
Checklist
Score
Data Used for SPEP Scores
► Amount
of Service: (NCALLIES)
 Duration of Service
►Target
number of weeks specified for each service
type
►% of youth who received targeted weeks of service
►Maximum number of points ~10
 Contact Hours
►Target
number of hours specified for each service
type
►% of youth who received targeted hours of service
►Maximum number of points ~10
23
Data
generated
from NC
ALLIES
Data Used for SPEP Scores
► Risk
level of Youth Served:
 Program’s use of a validated risk assessment
instrument/tool to provide juvenile risk level data on
each juvenile receiving services
► North
Carolina’s Assessment of Juvenile Risk of Future
Offending ~ Validated tool currently being used by Juvenile
Court Services. This risk assessment tool has been selected for
use by program providers. The department will train providers
on its use.
► NC ALLIES will be modified with the addition of a Risk Level
field. Risk Level Scores from referrals from Juvenile Court would
be entered as well as risk level scores generated by the
program.
 Future modifications will include incorporating the risk assessment
tool into NCALLIES.
25
Data Generated via a
Validated Risk
Assessment Tool and
entered into NC
ALLIES
The SPEP SCORE
► Examines
how a specific program is
performing compared to the effective
practice for that service type in the research
► Provides opportunities for program
improvement for programs that fall short of
the evidence-based effective practice
profiles.
27
Systematic
Correction to
Optimize
Recidivism reduction
Effectively
28
Scoring = Improvement Opportunity
► Initial
Score
► Understanding
► Implications
► Program
for Improvement
Improvement Plan
► Subsequent
► Repeat
the Score
Scoring
29
Program Improvement
► Program
Improvement Plans address the areas
needing improvement as identified through the
program’s SPEP score.
► Timeframes for program improvement will be
established
► Monitoring of the status of program improvement
will be incorporated into the JCPC
monitoring/planning processes and Consultant
Program Monitoring
► JCPCs would continue to evaluate the
effectiveness of their funded programs through
general oversight and supporting program
improvement plans.
30
Division of Adult Correction and Juvenile Justice
Questions?
Ronald Tillman
Central Area Consultant
Ronald.tillman@ncdps.gov
919-323-6845
31