“Just the facts, ma’am”:
In search of FACT fidelity
Joe Morrissey
University of North Carolina at Chapel Hill
Festschrift for Gary Bond
IUPUI, Indianapolis, IN
September 23, 2009
Funding from the NIMH and MacArthur Foundation’s Mental
Health Policy Research Network is gratefully acknowledged.
1
“Just the facts, ma’am”


Joe Friday’s
signature directive
in 1950-60s TV
docu-drama about
LAPD
Searching for FACT
fidelity has been a
lot like detective
work . . . cajoling
findings from
unruly data
2
Some facts about FACT




Forensic adaptations of ACT (FACT) are one of
the latest efforts to keep persons with severe
mental illness out of jail
Despite rapid dissemination, current evidence
about FACT’s public safety and mental health
effects is weak
Today, I’d like to add to that evidence base
and discuss future prospects for FACT fidelity
Throughout this work Gary Bond has been an
inspiration . . .
3
Gary Bond’s First Principles of
Mental Health Services Research

#1. “Any untested
service intervention
should not be
demonstrated because
it will fail.”

#2. “A social
experiment is a
contrivance that when
applied to a group of
people leads to a
scientific publication.”
4
Mentally ill & jails



People with mental illness in the criminal
justice system have become the new frontier
for community mental health interventions
Now, more than 1 million jail bookings of
people with SMI each year; SMI prevalence:
14.5% male & 31.0% female detainees
(Steadman et al., 2009)
Relative risk of persons with SMI being jailed
is 150% greater than being hospitalized
(Morrissey et al., 2007)
5
Current ACT evidence



Bond & colleagues (2001): 8 of 10 trials,
usual care equal-to-or-better-than ACT on
arrests & jail use
Calsyn & colleagues (2005): ACT no-betterthan usual care on range of CJ outcomes
Chandler & Spicer (2007): IDDT no-betterthan usual care on range of CJ outcomes
6
Significant ACT Outcomes in 25
RCTs pre-2000 [n of trials, %]*
Outcome
Better
No Difference
Worse
17 (74%)
8 (26%)
0
Housing stability
8 (67%)
3 (25%)
1 (8%)
Symptoms
7 (44%)
9 (56%)
0
Quality of life
7 (56%)
6 (42%)
0
Social adjustment
3 (23%)
10 (77%)
0
Jails/arrest
2 (20%)
7 (70%)
1 (10%)
Substance use
2 (33%)
7 (70%)
0
Vocational functioning
3 (37%)
5 (63%)
0
Psych hosp use
*Source: Bond, Drake, Mueser & Lattimer, 2001
7
ACT as an intervention platform



Bond & colleagues (2001) review also showed
weak effects of ACT on substance abuse &
vocational functioning outcomes
But when ACT teams were retrofitted to
address these issues (via IDDT & supported
employment) subsequent RCTs showed
positive effects
Would same thing happen if ACT was
retrofitted to prevent arrests and
incarcerations?
8
ACT to FACT Adaptations





DACT Core Items
Mobile/comprehensive services/teambased
Psychiatrist on team
1-10 S-to-C ratio
24/7 crisis response
Time unlimited
FACT





Same
Same
Same
Same
Same




Prior arrests
CJ referrals
CJ partners
Court sanctions
9
Other Differences
ACT
 Target: SMI @ risk of
hospitalization
 Goal: Prevent
hospitalization &
sustain community
living
 Vocational, AOD
staffing on team

DACT fidelity
standards
FACT
 Target: SMI @ risk of
arrest/ jail detention
 Goal: Prevent reincarceration



Less vocational,
linked AOD services
Probation officers as
team members
No clinical model or
fidelity standards
10
Current FACT Evidence


Separate pre-post studies (no control groups)
with small samples, FACT associated with
fewer jail days, arrests, hospital days, and
hospitalizations (Lamberti et al., 2001;
Weissman et al., 2004; McCoy et al., 2004)
Still no published reports based on rigorous
comparison group data clearly showing FACT
can improve both mental health & public
safety outcomes
11
Our efforts. . .
1. Birmingham study (2004-05)
abortive effort to retrofit the ‘first’ RCT (so we
thought) on a SAMHSA jail diversion site
2. FACT survey and site visits (2005-06)
surveyed 30 ACT & CJ programs, visited 12
FACT programs to document operating
characteristics and sustainability
3. Mentally Ill Offender Crime Reduction Grant
(MIOCRG) program (2006-09)
discovered 20+ RCTs and opportunity to reanalyze data from several counties in
California
12
MIOCRG initiative




California Board of Corrections: 30 county,
$80.5 million program 2000-04, Sheriffs
Assn. and MH Association bill sponsors
Goals: identify what works most effectively
in reducing recidivism among mentally ill
offenders
Local evaluations with random assignment;
individual data on ~8,000 enrollees
reported to BOC for cross-site evaluation
BOC Report: ACT-like programs improved
CJ and MH outcomes, but major sampling
and statistical problems not addressed
13
MIOCRG re-analysis saga
1. Find Calif. counties w true FACT models
2. Get local evaluators to share data
3. Get CA-DMH to agree to create linked, deidentified services data files Get IRB
approvals at UNC-CH & CA-DMH
4. Get county MHAs to approve re-analysis
plan & send study IDs to DMH for record
linkage
5. Obtain & link data across counties with
common prospective cohort format & with
common variables
6. Run individual site & pooled analyses
14
Found six MIOCRG sites that allowed for FACT v. FICM comparisons
Adaptations
County
Small
Ratio
24/7
Psychia
-trist on
Team
Daily
Mtgs.
Shared
Cases
In-house
Services
RCT
Sample
Sizes
(TX/
CTRL)

70/25

101/47


72/62


79/81


137/98


44/26
PO*
MH
Court
Forensic Assertive Community Treatment
Marin

San
Joaquin

Stanislaus

















Forensic Intensive Case Management
San Mateo

Butte

Solano



* Police or probation officer on the team
15
Best laid plans go awry . . .
1. CA-BOC failed to implement a true
experimental study; we had access to the
MIOCRG data for all 30 sites but we couldn’t
make sense of it
2. Ended up working with three sites with
same evaluator, but even then, the CA-BOC
design led to incomplete data and we were
unable to fix that for 2 of 3 FACT sites
3. So, we resorted to administrative data to
assess impact of FACT at 1 site
16
Setting and design





Mid-size city
FACT program (2000-03) enrolled consumers
from county jail; probation officers on team
DACT scores of 4.5 and 4.6
Retained MIOCRG randomized groups for our
analyses: FACT v. treatment as usual (TAU)
Followed both groups in administrative data
12 mos. pre and 12 & 24 months post
17
Administrative data elements
Mental Health
Public Safety

 Jail use & arrests

Service utilization



# psych hosp days
# crisis contacts
# outpatient visits
Costs

# bookings

# felony/misdemeanor
charges & convictions
# jail days

 Costs
18
Study sample



Participants had histories of frequent jail use
But they also had a lot of mental health
services use in baseline period
Random assignment to FACT and TAU worked
to produce two equivalent groups (age was
only significant difference but it didn’t matter
in multivariable analyses)
19
Sample & randomization results
Baseline
Characteristics
*p<.05
FACT (N= 72)
TAU (N= 62)
Demographic
% Caucasian
% African Am
% Hispanic
% Male
Age: mean (sd)
61
8
22
60
38.8 (10.9)*
65
8
21
58
34.4 (8.9)
Clinical
% Psychotic dx
% Affective dx
% Comorbid SA
61
29
77
72
24
64
Criminal Involvement
% Any booking
% Any felony charge
% Any conviction
96
57
65
95
66
67
Mental health services
% Any hospital use
% Any outpatient use
# hospital days
# outpatient visits
58
88
13.7 (19.6)
29.8 (35.7)
47
84
7.1 (14.1)
26.6 (35.1)
20
Results
Compared to TAU participants:


FACT participants had fewer bookings
(p<.01) and jail days (p<.05) in each year.
FACT participants had more outpatient
visits (p<.001) but fewer days of
hospitalization (p<.05) and incurred lower
overall costs for the county jail and the
county mental health service system.
21
Conclusion

A forensically-oriented, high-fidelity ACT
(FACT) team can alter the criminal justice
involvement of offenders with serious
mental illness, reduce their time spent in
inpatient psychiatric settings, while
providing more appropriate and less costly
outpatient services.
22
Implications for FACT fidelity




Single RCT is never definitive, but it helps
to elevate the evidence base
FACT works, but it is expensive and it
should be carefully targeted to those most
in need, not everyone who ends up in jail
More needs to be done to specify and test a
clinical model for FACT; then, fidelity issues
become meaningful
Some feel criminogenic needs should be
targeted via a CBT add-on to FACT; further
research needed here
23
An interplanetary traveler’s advice
 “In doing meaningful
services research, try
to . . . Fill what’s
empty. Empty what’s
full. And scratch
where it itches.”
 “The really important
thing is . . . not to
stop questioning.”
 “The best is yet to
come.”
24
Acknowledgements . . .
All personal attributions herein
are apocryphal . . .they have
been gleaned, stolen, modified,
invented, and filched from
various sources to fit the
occasion!
25