California Institute for Mental Health
Page 1
IMPLEMENTING SAMHSA EVIDENCE-BASED PRACTICE TOOLKITS
Illness Management and Recovery
Brief description of the practice
The SAMHSA Toolkit is titled Illness Management and Recovery. California constituencies have
opted to call the practice Wellness and Recovery, but this paper uses the original designation
because it is focused as much as possible on the interventions in the particular Toolkit. The
description of the contents of the Toolkit is more extensive than in the other EBP reviews because
of the lack of consistent definitions of particular interventions.
Target Group
The educational materials in the Toolkit are written for schizophrenia, bipolar, and major
depression. The intervention is better if done with consumers who are not in crisis, but the authors
say that it is worthwhile even if a consumer is always in crisis. Most of the research has been
conducted largely with schizophrenia, except for the cognitive- behavioral interventions.
Practice Components
The purpose of the practice is to help people to “develop personalized strategies for managing
their mental illness and moving forward in their lives….the focus of Illness Management and
Recovery is providing people with the information and skills they need in order to make informed
decisions about their own treatment.”
The Toolkit encompasses much of what is believed to be effective in working with consumers in
an outpatient rehabilitation and recovery-oriented system. As such it is based on the underlying
principles of recovery and includes the techniques and strategies that have been developed within
the broad rehabilitation and recovery movement. These are outlined here in some detail to
indicate the breath of the components which are included in the Toolkit.
The Toolkit stresses the importance of the core values that underlie the intervention:

Hope and optimism are the key ingredients

The person is the expert in her own experience of mental illness, how people react to
them, what works and not

Personal choice is paramount; give people the information and skills they need to
make choices regarding their own treatment

Practitioners are collaborators – nonhierarchical relationships are essential

Practitioners demonstrate respect for people who experience psychiatric symptoms –
means respecting person’s views, e.g. that don’t have a psychiatric illness; work to
build common understandings
California Institute for Mental Health
Page 2
The evidence-based skills that are taught: coping skills, relapse prevention, and behavioral
tailoring.



Coping skills
o
Develop more effective way of dealing with persistent and distressing symptoms
o
Identify symptom; behavioral analysis to determine situations in which symptom
is worse; identify coping strategies – then model and practice and do homework
Relapse prevention
o
Help identify early signs and develop plan to avert relapse or reduce severity if
do relapse
o
Often more effective when include someone else
Behavioral tailoring
o
“Behavioral tailoring involves helping people to develop strategies that
incorporate the taking of medication into their daily lives.”
o
Review person’s routines – identify an activity that can be modified to
incorporate taking meds
The intervention uses motivational, educational, and cognitive-behavioral strategies. Each of the
sections of the manual include each of these strategies.

Motivational strategies - “Motivational strategies involve helping people see how
learning information and skills will help them achieve short and long-term goals.”

Educational strategies which should be interactive not didactic with material
presented in small chunks with periodic reviews

Cognitive-behavioral strategies - “Cognitive-behavioral techniques involve the
systematic application of learning principles to help people acquire and use
information and skills in Illness Management and Recovery.” In addition it covers
cognitive restructuring: message that feelings are a byproduct of thoughts and can
change the former based on examination of evidence
The intervention is designed to be implemented over a nine-month period with weekly individual
or group sessions. The actual chapters in the Toolkit provide additional clarity about what is
included in the intervention (the number of sessions is in parentheses).:

Recovery strategies including goal setting (4)

Information about mental illness and specific disorders (4)

Stress vulnerability model (3)

Building social support (7)
California Institute for Mental Health

Using medications effectively (4)

Reducing relapse (4)

Coping with stress (5)

Coping with problems and symptoms (6)

Getting your needs met in the mental health system (3)
Page 3
Evidence for Illness Management and Recovery
A.
Systematic Reviews
Mueser, et. al. (2002)1 presents a narrative review of some of the major components in
the IMR Toolkit. While the correspondence between the activities in the Toolkit and
these broad areas of intervention research is not always clear, it represents a useful way
of examining the research evidence. While not in the original supporting article, social
skills training is included in this review since it is cited as one of the core elements of the
Toolkit in the Mueser, et al. (2006) article describing an implementation trail. Thus, three
of the areas reviewed here refer to approaches (psychoeducation, social skills training,
and cognitive-behavioral interventions) and two refer to the target outcome areas
(medication compliance and relapse prevention).
Psychoeducation
Mueser et al. conclude that education generally improves consumer knowledge but there
is little evidence that it results in changes in other outcomes. “Nevertheless,
psychoeducation remains important because access to information about MI is crucial to
people’s ability to make informed decisions about their own treatment and
psychoeducation is the foundation for more comprehensive programs.”
Pekkala &Merinder (2002)2 conducted a review using the Cochrane standards on
psychoeducation with consumers with schizophrenia.. Excluding studies which included
other interventions in addition to psychoeducation left only 10 small studies with
substantial heterogeneity of interventions and outcomes. They conclude there is a
possible positive impact on knowledge gained, mental state, global level of functioning,
but the scales used for measurements are difficult to interpret. They conclude, “Evidence
from trials suggests that psychoeducational approaches are useful as a part of the
treatment programme for people with schizophrenia and related illness. The fact that the
interventions are brief and inexpensive should make them attractive to managers and
policy makers.”
Medication-focused interventions.
Mueser reviews two basic approaches

Psychoeducation about medications: about benefits and side effects of medication
and teaching strategies for managing side effects was studied in eight studies. The
California Institute for Mental Health
Page 4
intervention “improves knowledge about meds, but little evidence indicates that it
improves taking medications as prescribed or affects other areas of functioning.”

Cognitive-behavioral programs. Included in this category are behavioral tailoring,
simplifying medications regimen, motivational interviewing, and social skills
training.3 These programs have better results with studies showing positive impacts
on medication adherence for all but the social skills techniques.
Other reviews confirm the basic finding that education by itself has no impact on
medication compliance, and that other approaches which rely more on cognitive and
behavioral interventions have greater success.4
Relapse prevention
Mueser reviews this intervention which “focuses on teaching people how to recognize
environmental triggers and early warning signs of relapse and taking steps to prevent
further symptom exacerbation.” Five studies were reviewed– two of which had only
consumers with schizophrenia and three of which had consumers only with bipolar
disorder. Additionally two of the interventions also had family groups as part of the
intervention. Summaries of the study findings indicate that 4 of the 5 studies had the
intervention performing better than standard care on relapse prevention while three
reportedly also had positive benefits on other social and functional outcomes. There is no
rating of the quality of the studies, and the descriptions are too brief to assess.
Social skills training.
Bustillo et al (2001)5 review included three models of social skills training: corrective
learning through things like role playing in natural setting; social problem-solving
through improving information processing in specific function areas; cognitive
remediation which focuses on more general cognitive impairments. They conclude that
the intervention “improves social skills but has no clear effects on relapse prevention,
psychopathology, or employment status,” and they note difficulties with generalization
beyond the specific skills trained in the specific setting.
A comprehensive review by Bellack (2004)6 Bellack of 8 narrative reviews and 4 metaanalyses of social skills training reached essentially the same conclusion: effectiveness
with learning of behavioral skills and to some but a lesser degree an impact on more
general social role functioning.
Cognitive-behavior approaches
There are a growing variety of techniques and approaches that fit into this general
category. Mueser et al (2001) appear to make a distinction between “coping programs
[which] aim to increase people’s ability to deal with symptoms or stress or persistent
symptoms” all of which employed some cognitive-behavioral techniques, and the CBT
approach which addresses the underlying beliefs of persons about their symptoms.1 An
California Institute for Mental Health
Page 5
examination of the content of the Toolkit suggest that the emphasis is on the former, i.e.
the development of coping strategies. But Mueser et al (2001) include the more formal
CBT in their literature review creating a somewhat murky situation. This is made more
complicated by the explosion in the last few years of interest and studies in CBT and its
potential applicability to schizophrenia.
Mueser et al (2002) find support for both definitions of CBT, but the quality of the
studies included in the review is not indicated. A number of other reviews with patients
with schizophrenia indicate promising results on positive symptoms when compared to
standard treatment, but more equivocal results when compared to groups with other nonspecific interventions. (Bustillo et al, 2001;7 Turkinton, et.al., 2004;8 Pilling et.al., 2002;9
Rector & Beck, 2001;10 Gould, et al, 2001;11 Tarrier, 2005;12 and Zimmermann, 2005.)
Many of the above reviews were done by proponents of CBT; and many of the authors
indicate the lack of rigor in some of the studies reviewed. Standing somewhat in contrast
to these is the Jones, et.al (2004)13 Cochrane review which using more rigorous criteria
for selection of studies found no effect on relapse or readmission; no consistent impact on
mental status or on symptoms. “At present, it is not possible to assert any substantial
benefit from CBT over standard care or supportive therapies…. CBT is a promising but
under evaluated intervention.”
C. Evidence regarding adaptability to cultural and other subpopulations
There is no discussion in the literature of the applicability of the various approaches and
techniques to other cultures. One would expect that that issues would be on importance given the
emphasis in the overall approach on the consumer’s understanding of mental illness and its
impacts on one’s concept of oneself – both of which can be heavily influenced by cultural beliefs.
There is research literature on the efficacy of the various interventions on other diagnostic
populations, particularly bipolar disorder. For example, a review by Vieta & Colom (2004)14,
however, found 4 studies with psychosocial interventions which met the standard of randomized
controlled studies with sufficiently large samples. They found evidence for the effectiveness of
the psychoeducation and cognitive-behavioral interventions in terms of reduced relapse and
rehospitalization. We have not reviewed the substantial literature on the effectiveness of CBT
with bipolar and major depression disorders.
D. Consensus Panel Recommendations Regarding Illness Management and Recovery
The PORT 2002 recommendations include “Patients with schizophrenia should be offered skills
training, the key elements of which include behaviorally based instruction, modeling, corrective
feedback, contingent social reinforcement, and homework assignments.” The PORT guidelines
also recommend offering cognitive behaviorally oriented therapy.
The United Kingdom National Institute of Clinical Excellence (NICE) recommends CBT.15
California Institute for Mental Health
Page 6
Capsule Summary of Evidence: Effective, Efficacious, Promising, or Emerging,
Not Effective, or Harmful
The inclusion of numerous often overlapping techniques in the IMR Toolkit and in the research
literature makes it difficult to define the actual Practice which is to be rated. We have attempted a
rating for the overall IMR Toolkit followed by ratings for its various components.

IMR Toolkit: Promising. The combination of a variety of interventions, most of which have
at least some evidence-base, certainly passes the “face” validity or common sense test for an
emerging practice. There is one report from Mueser et.al. (discussed below) which documents
positive pre- to post-change on a number of outcomes including overall functioning, coping
strategies, and some elements of recovery outcomes. But this is the only study we know of
which utilizes the specific package of interventions included in the Toolkit.
Below are summary ratings for the various components otfthe Toolkit.


Psychoeducation:
o
Effective with regard to increasing consumer’s knowledge.
o
Not effective when used by itself for any other outcome.
Medications-focused interventions:
o
Education efforts: Not effective when used by itself in enhancing medications adherence.
o
Cognitive-behavioral- motivational techniques: Promising. While studies document
positive, if small, effects from a variety of techniques, no clear results indicate their
relative merits.

Relapse-prevention: Promising. While the results from the few studies cited by Mueser, et.al
(2002) were generally positive there does not seem to be a sufficient body of research to
warrant a higher level of effectiveness rating.

Social skills training:

o
Effective regarding the particular social skills which are taught.
o
Promising with respect to more general social role functioning, generalization of
particular social skills to other situations, and social confidence.
o
Not effective with regard to other outcomes such as relapse prevention or symptom
reduction.
Cognitive-behavioral approaches: Promising practice. While the most rigorous review failed
to find support for the efficacy of this practice, the positive effects found in less well
controlled studies along with its widespread support in the United Kingdom and the
blossoming of the clinical use of the practice places it in the promising category. This is not
to say that there is not controversy in the literature about a possible premature acceptance of
the approach with schizophrenia.16
California Institute for Mental Health
Page 7
Qualifications
The above ratings need to be viewed within a broader understanding of both the nature of the
Toolkit and the variety of clinical and research activity in this area. Some of the issues are
highlighted below.

As noted throughout this review, this “practice” has the least clear definition and has virtually
no research which addresses the package of interventions included within the Toolkit. The
other practices generally had “innovators” and “proponents” who conducted early research on
their interventions so that even when the practices were further elaborated there was a core
element to what was included in the intervention. Some of the components of the IMR
Toolkit fit this pattern, most notably CBT, but the particular combination of elements in the
Toolkit is unique.

The research findings on the above areas are almost all based on a greater intensity of
intervention than appears to be reflected in the Toolkit. The nine-month Toolkit intervention
covers multiple strategies whereas many of the research studies utilized a similar intervention
period for just one approach. And as noted with some of them, for example, the cognitivebehavioral techniques are not nearly as substantive as in the interventions in the CBT
literature. So applying the efficacy ratings of the literature to the practices in the Toolkit can
be misleading.

There is no research that assesses recovery outcomes. Mueser et al. (2002) note that “…little
controlled research has examined the effect of interventions on the broader dimensions of
recovery, such as developing hope, meaning, and a sense of purpose in one’s life.” The major
outcomes in the studies in this review relate to symptoms severity, relapse, medication
adherence, and mental status. The Mueser, et.al. implementation study of the IMR Toolkit
cited below does include some recovery-oriented outcomes.

The Toolkit itself focuses primarily on illness management – on the coping with symptoms,
reducing stress, taking of medications, and preventing relapse. As noted above, while there is
a session on building social networks the authors implicitly acknowledge this as a less crucial
element in the Toolkit by not addressing the social skills literature in their literature reviews
which supports the Toolkit. While the toolkit embraces the underlying principle of recovery,
the importance of goal-setting, and certainly emphasizes the importance of consumer
involvement in decision-making, it does not include much about how to assist the consumer
towards accomplishing recovery-oriented personal goals.

As with the Toolkit, most clinical interventions that attempt to address illness management
and/or recovery include comprehensive mixes of strategies and the integration of various
techniques. Some recent studies explicitly include various combinations of interventions.
This makes evaluation of specific efforts extremely difficult. Two conclusions from the
research seem to support this approach: most of the specific strategies by themselves fail to
show strong effects, and there is no clear advantage to one technique over another when more
than one is tested. The Toolkit’s emphasis on utilizing educational, motivational, and
cognitive-behavioral approaches would appear to be a good, but untested, strategy given the
state of the research.

The magnitude of the effects may not be large. There are no studies which clearly indicate
that any one or combination of approaches is likely to yield large impacts. There is also scant
California Institute for Mental Health
Page 8
evidence on the durability of effects, and as noted with the social skills intervention on the
extension of the findings outside the treatment setting.
Information regarding implementation
Fidelity scale
Like the other SAMHSA Toolkit fidelity scales, the IMR Fidelity Scale has good reliability. Like
the fidelity scale for IDDT the validity of the scale is untested.
Extent of implementation
Mueser (2006)17 reports that 1700 Toolkits have been disseminated since 2003 plus an unknown
number of others have been downloaded. The NASMHPD Research Institute, Inc. reported in
2003 that 12 states had implemented "self-management" in part of the state and 9 were
implementing it statewide.
Success of implementation
Mueser (2006) studied implementation of the IMR Toolkit in three community sites in a small
pre-post study. While there were sizable drop-out rates (9 of 31 clients) the study documents
positive change in many of the measures with the biggest changes in coping effectiveness, the
overall scores on the IMR scale, and the GAF. There were also positive change in some of the
recovery-oriented items such as hope, goal-orientation, and not dominated by symptoms. He
suggests that the results are positive enough to warrant a RCT of the Toolkit.
Assistance available for implementing wellness and recovery

The SAMHSA Toolkit. Like all the toolkits, this one contains descriptions of the practice
from a number of perspectives as well as other implementation resources. It is available
in Spanish.

Developers able to assist. The Dartmouth Psychiatric Institute provides training and
consultation for all of the EBPs: http://dms.dartmouth.edu/prc/evidence/
California Institute for Mental Health
Page 9
Endnotes
1
Mueser, K.T., et al. Illness management and recovery: a review of the research. Psychiatr Serv. 2002.
53(10): p. 1272-84.
2
Pekkala, E. and L. Merinder. Psychoeducation for schizophrenia. Cochrane Database Syst Rev.. 2002(2):
p. CD002831.
3
While Mueser et al. and others include “social skills training” as a general terms for a variety of
interventions focused on behavioral interventions directed at improving the social functioning (broadly
defined) of consumers, we do not include a review of this literature in this review because it seems not to
be used extensively in the actual Tollkit curriculum. Attachment A includes a few review articles on social
skills training.
4
Zygmunt, A.,et al. Interventions to improve medication adherence in schizophrenia. Am J Psychiatry.
2002. 159(10): p. 1653-64. Nose, M. et al. clinical interventions for treatment non-adherence in psychoses:
a meta-analysis. Br J Psychiatry. 2003. 183: p. 197-206. Gray, R. et al. From compliance to concordance: a
review of the literature on interventions to enhance compliance with anti-psychotic medication. J of
Psychiatr and Mental Health Nursing. 2002. 9: p. 277-84. Ilott R. Does compliance therapy improve use of
anti-psychotic medications? Br J Community Nurs. 2005 10(11): p. 5124-19
5
Bustillo JR et al. The psychosocial treatment of schizophrenia: An update. Am J Psychiatry. 2001. 158:
163-75.
6
Bellack, A. Skills training for people with severe mental illness. Psychiatric Rehabilitation Journal. 2004.
27(4): p. 375-390.
7
Bustillo JR et al. The psychosocial treatment of schizophrenia: An update. Am J Psychiatry. 2001. 158:
163-75.
8
Turkington, D. et al. Cognitive behavioral therapy for schizophrena: A review. J of Psychiatr Practice. 10:
p. 5-16.
9
Pilling, S. et al. Psychological treatments in schizophrenia: I. Meta-analysis of family intervention and
cognitive behavioral therapy. Psychol Med. 2002. 32(5): p. 763-82. Piling, S. et al. Psychological
treatments in schizophrenia: II: Meta-analyses of reandomized control trials of social skills training and
cognitive remediation. Psychol Med. 2002. 32(5): p 783-91.
10
Rector NA. & Beck AT. Cognitive behavior therapy for schizophrenia: An empirical review. J Nerv
Ment Dis. 2001. 189: 278-87.
11
Gould R et al. Cognitive therapy for psychosis in schizophrenia.: An effect size analysis. Schizo
Research. 2001. 48: p. 335-42.
12
Tarrier N. Cognitive behavior therapy for schizophrenia: A review of development, evidence and
implementation. Psychother Psychosom. 2005 74: p. 136-44..
13
Jones C, et al. Cognitive behaviour therapy for schizophrenia. The Cochrane Database of Systematic
Reviews 2004, Issue 4. Art. No.: CD000524.
14
Vieta E. & F Colom. Psychological interventions in bipolar disorders: from wishful thinking to an
evidence-based approach. Acta Psychiatr Scand. 2004. 110(Suppl 442): p/ 34-38.
15
National Institute for Clinical Excellence. Core interventions in the treatment and management of
schizophrenia in primary and secondary care. National Institute for Clinical Excellence. 2003.
16
See for example McKenna PJ. What works in schizophrenia: Cognitive behavioral therapy is not
effective. BMJ. 2006.333: P 335. Durham RC et al. Long-term outcome of cognitive behavioral therapy
trails in central Scotland. Health Technol Assess. 2005. 9(42): p 1-174.
17
Mueser, KT, et al. The illness management and recovery program: Rationale, development, and
preliminary findings. Schiz Bull. 2006. 32(S1): p. S32-S43.