1
Conducting Evaluations of Client Outcomes and Satisfactions
Michael J. Lambert & Kara Cattani
Abstract
Psychotherapists improve their services to patients by attending to data that tracks patient
outcome. This chapter aims to assist clinicians in establishing a practice in which outcome data
is used to enhance routine clinical care. Information is provided to help clinicians select useful
measures to track patient outcome as well as information about collecting outcome data in a
practical manner.
Key Words: psychotherapy outcome, quality assurance, treatment planning
Psychotherapists have a scientific and ethical responsibility to learn whether they are
providing helpful services to their clients. Effective outcome assessment can let clinicians know
whether individual clients are deteriorating, remaining unchanged, or recovering and can thereby
improve the effectiveness of treatment. Although more and more psychotherapists are employing
outcome measures in their practices, most clinicians do not yet objectively assess psychotherapy
outcome in routine practice (Hatfield & Ogles, 2007).
Patient focused research, a movement toward measuring the effects of therapy on
individual clients, provides a strategy for clinicians who want to enhance client outcomes. This
strategy involves using session-by-session outcome to inform individual therapy in real time.
The measured outcome allows clinicians increased awareness about client welfare and progress.
Through such monitoring, for instance, the clinican has access to information about what seems
to be changing or not changing in psychotherapy. This type of feedback enhances clinician
performance and improves client outcome (Shimokawa, Lambert, & Smart, 2010).
2
This chapter summarizes recommended methods to (1) select potentially useful measures,
(2) collect client outcome and satisfaction data in a practical matter, and (3) use those data to
enhance routine practice.
Selecting Potentially Useful Measures
The following principles of selecting measures attempt to strike a balance between what is
practical for the everyday clinician and what is scientifically necessary in order to obtain useful
outcome data.
1. Make sure the measure covers broad, yet crucial, content areas: The three broad areas to
be assessed are the subjective state of the client (intrapersonal functioning, including
behavior, affect, and cognition), the state of the client’s intimate relationships
(interpersonal functioning), and the state of the individual’s participation in the
community (social role performance). Both symptomatic change and functioning are
important, if not essential, targets for outcome assessment. A compendium of suitable
measures has been edited by Maruish (2004).
2. Select a brief self-report measure: Most clinicians who are assessing client outcome are
using brief self-report measures. Outcome measures have been developed that can be
completed by the client, a parent/guardian/spouse, the therapist, or an independent judge.
However, because it is usually feasible to obtain only one perspective, in the case of an
adult, a self-report measure is ideal and in the case of a child/adolescent a parent-report
measure is recommended. Additionally, it should be kept in mind that instruments and
methods useful for diagnostic purposes and treatment planning are unsuitable for the
purpose of measuring patient change (Vermeersch, Lambert, & Burlingame, 2000).
3
Symptom-focused measures are most likely to reflect improvement and are therefore
highly recommended. Literally hundreds of measures are available for use. We
recommend the Brief Symptom Inventory (available at
www.pearsonassessments.com/tests/bsi.htm), a shortened version of the Symptom
Checklist-90-R that focuses on a wide variety of symptoms. The Short Form-36 Health
Survey (http://www.sf-36.org) is also a promising measure for adults. The Outcome
Questionnaire-45 (http://www.oqmeasures.com) is growing in popularity. It measures
symptoms, interpersonal functioning, social role performance, and quality of life, and has
been shown to be sensitive to treatment effects. For children, the Ohio Youth Problems,
Functioning, and Satisfaction Scales (http:// www.mh.state.oh.us/what-we-do/protectand-monitor/consumer-outcomes/instruments/index.shtml) or the Youth Outcome
Questionnaire (http://www.oqmeasures.com/) appear to be especially promising because
they are relatively short, sensitive to change, and available in parent-, self-, and otherreport formats.
3. Select measures that can detect clinically meaningful change: Methods have been
developed to set standards for clinically meaningful client change (Jacobson & Truax,
1991). The clinical significance methodology provides for the calculation of two specific
statistical indexes: a cutoff point between normal and dysfunctional samples and an
evaluation of the reliability of the change score. These indexes provide specific cut scores
for interpreting the importance of observed scores and some existing measures provide
such guidelines. When they are not available, however, the clinician can consult the work
of Jacobson for the formulas for establishing a cutoff score, as well as a reliable change
index.
4
4. Use caution if you tailor the change criteria to the individual in therapy: Tailoring
change criteria with individualized goals for a particular patient has been advocated
because it is likely to provide evidence for efficacy. The use of individualized change
measures enables the therapist to assess change from an idiographic and multifaceted
perspective, which is consistent with the wide range of problems presented by an
individual (e.g., Persons, 2007). However, such change criteria are often poorly defined,
subjective in nature, and have little credibility. The amount of change reflected by such
measures is often overly dependent on the therapist’s judgments. On the other hand, in
difficult-to-treat individuals, idiographic change criteria may be a necessary addition to
standardized outcome measures. Such individuals abound in residential, geriatric,
severely mentally ill, or neuropsychologically impaired populations that may be
atypically responsive or appear to be nonresponsive on standard measures.
Collecting Client Outcome Data In A Practical Manner
Practical concerns in routine practice demand that outcome assessment be painless and
resource effective (i.e. minimize demands such as money, time or energy). Brief self-report
measures that can be completed in 5-10 minutes are recommended. A measure that can be easily
administered and scored by computer or clerical staff is preferable. Some computer based
measures provide the practitioner with graphs that depict patient change over time. This visual
depiction of change over time is easy to read and is immediately available once the measure is
complete.
Once a measure is selected, it is best to have it completed prior to treatment. For
5
instance, clients complete the questionnaire when they arrive for their appointment and are
waiting for the clinician. Ideally, this data will be gathered prior to every therapy session. This
ensures that there will be at least one measure of change, provided that the client has a second
appointment. Since many patients improve rapidly, and most attend few sessions, delaying the
second assessment is likely to result in underestimating treatment benefits or failing to gather any
outcome data.
Using Outcome Data To Enhance Routine Practice
The initial assessment can be used to (1) determine the client’s incoming symptom
severity and forming an opinion about expected length of treatment, (2) highlight possible target
symptoms seen at the individual item level, and (3) identify particular strengths that might be
capitalized on. For example, the measure(s) you select will have several “critical” items that you
may want to routinely examine (e.g., “I have thoughts of ending my life”; “I have people around
me that I can turn to for support”).
The most important aspect of tracking change after the initial assessment (i.e. assessing
outcomes on a session-by-session basis) is assessing whether client scores tend to increase, stay
the same, or decrease in relation to the intake score. Research demonstrates that early positive
response to treatment foretells final success, while negative change foretells final failure (Haas,
Hill, Lambert, & Morrell, 2002). In any case, until the client’s functioning is within the normal
range, some kind of treatment is needed. As mentioned previously, measures are available that
graphically display client data across time or sessions. Many clinicians find this helpful as it
facilitates the visualization of score changes and general trends. If a client’s scores worsen after
beginning treatment, then the clinician can consider the causes and possibly modify treatment
6
(e.g., more frequent sessions, medication referral, change in treatment focus). On the other hand,
if the client’s scores indicate improvement and a return to normal functioning, the focus of
treatment could shift to preparing the client for termination and maintenance. The specific details
of what to do in the individual cases vary and are up to the treating clinician and the client.
One might ask how much a client’s score needs to change in order to be considered
meaningful. Although clinicians can try to rely on personal methods of detecting significant
change, standardized methods are available to better serve clinicians and clients. As already
noted, these methods operationalize meaningful change so clinicians can know how much a
client’s score must change in order to be considered clinically significant. If a client’s score
changes by at least the amount of the Reliable Change Index (RCI; individually calculated for
the particular instrument being used), then the client is considered to have reliably changed,
becoming symptomatically worse or better. However, reliable change by itself cannot be equated
with recovery. For example, a client’s score may change in the amount of the RCI and still be in
the dysfunctional range. If, however, a client’s score (1) moves from the dysfunctional range to
the functional range and (2) the amount of change is equal to or greater than the RCI, then the
client is considered to have made clinically significant improvement, sometimes labeled
recovery. Ogles, Lambert, and Fields (2002) have listed cutoff scores and Reliable Change
Indices (RCIs) for some of the most commonly used measures, such as the SCL-90R, BDI, and
CBCL.
Another way to use session-by-session data to enhance treatment outcome is to compare
a client’s treatment response to a typical or expected treatment response. By comparing a client’s
symptom course to the average symptom course among others that have the same initial
assessment score, you can know if they are progressing as “expected.” Although this technique is
7
more specific than relying on RCIs and cutoff scores, it is not yet readily available for most
outcome measures. Nevertheless, such data help to inform clinicians if the client is responding
faster or slower than similar clients. Psychotherapy research investigating this method is
promising and will probably become widely available for use in routine practice. The interested
reader can consult other sources (Lambert et al., 2002; Ogles et al., 2002;). This research
demonstrates that feedback to therapists about potential treatment failure (based on client
deviations from expected treatment response) improves outcomes and reduces deterioration for
the 20-30% of clients who are at risk for treatment failure (Lambert 2010). It is our hope that
clinicians will continue to improve the quality of their work through systematic assessment of
outcomes.
For More Information
Haas, E., Hill, R., Lambert, M. J., & Morrell, B. (2002). Do early responders to psychotherapy
maintain treatment gains? Journal of Clinical Psychology, 58, 1157–1172.
Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others
do not. Administration and Policy in Mental Health and Mental Health Services
Research, 34, 283-291.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining
meaningful change in psychotherapy research. Journal of Consulting & Clinical
Psychology, 59, 12–19.
Lambert, M. J. (2010). The prevention of treatment failure: The case for measuring, monitoring,
and feedback in routine care. Washington, DC: APA.
Lambert, M. J., Whipple, J. L., Bishop, M. J., Vermeersch, D. A., Gray G. V, & Finch, A. E.
8
(2002). Comparison of empirically-derived methods for identifying patients at risk for
treatment failure. Clinical Psychology and Psychotherapy, 9, 149–164.
Maruish, M. E. (2004). The use of psychological testing for treatment planning and outcomes
assessment (3rd ed.). Mahwah, NJ: Erlbaum.
Ogles, B. M., Lambert, M. J., & Fields, S. A. (2002). Essentials of outcome assessment. New
York: Wiley.
Persons, J. B. (2007). Psychotherapists collect data during routine clinical work that can
contribute to knowledge about mechanisms of change in psychotherapy. Clinical
Psychology: Science and Practice, 14, 244-246.
Shimokawa, K., Lambert, M.J., & Smart, D.W. (2010). Enhancing treatment outcome of patients
at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy
quality assurance system. Journal of Consulting & Clinical Psychology, 78, 298-311.
Vermeersch, D. A., Lambert, M. J., & Burlingame, G. M. (2000). Outcome questionnaire: Item
sensitivity to change. Journal of Personality Assessment, 74, 242–261.