How To Be An
Evidence-Based
Psychologist
John Hunsley
hunch@uottawa.ca
OVERVIEW
• What Is Evidence-Based Psychological Practice (EBPP)
• Evidence-base practice, empirically supported treatments,
clinical practice guidelines
• Examining the research evidence
• Evidence-based treatments, therapeutic relationships, and
assessment
• Dissemination and implementation of EBPP
• Implications, challenges, and opportunities
SOME DEFINITIONS
• Evidence-Based Practice (and Evidence-Based
Practice in Psychology)
• Empirically Supported Treatments
• Practice Guidelines
• Randomized Controlled Trials (RCTs)
• Efficacy Studies & Effectiveness Studies
• Treatment As Usual (TAU)
EVIDENCE-BASED PRACTICE
The use of systematically collected data, clinical
expertise, and patient preferences by
decision-makers (including clinicians,
administrators, and policy makers) when
considering service options.
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson,
W. S. (1996). Evidence based medicine: What it is and what it isn’t. British
Medical Journal, 312, 71-72.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for
the 21st century. Washington, DC: National Academy Press.
American Psychological Association Presidential Task Force on Evidence-Based
Practice. (2006). Evidence-based practice in psychology. American
Psychologist, 61, 271-285.
EVIDENCE-BASED PRACTICE
• Providing the right health care services—services that have
been demonstrated to work—for each client’s needs
• Services are based on empirical evidence but are individually
tailored to take into account client characteristics, needs, and
resources
• Services may also need to be adjusted in order to fit the
demands and constraints of real world clinical practice
IMPLEMENTATION OF EVIDENCE-BASED
SCURVY PREVENTION PROGRAMS: AN
EXAMPLE WE DO NOT WANT TO REPEAT
•
•
•
•
•
•
1497 Vasco da Gama sailed around Cape of Good Hope:
100 of 160 sailors died of scurvy (due to deficiency of vitamin C)
1601 James Lancaster RCT (4 ships England to India, 1 on which
sailors received 3 tsps of lemon juice each day):
Halfway, 110/278 died of scurvy on 3 ships; 0 on the “lemon” ship
1747 James Lind RCT of 6 treatments (including citrus juice) replicates
Lancaster finding
1768-1780 James Cook (early adopter), 3 voyages, required
sauerkraut in the rations
Only 3 sailors died of scurvy
1795 British Navy required citrus fruit as part of diet (limey)
1865 British Board of Trade required citrus fruit as part of diet
Berwick. (2003). Disseminating innovations in health care. Journal of the American Medical
Association
THE EBP MODEL
Best
Available
Evidence
EBP
Client/Patient
Preferences
Clinical
Expertise
BEST AVAILABLE RESEARCH EVIDENCE
(TREATMENT)
Systematic
Reviews and Meta-Analyses
RCTs
Cohort Studies
Single Case Designs
Case Studies
Expert Opinions
EVIDENCE-BASED PSYCHOLOGICAL
PRACTICE
• Influenced by multiple factors in North America
– Evidence-Based Medicine
– Scientist-Practitioner Model
– Accountability/Quality Assurance
– Empirically Supported Treatments
EBP IN PSYCHOLOGY:
THE CONTROVERSY
• Concerns:
– loss of professional autonomy
– takeover of professional psychology by specific
interest groups
– dehumanization of psychological services
– inadequacy of the research base
– impossibility of basing care on research evidence
• Similar concerns raised in other health professions
APA DIVISION 12 EMPIRICALLY SUPPORTED
TREATMENTS: CRITERIA FOR WELLESTABLISHED TREATMENTS
• I. At least 2 good between group design experiments
demonstrating efficacy in one or more of the following ways:
– A. Superior (stat. sign.) to pill or psychological placebo or to
another treatment
– B. Equivalent to an already established treatment in
experiments with adequate sample sizes
OR
• II. A large series of single case design experiments (n > 9)
demonstrating efficacy. These experiments must have:
– A. Used good experimental designs and
– B. Compared the intervention to another treatment.
CRITERIA FOR WELL-ESTABLISHED
TREATMENTS
FURTHER CRITERIA FOR BOTH I AND II:
• III. Experiments must be conducted with treatment manuals or
equivalent clear descriptions of treatment
• IV. Characteristics of the client samples must be clearly
specified
• V. Effects must have been demonstrated by at least 2 different
investigators or teams
CAUTION: ESTs ≠ EBT
• Difference between (a) research on a treatment
meeting a pre-established set of criteria and (b)
determining which treatments have strongest support
for a specific condition
• Despite this, terms are now being used almost
interchangeably
– can be confusing, especially as trend in
professional psychology has been to use lists
rather than encourage the use of practice
guidelines or having individual clinicians
conducting their own literature searches
EBTs FOR CHILDREN &
ADOLESCENTS (Multiple Single Case
Designs or Higher)
•
•
•
•
•
•
•
•
•
•
Autistic Disorder
Attention-Deficit/Hyperactivity Disorder
Anxiety Disorders
Chronic Pain
Conduct Problems & Oppositional Defiant Disorder
Major Depressive Disorder
Eating Disorders
Elimination Disorders
Obesity
Tic Disorders
EBTs FOR ADULTS
• Anxiety Disorders
– Specific Phobias
– Social Phobia
– Panic Disorder (with/without Agoraphobia)
– GAD
– OCD
– PTSD
• Major Depressive Disorder
• Bipolar Disorder
EBTs FOR ADULTS
• Eating Disorders
– Anorexia Nervosa
– Bulimia Nervosa
– Binge-Eating Disorder
• Sleep Disorders
• Substance-Related Disorders
– Alcohol Abuse
– Cocaine Abuse
– Opiate Abuse
EBTs FOR ADULTS
•
•
•
•
•
Tic Disorders
Sexual Disorders
Schizophrenia
Marital/Couple Conflict
Personality Disorders
– Avoidant PD
– Borderline PD
• Somatoform Disorders
– Pain Disorders
– Body Dysmorphic Disorder
– Hypochondriasis
SOME ADDITIONAL EBTs
•
•
•
•
•
•
•
Anger Management
Anxiety/fear associated with medical/dental procedures
Assertiveness Skills
Parent Training
Social Skills
Stress Management
Many also available for specific illnesses/chronic health
conditions (e.g., Chronic Fatigue Syndrome, Irritable Bowel
Syndrome)
• Psychometrically strong assessment measures available
for assessing outcomes from all EBTs
CLINICAL PRACTICE GUIDELINES
• Common in many health professions; in professional
psychology, they are almost totally absent (but coming soon
from APA)
• In general, they are consensus statements from
experts/professional organizations/healthcare organizations that
present best clinical practices (screening, assessment,
consultation, treatment, referral, etc.)
– Not the same as guidelines for reimbursement or other
administrative purposes
• Usually encourage appropriate initial assessment, without any
further guidance on how to do this (more on this later)
CLINICAL PRACTICE GUIDELINES
• National Institute for Health and Clinical Excellence
(NICE) sponsored by National Health Service in
England and Wales
• Use of explicit evidence hierarchy
• Extensive consultations undertaken with stakeholder
organizations (both consumer and professional
groups)
• Each guideline has a limited “life,” ensuring review in
near future
http://www.nice.org.uk/
CLINICAL PRACTICE GUIDELINES:
NICE RECOMMENDATIONS
Some current guidelines recommending use of a psychological
intervention
– ADHD
– Anxiety Disorders (Panic Disorder, Agoraphobia, Generalized
Anxiety Disorder)
– Bipolar Disorder
– Chronic Fatigue Syndrome
– Conduct Disorder
– Depression (child, adolescent, & adult)
– Eating Disorders
– Obsessive-Compulsive Disorder & Body Dysmorphic Disorder
– Personality Disorders (Antisocial, Borderline)
– Posttraumatic Stress Disorder
– Schizophrenia
BUT AREN’T ALL THERAPIES
EQUIVALENT?
• Most meta-analyses suggest that this is not the case for most
disorders/conditions
• Wampold et al. (1997) meta-analysis of bona fide comparative
treatment studies
• Mean effect size = .19
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A metaanalysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must
have prizes.” Psychological Bulletin, 122, 203-215.
– But, over ¾ of comparisons within types of CBT
– NNT was 9 (for comparison, interferon vs. placebo to slow
progression of multiple sclerosis has NNT of 9)
• For depression, marital conflict, PTSD, & bipolar disorder,
however, there do appear to be multiple treatments with similar
results
EBT COMPARED WITH TAU
• Meta-analysis of 32 studies of youth interventions
found majority of EBTs yielded outcomes superior to
TAU (most studies focused on externalizing
disorders)
– Mean effect size for EBT versus TAU was 0.30
– This indicates that the average “EBT” youth was
better off after treatment than 62% of “TAU” youth
– Alternatively, this can be represented as a NNT of
6 (similar NNT for average effects of adding
radiation treatment to chemotherapy for a range of
cancers)
Weisz, Jensen-Doss, & Hawley (2006). Psychological Bulletin.
EBT COMPARED WITH TAU
• Data on treatment outcome for adults, in real world
clinics:
– Of over 6000 clients, 35% rate of
improvement/recovery (Hansen, Lambert, & Forman, 2002, Clinical
Psychology: Science and Practice, 9, 329-343)
– Of over 6100 clients, 29% rate of
improvement/recovery (Wampold & Brown, 2005, Journal of
Consulting and Clinical Psychology, 73, 914-923)
• Data on treatment outcome for adults, in RCTs with
EBTs
– Of over 2100 clients, 67% rate of
improvement/recovery (Hansen, Lambert, & Forman, 2002, Clinical
Psychology: Science and Practice, 9, 329-343)
BUT ARE EBTs CLINICALLY
RELEVANT?
• Are the participants in treatment studies similar
enough to patients routinely seen in practice settings
to warrant generalizing research results to clinical
practice?
• Is the research literature on psychological treatments
sufficiently developed to be applicable to the broad
range of conditions encountered in practice?
BUT ARE EBTs CLINICALLY
RELEVANT?
• Participants in RCTs/efficacy trials are unlike “real”
clients because ………..
– They are only the “worried well”
– They are only “pure” cases
BUT ARE EBTs CLINICALLY
RELEVANT?
• Patients included in RCTs for cocaine dependence
had symptoms comparable to, but more severe than,
those found reported by patients receiving outpatient
services
Carroll et al. (1999, Drug and Alcohol Dependence)
• Compared patient files from a managed behavioral
health care network to the inclusion and exclusion
criteria used in numerous RCTs for adults with
mental disorders
– Over ½ of patients would have been ineligible for
RCTs because symptoms were not severe
enough to warrant inclusion (in most cases, due to
patient diagnosis of adjustment disorder)
Stirman et al. (2003, Journal of Consulting and Clinical Psychology)
BUT ARE EBTs CLINICALLY
RELEVANT?
• For youth RCTs, approx. half used no exclusion
criteria related to comorbidity
Weisz et al. (2004, Child and Adolescent Psychiatric Clinics of North America)
• In adult disorder efficacy RCTs, only “clinical
appropriate” exclusion criteria used for many years
now
– Can see this even almost 20 years ago in US
NIMH Collaborative Depression Treatment study,
with approximately ¾ having personality disorders
BUT ARE EBTs CLINICALLY
RELEVANT?
• Single RCTs, or more, available for most DSM IV
Axis I and Axis II disorders, and many
nondiagnosable conditions
• Hawaii study of 2,200 youth receiving services
– 89% had a primary diagnosis for which an EBT
was available
– In terms of treatment targets, 90% had 1 or more
problems for which an EBT was available
– On the other hand, for only 3% were there EBTs
for all treatment targets
Schiffman et al. (2006). Evidence-based services in a statewide public mental health system: Do
the services fit the problems? Journal of Clinical Child and Adolescent Psychology, 35, 1319.
EFFECTIVENESS RESEARCH
• Reviewed EBT effectiveness studies published prior
to April 2006
– 21 studies of adult treatment and 13 of
child/adolescent treatment met criteria (including
at least 2 effectiveness trials)
• Compared results to benchmarks from reviews of
efficacy studies (mainly meta-analysis)
Hunsley & Lee (2007). Research-informed benchmarks for psychological treatments: Efficacy
studies, effectiveness studies, and beyond. Professional Psychology: Research and
Practice, 38, 21-33.
EFFECTIVENESS RESEARCH
• Completion rates higher than usually reported in
studies of “real world” psychotherapy
• For both adult and youth disorders, the average
improvement rates were similar to efficacy
benchmarks
• Some examples:
– Adult Depression: 74% completed, 51% improved
– Adult OCD: 88% completed, 64% improved
– Youth Anxiety Disorders: 87% completed, 63%
improved
EFFECTIVENESS RESEARCH
PTSD Treatments
Omagh Bombing
Gillespie et al. (2002). Behaviour Research and Therapy, 40, 345-357.
9/11 World Trade Center
Levitt et al. (2007). Behaviour Research and Therapy, 45, 1419-1433.
• Vast majority completed treatment, with results very
similar to efficacy RCTs
• Up to 25-30 sessions; most clinicians had limited
CBT background, received training over several
days, weekly/monthly supervision
EBTs: CONCLUSIONS
• Does treatment research generalize to practice?
– Compelling evidence that it does, but always need
to exercise caution in applying results to a
particular individual
– Much more research on “mild” conditions and Axis
II conditions needed
• How effective are evidence-based treatments (EBT)
in clinical practice?
– More evidence needed, but EBTs are usually as
effective as in efficacy trials
Hunsley, J. (2007). Addressing key challenges in evidence-based practice in psychology.
Professional Psychology: Research and Practice, 39, 113-121.
Hunsley, J. (2007). Training psychologists for evidence-based practice. Canadian
Psychology, 47, 32-42.
EBPP: “FLEXIBILITY WITHIN
FIDELITY”
• Treatment manuals now focus attention on key elements of
treatments, not just session by session list of activities/strategies
• Increased recognition of importance of tailoring treatment to
clients, especially aspects related to cultural diversity and
presence of multiple disorders/problems components
• Call for attention to principles of change and commonly used
techniques/strategies, not “trademarked” therapies
• Some examples:
David Barlow: Unified treatment for mood and anxiety disorders
Bruce Chorpita: Modular CBT for youth disorders
EBPP: INCLUDING CLINICAL
EXPERTISE
• Evidence-Based Therapy Relationships
Norcross, J. C. (Ed.). (2011). Psychotherapy relationships that work: Evidence-based
responsiveness (2nd ed.). New York: Oxford University Press.
• There is more to EBT than RCTs
– Client expectations
– Therapist empathy
– Therapeutic alliance
– Repairing therapeutic ruptures
– Culturally adapted treatments
EVIDENCE-BASED ASSESSMENT (EBA)
Use of research and theory to guide:
• The selection of constructs to be assessed for a
specific assessment purpose
• The methods and measures to be used in the
assessment
• The manner in which the assessment process
unfolds (including integration and interpretation of
assessment data)
Hunsley, J., & Mash, E. J. (2007). Evidence-based assessment. Annual Review of Clinical
Psychology, 3, 29-51.
WHY IS EBA NEEDED?
“…blanket recommendations to use reliable and
valid measures when evaluating treatments are
tantamount to writing a recipe for baking
hippopotamus cookies that begins with the
instruction “use one hippopotamus,” without
directions for securing the main ingredient.”
Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent
disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology,
34, 362-379.
WHY IS EBA NEEDED?
• Almost no overlap among the psychological measures found in
surveys (USA, UK) to be commonly used by psychologists (and
those commonly taught to clinical graduate students) and the
measures necessary to
– implement and monitor EBTs
– adapt treatments based on the consideration of EvidenceBased Therapy Relationship (EBTR) elements
• How can we use EBT and EBTR information without using
appropriate assessment tools in practice?
A GUIDE TO ASSESSMENTS THAT
WORK
Hunsley and Mash (2008) “good enough” criteria for use of
instruments
• Must balance psychometric ideals with clinical realities
• Must keep in mind issues such as age, gender, ethnicity in
determining relevance of instruments and supporting data
• Presented criteria used in rating norms, reliability indices
(internal consistency, inter-rater reliability, test-retest reliability),
validity indices (content validity, construct validity, validity
generalization, treatment sensitivity), and clinical utility
A GUIDE TO ASSESSMENTS THAT
WORK
Youth: ADHD, Conduct Problems, Depression, SelfInjurious Thoughts and Behaviors, Anxiety Disorders,
Pain
Adults: Depression, Bipolar Disorder, Self-Injurious
Thoughts and Behaviors, Anxiety Disorders,
Substance Abuse Disorders, Alcohol Use Disorders,
Gambling Disorders, Schizophrenia, Personality
Disorders, Couple Distress, Sexual Dysfunction,
Paraphilias, Eating Disorders, Sleep Disorders, Pain
Hunsley, J., & Mash, E. J. (Eds.). (2008). A guide to assessments that work. New
York: Oxford University Press.
WHY IS EBA NEEDED?
• When asked to rate 3 most important problems to
address in treatment, 77% of child-parent-therapist
triads failed to agree on a single problem
• Correlations between clinician and youth symptom
and self-esteem measures <.23
Hawley, K. M., & Weisz, J. R. (2003). Child, parent, and therapist (dis)agreement on target problems in
outpatient therapy: The therapist’s dilemma and its implications. Journal of Consulting and Clinical
Psychology, 71, 62-70.
Love et al. (2007). Meeting the challenges of evidence-based practice: Can mental health therapists evaluate
their practice? Brief Treatment and Crisis Intervention, 7, 184-193.
.
WHY IS EBA NEEDED?
• In a population-based study of Canadian adults who received
psychotherapy in past year, 43% terminated services because
they felt better
– But, 14% terminated because they felt therapy was not
helping, and 7% were not comfortable with the therapist’s
approach
• Clinicians identified <50% of treatment successes (as rated by
patients) and failed to identify the 10% of patients who
terminated because they felt treatment was worsening their
problems
Westmacott & Hunsley. (2010). Reasons for terminating psychotherapy: A general population study. Journal of
Clinical Psychology, 66, 965-977
Hunsley et al. (1999). Comparing therapist and client perspectives on reasons for psychotherapy termination.
Psychotherapy, 36, 380-388.
.
WHY IS EBA NEEDED?
Limited evidence of clinical utility for commonly used instruments
• MMPI-2 completed by all clients prior to treatment
• Half of clinicians (randomly assigned) received test results; half
did not
• All clients received an appropriate EBT (based on diagnosis)
• Having MMPI-2 information had no impact on
– the number of sessions patients attended
– whether therapy ended prematurely
– overall patient improvement in functioning assessed in the
end of treatment.
Lima et al. (2005). The incremental validity of the MMPI-2: When does therapist access not enhance treatment
outcome? Psychological Assessment, 17, 462-468.
A GUIDE TO ASSESSMENTS THAT
WORK
• Focus on specific assessment purposes directly pertinent to
clinical interventions:
– diagnosis (including screening issues and the importance of
addressing comorbidity)
– case conceptualization and treatment planning
– treatment monitoring and treatment evaluation
DIAGNOSIS
•
Diagnostic information allows access to relevant research on
psychopathology, epidemiology, prognosis, and treatment
•
Client characteristics (common comorbid conditions, likely health
concerns) and social/interpersonal characteristics (common problems
or limitations associated with social networks and intimate
relationships, work functioning, and healthcare utilization) that are likely
to merit further evaluation or consideration in treatment planning
– e.g., clients meeting criteria for a substance abuse disorder are
likely to abuse additional substances, and those who abuse
multiple substances are least likely to benefit from treatment
Rohsenow, D. (2008). Substance use disorders. In J. Hunsley & E. J. Mash (Eds.), A guide to
assessments that work (pp. 319-338). New York: Oxford University Press.
DIAGNOSIS
• Diagnosis has utility for EBTs
– i.e., points in the direction of treatment options based on
research evidence (e.g., NICE guidelines)
• Can give directions for treatment planning with respect to
comorbidity
– e.g., presence of depression among people with OCD can
diminish the effectiveness of exposure and response
prevention
Abramowitz, J. S., Franklin, M. E., Kozak, M. J., Street, G. P., & Foa, E. B. (2000). The
effects of pre-treatment depression on cognitive-behavioral treatment outcome in OCD
clinic patients. Behavior Therapy, 31, 517-528.
DIAGNOSIS
•
Symptom profile can be used to guide treatment selection
– e.g., bipolar disorder treatments to reduce manic symptoms should
address medication adherence and recognition of mood changes;
treatments focusing on cognitive and interpersonal coping
strategies reduce depressive symptoms
Miklowitz, D. J. (2008). Adjunctive psychotherapy for bipolar disorder: State of the
evidence. American Journal of Psychiatry, 165, 1408-1419.
•
Symptom profile can also lead to emphases in treatment strategies or
addition of treatment strategies
– Chorpita’s modular treatment approach
Chorpita, B. F. (2006). Modular cognitive– behavioral therapy for childhood anxiety
disorders. New York: Guilford Press.
Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children
and adolescents: Application of the distillation and matching model to 615 treatments
for 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566-579..
DIAGNOSIS
• Study of community based services for adolescents
• Disagreement between clinician-generated and research-based
diagnoses were associated with a host of treatment
implementation problems including:
– Increased number of client “no-shows”
– Cancelled treatment appointments
– Treatment drop-outs
– Inaccurate clinician-generated diagnoses were also
associated with smaller treatment gains
Jensen-Doss, A., & Weisz, J. R. (2008). Diagnostic agreement predicts treatment process
and outcomes in youth mental health clinics. Journal of Consulting and Clinical
Psychology, 76, 711-722.
TREATMENT MONITORING AND
EVALUATION
• What do all RCTs, used as evidence for EBTs, have in
common?
• Routine collection of monitoring data, typically reviewed by
clinicians during the supervision of services being provided
• Hypothesis: The fact that treatment is repeatedly monitored, and
information provided to clinicians, is one of the most important
contributors to successful treatment outcome in EBTs
– Collection of assessment data as treatment unfolds allows
for making any needed adjustments as required
• Does this apply to routine practice?
TREATMENT MONITORING AND
EVALUATION
• The potential benefits of treatment monitoring
– Outcome Questionnaire – 45
http://www.oqmeasures.com
– Over 2,500 clients from a range of clinics
– All completed OQ weekly, half of clinicians
received feedback, namely RED, YELLOW, &
GREEN dots
– No feedback: 21% improved, 21% worsened
– Feedback: 35% improved (i.e., 66% increase),
13% worsened (i.e., 33% decrease)
Lambert et al. (2003). Is it time to track patient outcome on a routine basis? A metaanalysis. Clinical Psychology: Science and Practice, 10, 288-301.
TREATMENT MONITORING AND
EVALUATION
• Most recent meta-analyses
Lambert & Shimokawa. (2011), Collecting client feedback. In Norcross (Ed.). Psychotherapy relationships that
work (2nd ed.).
• Outcome Questionnaire (4 studies, over 6100 clients)
– 2.6X greater likelihood of improvement
– Less than half the likelihood of deterioration
• Partners for Change Outcome Management System (3 studies,
over 550 clients)
– 3.5X greater likelihood of improvement
– Less than half the likelihood of deterioration
TREATMENT MONITORING AND
EVALUATION
Three specific categories of client and treatment variables that
should be given particular consideration:
• Treatment targets and goals (intermediate and ultimate)
• Causal mechanisms believed to be maintaining client problems
• Therapeutic context or process variables are particularly
relevant to enhancing treatment services
• In all instances, validity evidence for sensitivity to treatment
change is particularly relevant in selecting measures
TREATMENT MONITORING AND
EVALUATION
• To practice in an evidence-based manner, it is
inappropriate to claim that the treatment that a client
is receiving is effective simply because it is an EBT—
to determine the impact of any treatment, including
an EBT, it is essential that data are collected in order
to accurately determine its effects for individual
clients
Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental
health outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 47,
1114-1119.
INTEGRATION AND INTERPRETATION OF
ASSESSMENT DATA
• Clear evidence that clinicians (like people in general)
are not very accurate in self-assessment, have
limited awareness of biases, and are influenced by
numerous heuristics in the decisions they make
Ægisdóttir et al. (2006). The meta-analysis of clinical judgment project: Fifty-six years of
accumulated research on clinical versus statistical prediction. The Counseling Psychologist,
34, 341-382.
Davis et al. (2006). Accuracy of physician self-assessment compared with observed measures of
competence: A systematic review. Journal of the American Medical Association, 296, 10941102.
Garb, H. (2005). Clinical judgment and decision making. Annual Review of Clinical Psychology,
1, 67-89.
INTEGRATION AND INTERPRETATION OF
ASSESSMENT DATA
•
•
•
•
•
•
•
Use decision aids (such as decision trees in DSM-IV-TR, research
based data, norms, practice guidelines)
Use standardized, psychometrically strong instruments to collect data
Collect data from multiple sources (remembering that different
informants have differing experiences, perspectives, & attributions)
Ensure all aspects of assessment processes are sensitive to diversity
factors
Continue to evaluate and adjust hypotheses based on data
Treat all formulations as tentative
Remember heuristics (e.g., primacy effect, availability heuristic) and
biases (e.g., attributional biases, orientation influence) and actively
address them
EBA INITIATIVES
• Antony, M. M., Orsillo, S. M., & Roemer, L. (Eds.). (2001).
Practitioner's guide to empirically based measures of anxiety.
New York: Plenum.
• Nezu, A. M., Ronan, G. F., Meadows, E. A., & McClure, K. S.
(Eds.). (2000). Practitioner’s guide to empirically based
measures of depression. New York: Kluwer Academic.
• Journal of Clinical Child and Adolescent Psychology (2005)
special section
• Psychological Assessment (2005) special section
• Journal of Pediatric Psychology (2008) special issue
• A Guide to Assessments That Work (2008) Hunsley & Mash
• Handbook of Child and Adolescent Diagnostic and Behavioral
Assessment (in press) McLeod, Jensen-Doss, & Ollendick
THE PROFESSIONAL CHALLENGES OF
EBPP
•
•
•
•
•
Critical self-evaluation of knowledge and skills
Critically evaluate limits of competence
Time constraints for learning and skill development
What is sufficient training (peer supervision/consultation)
Competence + Frequency = ↑ Positive Outcomes
– implications for specialist vs. generalist?
• Possible impact on income/job requirements (referring to others
vs. time for training vs. client hours)
• Balancing the ethical/professional issue of competence with
reality of availability of appropriately trained mental health
professionals
EBPP: KEEPING UP TO DATE
• Example of CPA Code of Ethics (2000)
II.9 (Psychologists) Keep themselves up to date with a broad range
of relevant knowledge, research methods, and techniques, and
their impact on persons and society, through the reading of
relevant literature, peer consultation, and continuing education
activities, in order that their service or research activities and
conclusions will benefit and not harm others.
• Information overload: Primary care physicians need 21
hours/day to read all relevant articles that are published!
Alper et al. (2004). Journal of the Medical Library Association.
• Clearly some strategies and summaries are necessary
OXFORD UNIVERSITY PRESS SERIES
EBPP RESOURCES
•
Antony & Barlow (Eds.), (2010). Handbook of Assessment and
Treatment Planning for Psychological Disorders (2nd ed.). Guilford
Press.
• Clinician’s Research Digest
“Clinicians don't have to read all the journals publishing research of interest
to them—the Editor and staff of Clinician's Research Digest do it for
them. CRD reviews over 100 journals each month and highlights the
most relevant articles in this 6-page monthly newsletter.”
• Evidence-Based Mental Health
“Evidence-Based Mental Health journal surveys a wide range of
international medical journals applying strict criteria for the quality and
validity of research. Practising clinicians assess the clinical relevance of
the best studies. The key details of these essential studies are
presented in a succinct, informative abstract with an expert
commentary on its clinical application.”
TREATMENTS THAT WORK SERIES
http://www.oup.com/us/companion.websites/umbrella/treatments/?view=usa
•
•
•
•
•
•
•
•
•
•
•
•
Some examples:
Stress Management
Hoarding
Coping with Chronic Illness
Social Anxiety
Adult ADHD
Fears and Phobias
Panic
Worry
Eating Disorders
Pathological Gambling
Insomnia
School Refusal
ADVANCES IN PSYCHOTHERAPY:
EVIDENCE-BASED PRACTICE SERIES
http://www.hogrefe.com/program/books/book-series/advances-inpsychotherapy-evidence-based-practice.html
Some examples:
• Bipolar Disorder
• Heart Disease
• Obsessive-Compulsive Disorder
• Childhood Maltreatment
• Schizophrenia
• Treating Victims of Mass Disaster and Terrorism
• Attention-Deficit/Hyperactivity Disorder in Children and Adults
• Problem and Pathological Gambling
• Chronic Illness in Children and Adolescents
• Alcohol Use Disorders
• Borderline Disorder
• Prostate Cancer
• Diabetes
EBPP RESOURCES: WEB SITES
• The Cochrane Collaboration
http://www.cochrane.org/
• The Cochrane Library
http://www.thecochranelibrary.com/view/0/index.html
“The Cochrane Library contains high-quality, independent evidence
to inform healthcare decision-making. It includes reliable
evidence from Cochrane and other systematic reviews, clinical
trials, and more. Cochrane reviews bring you the combined
results of the world’s best medical research studies, and are
recognised as the gold standard in evidence-based health care.”
EBPP RESOURCES: WEB SITES
• TRIP Database
http://www.tripdatabase.com/index.html
“Welcome to the TRIP Database, the Internet's leading resources
for Evidence-Based Medicine. Allowing users to rapidly identify
the highest quality clinical evidence for clinical practice.”
• Enter a term (e.g., adolescent anxiety, postpartum depression,
psychosis) and search engine will list systematic reviews,
evidence based synopses, guidelines (international), etextbooks, etc.
EBPP RESOURCES: WEB SITES
• National Registry of Evidence-Based Programs and Practices
http://www.nrepp.samhsa.gov/
“Welcome to the National Registry of Evidence-based Programs
and Practices (NREPP), a service of the Substance Abuse and
Mental Health Services Administration (SAMHSA).
NREPP is a searchable database of interventions for the
prevention and treatment of mental and substance use
disorders. SAMHSA has developed this resource to help people,
agencies, and organizations implement programs and practices
in their communities.”
EBPP RESOURCES: WEB SITES
•
Society of Clinical Psychology (APA Division 12)
http://www.div12.org/PsychologicalTreatments/index.html
“The purpose of this website is to provide information about effective
treatments for psychological disorders. The website is meant for a wide
audience, including the general public, practitioners, researchers, and
students. Basic descriptions are provided for each psychological
disorder and treatment. In addition, for each treatment, the website lists
key references, clinical resources, and training opportunities.
The American Psychological Association has identified "best research
evidence" as a major component of evidence-based practice (APA
Presidential Task Force on Evidence-Based Practice, 2006). This
website describes the research evidence for psychological treatments,
which will necessarily be combined with clinician expertise and patient
values and characteristics in determining optimum approaches to
treatment.”
EBPP RESOURCES: WEB SITES
• Evidence-Based Mental Health Treatment for Children and
Adolescents
http://effectivechildtherapy.com/sccap/
“The information on this website is offered as a completely free
service to families and mental health professionals to help
ensure that children and adolescents benefit from the most upto-date information about mental health treatment. We request
absolutely no information from visitors to this site, and hope this
service will help all learn more about important differences in
mental health treatments. Families want their children to get the
best possible treatment, and this site maintains an updated list
of treatments with strong scientific support.”
HOW TO BE AN EVIDENCE-BASED
PSYCHOLOGIST
• Accurately assess and diagnose clients, paying close attention
to:
– Comorbidity
– Dominant client concerns
– Cultural factors
• Collaborate with clients to develop treatment goals, paying close
attention to client expectations and barriers to treatment
involvement
• Provide the most appropriate evidence-based treatment
possible, flexibly tailoring to client circumstances
HOW TO BE AN EVIDENCE-BASED
PSYCHOLOGIST
• Work to ensure a strong therapeutic relationship, and move
quickly to repair any relationship difficulties that arise
• Monitor treatment progress, both in terms of
symptoms/problems and the therapeutic relationship
• Regularly review treatment progress with clients, modifying
treatment as needed
• If treatment progress is suboptimal, be prepared to provide a
different EBT or refer to another professional who can offer a
different EBT
HOW TO BE AN EVIDENCE-BASED
PSYCHOLOGIST
Questions or comments?
For copies of the presentation, contact hunch@uottawa.ca