CLINICAL RESEARCH: USING EVIDENCE IN CLINICAL DECISION MAKING (44501.04) Spring Quarter 2014 Tuesdays 9:30 – 11.50 a.m., Room E-IV Instructor: Email: Tel: Office: Office Hours: Leyla Ismayilova, PhD leyla@uchicago.edu 773-834-0401 SSA, E-2 Tuesdays 2:00 – 3:00 pm Course Description The overall goal of Clinical Research course is to provide students with an understanding of how research is used to evaluate social work practice. The course also seeks to foster the development of critical and analytical skills for assessing and translating knowledge about effective social work interventions into real-world practice situations. The course is designed to provide students with an overview of the methods, concepts, and principles of practice evaluation research in order to become practitioners who can engage in self-directed evaluation of their own practice and to make meaningful contributions to program evaluation efforts. The course will expand students’ empirical knowledge, capacity for critical assessment, and research skill development in the generation, analysis, and use of information during each phase of the assessment and intervention process. Students will learn: 1) sound assessment procedures to identify and clarify the problem for intervention; 2) the ongoing evaluation of practice as it occurs. Building on principles of social scientific inquiry and basic research methodology covered in Introduction to Social Work Research, this course will reinforce and extend understanding of a scientific, analytic approach for social workers in advanced clinical practice so that they may build knowledge for practice and evaluate their service delivery. In addition, a systematic approach to knowledge building and evaluation of service delivery relevant to the advanced clinical practice method is reinforced and enhanced by other advance curriculum areas, including required advanced Clinical Practice courses; field of practice platform courses; and advanced practice electives. Students are expected to draw on previously acquired knowledge from prior research and clinical coursework regarding problem formulation, measurement, and research design as they learn to formulate and analyze research questions for clinical practice. Strict adherence to ethical standards of scientific inquiry will be emphasized, with particular attention to protecting and promoting the well-being of vulnerable and oppressed populations. 1 Students will be expected to demonstrate competency in “process evaluation” by engaging in ongoing qualitative assessment and monitoring as an essential part of the intervention. Furthermore, students will be expected to construct, critically assess, and adapt qualitative and quantitative tools for “outcome evaluation” involving monitoring and evaluation of client progress in reaching their goals. Such tools include various types of individualized and standardized measures, such as individualized rating scales, logs, observational measures, and rapid assessment instruments. Students will be expected to demonstrate competence in the critical examination and development of tools, within an agency context, either to assess unmet client or agency needs or evaluate clinical practice. These tools will include developing skill in designing a single-system evaluation for use with a single client, or client system, such as a couple or family, to assess clinical progress over time; and designing an original and “well-crafted” questionnaire for use with a small group of clients or clinicians engaged in a common clinical purpose. Examples of a common clinical purpose include clients participating in a therapeutic or support group or agency clinicians providing feedback about unmet needs or underutilized interventions for a particular group of clients. Finally, students will be introduced to software for statistical analysis and graphing of data from singlesystem designs and questionnaires. Students will apply one or more of these tools in a final evaluation project which will be conducted, whenever possible, within a specific clinical context from their field placement settings. The final project will address the planning, design, and execution of the evaluation, including data collection and analysis, as well as the reporting and application of findings to influence practice decisions. Course Content The role of theory (social work, social science, programmatic) in guiding accountable social work practice; The history, steps, and rationale for Evidence-Based Practice (EBP) and how EBP relates to the ethical practice of social work and practice evaluation; Qualitative assessment of competing issues to choose target(s) for intervention Measurement o Review of reliability and validity o Wording items for single cases versus for small-group questionnaires o Measurement reactivity o Use of multiple measures and perspectives Selecting measures of client change, including o Behavioral observations o Qualitative and quantitative logs o Individualized “self-anchored” scales o Standardized instruments for rapid assessment Critical appraisal of empirically supported treatments and best practices Client self-monitoring and clinician monitoring o To assess targeted problem at baseline as well as changes during intervention and at follow-up o To maintain fidelity and integrity of practice intervention over time Design options for core practice evaluation methods Qualitative, quantitative, visual, and statistical analysis of data 2 Course Objectives By the completion of this course, students are expected to demonstrate: 1. An understanding of the importance of evaluating clinical practice in social work, including strengths and limitations; 2. Knowledge and skills in accessing, critiquing, and utilizing empirical research to formulate practice-related research questions and evaluate client outcomes; 3. An appreciation of the importance of racial, ethnic, and cultural diversity and oppression within the context of conducting evaluation of social work practice; 4. Comprehension and adherence to ethical standards of scientific inquiry, including those in the NASW Code of Ethics and university/agency Institutional Review Boards, especially with regard to the well-being of vulnerable and oppressed clients; 5. Skill in clinical assessment and specification of client problems and goals; 6. An ability to operationalize client problems and goals; 7. An understanding of the steps involved in evaluating 1) an individual clinical case and 2) a small group of clients or clinicians engaged in a common clinical purpose, such as a support group or clinicians’ assessments of unmet client or clinical needs; 8. An ability to select or develop measurement tools appropriate for use with 1) a specific client and their client system; and 2) a group of clients or clinicians; 9. Proper choice of evaluation design options; 10. Critical selection of intervention method/s appropriate for use with specific client problems/issues and systems; 11. Collection of data and the use of actual data (primary or secondary) whenever feasible; 12. Skill in documenting and analyzing the effects of social work intervention; 13. Ability to appropriately and effectively apply evaluation findings to practice; 14. Ability to report clinical findings to clients and to a larger audience of practitioners. Note: We will adhere to University and School policies regarding accommodations for students with disabilities, religious holidays, incomplete work, plagiarism, and students' evaluations of the course and its instruction, as stated in the SSA/University of Chicago Student Handbook. Except when an "incomplete" grade will be granted, the final grade will be lowered if absence from class is excessive (i.e. more than two classes including any excused absence). Students with disabilities are strongly encouraged to follow University procedures to ensure that necessary accommodations are made at the beginning of the semester. Course Format This is an advanced level course designed with strong emphasis on lectures, class discussions, group exercises, class presentations, and computer lab sessions. Audiovisual tapes may occasionally be used to supplement the class lectures. Student Evaluation The written assignments, in-class presentation and quizzes are designed to assist you in: (a) integrating course material with field work content, (b) developing skills in differential assessment and intervention, and (c) integrating research concepts, theory, and practice. It is expected that student's assignments and class participation will reflect an understanding of both required and independent readings and critical thinking. Assignments are due at the beginning of class on the stated dates. Grades will be lowered for failure to follow directions from the syllabus 3 and the specific assignments. An “incomplete” grade must be requested in advance and will be granted only in extraordinary circumstances. The grade value of assignments is as follows: Assignments 1. Quizzes 2. Mid-term paper: Evaluation Proposal 3. Final paper: Evaluation / Self-monitoring Findings 4. Class Presentation 5. Class Attendance and participation in class activities Value 20 points 30 points 30 points 10 points 10 points 100 points total The grade for Class Attendance and Participation will be based upon promptness, attendance, and the quality and quantity of class participation. This does not mean that excessive participation, such as monopolization during class discussions, will be valued. For full credit in this area, I will look for a reasonable level of quality participation, beyond formally required class presentations. This will include your willingness to share during class discussions with your colleagues: 1) perceptions, feedbacks and questions from assigned readings; 2) measures and empirical clinical research from independent literature searches you conduct; and 3) case material and issues from your field placement, outside reading, or other sources. Presentations: Students will be required to make one class presentation of their projects. Students will have a choice of presenting their mid-term evaluation proposals or final results of their projects. Students who miss presentations by classmates will have 2 points deducted from their final grade for each day of presentations missed. Quizzes: To better facilitate students’ understanding of this course and prepare for the SW license exam, students are expected to take two quizzes given throughout the semester. Quizzes will include multiple-choice questions related to the assigned readings and lectures. Percentage and Letter Grading Scale: 94-100=A 90-93.9=A87.5-89.9=B+ 83-87.4=B 80-82.9=B77.5-79.9=C+ 73-77.4=C 70-72.9=CBelow 70=Failing. Required Texts: Rubin, A. and J. Bellamy (2012). Practitioner’s Guide to Using Research for Evidence Based Practice. 2nd Edition. Hoboken, NJ: John Wiley & Sons, Inc. Other required readings can be retrieved from the Chalk site or will be available on reserve at the SSA library. 4 SESSION OUTLINE AND READING LIST Note: Articles are available for download on the Chalk site WEEK 1 Overview of the Course. Introduction to Evidence-Based Practice and Practice Evaluation. Locating the Evidence. Integrating evaluation and practice Arguments for and against evaluation Similarities and differences between evaluation and research Overview of the steps involved in practice evaluation A first run through the single-subject approach Practice wisdom vs. evidence-based knowledge In-class Activity: Introducing EBP to an agency. REQUIRED READINGS: *Go to the Evidence-based Behavioral Practice Project /EBBP website (www.ebbp.org/training.html), register for the EBBP.org training portal, and complete two modules EBBP Process Module and Search for Evidence Module. *Gibbs, L., & Gambrill, E. (2002). Evidence-based practice: Counterarguments to objections. Research on Social Work Practice, 12(3), 452-476. Rubin, A. & Bellamy (2012). Practitioner's Guide to Using Research for Evidence-Based Practice. Chapter 1: Introduction to Evidence-Based Practice Chapter 2: Steps in EBP Process WEEK 2 Moving Beyond Empathy and Rapport: Empirically Supported Interventions. Assessing the Quality of Evidence Hierarchy of Evidence Critical selection of interventions Campbell and Cochrane Collaborations The Campbell Collaboration (C2): Systematic Reviews for psychosocial, behavioral, and educational interventions www.campbellcollaboration.org Use of manual-based treatments with empirical evidence of effectiveness Systematic Reviews and Meta Analyses In-class Activity: Review of systematic reviews, practice guidelines, and treatment manuals in small groups REQUIRED READINGS: *EBBP.org: Complete the following modules: 1. Systematic Review Module 2. Critical Appraisal Module 5 Rubin, A. & Bellamy (2012). Practitioner's Guide to Using Research for Evidence-Based Practice. *Chapter 3: Research Hierarchies *Chapter 4: Criteria for Inferring Effectiveness Chapter 8: Critially Appraising Systematic Review and Meta-Analyses *Review one systematic review of your choice: Bisson J, Andrew M (2009) Psychological treatment of post-traumatic stress disorder (PTSD). Systematic Review. The Cochrane Collaboration. JohnWiley & Sons, Ltd. Jones C, Hacker D, Cormac I, Meaden A, Irving CB (2012). Cognitive behavioural therapy versus other psychosocial treatments for schizophrenia. Systematic Review. The Cochrane Collaboration. JohnWiley & Sons, Ltd. De Silva MJ, Cooper S, Li HL, Lund C, Patel V (2013). Effect of psychosocial interventions on social functioning in depression and schizophrenia: meta-analysis. British Journal of Psychiatry, 202:253-260. Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Perspectives on Psychological Science, 2(1), 53-70. Recommended readings: Greenhalgh, T. (1997). How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). British Medical Journal, 315(7109), 672-675. WEEK 3 Assessment and Research Tools Incorporating different approaches within interviewing, assessment, and intervention. Attention to race, gender, ethnicity, power and life history In-class activity: Assessing a client using SCID (Structured Clinical Interview for DSM Disorders). REQUIRED READINGS: Find a systematic review, best model, or practice guidelines pertinent for one of two clients (brother or sister) from the Prince of Tides movie. Be prepared to discuss in class why the chosen intervention is the most appropriate for this client. To conduct the search, use online databases presented in class (Campbell Collaboration library, Cochrane Collaboration library, SAMHSA’s National Registry of Evidence-based Programs and Practices, etc). Link to watch The Prince of Tides movie for free. http://www.tubeplus.me/search/movies/The_prince_of_tides/0/ Do not download it. Just hit "Play" otherwise the site will request payment. *Tuerk, P., Brady, K. T., & Grubaugh, A. L. (2009). Clinical case discussion: Combat PTSD and substance use disorders. Journal of addiction medicine, 3(4), 189-193. *Corcoran J. and Walsh J. (2010). Clinical Assessment and Diagnosis in Social Work Practice. 6 Chapter 10: Depression Chapter 11: Substance Use (Focus primarily on the assessment and measurement instruments described in Appendix at the end of each chapter) *Jordan C. & Franklin, C. (2003). Clinical Assessment for Social Workers. Chapter 2: Linking Assessment and Intervention Chambless, D. L. & Holton, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7-18. WEEK 4 Treatment Objectives: Deciding on a Target for Intervention The interview as a focused clinical assessment method Problem identification and formulation Formulating questions from practice Setting measurable goals Choosing, implementing, and monitoring the intervention In-class activity: Reviewing treatment planners and setting treatment objectives (Jongsma, A.E., Jr., Peterson, L.M., Bruce T.J. (2014). The complete adult psychotherapy treatment planner. New York: Wiley, 4th ed.) REQUIRED READINGS: Assess your client’s problem using SCID interview or any other standardized assessment tool. Be prepared to discuss the results of assessment in class. Feel free to share copies of the assessment tool with the rest of the class. *Bloom and Brittner (2012) Chapter 2: Conceptualization: Naming what we see in the client situation. *Shonin, E., Van Gordon, W., & Griffiths, M. D. (2014). Cognitive behavioral therapy (CBT) and meditation awareness training (MAT) for the treatment of co-occurring schizophrenia and pathological gambling: A case study. International Journal of Mental Health and Addiction, 12(2), 181-196. *Chorpita, B.F., Becker, K.D., & Daleiden, E.L. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 647-652. Brophy, G. (2000). Social Work Treatment of Sleep Disturbance in a 5-Year-Old Boy: A Single-Case Evaluation. Research in Social Work Practice, 10(6), 748-758. Lau, A. (2006). Making the case for selective and directed cultural adaptations of evidence-based treatments: examples from parent training. Clinical Psychology, 13(4), 295-310. 7 WEEK 5 Measurement tools I: Basic Principles & Using standardized measures Measuring Change: What are we looking for in a measure? Review of basic measurement principles Reliability, validity, reactivity, utility, directness, and sensitivity to change Use of multiple measures Standardized instruments for rapid assessment Selecting Standardized Measures for Clinical Planning, Assessment, Monitoring, and Evaluation The ETS Test Collection (www.ets.org/testcoll) In-class Exercise: Reviewing standardized instruments. IN-CLASS QUIZ REQUIRED READINGS: *Berkowitz, S. J., Stover, C. S., & Marans, S. R. (2011). The child and family traumatic stress intervention: Secondary prevention for youth at risk of developing PTSD. Journal of Child Psychology and Psychiatry, 52(6), 676-685. *Rubin, A. & Bellamy (2012). Practitioner's Guide to Using Research for Evidence-Based Practice. Chapter 11: Critically Appraising and Selecting Assessment Instruments *Fonagy, P., Target, M., Cottrell, D., Phillips, J., & Kurtz, Z. (2002). Chapter 1: Introduction and review of outcome methodology: The importance of outcome measurement (pp. 1-41). In What Works for Whom?: A Critical Review of Treatments for Children and Adolescents. New York: Guilford. Download at www.guilford.com/excerpts/fonagy.pdf [Focus primarily on Levels of Outcome Measurement pp. 5-13] Jordan C. & Franklin, C. (2011). Clinical Assessment for Social Workers. Chapter 3: Quantitative Clinical Assessment Methods Chapter 4: Standardized Assessment Measures Recommended reading: Minton M.E. & Barron C.R. (2008). Spousal Bereavement Assessment: A Review of BereavementSpecific Measures. Journal of Gerontological Nursing, 34(8), 34. WEEK 6 Measurement tools II: Developing your own instruments Types of measurement tools, their strengths and limitations Observational Measures 8 Individualized or "Self-Anchored" Rating Scales Goal Attainment Scaling Client logs Collecting baseline SSE data and questionnaire data Developing a measurement and recording plan Coping with methodological and practical limitations; Self-management strategies and creating less stressful conditions for participants and researcher in the course of data collection. Ethical issues around collecting information from clients belonging to different race, gender, ethnicity, sexual orientation, religion, physical or mental abilities. In-Class Exercise: Pilot-testing an instrument In-Class Exercise: Crafting a questionnaire REQUIRED READINGS: Review a few standardized instruments pertinent to your clients/interests and choose one instrument that is the most appropriate for your client. Be prepared to explain your choice. Find an article describing or testing this instrument and be prepared to summarize and discuss it in class. Online databases and books listed below can assist you in searching for the instrument. Instruments for couples, families, and children Corcoran, K., and Fischer, J. (2013). Measures for clinical practice: A sourcebook, Volume One. Oxford University Press, USA. Instruments for adults Corcoran, K., and Fischer, J. (2013). Measures for clinical practice: A sourcebook, Volume Two. Oxford University Press, USA. *Rubin, A. & Bellamy (2012). Practitioner's Guide to Using Research for Evidence-Based Practice. Chapter 12: Monitoring Client Progress Bloom and Brittner (2012). Chapter 6: Individualized Rating Scales (IRS) and Structured Logs *Chapter 7: Behavioral observations *Roach, A. T. & Elliott, S. N. (2005). Goal attainment scaling: An efficient and effective approach to monitoring student progress. Teaching Exceptional Children, 37(4), 8-17. *Orgnero, M. I., & Rodway, M. R. (1991). AIDS and social work treatment: A single-system analysis. Health & Social Work, 16(2), 123-141. (Focus primarily on the description of instruments) 9 WEEK 7 Design Issues Involving Interventions and Questionnaire Data The basic AB design The withdrawal designs (ABA, ABAB, BAB) Multiple baseline designs and Multiple Target designs In-class Activity: Reviewing studies and identifying types of designs REQUIRED READINGS: *EBBP.org: Randomized Controlled Trials Module Rubin, A. & Bellamy, J. L. (2012). Practitioner’s Guide to Using Research for Evidence Based Practice. Chapters 5, 6, 7 (focus more on chapters 5 and 7) *Jensen C. (1994). Psychosocial treatment of depression in women: nine single-subject evaluations. Research on Social Work Practice, 4(3):267-282. Hunter JA, Ram N, Ryback R (2008). Use of Satiation Therapy in the Treatment of AdolescentManifest Sexual Interest in Male Children: A Single-Case, Repeated Measures Design. Clinical Case Studies, 7(1):54-74. MID-TERM PAPER DUE WEEK 8 Analysis of Evaluation Findings: Basic principles of data analysis Using Excel for statistical analysis and to graph results REQUIRED READINGS: *Bloom and Brittner (2012). Chapter 4: Graphing: the Basics; Exceptions to the Rules Chapter 14: Analysis of Data: A systematic and Holistic Approach *Norcross, J.C., Hogan, T.P., & Koocher, G.P. (2008). Chapter 5: Numbers and Measures *Bradshaw, W., & Roseborough, D. (2004). Evaluating the effectiveness of cognitive-behavioral treatment of residual symptoms and impairment in schizophrenia. Research on Social Work Practice, 14(2), 112-120. Cardaciotto, L., & Herbert, J. D. (2005). Cognitive behavior therapy for social anxiety disorder in the context of Asperger's syndrome: a single-subject report. Cognitive and Behavioral Practice, 11(1): 75-81. 10 Dixon MR., Jackson JW., Small SL. et al (2009).Creating single-subject design graphs in Microsoft ExcelTM 2007. Journal of Applied Behavioral Analysis, 42(2): 277–293. (if unfamiliar how to create graphs in Excel). WEEK 9 Data Analysis Lab Deciding if your client changed: A guide to clinical decision-making Results by inspection Clinical vs. statistical significance Reporting the results and presenting your findings In-class activity: Data Analysis Lab - students work on analyzing data for their final projects. IN-CLASS QUIZ REQUIRED READINGS: *Bloom and Brittner (2012). Chapter 15: Decision Making Mullane, J. Corkum, P. (2006). Case Series: Evaluation of a Behavioral Sleep Intervention for Three Children with Attention-Deficit/Hyperactivity Disorder and Dyssomnia. Journal of Attention Disorders, 10 (2), 217-227. Norcross, J.C., Hogan, T.P., & Koocher, G.P. (2008). Chapter 7: Translating Research into Practice Chapter 8: Integrating the Patient and the Clinician with the Research WEEK 10 Student Presentations and Course Wrap-up FINAL PAPER DUE 11 Mid-term Paper: Evaluation Proposal 1. Assessment. Select a client (current, past or hypothetical) and provide a brief but comprehensive assessment of the client’s problem. Include only necessary background information (e.g., client’s socio-demographic characteristics such as race, socioeconomic status, insurance status, sexual orientation, and legal aspects of treatment--voluntary, involuntary, court-ordered) and focus primarily on the assessment of client’s problem. 2. Treatment Objectives. Select and describe briefly one of the problems/issues/behaviors you have contracted to work on with your client and transform that problem into treatment objective(s). 3. Intervention. Provide a clear description of the intervention you are planning to use with your client/group to achieve the treatment objectives. Search the empirical treatment/intervention literature related to the client-specific problem and discuss if there is any research that supports the use of your intervention in this particular context or situation. It is important to include the intervention type/name and authors, key intervention components and strategies, evidence of intervention effectiveness compared to other treatment options and applicability to the client (issues related to age, ethnicity, gender identify, subpopulation). Based on the literature review, discuss why the selected intervention is most appropriate for your client and whether it is necessary to modify the intervention you are currently using with your client. 4. Outcomes and Measurement Instruments. List and operationally define the specific outcomes you think would be important to attain (so that others will understand it and will know when it is occurring). Remember: our clients, like all of us, seek not only symptomatic improvement, but improvement in subjective experiences of well-being and health. With regard to these outcomes, please address the following: • Provide a rationale as to why these outcomes are significant; • Briefly discuss how your clients demographic characteristics might influence these outcomes; • Discuss how these outcomes might vary from the following perspectives: client, family, employer, service-provider (i.e. you). Also, are there any additional perspectives to consider when thinking about outcomes? For each outcome, describe the measurement instrument/s you will use to monitor client change. Provide an example for two of four instruments: a) an individualized rating scale (IRS), b) a rapid assessment instrument (RAI or a standardized scale), c) an observational measure that you could use to measure change in this particular problem/issue, or d) a client log that could be used to monitor change in this particular problem/issue. Provide a discussion of strengths and weaknesses of these measures with regard to reliability, validity, and reactivity, and a brief rationale why you chose each of these measures. Attach a copy of both measures. 5. Measurement plan. Indicate who will fill out the measure and discuss your rationale for this choice. • How often will the client(s) be filling out the measure(s)? 12 • When and under what situation(s) will the client be filling out the measure(s)? • What steps have you taken to try to increase the consistency of data collection? Length: 1-1.5 single-spaced pages (12-font) (body of text) plus References and Appendixes. Include measurement instruments in Appendix. For citations in the text you can use Footnotes, Endnotes, Numbered or JAMA format. Please, use SCID interview or any other standardized tools to assess your client's problems. Use Treatment Planners to develop treatment objectives and Systematic Reviews, Meta-analyses or other sources to review interventions. 13 Final paper: Evaluation Findings In the final paper provide a brief description of the client (or group) focusing primarily on the results of the evaluation study conducted with this client or group. If you can’t collect data from your client, you can use yourself as a client or monitor your own behavior as a therapist/clinician. Please, provide specific answers to the following sections: I. Case Introduction Briefly present client’s presenting problem, targeted outcomes and intervention. II. Data Analysis & Results · · · · · Collect and analyze the data for the single client or group. Use the Excel spreadsheets to conduct statistical analysis of client's data. Briefly describe the data analytical techniques. Present the results in a graph and/or table. Attach graphs and/or tables in the Appendix. III. Interpretation and Conclusions · · Describe and interpret your main findings. What are the strengths and limitations of your findings? Describe how the results can inform clinical decisions for this client/group. Be thoughtful and creative. Incorporate feedback from the mid-term paper and presentation. Support your statements and clinical decisions with literature. Use sources presented in class - SCID interview, Treatment Planners, Systematic Reviews, Meta-analyses and Practice guidelines. Length: 1 single-spaced page (12-font, body of text) plus References and Appendixes. Include graphs and/or tables in Appendix. Final presentation should have similar structure as the final paper and be max 10 minutes in length. Use PowerPoint presentations and/or handouts. 14