Evolving the EBP Process Model: Advancing a Brown School Approach An Alternative Re-Affirmation Project Proposal Submitted to the Commission on Accreditation, Council on Social Work Education December 1, 2008 Submitted by George Warren Brown School of Social Work, Washington University in St. Louis authors Brett Drake, PhD Associate Professor Re-Affirmation Project Director Tonya Edmond, PhD Associate Professor Associate Dean for Academic Affairs Re-Affirmation Project Director Ellen Rostand, MBA Assistant Dean for Communications our vision To create positive social change through our path-breaking research and educational excellence. our mission •To educate and prepare future social work and public health leaders in areas of policy, practice, and research. •To pioneer research and apply results to impact policy and practice locally, nationally, and internationally. •To collaborate with organizations to use evidence to improve access to and quality of social services and to address social and economic justice. Contents A History of the Brown School...............................................................................1 The Evolution of the EBP Process Model...........................................................3 A Review of the Literature................................................................................3 The Brown School Approach.............................................................................6 Reaffirmation Project: Goal, Objectives, Strategies, and Evaluation..............9 Implementation Tools and Trainings....................................................................16 Product Development and Dissemination.............................................................20 Timeline.........................................................................................................................23 Why Now? Why the Brown School?.....................................................................25 Reaffirmation Project Leadership.................................................................... 25 Resource Commitment....................................................................................... 26 Appendices....................................................................................................................29 Appendix A: Faculty EBP publications over past 5 years or longer....... 29 Appendix B: Members of Brown School EBP Task Force....................... 39 Appendix C: References..................................................................................... 40 Appendix D: Members of Brown School Macro Work Group................ 41 Appendix E: Parish/Rubin Instrument.......................................................... 42 Appendix F: Members of Brown School Curriculum Committee.......... 50 Appendix G: Members of Brown School Foundations Work Group..... 51 Appendix H: Sample syllabus from a foundation course............................ 52 Appendix I: Members of Brown School EBP Steering Committee........ 61 Appendix J: Members of Brown School National Council....................... 62 Appendix K: Members of Dean’s Professional Advisory Council.......... 64 the brown school A History of the Brown School Is social work a profession? This is the question that Abraham Flexner, a champion of medical school education reform, posed at the 1915 National Conference of Charities and Corrections. His stinging criticism of the lack of scientific rigor of early social work education helped spark a pedagogical evolution that continues today. Ironically, at this same time while social work programs began to flourish, Washington University in St. Louis, established in 1853, was getting out of social work education altogether, having offered social work training courses since 1909. Financial constraints and strained relations between the program director and the University Chancellor resulted in the demise of the fledgling program. But the University eventually heard the community’s call for trained social workers, and established a formalized social work program in 1925, and later our School in 1945. Today the Brown School is ranked as the top graduate school of social work in the country according to U.S. News & World Report, and regularly ranked among the top schools in terms of faculty productivity. We have been continually accredited, first by the American Association of Schools of Social Work and then by the Council on Social Work Education. Our last three reaccreditation reviews (2002, 1994, 1986) resulted in approval of our submitted documents without revision. Most recent affirmation of accreditation was formally granted on February 27, 2003, extending until February 28, 2011. Although the language we have used over the past century has changed, we have never wavered from our commitment to advancing our curriculum and identifying and applying the best available evidence to tackle challenging social problems. Evidence-based practice, however, did not become an official Brown School watchword until a little more than a decade ago. Since that time, we have been at the forefront of the movement to advance the utilization of evidencebased practice in the social work profession, both inside and outside the classroom. In 2000, we became one of the first U.S. schools of social work to embrace an EBP approach for its MSW curriculum. This same year, we convened and hosted a national conference on advancing EBP in social work. Our Office of Field Education has provided field instructor workshops and revised our practicum evaluation forms to include questions about students’ ability to apply EBP in their agency work. We also have sponsored several highly subscribed sessions on EBP for local agencies, and our orientation for incoming MSW students now includes a session on the role of research in social work. In our most recent reaccreditation application (2002), we adopted evidence-based practice and capacity building as core teaching themes and over the past 10 years our faculty has published evolving the ebp process model-december 1, 2008 -1- the brown school numerous articles on critical issues pertaining to evidence-based practice (See Appendix A). And in fall 2005, our new dean, Edward F. Lawlor, convened an EBP Task Force (See Appendix B) to assess our status with regard to evidence-based practice and consider how EBP cross cuts several areas of school activity and institutional next steps. The Task Force was critical of our efforts to date, finding that despite the strides we have made in gaining visibility around EBP, we needed a much more comprehensive systematic restructuring of our curriculum, including field education, to effectively train our students as true evidence-based practitioners. We want to continue to endorse and advance EBP with energy, innovation, and vision. The goal of our re-affirmation submission is to address this curricular challenge by designing, disseminating, and evaluating a model MSW curriculum that will further advance the capacity of evidence-based social work practitioners. By advancing a “Brown School Approach” to the EBP Process Model, we aim to prepare social work students here and elsewhere to employ the best available evidence when making practice decisions. Specifically we plan to: The goal of our re-affirmation submission is to address this curricular challenge by designing, disseminating, and evaluating a model MSW curriculum that will further advance the capacity of evidence-based social work practitioners. 1. Build capacity of all Brown School educators (current full-time faculty, new faculty, adjunct faculty, doctoral teaching fellows) to cogently and consistently teach our MSW students the Brown EBP Process Model (the Brown School Approach). 2. Restructure our MSW curriculum, including both foundation and concentration courses, to systematically and progressively teach the Brown School Approach, ensuring effective horizontal and vertical integration. 3. Build capacity of field education/practicum sites to model, supervise and support the implementation of the Brown School Approach. evolving the ebp process model-december 1, 2008 -2- the brown school The Evolution of the EBP Process Model Although we begin our description of evidence-based practice with recent usage of the term, we are cognizant of the longstanding commitment of social work to the use of evidence. Many of our profession’s best and most defining moments have centered on the thoughtful generation and/or use of evidence. Nonetheless, there has been a long standing interest in and many efforts toward strengthening the links between research and practice. We are mindful of previous efforts in our profession to advance the use of scientific evidence in social work practice that have been met with limited success (single-subject designs, the practitionerresearcher model). Consequently, some have argued that the current excitement about EBP is not entirely new, and concerns exist that this is yet another fad that will eventually fade away as others have. We believe that EBP is essential to the field of social work, that we have an ethical imperative to employ evidence, and that to fail to do so puts our profession at risk of irrelevancy. Medicine, nursing, public health, and psychology, important allied disciplines, are all making rapid strides in advancing the use of evidence-based practice and social work is perhaps in its strongest position ever to join in these efforts to ensure that clients, consumers, and other constituencies receive the highest quality of service. Through our reaffirmation project, we aim to extend the next chapter of EBP into the mid-century. A Review of the Literature EBP has gained considerable momentum in social work over the past two decades. What follows is a brief overview of the literature and a description of some of the guiding principles informing our reaffirmation project (See Appendix C for References). 1992- 1996: Origins of the Contemporary Term “Evidence-based practice.” The term EBP has its roots in medicine; coming to the national forefront in the early 1990’s when the first article on evidence based medicine was published in the November 4, 1992 issue of JAMA by the Evidence Based Medicine Working Group (EBMWG, 1992). EBM was further codified with the publication of the medical text Evidence Based Medicine: How to Practice and Teach EBM (Sackett, Richardson, Rosenberg & Haynes, 1996). During this time, EBP represented a radical departure from prior forms of medical and social service delivery because it placed a “much lower value on authority” (EBMWG 1992, p. 2421). Historically (EBMWG, 1992; p. 2420), practitioners had no choice but to rely mainly on tradition and opinions of those in authority when making decisions. Driven by a growing body evolving the ebp process model-december 1, 2008 -3- the brown school of high quality scientific evidence and technological advancements, such as the Internet and search engines, practitioners had the ability to access good evidence in real time to make their own practice decisions. According to Sackett and his colleagues (Sackett et. al. 1996), EBP was a formalized five-step process: 1. 2. 3. 4. Converting the need for information into a question Tracking down the best evidence to answer the question Appraising that evidence for validity, impact, and applicability Integrating this appraisal with our expertise and client’s unique biology, values, and circumstances 5. Evaluating effectiveness and efficiency in completing steps 1-4 The circles in Sackett’s model (see below) visually emphasize how critically important professional judgment and client factors are to the EBP Process. Too often the discussions of EBP become myopically focused on the best available evidence circle, but the reliance on evidence without the application of good professional judgment or consideration of client values, preferences, and context is problematic, and not equivalent to true evidence-based practice. EBP’s focus on the primacy of the individual practitioner is very desirable in graduate social work education because the MSW is the terminal practice degree, a prerequisite for licensure, and is geared toward creating social workers who can practice autonomously. Best Available Evidence EBP Professional Judgment Client Factors evolving the ebp process model-december 1, 2008 -4- the brown school Mid 1990’s – Mid 2000’s: A Different EBP Model Emerges Although the process model developed by Sackett and his associates is clearly delineated, during the 1990’s, a different and much more limited definition emerged. During this time, EBP was sometimes employed to refer to the use of empirically supported treatments (ESTs), which were often confusingly described as “evidence-based practices.” This was particularly true in the field of mental health where there was much discussion occurring regarding the role of practice guidelines in clinical decision making. While guidelines provided very useful information about recommended evidence-based intervention options, there were concerns that they would potentially reinforce the over reliance of expert opinion and authority. Certainly ESTs are an essential resource in evidence-based practice and are highly valuable, but this restricted definition led to some confusion and debate about what constituted evidence-based practice. A common question was: Is EBP a process or a product? (Rubin, 2007). Virtually all of the criticisms of EBP in social work have been criticisms of the “practices” model (Gibbs & Gambrill, 2002) and involve concerns that clinical judgment, client diversity and preferences, context, and other critical factors would be downplayed in favor of “cookbook” application of validated treatments. The irony is that the focus on clinical judgment and client factors are strengths of the original process definition of EBP, which when described rarely fosters resistance among practitioners. 2006–Present: A Return to the Process Model Today the tide has turned. Thanks to two key milestones – the “Austin Conference” and a groundbreaking issue of the Journal of Social Work Education – the social work profession is moving toward consensus of EBP’s definition and a return to EBP’s process-oriented roots. First, the “National Symposium on Improving the Teaching of Evidence-based Practice” (aka “The Austin Conference”) hosted by Allen Rubin at UT Austin in October 2006 brought together many of the key social work leaders in EBP, including one of our own faculty members, Dr. Enola Proctor. The papers presented remain one of the most concentrated sources of “must read” EBP material in our field. This conference is not only useful for its indepth analysis of social work educational issues stemming from EBP, but it is invaluable as a frozen “moment in time” capturing disparate views of the “practices” and “process” versions of EBP. The second milestone was the fall 2007 special issue (43[3]) of the Journal of Social Work Education. In the lead editorial article, we begin to see that social work is reaching a consensus, embracing the “process” model of EBP. This article (“Implementing and Sustaining evolving the ebp process model-december 1, 2008 -5- the brown school Evidence-based Practice in Social Work,” Walker, Briggs, Koroloff & Friesen (2007)) remains the most clear and concise source on the current status and future direction of EBP in social work. Essentially, this article, and the journal issue as a whole, suggests the necessity of embracing the “process” model of EBP, and includes substantial general guidance on how EBP can be used in social work. It includes important content concerning current challenges related to EBP (Rubin & Parrish, 2007) and curricular issues with implementation of EBP at the MSW level (Drake, Hovmand, Jonson-Reid & Zayas, 2007). The Brown School Approach: Our EBP Process Model As a result of our involvement in the scholarly and intellectual examination of evidencebased practice, we have come to a number of conclusions that inform our approach to teaching evidence-based practice and subsequently this proposal. 1. EBP is a process. Our work is aligned with the emerging consensus that EBP is a process. We view empirically supported treatments as an important element of social work practice that is easily and naturally subsumed under the EBP process model. 2. EBP can bridge the research-practice gap. We view EBP as an approach that has the potential to effectively bridge the gap between research and practice. True EBP practitioners need a strong grasp of research methods, an ability to assess and integrate existing research in practice decision-making, and the ability to generate needed evidence from one’s own practice when possible. Similarly, under an EBP framework, researchers can best claim utility for their work by showing how their work can answer questions that practitioners generate in the field. One of the critical features of the EBP Process Model that has been missing from previous models that promised to bridge the research-practice divide is an integrated understanding of and explicit expectation for the importance of the practitioner’s judgment and experience. 3. The EBP Process Model is a framework for social work practice. We view the EBP Process Model as a coherent framework for social work practice, not a competitor with or replacement for traditional professional touchstones. Core social work touchstones, such as social and economic justice, diversity, and the “person-in-environment” perspective are in fact strengthened under the EBP framework, which requires attention to these elements as part of all practice decisions. In fact, it provides a means for showcasing and supporting social work’s unique strengths. The EBP Process Model also supports social work’s long standing commitment to critical thinking (Gambrill, 1997, 2000). Futhermore, the model is consistent with the value social work has always placed on lifelong learning, as the process requires social workers to engage in lifelong learning as a standard part of everyday practice. evolving the ebp process model-december 1, 2008 -6- the brown school 4. The EBP Process Model applies directly to macro practice. Although the EBP Process Model works well for all forms of social work practice, too often the language used in the literature is heavily skewed towards direct practice, which can be a barrier to receptivity to and adoption of evidence-based practice. To address this shortcoming, in fall 2007, we formed a macro practice work group to ensure that our approach and model of EBP practice was truly applicable to all fields of social work practice. The group concluded that the model needed a modification to more explicitly depict the importance of social context and the formal and informal structures influencing all involved persons. (See Appendix D for list of members of Macro Practice Work Group) In addition we created an acronym, “FLAIR,” to help students and practitioners remember the five steps on the process. The Brown School Model of EBP is graphically depicted below. Brown School EBP Process and “Circles” Best Available Evidence Practitioner Judgment and Experience Involved Persons (clients and others) Social Context (formal and informal) 1. Formulate Question 2. Locate Best Available Evidence 3. Assess Quality of Evidence 4. Integrate Evidence with judgment, involved persons and social context 5. Review process evolving the ebp process model-december 1, 2008 -7- the brown school Although social workers engage in a wide array of practice activities with varying emphasis on macro, mezzo and micro levels (e.g. lobbying vs. direct mental health practice), through the Brown School Approach to EBP, they will share a common practice framework that includes our five-step “FLAIR” process; and consideration of best available evidence, practitioner judgment, involved persons and social context. Our model of EBP also assures the explicit use of the ecological model, one of the hallmarks of social work practice. The model stresses explicit consideration of the social context, both formal and informal, requiring the student to consider and address formal structural factors (e.g. organizational context and dynamics, availability of employment opportunities, policies that influence access to and availability of resources, etc.) and informal factors (e.g. community or societal values and norms that might bear on the situation) affecting all involved persons. To better include the macro level of social work practice and to acknowledge that social workers often operate in the context of multi-disciplinary teams, we have moved away from using the word “client” to “involved persons.” For example, in a community development project, involved persons might include community members, members of organizations serving the community, experts in other disciplines like public health, and policy makers. In a policy context, involved persons might range from those impacted by a policy, to advocates, persons in relevant organizations, policy makers and even business leaders. Our EBP approach will provide a structured framework to help new students learn to consistently apply an ecological approach to their work, making both models a reflexive part of their social work practice. Our EBP model also assures the centrality of social justice and diversity in social work practice. Framing a question (Step 1) will often be done differently in different contexts and depending on the population/community being served. Steps two and three (finding and critiquing best available evidence) will also change radically based on diversity issues: • Is there evidence on treatment effectiveness in which the study samples included the population you will apply the intervention to? • Are populations broken out in the literature with regard to differential effectiveness? • Are there studies available exploring how different groups may perceive or react to the intervention? As diversity and the environmental perspective are highlighted in Step 4, so too is social and economic justice. All social work students are trained to be sensitive to, recognize, and combat unjust formal and informal structures. To embrace this social justice perspective requires a broad view of the word “evidence” as well. So, Steps 2 and 3 must also change to address the nature of the problem. Using our EBP Process Model, we will teach our students exactly how these necessary and defining social work acts fit into every practice decision. evolving the ebp process model-december 1, 2008 -8- the brown school Reaffirmation Project: Goal, Objectives, Strategies and Evaluation Our reaffirmation proposal The literature on EBP continues to flourish, both within and is geared to infuse the EPB beyond social work. We believe that social work education in Process Model in all aspects the next twenty years will feature a substantial shift toward the of our curriculum with the process model of EBP as described by Walker et. al. (2007) and goal of graduating true EBP Drake et. al. (2007). Our reaffirmation proposal is geared to infuse practitioners. the EPB Process Model in all aspects of our curriculum with the goal of graduating true EBP practitioners. By advancing a “Brown School Approach” to the EBP Process Model, we aim to prepare our own students, and hopefully other social work students to employ the best available evidence when making practice decisions. In fact, work is already underway to ensure that if our reaffirmation proposal is approved, our first MSW cohort under this model will have graduated just prior to our CSWE site visit in 2010. Consequently, we felt it critical to start now to ensure that we design our program to complement the 2009 version of EPAS. Below we have outlined our three main objectives and a set of implementation strategies and evaluation measures for each. Where appropriate, we have included an update of the groundwork completed to date. Objective 1: Build capacity of all Brown School educators (current full-time faculty, new faculty, and adjunct faculty, as well as doctoral teaching fellows) to cogently and consistently teach our MSW students the Brown School EPB Process Model. Implementation Strategies: 1. Develop specific training modules on the EBP Process Model for each target group. Dr. Danielle Parrish and Dr. Allen Rubin from the University of Texas at Austin have developed a day-long training module to teach the EBP Process Model to social work practitioners. In spring 2008 we piloted this model with a small group of full time faculty, adjunct faculty and teaching fellows to assess how it would work for these target groups. The model worked very well and we received excellent feedback on the value and quality of the training from our pilot group. However, we plan to strengthen the applicability of the training evolving the ebp process model-december 1, 2008 -9- the brown school for macro social work practice. We will modify the training to match the specific training needs of each target group. For example, full time faculty and doctoral students in most cases will not need training to understand how to conduct a search of the literature or to critically appraise the methodological quality of the literature obtained, but adjunct faculty most likely would. We will require this training for all new full-time faculty, current and future adjunct faculty, and doctoral students that serve as TAs or Teaching Fellows (independent instructors). Current full-time faculty has already had extensive exposure to our EBP Process Model through faculty retreats, the EBP Taskforce, and various EBP-related work groups. Evaluation: 1. We will conduct a pre-training/post-training assessment utilizing an instrument developed by Dr. Danielle Parrish and Dr. Allen Rubin at the University of Texas at Austin to evaluate the effectiveness of the EBP Process training they developed for social work practitioners (See Appendix E). We will modify as needed to reflect the changes that are made to construct the specific training modules. Objective 2: Restructure our MSW curriculum, including both foundation and concentration courses, to systematically and progressively teach the EBP Process Model ensuring effective horizontal and vertical integration. Implementation Strategies: 1. Restructure foundation courses. Beginning in fall 2007, we formed a work group of all the lead teachers for the Foundations Curriculum and tasked the group with the responsibility of developing a strategy for infusing the Brown EBP Process Model and the new EPAS competencies across the foundation courses. The group met weekly for nine months and diligently crafted a systematic approach for introducing our model and establishing specific responsibilities for each foundation course related to key knowledge and skills needed to develop competencies with the EBP Process Model. In the process we: • restructured the content and format of all foundation courses syllabi to make explicit the core EPAS and EBP competencies expected for each course; • developed common pedagogical elements for courses with multiple sections to ensure consistency in learning opportunities; and • developed recommendations for a required sequencing of courses to facilitate the progressive acquisition of requisite knowledge and skills, as well as good horizontal and vertical integration of course content. evolving the ebp process model-december 1, 2008 -10- the brown school Our Foundations Committee regularly presented reports of their work to the Curriculum Committee, which ultimately gave full approval to their recommendation of these curricular changes. In May 2008, the full faculty unanimously approved the recommended changes to the foundation curriculum. We are currently pilot testing the restructured foundation curriculum to identify potential modifications that might be warranted for our reaffirmation project. 2. Increase exposure to Brown EBP Process Model for incoming advanced standing students. The newly restructured foundation curriculum is well positioned to ensure that all incoming MSWs who do not have a BSW receive rigorous training in the EBP Process Model. However, BSWs can receive up to 19 hours of credit for foundation level courses leaving them without the necessary exposure to the EBP Process Model and the opportunity to develop the knowledge and skills needed to implement it in practice. To address this concern we have developed a new course exclusively for BSWs - “Evidence Based Social Work for BSWs.” We are currently piloting this course to identify potential modifications we may need to make for our reaffirmation project. 3. Restructure concentration courses. Beginning fall 2008 our Curriculum Committee began discussing how to build on the work of the Foundations Work Group to ensure vertical integration of both the new EPAS competencies and opportunities to develop the knowledge and skills needed to implement the EBP Process Model within the context of a specific concentration. Each of the Concentration Chairs, who are members of the Curriculum Committee, will be working with faculty within their respective concentrations to review current EBP relevant content covered in the current syllabi and to identify potential modifications that may be needed. This preliminary work is expected to continue through the 2008-2009 academic year and to produce substantive recommendations for curriculum changes to advance the goal of producing high quality EBP social work practitioners within each of our concentrations. We plan to pilot recommended changes in the 2009-2010 academic year to identify any modifications that we may need to make. Faculty will formally approve these changes later in the academic year. (See Appendices F and G for members of Curriculum Committee and Foundations Work Group) 4. Develop Empirically Supported Treatments Skills Labs. According to Weissman, Verdeli, Gameroff, Bledsoe, Betts, Mufson, Fitterling, & Wickramaratne (2006), very few graduate students in social work or psychology graduate with any real exposure to empirically supported treatments. They conducted a national survey on psychotherapy training that included psychiatry, psychology and social work. Although social work and PsyD programs train the largest number of clinical students, they have the lowest rates evolving the ebp process model-december 1, 2008 -11- the brown school (10%) of meeting the gold standard for learning a new treatment approach—combining didactic programs with clinical supervision. Weissman and her colleagues state that “until the training programs in the major disciplines providing psychotherapy increase training in EBP [evidence based treatments], that gap between research evidence and clinical practice will remain (p. 925).” We are committed to ensuring that all of our students will graduate with a thorough understanding and ability to implement the EBP process, and they will be knowledgeable about specific empirically supported treatments or practice approaches that are viewed as critically important within their field of practice. Our students will gain much of this knowledge in their practice methods courses within their concentrations. To supplement this knowledge and to create more focused opportunities for practice skill development, we will develop a series of skills labs on empirically supported treatments. The Curriculum Committee approved this idea in spring 2008, and we are currently piloting the idea this academic year by offering 1.5 credit hour skills lab courses. This fall we offered Motivational Interviewing and Assertive Community Treatment, and in the spring we will offer Dialectical Behavior Therapy and Interpersonal Therapy. To support the interests of our macro students we are offering a skills lab in GIS (Geographical Information Systems). All of the Concentration Chairs have been encouraged to identify specific skills labs that they would like to make available to their students. 5. More fully integrate field education efforts. We cannot educate competent EBP practitioners without the inclusion and integration of field education. Our field education staff has been actively engaged on every internal work group related to EBP that has existed for the past eight years. They have been at the forefront of our efforts to advance evidence-based practice inside the classroom and within the community. As a part of the Foundations Work Group, the field education staff integrated the EBP Process into the Integrative Seminar and they are currently piloting those changes. In preparation for our reaffirmation project, we are revising virtually all of our practicum related documents to support students’ learning of the Brown EBP Process Model and capacity building within the practice community. We are planning to modify our Educational Learning Agreements that students complete as a contract with their practicum site to ensure that EBP Process activities are explicitly described. The midterm and final evaluations that practicum instructors complete, and student evaluations of the site and their field instructor will also be modified. evolving the ebp process model-december 1, 2008 -12- the brown school Evaluation: 1. Each lead teacher and concentration chair will conduct a content analysis of all syllabi under their responsibility to assess degree of compliance with course competencies and expectations for common pedagogical elements, and explicit EBP relevant content and assignments. 2. We will modify course evaluations to gather student feedback on their self reports of learning the EPAS competencies and the EBP Process Model. 3. We will collect data from field instructor evaluations of student implementation of EBP Process Model by modifying our current field instructor mid-term and final evaluations. 4. We will assess current students’ knowledge of the Brown EBP process. Because we are testing a number of changes in an incremental process prior to the start of our reaffirmation project, we plan to collect data from several cohorts at different points in time. In May 2009 we will have a cohort of students graduating who have been exposed to the general principle of the importance of using evidence to guide practice that has been in place since our last reaccreditation in 2002. These students have heard repeatedly about the value of the best available evidence but they have not been trained in the EBP Process Model in a systematic and progressive manner. We will assess their knowledge of the EBP Process with a post-test using the instrument previously discussed that was developed by Dr. Parrish and Dr. Rubin. We will also use this instrument as a post-test for the students graduating in May 2010 who will have only been exposed to an incomplete version of the piloted revised curriculum. This will allow us to make preliminary comparisons in the effectiveness of teaching students to be EBP practitioners between our old model and our revised model. 5. We will pre-test knowledge of incoming students. In August 2009 we will be able to pre-test our incoming students who will be the first cohort to receive our fully revised curriculum. Using the same instrument, we will compare these data to the 2010 exit cohort scores (pre/post using different cohorts). Later we will also compare the 2009 entry cohort data to the May 2011 exit data (pre/post using same cohort). This will allow for a comprehensive evaluation of the effectiveness of our curriculum. We will supplement this data with self assessment information gathered from a modified version of an exit survey we currently give to our graduating students. evolving the ebp process model-december 1, 2008 -13- the brown school Objective 3: Build capacity of field education/practicum sites to model, supervise and support the implementation of the EBP Process Model. Implementation Strategies: 1. Develop a training module on the EBP Process Model for field instructors (Practicum site supervisors). We will build on the training module developed by Dr. Parrish and Dr. Rubin previously described. In spring 2008 when we piloted this model, we included several practicum instructors from a range of agencies and they all provided very positive feedback about their training experience. 2. Hold mandatory day long training for all local affiliated field instructors. Specifically, we will hold a series of workshops limited to 20 practitioners at a time. Given that we have approximately 200 field instructors we anticipate that it will take two years to train everyone. Consequently, our field instructors will be given two years to complete the mandatory training. 3. Allow field instructors to attend the Empirically Supported Treatment Skills Labs. As Weissman and her colleagues (2006) point out, the gold standard for learning a new treatment requires clinical supervision in the method. To strengthen the capacity of our field instructors to provide clinical supervision in empirically supported treatments we will invite them to attend these skills labs, create incentives for them to attend and reduce barriers that might inhibit their participation. These 1.5 credit courses will only be offered to field instructors at no cost and will only be held during weekends to reduce potential scheduling barriers that might inhibit field instructor participation. They will also be given parking passes free of charge as an expression of appreciation for their service to our students and the profession. They will participate jointly in the lab with MSW students, an effective approach used by the University of Michigan. 4. Offer Empirically Supported Treatments Skills lab through our ongoing Professional Development Series that is well attended by alumni, field instructors, and other community practitioners. Evaluation: 1. We will conduct a pre-training/post-training assessment utilizing the modified instrument originally developed by Dr. Parrish and Dr. Rubin following completion of the mandated day long training on the Brown EBP Process Model. 2. We will modify the current instrument that our students use to evaluate their field instructor and practicum site. We will make modifications to incorporate the student’s evolving the ebp process model-december 1, 2008 -14- the brown school evaluation of the field instructor’s knowledge of and receptivity to the use of the EBP Process Model. 3. We will track the number of field instructors who elect to take one of our skills labs or EBP-related professional development workshops and evaluate increase in participation over time. 4. We will develop and implement a survey that assesses the implementation of any empirically supported treatments that field instructors obtain through one of the skills labs. A note about human subjects and Institutional Review Board. We will implement our reaffirmation proposal with the highest ethical regard and in compliance with all applicable guidelines. All measures except the EBP knowledge instrument (Parrish/ Rubin) are standard in-house measures (e.g. course evaluations, exit survey, field evaluation) which would be given in any case. However, the reaffirmation proposal is distinct from normal administrative surveys directed towards improving and/or developing Brown School services and programs. Given the potential for publication and dissemination, we fully expect that our project will be generalizable and therefore meets the federal definition to be considered research with human subjects. Accordingly, we will seek human subjects approval from the Washington University Human Research Protections Office (HRPO). We will seek approval under exempt category 1 for research in educational settings. Faculty and staff undertaking this project have significant experience conducting research in educational settings and understand the confidentiality and coercion risks attendent to this type of work. All materials will be submitted to HRPO for review and approval. The protocol will thoroughly describe the purpose of the project, data to be obtained, standards for protecting data and publication goals. evolving the ebp process model-december 1, 2008 -15- the brown school Implementation Tools and Training To help support the implementation of the Brown School Approach in the classroom and the field, we will develop the following trainings and tools. • EBP Process Model Introduction Workshop: We will develop and make this eight hour workshop available to classroom teachers and field instructors. • New Faculty “Up-to-Speed” EBP Workshop: We will develop and make this three-hour training available to all new faculty. • New course syllabus structure: We have restructured our course syllabi in order to make transparent (Gambrill, 1997, 1999) the EBP and CSWE required competencies that will be expected for each specific course along with common pedagogical elements designed to ensure consistency in teaching across multiple sections of one course. We have already revised syllabi for our foundation courses. Syllabi for all concentration courses will be restructured by August 2009. These syllabi include three sections: o Course Domain that outlines the purpose and content of the course. o Course Competencies that outline those specific competencies taught in the course. Viewed across classes, this section provides a structured way of presenting or evaluating the curriculum as a whole, and are directly linkable to 2008 EPAS competencies. o Common Pedagogical Elements that offer specific guidance to teachers regarding assignments, activities and instruction within the course. In short, this section allows us to provide some concrete guidance about activities that need to occur in the classroom within each section of the course. (See Appendix H for sample syllabus from foundation course) • New course evaluations. We will revise our course evaluations to enhance teachers’ ability to gain feedback on how students perceive their teaching relative to EBP. • New field education documents. We will revise all field education documents, specifically Educational Learning Agreements, mid-term and final evaluations of student performance, student evaluation of practicum instructor and practicum site to highlight EBP content. To provide additional 24/7 student and faculty support to this effort, we have expanded our intranet, Inside Brown, to include an EBP online support system that links together all aspects of the curriculum, including field. This fall we created two online elements in this system: evolving the ebp process model-december 1, 2008 -16- the brown school • An EBP Literature Retrieval Guide developed by the director of the Brown School Library, in consultation with faculty. This guide serves as a reference that our MSW students can use throughout their studies and instills an understanding of the Brown School Approach that they can use in future practice. The guide: o Illustrates how to formulate questions o Provides definition and core materials to be consulted for finding current evidence o Assists in formulation of search strategies for seeking literature in various databases o Emphasizes the questions related to evaluating current literature o Assists in analyzing articles for applicable tools o Provides suggestions for what to do with evaluations of the applications evolving the ebp process model-december 1, 2008 -17- the brown school • Course-specific EBP resources such as links to key guidelines, data sources and related databases, tests and measures, and other relevant web sites. evolving the ebp process model-december 1, 2008 -18- the brown school Since we launched the first components of our online resource this fall, our online support system has been viewed more than 829 times by students, faculty, and staff; all or portions of the Literature Retrieval Guide have been downloaded 287 times. (Data as of November 15). Over time, this resource will grow to support all courses and to provide support for areas of EBP efforts, including space for on-demand training (video, audio, etc) around EBP. Because of the investment of time and talent in developing this online system, we plan to evaluate its effectiveness as well. Specifically we will: • Incorporate questions about the system in course evaluations to understand how the system is supporting the goals of the classroom. • Hold discussion groups each semester with students, faculty, and relevant staff to understand their perceptions, attitudes, and use of the system, as well as ways in which the resource can be enhanced. Brown School’s Office of Communications, in consultation with Reaffirmation Project Directors, the Brown School’s Library, and Office of Information Technology will structure and oversee the groups. • Complete an analysis of usage statistics each semester to understand the number and type of resources accessed and the number of returning visitors to the site. • Showcase this internal resource to field instructors and other agency partners to begin to understand if and how we might be able to provide this type of online support tool to the field overall. In addition, we will create the Brown FLAIR Blog, to help enhance regular dialog among students, field educators, and Brown School Academic Affairs staff. Managed by Academic Affairs, lead faculty and staff will post regular (at least weekly) postings discussing specific EBP topics, issues, and opportunities. We will promote the blog through regular Brown School communications channels (see next section) as well as online outreach to our audiences. We will also use the blog to reinforce content included in our other communications channels. evolving the ebp process model-december 1, 2008 -19- the brown school Product Development and Dissemination We want to teach EBP effectively in the classroom and field, but also assist in promulgating EBP, specifically the process model, in the community, nation, and world. As a result, we are committed to sharing our learnings, and the tools and resources from our reaffirmation initiative, with other social work educators. Specific products we plan to develop and disseminate include: The Brown School EBP Process Tool Kit. We will create and make available a tool kit for social work schools interested in learning more about and possibly adopting the Brown School Approach to the EBP Process Model. Contents of the tool kit include, but are not limited to: • • • • Digestible overview of our philosophy and approach to EBP Key learnings and success factors Relevant articles including monographs and peer-reviewed journal articles Examples of all tools developed – including course syllabi, classroom activities and assignments, course and field evaluation forms, field education documents, exit survey of graduating students, and pre/post-tests for EBP knowledge • Training resources including power points and audio/video for both the EBP General Introduction Workshop and our New Faculty “Up-to-Speed” EBP Workshop To conserve resources and minimize printing costs, the tool kit will be available both online and via CDROM or jump drive. We will develop and execute a promotional plan to ensure visibility of and access to the tool kit by social work educators. Publish peer-review articles. We will use the data obtained through our knowledge surveys, course and field evaluations, exit surveys and other tools to share the successes and lessons learned from this initiative via journal articles. Specifically, we will submit 2-5 articles for publication by fall 2010. Probable topics: • • • • • “The role of theory in EBP” “Advancing the EBP process in practice methods courses” “The application of the EBP process model in macro social work courses” “Building capacity for EBP in practicum sites” “Supporting EBP through technological adoption of online tools” evolving the ebp process model-december 1, 2008 -20- the brown school Speaking engagements and workshops. Brown School faculty will take the learnings from the development and implementation of its EBP Process Model on the road, showcasing lessons learned at industry conferences such as CSWE’s Annual Program Meeting and SSWR. We will also promote Brown School faculty as being available to present at other schools of social work interested in developing similar approaches to teaching EBP. In addition, we will take advantage of a number of existing Brown School communications tools that target a wide range of social work practitioners, educators, researchers and policy makers, including the following: • Social Impact. Social Impact is our award-winning magazine dedicated to sparking dialogue and debate about issues impacting social work, social policy, and public health. We distribute the publication two times a year to approximately 24,000 individuals, including graduates, field instructors, community partners, policy makers, and funders. The magazine is distributed by mail, but we make key articles available on our web site. • Knowledge Monograph Series. Periodically, we distribute a 6-8 page monograph dedicated to a single issue or specific piece of research. We will leverage this series to showcase the progress, process, and outcomes of our reaffirmation project. The monograph targets educators and is distributed by mail and is available online. • Nexus. Each semester, the Office of Field Education publishes its newsletter, Nexus. We will use this newsletter as a key vehicle for communicating information about our new curriculum, as well as training opportunities and additional resources for field educators. The newsletter is distributed by mail and it is always available online. Field education will supplement this publication with periodic electronic communications to field instructors, to keep news and resource information top-of-mind. evolving the ebp process model-december 1, 2008 -21- the brown school • Impact Bulletin. In January, we will launch an e-newsletter titled Impact Bulletin, which is a bi-monthly supplement to Social Impact. We will showcase news and developments related to our reaffirmation project through this channel. • Media outreach. Where appropriate, Washington University’s media relations team will conduct outreach to target publications geared to social work (e.g. Social Work Today) and/ or higher education (e.g. Chronicle of Higher Education). Also, as appropriate, we will explore ways to pitch this story to more mass media outlets. evolving the ebp process model-december 1, 2008 -22- the brown school Timeline The timeline below outlines at a high level the timeframe for implementing each of the strategies, as well as tactical tools and trainings. We’ve included evaluation activities as well. The colors indicate various stages of activities, specifically planning, piloting, and implementing. 2007-08 2008-09 2009-10 2010-2011 FL SPR SU FL SPR SU FL SPR SU FL SPR SU 07 08 08 08 09 09 09 10 10 10 11 11 Develop EBP Process Introductory workshop X X X X X X X X X X Develop new faculty “Up-to-Speed” workshop X X Objective 1: Build capacity of all Brown School educators Expand and enhance online support tools X X X X X X X X X X X X X X X X Create, launch, and promote blog Conduct pre-training and post-training assessments XX X X X X X X X X X X X X X X Objective 2: Restructure MSW curriculum Revise foundation courses Increase exposure to model for advanced standing students Develop macro practice work group X XX XX X XX XX XX X X X X X X XX XX X XX XX XX X X X X X X X X X X X X X X X X X X X X X X X X X Revise concentration courses Develop EST skills labs X X Conduct content analysis for syllabi Fully integrate field ed X X X Modify course evaluations to get student feedback X X Evaluate and enhance online support tool X X X X X X X X X X X X X X X X X X Collect data from field instructor evaluations X X X X X Assess current students’ knowledge X X X X X X X X Pre-test knowledge of incoming and graduating students evolving the ebp process model-december 1, 2008 -23- X X X the brown school 2007-08 2008-09 2009-10 2010-2011 FL SPR SU FL SPR SU FL SPR SU FL SPR SU 07 08 08 08 09 09 09 10 10 10 11 11 X X Objective 3: Build capacity of practicum sites Develop training module for field instructors Hold day-long training for field instructors X X X Allow field instructors to attend EST skills labs X X X X X X X Offer EST skill labs through Professional Dev. Series X X X X X X X Conduct pre-post training assessments X Modify instrument that students use to evaluate X practicums X X Track number of field instructors who take trainings, labs, etc. X X X X X Develop survey that assesses the implementation of ESTs X X X X X X X X X X X X XX X X X X X X Product Development & Dissemination Develop promotional plan X X Create and package tool kit Promote tool kit availability Submit and publish peer review articles X Create, launch, and promote EBP implementation blog Promote in Social Impact, Knowledge Monographs, and/ or Nexus X X X web, #tool kits distributed, # inquiries, etc) X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Promote in Impact Bulletin Track general dissemination process measures (activity on X XX X evolving the ebp process model-december 1, 2008 -24- the brown school Why now? Why the Brown School? Economic insecurity. Greater cultural diversity. Aging population. Changing demographics. Global climate change. Our society needs well-trained social work graduates with critical problemsolving and analytical skills now more than ever. The School, as part of a larger University-wide planning process, has just completed an ambitious ten-year strategic plan which reaffirms our commitment to not only evolve, but where needed, transform our curriculum to meet the changing demands of this dynamic profession. As a result, this initiative has the full support of the entire Brown School community. Reaffirmation Project Leadership: Advancing the “Brown School Approach” has been and will remain a major aim of our Academic Affairs arm. Tonya Edmond, associate dean for academic affairs and Brett Drake, associate professor, will direct the work of this reaffirmation project, working closely with the curriculum and concentration committees, adjunct faculty, field education and career services that will support and inform this work. Impact 2020 A BLUEPRINT FOR BROWN “(the) systematic approach and use of evidence will be the defining characteristic of our work and specifically of future graduates of both our master’s and doctoral programs.” – Impact 2020, pg. 12, December, 2007 Our faculty is the bedrock of the project and they will continue to be fully engaged in the process. Dr. Brett Drake and Dr. Tonya Edmond, our Reaffirmation Project Co-Directors, have been instrumental in the design of this proposal and will continue to provide leadership in the implementation of this project. A review of the appendices illustrates the depth of faculty engagement across our EBP related committees: EBP Taskforce, EBP Steering Committee, Foundations Work Group, Macro Work Group, and the Curriculum Committee. Collectively the work of these committees has involved the full participation of six full professors, eight associate professors, seven assistant professors, and three associate deans. Virtually twothirds of our faculty has been actively working on advancing our EBP initiatives. Taking full advantage of committee assignments, this project will continue to leverage the administrative responsibilities of participating faculty. These assignments are important and are valued in the annual performance review process and in promotion/tenure applications. It is important to note that our governance structure ensures faculty ownership of this process. Each committee referenced reports either directly to the Curriculum Committee and/or the full faculty for approval of their curricular recommendations. evolving the ebp process model-december 1, 2008 -25- the brown school Our senior leadership will be closely involved in the progress of the project. Faculty meetings will regularly focus on elements of the plan, as well as on the overall progression of activities. Dean Edward F. Lawlor and the School’s associate and assistant deans meet on a biweekly basis. These meetings will serve as forums to discuss the project, to brainstorm challenges, consult on decision points and, most importantly, help ensure resources are consistently available and managed to provide the continued support necessary to advance the work. (See Appendix I for members of EBP Steering Committee) Our National Council and Dean’s Professional Advisory Council will provide meaningful opportunities to obtain input from external constituents to provide insight in developments in the field and objective evaluations from outside stakeholders. (See Appendices J and K) Resource Commitment: EPB is a central theme of our educational focus and the future direction of the School. As a result, our administrative resources are tactically and strategically aligned to provide the necessary capacity to support the program outlined in this proposal. Administrative units, especially the library, information technology, and communications, will provide critical support in advancing EBP and attaining the stated goals of the project Libraries The Washington University Libraries provide a wide range of computerized services and availability of all relevant periodicals to the project. Washington University libraries have been partial government documents depositories since 1912. The government document holdings of Olin Library also include access to a select part of the Government Accounting Office series and statistical information on current populations in the United States. Investigators can also access the Missouri Institute for Mental Health (MIMH) Center for Policy, Research and Training library within the University of Missouri, which is one of the most extensive mental health services research facilities in the mid-west. The Brown School library houses one of the finest collections in the nation for the fields of child welfare, community development, family therapy, mental health, children and youth, gerontology, public welfare, management of human services, and social policy. It maintains a collection of approximately 51,000 books, journals, audio visual materials, and U.S. government documents. The library has more than 450 current periodical subscriptions; over 1,000 bound volumes are added to the collection each year. The library has a comprehensive print and electronic reference collection. It also provides access to a broad collection of online journal indices (including Social Sciences Citation Index, MEDLINE, PsycInfo, and Academic Search Premier), as well as hard copy indices and abstracts. evolving the ebp process model-december 1, 2008 -26- the brown school We have two professional librarians available to assist investigators and students with online database searching, accessing electronic resources, reference queries, searching the Washington University Libraries’ catalog, bibliographies, and any other library need. Bibliographies on the following topics are available at the circulation desk: African Americans; alcoholism; Asian Americans; children/families and youth; family therapy; gay/lesbian issues; gerontology; health; Hispanic Americans; international, social, and economic development; managed care; mental health; minorities; Native Americans; reference; social and economic development; and women. The Bernard Becker Medical Library integrates a modern health sciences library, a media/ learning resources center, a computer teaching and information management laboratory, and a health information network that links regional, national, and international information resources. The 8-level, 114,000 square foot facility houses more than 269,000 volumes and an extensive media collection, and is also one of the most technologically advanced health sciences libraries in the world. Office of Information Technology We maintain a state-of-the-art technological support system for faculty and staff. Our local area network (LAN) consists of a Windows 2003 file server, a Microsoft Exchange 2003 mail server, and a Windows 2003 web server, as well as a connection to the University’s T3 backbone, providing access to the Internet. Available network applications include Telnet, FTP, and Outlook 2007 for electronic communications. Available software includes Windows 2000, Windows XP, Lisrel 8.0, Stata 10, NVivo, SPSS 15, SAS 9.1, Microsoft Office 2007, SNAP 9, and ARCVIEW/GIS 8.0. The Office of Information Technology assists with software licensing, obtaining hardware resources, and technological consulting. Our network firewall enhances data and communication security for the perimeter of our network, and allows administrators to control-- on a very detailed level-- the information and resources accessible to external sources. Washington University, the Brown School, and select research centers and initiatives have websites to assist in raising the visibility of our work. Web site use and development is a shared responsibility between the Office of Information Technology and the Office of Communications. We use Microsoft Office SharePoint Server (MOSS) 2007 as the platform for our web operations, including the management of all of our web sites, including Inside Brown, our intranet which is home to our EBP online support system. evolving the ebp process model-december 1, 2008 -27- the brown school In addition to serving as an efficient method of centrally managing web content, MOSS enables access to organized and aggregated information in one central, web-based application. MOSS 2007 integrates with Microsoft Office applications, offers information search tools, and provides web content management, as well as more specialized document and record management. MOSS also offers Web 2.0 collaboration functionality like blogs, wikis and delivery of information via RSS. MOSS is the platform that we are using for web-based collaboration helping to facilitate interdisciplinary collaborative research among researchers at the Brown School, across campus, and at other institutions. Communications Our Office of Communications provides school-wide support to ensure visibility for all facets of the School’s teaching, research, and service mission. The office is comprised of an assistant dean for communications, a communications coordinator specializing in graphic design, print production, and copy editing; and a web content manager, who oversees content development and editing for our intranet and main external web site, and provides training and consultation to our research centers on the management of their web sites. A half-time media relations specialist works with faculty to promote their research through print, broadcast, and online news outlets. The office works closely with public affairs and media relations representatives from across the University. evolving the ebp process model-december 1, 2008 -28- the brown school Appendix A Evidence-based social work practice: Abstracts of Brown School faculty publications and articles under review 2008 Zayas, L. H., Drake, B., & Jonson-Reid, M. (under review). Consequences of Misapplying Evidence-Based Practice in Social Work: Getting Ensnared in Category Fallacies and Looping Effects. Abstract: Current implementation of the evidence-based practice (EBP) model in social work has focused heavily on the application of evidence from randomized clinical trials (RCT). There has been far less attention to the two other core aspects of EBP: the importance of practitioner judgment and client values. The present paper suggests that a narrow focus on EBP as the promulgation of evidence derived from RCTs can result in category fallacies (i.e., problems defined and treatments created by holders of power) and looping effects (i.e., humans institution transforming conceptual categories into social realities). Reasserting EBP’s focus on clinical judgment and the centrality of clients’ experience will assist the practitioner in recognizing the variety of human problems and suffering. This is especially important in the context of globalization and the diversity of clients we serve. The result should be a much more humble and clear-eyed view of EBP’s utility and a renewed focus on its abundant potential. Drake, B. & Jonson-Reid, M. (2008). Social Work Research Methods: From Conceptualization to Dissemination, Allyn Bacon. This is, to our knowledge, the first research methods textbook to integrate EBP content into individual chapters through use of a set of extensive “EBP Modules”. 2007 Drake, B., Hovmand, P., Jonson-Reid, M & Zayas, L. (2007). Adopting and Teaching Evidencebased practice in Masters Level Social Work Programs. Journal of Social Work Education. 43(3), 431-446. This article makes specific suggestions for teaching evidence-based practice (EBP) in the master’s-in-social-work (MSW) curriculum. The authors use the model of EBP as it was originally conceived: a process for posing empirically answerable questions, finding and evaluating the best available evidence, and applying that evidence in conjunction with client characteristics and practitioner judgment. The authors suggest that EBP, in its original evolving the ebp process model-december 1, 2008 -29- the brown school form, is both sufficiently well operationalized and flexible to serve as a primary paradigmatic component of social work education. Furthermore, EBP carries a series of distinct advantages for MSW education that have not been widely recognized to date. These include the provision of a structure for more explicitly recognizing client factors, bridging the micro-macro and researcher-clinician divides, and emphasizing the professionalism of MSW-level practice. Specific curricular components are proposed and discussed. 2006 Drake, B., Jonson-Reid, M., Hovmand, P. & Zayas, L. (In Progress, January 2006). Correcting the Drift in Evidence-based practice. Abstract: Evidence-Based Practice (EBP) is increasingly discussed as an approach to Social Work Practice (Gambrill, 2001, 2004; Gilgun, 2005; Proctor, 2004, Thyer, 2004). As yet, few writers in the United States have extended this discussion to the multiple dimensions of social work outside clinical practice. This paper began as an attempt to extend thinking about EBP to other aspects of social work-like case management, community development, program and policy. Along the way it became evident that much of the difficulty in applying EBP was due to a common misunderstanding of the original model as presented in Evidence Based Medicine (Gambrill, 2005; Liberati, 2004; Michelson, 2004; Rosenfeld, 2004; Thyer, 2004). We assert that the way to minimize the current problems and maximize the promise of EBP is to re-emphasize Evidence Based Medicine’s (EBM) original focus on the practitioner as a critical consumer and active synthesizer of various forms of evidence in the context of professional experience and client context. Drake, B., Jonson-Reid, M., & Hovmand, P. (In Progress, January 2006) The Pedagogy of Evidence-based practice in Social Work Abstract: This article is paired with the preceding article “Correcting the Drift in Evidencebased practice (Jonson-Reid, Drake, Hovmand, Zayas, In Progress) and should follow that article. This article proposes a specific format for teaching EBP in Master’s level social work education. Edmond, T., Megivern, D., Williams, C., Rochman, E. & Howard, M. (2006). Integrating evidence-based practice and social work field education. Journal of Social Work Education, 42, 2 (Spring/Summer), 371-390. Abstract: Field instructors are a vital part of practicum education for the social work profession. This paper reports the results of a survey that was conducted to deepen our understanding of the views and attitudes that field instructors have toward evidence-based practice, and the degree to which they appear to currently be in use within practicum sites. A evolving the ebp process model-december 1, 2008 -30- the brown school 49% response rate was achieved based on completed questionnaires from 283 field instructors within 180 agencies. Most field instructors view EBP as a useful practice idea; a smaller number actually implements these methods in their practice with any regularity. Pollio, D. The Art of Evidence-Based Practice. Research on Social Work Practice, Vol. 16 No. 2, March 2006 1-9. Abstract: The purpose of this article is to discuss evidence-based practice (EBP) from the perspective of a self-identified evidence-based practitioner. Discussion of EBP includes choosing an initial intervention and evaluation procedures, the iterative process of rechoosing and refining an intervention over the treatment lifespan, the importance of evidence within the specific clinical situation. Two illustrative case studies are presented. Practice principles include (a) explaining EBP clearly, including an ability to deconstruct key elements; (b) creating an evaluation that yields useful outcome data for practitioner and client and is realistic given the characteristics of the client system; (c) refining intervention and evaluation efforts, based on increased knowledge of the client system and as their willingness to participate changes; and (d) understanding relevant evidence about specific techniques, incorporating evidence developed as part of the intervention, and being critical consumers of both types of evidence in specific situations with clients and client systems. 2005 McMillen, J.C., Proctor, E.K., Megivern, D., Striley, C.W., Cabassa, L.J., Munson, M.R., Dickey, B. (2005) Quality of Care in the Social Services: Research Agenda and Methods. Social Work Research, 29-3, 181-91. Abstract: In an era of heightened accountability, remarkably little is known empirically about the quality of social work services. This article applies insights from health services research to propose a research agenda on the quality of care in the social services. The agenda calls for studies that address the definition of quality service, variations in quality, the relationship between quality service and outcomes, structural influences on quality, and ways to improve quality. The article also details specialized research methods for implementing this agenda, including the use of administrative data, risk-adjusted outcomes, case vignettes, standardized consumers, and stakeholder preference assessments. Although social work is currently underrepresented in quality research, social workers’ research skills, their traditional academicagency partnerships, and their accent on consumer experiences position the profession to make quick strides in developing the information needed for quality improvement efforts. evolving the ebp process model-december 1, 2008 -31- the brown school Proctor, E.K. (2005). Social work practice research: A question of questions. Paper presented at the annual program meeting, National Association of Deans and Directors, September 2005, San Antonio, TX and at the International Practice Research Symposium, The University at Albany, June 2, 2005. Proceedings under review by Columbia University Press. Abstract: The paper explores the knowledge needs of social work as a profession, asserts the primacy of five research questions whose pursuit can inform and improve the delivery of social work services and demonstrate the profession’s social value. Those questions are: (1) what are the practices in social work practice? (2) How does social work practice vary? (3) What is the value of social work practice? (4) What practices should social workers use? And (5) how can social work practice be improved? The pursuit of these questions in social work research is encouraged as a means to strengthen the social work knowledge base. 2004 McMillen, J.C., Morris, L., Sherraden, M. (2004). Ending Social Work’s Grudge Match: Problems versus Strengths. Families in Society, 85, 317-325. Abstract: Some in social work have called for a paradigm shift away from a focus on problems to a focus on strengths, empowerment, and capacity building. This call sets up an unnatural dichotomy, asking social workers to identify with one side or another. In this article, we review social work history to argue that the best social work practice has always maintained a dual focus on both problems and capacity building. Throughout our history, those who championed a problem-oriented practice also emphasized strengths and growing client capacity, and today’s strength-based, capacity-oriented practitioners typically advocate for the solving of consumer’s presenting problems. Proctor, E.K. (2004). Leverage points for the implementation of evidence-based practice. Brief Treatment and Crisis Intervention 4(3), 227-242. Abstract: Adoption of evidence-based practice (EBP) is an increasingly advocated yet formidable challenge. Much work on EBP has implied simplistic solutions: if researchers would produce practice-relevant evidence-based approaches, practitioners would find, adopt, and use them. Blaming researchers for problems in supply, and practitioners for resistance in adoption, will only thwart progress at improving the quality of service. The dissemination and implementation of evidence-based practice requires a more discerning analysis of issues in agency, research, and professional cultures. Drawing on literature on knowledge diffusion, innovation, and quality improvement, this paper proposes a conceptual framework for the multiple tasks, participants, and leverage points required for the adoption of EBP. evolving the ebp process model-december 1, 2008 -32- the brown school Evidence-based practice requires attainment of four intermediate outcomes—access, adoption, implementation, and assessment—each with distinct interventions required for attainment. The framework reveals action points and leverage points for researchers, agency administrators, educators, and individual practitioners. Implementation of EBP requires supportive research, training, and organizational infrastructures. Proctor, E., & Rosen, A. (2004). Using empirically supported treatments in practice. In A. Robert and K. Yeaker (Eds.), Evidence-based practice manual: Research and outcome measures in health and human services (pp. 193-199). Oxford University Press. Abstract: Recognizing the many challenges inherent in the application of evidence-based practices in day to day practice, this chapter addresses four types of actions and decisions required by practitioners: (1) identify potential empirically supported treatments (ESTs) that are relevant to the outcomes for pursuit; (2) select the best-fitting intervention (EST) in view of the client problems, situation, preferences, and outcomes; (3) supplement and modify the EST as needed, drawing on practitioner experience and knowledge; and (4) monitor and evaluate intervention effectiveness. 2003 Howard, M., Bricout, J., Edmond, T., Elze, D., & Jenson, J. (2003). Evidence-based practice guidelines. Encyclopedia of Social Work, 19th Ed. (2003 Suppl.), Washington, DC: National Association of Social Workers Press. Abstract: Evidence-based guidelines have proliferated widely over the past decade. This review examines the nature and methods of current guideline development activities in medicine and allied health professions, factors contributing to the rapid growth of guidelines, desirable attributes of practice guidelines, guideline effectiveness, dissemination and implementation issues, strengths and limitations of guidelines, and barriers to guideline development in social work. Howard, M.O., McMillen, J.C., & Pollio, D.E. (2003). Teaching evidence-based practice: Toward a new paradigm for social work education. Journal of Research on Social Work Practice, 13, 234-259. Abstract: The scientific literature relevant to social work practice has grown expansively in recent years. Corollary developments, including the widespread availability of electronic bibliographic databases, improved indexing services, and increased acceptance of systematic reviews and evidence-based practice guidelines, have made research findings increasingly accessible to practitioners. For the first time in the history of the profession, social work educators are confronted with the challenges posed, and opportunities afforded, by this evolving the ebp process model-december 1, 2008 -33- the brown school accumulating body of practice-relevant scientific information. Evidence-based practice is a new paradigm that promotes more effective social interventions by encouraging the conscientious, judicious, and explicit use of the best available scientific evidence in professional decision making. Pedagogically, evidence-based practice involves teaching students the values and skills they need to identify, critically appraise, and apply practice-relevant scientific evidence over the course of their professional careers. This article describes the potential benefits of evidencebased social work professional education and ongoing efforts of the George Warren Brown School of Social Work at Washington University to implement curriculum-wide changes supportive of evidence-based professional practice education. Proctor, E.K. (2003). Developing knowledge for practice: Working through “trench-bench” partnerships (Editorial). Social Work Research, 27(2), 67-69. Abstract: In many areas of biological and social sciences and in medicine, knowledge development has traditionally developed from “bench to trench.” The academic setting has been the “bench,” where hypotheses and research questions are formulated, rooted in theory and prior published research. Practice or policy has been the “trench,” to which knowledge dissemination is directed through “last” but all-too-often neglected, steps in the research process. The sufficiency of this approach for social work is challenged by persistent evidence that research findings have very little “uptake” by practice, and that new practices are rarely sustained in agency after an intervention trial or prevention program ends. Academic practice partnerships need to be cultivated, respected, and nurtured from the outset of the research process. Rosen, A., & Proctor, E.K. (Eds.). (2003) Developing practice guidelines for social work interventions: Issues, methods, and research agenda. New York: Columbia University Press. Abstract: Addressing the ultimate challenge of increasing the effectiveness of social work practice, this book asserts that as a publicly sanctioned profession, social work has responsibility to develop, elucidate, and organize in a manner accessible to practitioners its practice knowledge. Practice guidelines constitute a set of systematically compiled and organized knowledge statements designed to enable practitioners to find, select, and use those interventions that are most effective and appropriate for a given practice challenge. The content and structure of guidelines should reflect the uncertainty and tentativeness inherent in a science-based profession. The book addresses the infrastructure and research challenges of developing empirically based practice guidelines for social work. Contents include: “Practice Guidelines and the Challenge of Effective Practice,” by Aaron Rosen and Enola K. Proctor; “Precursors of Guidelines: Intervention Research and Evidence-Based Practice,” “Intervention Research in Social Work: A Basis for Evidence-Based Practice and Practice Guidelines,” by Mark W. Fraser; “Evidence-Based Practice: Implications for Knowledge Development and Use evolving the ebp process model-december 1, 2008 -34- the brown school in Social Work,” by Eileen Gambrill; “Empirical Foundation for Practice Guidelines in Current Social Work Knowledge,” by William J. Reid and Anne E. Fortune; “Practice Guidelines for Social Work: Need, Nature, and Challenges;” “Clinical Guidelines and Evidence-Based Practice in Medicine, Psychology, and Allied Professions,” by Matthew Owen Howard and Jeffrey M. Jenson; “The Structure and Function of Social Work Practice Guidelines,” by Enola K. Proctor and Aaron Rosen; “Social Work Should Help Develop Interdisciplinary Evidence-Based Practice Guidelines, Not Discipline-Specific Ones,” by Bruce A. Thyer; “The Role of Diagnostic and Problem Classification in Formulating Target-Based Practice Guidelines,” by Stuart A. Kirk; “Constructing Practice: Diagnosis, Problems, Targets, or Transactions?,” by Mark A. Mattaini; “Responsiveness of Practice Guidelines to Diversity in Client Populations and Practice Settings: The Idiographic Application of Normative Generalizations;” “Accounting for Variability in Client, Population, and Setting Characteristics: Moderators of Intervention Effectiveness,” by Lynn Videka; “Service-Delivery Factors in the Development of Practice Guidelines,” by Luis H. Zayas; “Performance Standards and Quality Control: Application of Practice Guidelines to Service Delivery,” by Wilma Peebles-Wilkins and Maryann Amodeo; “Practitioner, Organizational, and Institutional Factors in the Utilization of Practice Guidelines;” “Practitioner Adoption and Implementation of Practice Guidelines and Issues of Quality Control,” by Edward J. Mullen and William F. Bacon; “Organizational and Institutional Factors in the Development of Practice Knowledge and Practice Guidelines in Social Work,” by Jeanne C. Marsh; “Social Work Practice Guidelines in an Interprofessional World: Honoring New Ties That Bind,” by Nina L. Aronoff and Darlyne Bailey; and “Advancing the Development of Social Work Practice Guidelines: Directions for Research,” by Enola K. Proctor and Aaron Rosen. Rosen. A., Proctor, E.K., & Staudt, M. (2003). Targets of change and intervention in social work: Empirically-based prototype for developing practice guidelines. Research on Social Work Practice, 13(2), 208-233. Abstract: Studies evaluating interventions were used as data to illustrate a rudimentary prototype of practice guidelines comprising two components: taxonomy of outcome targets, and their associated array of intervention. We use the data to discuss the adequacy of intervention research in social work for contributing to practice guidelines. Data were all the outcomes and their associated interventions investigated and reported in 13 social work journals over a four and a half year period, totaling 126 studies. Results: The outcomes and interventions were classified according to eight outcome target domains, yielding the practice guidelines prototype. The research was unevenly distributed across the eight target domains, and a pattern of ‘single outcome-single intervention’ design was evident across all target domains. Data suggest insufficient attention to comparative evaluation of interventions and to assessing effectiveness across client populations and service settings, and need for betterdesigned and prioritized intervention research overall. evolving the ebp process model-december 1, 2008 -35- the brown school Zayas, L., Gonazalez, M., & Hanson, M. (2003). What do I do now? On teaching evidencebased interventions in social work practice. Journal of Teaching in Social Work, 23 (3/4). Abstract: Recent advances made by medicine and psychiatry in the development of practice guidelines and evidence-based interventions are spreading to social work and other helping professions. As the movement toward evidence-based practice grows, more demands for the use of tested interventions will be placed on practitioners. In this paper, we argue that social work practice education must change to respond to this demand. Specifically, evidencebased interventions must be incorporated into the teaching of the middle phase of social work practice more than it is now. Social work training has traditionally relied on untested interventions that have come from the tacit knowledge of accumulated practice wisdom. Although tacit knowledge has an important place in social work practice, instructing students about evidence-based, empirically derived interventions provides them with critical knowledge and skills they will need to enter the competitive world of social work practice. Moreover, this approach amplifies the importance of launching practitioners with skills of evidencebased thinking; that is, the identification, evaluation, and selection of interventions with demonstrated effectiveness. 2002 Pollio D.E. (2002) The evidence-based group worker. Social Work with Groups, 25(4), 57-70. Abstract: This essay reflects on recent practice and educational trends contributing to the current impetus for consistent incorporation of evidence into practice. Evidence-based group work is defined as the conscientious and judicious use of evidence in current best practice. In operationalizing this definition, the evidence-based group worker (a) appraises critically systematically-collected evidence from all sources; (b) acquires knowledge through reading professional journals, attending conferences and presentations, and writing/presenting knowledge that he/she develops; (c) evaluate the outcomes of practice, using consistent methods; (d) implements models, either existing or developed by the practitioner, consistently and rationally; (e) attends to the impact of individual differences in making evidence-based practice decisions; and (f) incorporates evidence in understanding group process, leadership, and development. These principles lead to four behaviors associated with being an evidencebased group worker: (1) willingness to communicate evidence behind treatment decisions and techniques chosen with group members; (2) building time to learn into practice; (3) practicing from a developmental perspective, building knowledge through incorporating outside learning and results of his/her own practice evaluations; (4) adapting existing evidence to each practice situation. Implications for teaching group work are discussed. evolving the ebp process model-december 1, 2008 -36- the brown school Rosen, A., & Proctor, E. (2002). Standards for evidence-based social work practice: The role of replicable and appropriate interventions, outcomes and practice guidelines, In A.R. Roberts and G.J. Greene (Eds.), Social workers’ desk reference (SWDR) (pp. 743-747). Oxford University Press. Abstract: The fundamental issue addressed by this chapter is how practitioners can best use in practice the products of research in order to ensure effective practice. Social work has long recognized the scientific method as the preferred basis for its knowledge and the source of criteria for assessing the effectiveness of its practice. The essence of the scientific method as applied to practice is that the appropriateness of interventions be ascertained on the basis of evidence as to their effectiveness. Such evidence must consist of empirically demonstrated linkages between interventions and the outcomes toward which they are directed. 2001 and earlier Rosen. A., Proctor. E.K., & Staudt, M. (1999). Social work research and the quest for effective practice. Social Work Research 23(1), 4-14. Abstract: This article reports a study assessing the contribution of current social work journal publications to a cumulative body of research that is capable of informing interventions. Thirteen journals published from 1993 to mid-1997 were reviewed. Analysis was guided by the view that the focus and design of studies should correspond to and supply the knowledge needed to perform different practice tasks. Accordingly, journal research articles were classified by their potential contribution to practice decisions requiring descriptive, explanatory, or control knowledge. Research aiming at control knowledge, informing the selection and implementation of interventions, was further evaluated by whether the interventions and outcomes studied were sufficiently specified to permit reliable research replication and application in practice. Very few of the published articles were found to report research on interventions, and even fewer used designs that enable replication. Factors affecting the rate and quality of research and interventions are also discussed. Lawlor, E.F., & Raube, K. (1995). Social interventions and outcomes in medical effectiveness research. Social Service Review, 69(3): 383-404. Abstract:. The article focuses on the principles and approaches characteristic of medical effectiveness research and the treatment of social factors in the current generation of studies. A revolution in the design, administration, organization, and financing of health services is underway. The “medical outcomes movement,” a national program of research, practice guideline development, and administration, is radically reshaping the way health services are understood, justified, and delivered. The development of practice guidelines--standardized evolving the ebp process model-december 1, 2008 -37- the brown school protocols that indicate the appropriate practice given a set of clinical indications--will improve quality of care, patient satisfaction, and clinical outcomes and ultimately reduce costs. In its early stages, this movement focused on conventional and often high technology medical interventions where there was evidence of high degrees of physician discretion and variability in decision making. Now, research and guideline development are shifting to more intransigent health problems that are not purely medical in nature. Rosen. A.., Proctor E.K., & Staudt, M. (1995). Rationales for practice decisions: Variations in knowledge use by decision task and social work service. Research on Social Work Practice, 54(4), 501-523. Abstract: This article addresses social workers’ use of knowledge by studying the rationale they provided for their actual practice decisions. Extent of knowledge use was compared across four practice decision tasks and between medical and psychiatric social work services. Data were obtained from the records of 297 clients treated by 34 experienced M.S.W. social workers in medical and psychiatric services units of two mid-western hospitals. Workers were previously trained in the approach and procedures of Systematic Planned Practice (SPP). Data were obtained from SPP forms on which workers recorded their treatment decisions and rationale for each decision. Data analyzed by repeated measures (MANOVA and ANOVA’s) revealed that giving of rationales, and the types of knowledge it reflected, varied significantly by the decision task and social work service. Intervention decisions were least likely to be supported by rationales, and medical services workers provided fewer rationales than psychiatric services workers. Conceptual rationales were the most frequently used, whereas rationales based on practice experience, values, research, or client wish were very little used The implications for practice, professional education, quality assurance, and for further research are discussed. evolving the ebp process model-december 1, 2008 -38- the brown school Appendix B Members of Brown School EBP Task Force Members – 2005-06 Enola Proctor (Chair), Professor & Associate Dean for Research Tonya Edmond, Associate Professor Peter Hovmand, Assistant Professor Melissa Jonson-Reid, Associate Professor Amanda Moore McBride, Assistant Professor David Pollio, Associate Professor Nancy Vosler, Associate Professor Gautam Yadama, Associate Professor James Herbert Williams, Associate Professor & Associate Dean for Academic Affairs evolving the ebp process model-december 1, 2008 -39- the brown school Appendix C References Bloom, B. (1984). Taxonomy of Educational Objectives, Boston, Allyn Bacon. Drake, B., Hovmand, P., Jonson-Reid, M., & Zayas, L. (2007). Adopting and teaching evidencebased practice in master’s level social work programs. Journal of Social Work Education, 43(3), 405-428 Evidence Based Medicine Working Group (1992). A new approach to teaching the practice of medicine. JAMA, 268, 2420-2425 Gambrill, E. (1997). Social work practice: A critical thinker’s guide. New York: Oxford University Press Gambrill, E. (1999). Evidence-based practice: An alternative to authority-based practice. Families in Society, 80, 341-350. Gibbs, L. & Gambrill, E. (2002). Evidence-Based Practice: Counter-arguments to objections. Research on Social Work Practice, 12(3), 452-456. Rubin, A. (2006). Highlights of Symposium Papers. National Symposium on Improving the Teaching of Evidence-based practice, UT Austin on October 16th-18th, 2006 (http://www. utexas.edu/ssw/ceu/practice/papers/EBPRubinHighlights.pdf) Rubin, A. & Parrish, D. (2007). Challenges to the future of evidence-based practice in social work. Journal of Social Work Education, 43(3), 405-428 Sackett, D., Richardson, S., Rosenberg, W., Haynes, B. (1996). Evidence-Based Medicine: How to practice and teach EBM David L Sackett, W Scott Richardson, William Rosenberg, R Brian Haynes. Edinburgh: Churchill Livingstone. 1996. 250pp. ISBN 0-443-05686-2.) evolving the ebp process model-december 1, 2008 -40- the brown school Appendix D Members of Brown School Macro Work Group Melissa Jonson-Reid (Chair), Associate Professor Stephanie Boddie (2007-08), Assistant Professor Yunju Nam, Assistant Professor Shanta Pandey, Associate Professor Ramesh Raghavan, Assistant Professor Paul Shattuck, Assistant Professor evolving the ebp process model-december 1, 2008 -41- the brown school Appendix E Parish/Rubin Instrument Evidence-Based Practice Knowledge Questions Please read each statement and circle the letter next to the response that best represents your answer. ID#___________________ POSTTEST **Please do not complete until after the training. The posttest is comprised of two sections: 1) The EBP process scale and 2) the EBP knowledge questions. Both sections are included in this handout. Please complete both sections. Thank you! EBP PROCESS ASSESSMENT SCALE Purpose: The purpose of this scale is to assess your views about the Evidence-Based Practice (EBP) process. Definition: The EBP process includes considering the best research evidence available as part of the basis for making practice decisions. It does NOT mean just providing an evidence-based treatment; rather it means engaging in each of the following five steps in your practice: 1) Formulating a practice question that can be answered by searching for research evidence; 2) Tracking down the best research evidence to answer the question, 3) Critically appraising the evidence, 4) Integrating the critical appraisal with practitioner expertise and client attributes to guide your practice decision, and 5) Evaluating the outcomes of the practice decision. Instructions: The scale contains five sections. For the first three sections (I-III), please circle the response to the right that best fits how much you agree or disagree with statements regarding the EBP process. For the final section (IV), please circle the response that best fits the frequency with which you intend to and currently engage in the EBP process. evolving the ebp process model-december 1, 2008 -42- the brown school EBP is a relatively new concept. Therefore, like many other practitioners, you may know little about it. Nevertheless, please answer all items, even if you are unsure of your answer or have no opinion. Please circle N (Neutral) for every item for which you are neutral, uncertain, or feel that you don’t know enough about EBP to respond in an informed manner. All responses are anonymous; please answer each item according to how you really view the EBP process and its feasibility in your practice. Thank you! Section I. Familiarity with the Evidence-Based Practice (EBP) Process Level of Agreement Statement I know how to skillfully apply the steps of the EBP process. I understand how to formulate questions about practice that can be answered with research evidence. I feel confident in my ability to find the best research evidence to guide my practice decisions. I know how to find systematic reviews. I understand how to appraise the research evidence pertaining to my practice question. I can differentiate between very weak evidence and very strong evidence. I know what factors to consider in addition to the research evidence when making practice decisions. I understand how to evaluate the outcomes of my practice decisions. I understand what is meant by the term research-based practice guidelines. I know how to use the internet to facilitate my search for research evidence. Strongly Disagree Disagree Neutral Agree Strongly Agree SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA Please proceed to the next page. evolving the ebp process model-december 1, 2008 -43- the brown school Section II. Attitudes About the Evidence-Based Practice (EBP) Process Level of Agreement Statement Strongly Disagree Disagree Neutral Agree Strongly Agree EBP is nothing more than a way to cut treatment costs. SD D N A SA EBP helps improve clients’ outcomes. SD D N A SA Engaging in EBP hinders the use of practitioner judgment. SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA Practitioners who engage in the EBP process show greater concern for client well being than practitioners who do not engage in EBP. Engaging in the EBP process makes practice too mechanistic. The EBP process allows enough room for considering unique client circumstances or preferences. The judgment of esteemed colleagues or supervisors offers a better basis than research evidence for improving practice effectiveness. EBP helps clients meet their goals. Engaging in the EBP process hinders the practitioner-client relationship. Trying to engage in EBP is more ethical than refusing to engage in it. I know what is best for my clients without examining the research evidence. Experienced practitioners should disregard research evidence when it conflicts with their intuition. Engaging in the EBP process will improve one’s practice. Engaging in the EBP process means using interventions that won’t apply to the kinds of clients I see. Please proceed to the next page. evolving the ebp process model-december 1, 2008 -44- the brown school Section III. Feasibility for You to Engage in the Evidence-Based Practice (EBP) Process Level of Agreement Statement Strongly Disagree Disagree Neutral Agree Strongly Agree I have enough time to engage in the EBP process. SD D N A SA I wish I had more time to engage in the EBP process. SD D N A SA I have enough access to the research literature to engage in EBP. SD D N A SA SD D N A SA SD D N A SA SD D N A SA SD D N A SA I am too busy to think about incorporating anything new into my practice. It is too much to learn more than one kind of intervention. I have enough time to evaluate the outcomes of my practice decisions. The constraints of my practice setting preclude me from engaging in the EBP process. Please proceed to the next page. evolving the ebp process model-december 1, 2008 -45- the brown school Section IV. Intentions to Engage in the Evidence-Based Practice (EBP) Process Instructions: For this section, please circle the number to the right of each item that best describes how often you intend to engage in the specified behavior. Frequency Behavior I intend to use the internet to search for the best research evidence to guide my practice decisions. I intend to read about research evidence to guide my practice decisions. I intend to read research-based practice guidelines to guide my practice decisions. I intend to rely on research evidence as the best guide for making practice decisions. I intend to rely on the practice wisdom of esteemed colleagues or supervisors as the best guide for making practice decisions. I intend to inform clients of the degree of research evidence supporting alternative intervention options. I intend to involve clients in deciding whether they will receive an intervention supported by the research evidence. Never Rarely Some of the time Often Very Often 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 I intend to stick to providing the interventions with which I am more comfortable, even if research shows others to be more effective. I intend to evaluate the outcomes of my practice decisions. 1 2 3 4 5 I intend to engage in all steps of the EBP process 1 2 3 4 5 Please proceed to the next page. evolving the ebp process model-december 1, 2008 -46- the brown school Evidence-Based Practice Knowledge Questions Please read each statement and circle the letter next to the response that best represents your answer. 1. Practitioners who engage in evidence-based practice will: a. rely most heavily on the practice wisdom of supervisors and experts. b. scientifically monitor client outcomes. c. utilize one major theoretical framework and intervention strategy with most clients. d. only select interventions that are on the list of empirically supported treatments. e. Only b and d 2. The best scientific evidence might: a. be only good enough to guide practice in a tentative manner. b. not be relevant to a client. c. indicate what not to do. d. Only a and b e. All of the above 3. Which of the following statements is/are true about applying the intervention supported by the best research evidence? a. Because it is the most effective, you should implement this treatment the best that you can, even if you cannot get training or supervision. b. Because it is the most effective, you should make the clinical decision to select this intervention for your client. c. If you decide to implement this intervention, you should monitor client progress with it. d. All of the above. 4. In appraising the evidence to be strong in an outcome evaluation of the effectiveness of an intervention, it is important to consider: a. how the outcome was measured. b. whether a control group was used c. whether participants across different treatment conditions are really comparable to begin with d. All of the above Please proceed to the next page. evolving the ebp process model-december 1, 2008 -47- the brown school 5. Which of the following statements is/are true about searching for and critically appraising evidence? a. You should critically appraise all research articles that relate in some way to your EBP question. b. It is okay to use keyword terms that limit the kinds of studies that come up to those that are likely to be most useful to you. c. It is okay to just read the study abstract as a basis for deciding whether the study is sufficiently relevant to merit reading in its entirety. d. Only b and c 6. A good systematic review: a. fully discloses its search and inclusion strategy. b. assesses the quality of selected outcome studies. c. is done by someone within the field who has a vested interest in the findings of the review. d. All of the above e. Only a and b 7. Which hierarchy of evidence below is the most accurate order from highest to lowest for locating an effective intervention (assuming that all studies are implemented well with good measurement and are unbiased)? a. Experimental studies, quasi-experimental studies, meta-analyses, qualitative studies. b. Quasi-experimental studies, meta-analyses, qualitative studies, experimental studies. c. Meta-analyses, quasi-experimental studies, experimental studies, qualitative studies. d. Meta-analyses, experimental studies, quasi-experimental studies, qualitative studies. Please proceed to the next page. evolving the ebp process model-december 1, 2008 -48- the brown school True/False Items: 8. You should avoid using an asterisk (*) in your search term because it will make the search too narrow. T F 9. If a study has been published and it appears in a reputable literature base, you can be sure it has been conducted objectively and rigorously. T F 10.When measuring client outcomes, it might be okay to develop your own individualized rating scale instead of using an instrument that researchers have validated. T F 11.Evidence-based practice questions are restricted to asking about the effectiveness of interventions, programs or policies. T F Thank you for your time. Please place this pretest in the anonymous drop box. evolving the ebp process model-december 1, 2008 -49- the brown school Appendix F Members of Brown School Curriculum Committee Tonya Edmond, Chair, Associate Professor & Associate Dean for the MSW Program Peter Hovmand (Foundations), Assistant Professor Melissa Jonson-Reid (CYF), Associate Professor Carolyn Lesorogol (SED), Assistant Professor Timothy McBride (Health), Professor & Associate Dean for Public Health Curtis McMillen (Mental Health), Professor Nancy Morrow-Howell (Gerontology), Ralph and Muriel Pumphrey Professor of Social Work, ex officio Diane Beckerle O’Brien (Gerontology & Field Education), Associate Director of Field Education Estelle Rochman, Director, Office of Student Affairs Minchao Jin, MSW Student Representative Kay McChesney, MSW Student Representative evolving the ebp process model-december 1, 2008 -50- the brown school Appendix G Members of Brown School Foundations Work Group Brett Drake (Chair), Associate Professor & Reaffirmation Project Director Lorien Carter, Lecturer Tonya Edmond, Associate Professor & Associate Dean for Academic Affairs Peter Hovmand, Assistant Professor Barbara Levin, Program Coordinator, Alliance for Building Capacity Amanda McBride, Assistant Professor Yunju Nam, Assistant Professor Sylvia Toombs, Library Director Cynthia Williams, Director Luis Zayas, Shanti K. Khinduka Distinguished Professor of Social Work evolving the ebp process model-december 1, 2008 -51- the brown school Appendix H Sample Syllabus from a foundation course GEORGE WARREN BROWN SCHOOL OF SOCIAL WORK WASHINGTON UNIVERSITY Fall 2008 RESEARCH METHODS FOR SOCIAL WORK PRACTICE S15-5005 CREDIT HOURS: 3 INSTRUCTOR: GRADE: L/G OFFICE: ROOM: OFFICE HOURS: TIME: 3 HOURS RECOMMENDED PHONE: E-MAIL: NOTE: All foundation courses are being revised this semester as part of a school-wide effort to better incorporate Evidence-based practice. Due to this, all foundation (S15-XXXX) syllabi are subject to revision. I. COURSE DOMAIN AND BOUNDARIES This course has three main goals: 1. Students will understand the basics of social research. Core research concepts are presented including specification, design, measurement, qualitative and quantitative research, clinical research (multisubject and single subject) use of computers in data retrieval, data management and data analysis, research implementation, interpretation and dissemination. 2. Students will understand how research is used in evidence-based practice. This includes the ability to frame empirically answerable questions, locate data relevant to those questions, critically evaluate such data and apply it to practice situations. 3. Students will develop a deeper understanding of the research process through designing and executing a research project. “Tell me, I’ll forget. Show me, I’ll remember. Involve me, I’ll understand.” – Chinese Proverb evolving the ebp process model-december 1, 2008 -52- the brown school As core components of the course, values are explored and ethics are taught as they apply to research, and the intertwined nature of diversity, social and economic justice, and research is presented and explored. II. COURSE COMPETENCIES Competencies required in all foundation courses 1. Know and apply appropriate ethical codes (e.g. NASW, etc…) and laws (e.g. mandated reporting) 2. Work through ambiguity in resolving ethical conflicts. 3. Recognize and manage personal biases and values so that professional values guide practice, including issues of diversity 4. Apply ethical reasoning to reach decisions 5. Recognize global, social, economic, political and cultural factors which result in oppression or advantage. 6. Understand how age, class, color, culture, disability, ethnicity, gender, gender identity and expression, immigration status, political ideology, race, religion, sex, and sexual orientation shape clients experiences and affects practice with those clients. 7. Can frame empirically answerable questions from needs/problems encountered in practice. 8. Can locate the best available empirical evidence of any type 9. Can critically evaluate empirical evidence 10.Can integrate and apply best available evidence with professional judgment, involved persons, and social context. Additional competencies taught in Research Methods: 11.Can critically analyze and select and implement practice models, including evidence supported practices. 12.Can generate empirical data from own practice and evaluate own practice. 13.Can specify, design, execute, analyze and communicate empirical research at multiple levels. (contribution to EBP data) 14.Can use practice experience to inform scientific inquiry 15.Can assess client systems, including strengths, and limitations 16.Can develop agreed upon intervention plans and goals. 17.Engages in practice which is as transparent as possible given the situation evolving the ebp process model-december 1, 2008 -53- the brown school III.COMMON PEDAGOGICAL ELEMENTS Pedagogical elements common to all sections of this course include: 1. This course includes in-class demonstrations of computerized information retrieval systems. These are shown at general (e.g. Wikipedia and/or Google), scientific (e.g. Google Scholar, PsychInfo), and specialty (e.g. Cochrane Collaboration, NGO) levels. Students will complete assignments demonstrating mastery of the above kinds of search engines. 2. Instructors will demonstrate critical evaluation of existing research which is relevant to social work practice. Students will complete assignments demonstrating their ability to criticize existing research at a basic level. 3. Students will complete a research project (possibly in groups) demonstrating their ability to formulate a question, choose and execute a design, implement the research, analyze and interpret resultant data, and create a product in a format fit for dissemination (e.g. PowerPoint, poster, paper, agency presentation, etc…) 4. Instructors will demonstrate/model the EBP process, showing how specific field-generated questions can be (1) formulated and operationalized, (2) how best available evidence can be located, (3) critically evaluated, and applied to practice. Main emphasis will be on steps 1 -3. 5. Instructors will provide students with examples of critical ethical failures in the history of social research. Students will demonstrate compliance with ethical standards in their assignments. IV. TEXT/REQUIRED READINGS Drake & Jonson-Reid (2008). How to do Social Research: From Conceptualization to Dissemination. Drake & Jonson-Reid (2008). Student Workbook for How to do Social Research: From Conceptualization to Dissemination. V. ORGANIZATION OF THE COURSE Course content will be covered through readings, individual and group exercises, lectures, and discussions. Handouts will be provided in class where appropriate. VI.ROLE OF INSTRUCTOR AND STUDENTS Course Expectations: The instructor will: prepare and deliver course material; be available to students during office hours, after class, and by appointment for consultation; and provide timely and clearly explained feedback on student performance. The instructor expects students to: attend each class on time; complete all assignments in a timely manner; come to class evolving the ebp process model-december 1, 2008 -54- the brown school prepared, having read all assignments; participate in class discussions; seek any necessary clarification regarding course expectations from the instructor; and provide the instructor with feedback about the effectiveness of the course. Any problems with attendance, meeting deadlines, or completing assignments should be discussed promptly with the instructor. Special Needs: Students who have a learning disability, sensory or physical disability or impairment, or English as a second language and need special assistance in lecture, reading assignments, and/or testing should contact the instructor as soon as possible. VII.ASSIGNMENTS AND GRADING CRITERIA Weekly assignments (Weeks 2 through 7): 36 Points (6 Points each) Class participation 32 Points Final group product 32 Points Grading Scale: 92-100:A, 90-91: A-, 88-89: B-, 82-87: B, 80-81: BIf your grade is lower than 80, we’ll have to talk and figure out what to do. I doubt this will happen. I do not anticipate grading the group project harshly, so don’t be too worried about being “dragged down” by your group. VIII. COURSE OUTLINE WEEK 1: “Hello” and Brief Introduction to Evidence Sources Overview of course, introductions, familiarization with each other’s general areas of interest (2 hrs). In-class demo on evidence/literature retrieval, includes introductions to Wikipedia, Google, Google Scholar, PsychInfo and Guideline.gov (1 hr). This in-class literature retrieval demonstration is preliminary, and is meant to provide students with an initial sense of the various kinds of data available. As a final exercise, we will review an article I supply, and I will get your views on if this is a good or a poor article to use as a basis for practice WEEK 2: Values and Ethics, Defining Your Area of Research Interest Detailed description of research values and ethics. Students are expected to begin to specify general areas in which they might have questions or research interests. Students should pay particular attention to the EBP module in Chapter 1 (pages 15-20), as we will focus on how the skills learned in this class will be part of students’ everyday practice. Readings to be completed before class session: D&JR Preface, Chapters 1 and 2 and part of Chapter 3 (sections 3.1 and 3.2 only). Pages xix through 41. evolving the ebp process model-december 1, 2008 -55- the brown school Assignments due this week: Workbook: Pages 4 (EBP), 5, 7, 8 & 24 (your project), 6, (general) Videos: NOVA: The Deadly Deception (about the Tuskegee Syphilis Study) or or Quiet Rage (regarding Zimbardo’s Stanford Prison Experiment) may be shown if time permits. WEEK 3: Obtaining and Reviewing Literature Last week you began to specify a general area of interest. This week, you will begin to learn more about that area, and will spend some time locating at conceptual, review, and empirical sources (see page 42). We will review the evidence/literature retrieval skills covered in the book using in-class examples, and will pay particular attention to . Students will have the chance to discuss their successes and failures in finding needed sources. Readings to be completed before class session: D&JR Chapter 3.3-3.5. Pages 41 through 55. Read your own eight articles (workbook assignment on page 25) Assignment due this week: Workbook: Pages 10,11, 22,(EBP), 12, 13, 25 (your project), 10,11(general) WEEK 4: Your Conceptual Framework This week will build on last week’s session and feature the very difficult issue of how we begin to formalize our conceptual framework. Notice that you are only assigned 14 pages of reading this week. This is because I expect you to put a great deal of work into trying to nail down your conceptual framework. I will be looking at your answers to the questions in the workbook on pages 26 and 27 very carefully. Readings to be completed before class session: D&JR: Chapter 3.6 – 3.7. (pages 55-68) Assignments due this week: Workbook: Pages 23 (EBP) 26 ,27 (your project) Lab Component (1 hr): We will help each other review our assignments. We will do this kindly but critically to help you get used to taking the helpful (and painful) advice of others. WEEK 5: Specifying Aims, Questions and Hypotheses: Introduction of Seven Key Dimensions of Research, Introduction to Control, Subjects and Sampling We will move from our conceptual frame work to specified aims, questions and hypotheses. Readings to be completed before class session: D&JR: Chapter 4, up to and including 4.24. Your own nine articles (workbook page 25) evolving the ebp process model-december 1, 2008 -56- the brown school Assignments due this week: (Staple it all together) Redo the workbook assignments on page 26 and 27. If you are still pleased with these assignments as you turned them in before, just resubmit with the phrase “no changes” written at the top. Do the workbook assignments on pages 53,54 & 55 (your project). Redo the workbook assignment on page 25, finding 8 new articles. WEEK 6: Timeframe, Data, Measurement, Internal Validity and External Validity A number of issues will be covered this week, where we spend time learning the “nuts and bolts” of how research is done, and what key threats exist to doing quality work. Readings to be completed before class session: D&JR: Chapter 4.25 up to and including 4.4., also Chapter 5. Assignments due this week: Workbook Pages 57, 58 (Exercise 1 only). Redo the workbook assignments on page 26 and 27. If you are still pleased with these assignments as you turned them in before, just resubmit with the phrase “no changes” written at the top. This one is important, since it will form the basis for which projects are selected next week. WEEK 7: Organization of Group Projects and Mid-Semester Review. We will spend the majority of the time selecting your semester project topics and assigning you to groups. The projects will be discussed in detail, with prior course content being related clearly to each. We will spend time reviewing the Littell & Schuerman article as a way of reviewing core methods concepts and previewing next week’s content on experimental designs. Readings to be completed before class session: Littell, J., Schuerman, J., A Synthesis of Research on Family Preservation and Family Reunification Programs. (http://aspe.hhs.gov/hsp/cyp/fplitrev.htm). If they’ve moved the darn thing (again!) just find it online. You have the skills, Google the title and author’s names. Classroom Component : We will decide which of your projects will be selected to be used for the remainder of the class. Probably 4-5 will be chosen if we have 25 students. I may choose them before class, or I may ask your input during class. It depends on how feasible your projects are. There are plenty of really good ideas which just won’t work for this class, so don’t get upset if you get eliminated (it’s sort of like “American Idol” or “Top Chef ”). We will break into permanent groups, and each group will execute one of the projects during the remainder of the semester. evolving the ebp process model-december 1, 2008 -57- the brown school WEEK 8: In-Class Introduction to Excel, NVivo, and SPSS. Instructor will demonstrate these databases consistent with material presented in Chapter 10. Readings to be completed before class session: D&JR, Chapter 10. Assignment due this week: Workbook, Pages 78 and 79 (Excel examples). Print out products from Excel and stable together. WEEK 9: Experimental and Quasi-Experimental Designs Readings to be completed before class session: D&JR: Chapter 4.4 through 4.6. D&JR: Chapter 9 (Yuan and Kathy sections only) DJ&R: Chapter 12 (Yuan and Kathy sections only) DJ&R: Chapter 15 (Yuan and Kathy sections only) Assignment due this week: You need to increase compliance for visits (decrease no-shows) at your mental health clinic (near this university) where you mainly treat people for depression. You find the following article: “Effects of Group Psychoeducation (GPE) on Compliance with Scheduled Clinic Appointments in a Neuro-Psychiatric Hospital in Southwest Nigeria: A Randomized Control Trial (RCT)” at http://annals.edu.sg/PDF/36VolNo4Apr2007/V36N4p272.pdf. On one double-spaced page, answer the following questions: • What is your evaluation of the quality of this study? Address the sample, the design and if you think the author’s conclusions are justified. • Is this study useful to you based on the above-described circumstances? Why? WEEK 10: Non-Experimental Designs Readings to be completed before class session: D&JR: Chapter 4.7 D&JR: Chapter 9 (Maria and Abigail sections only) DJ&R: Chpater 12 (Maria and Abigail sections only) DJ&R: Chapter 15 Assignment due this week: Your practicum site deals with eating disorders and just had a pair of fatalities. People are not happy. This will not happen again. Your supervisor tells you to find information which will help to prevent future fatalities. You find: “Mortality in Eating Disorders: A Descriptive Study”. http://www3.interscience.wiley.com/cgi-bin/ fulltext/72500089/PDFSTART On one double-spaced page, answer these questions: evolving the ebp process model-december 1, 2008 -58- the brown school • What is your evaluation of the quality of this study? Address the sample, the design and if you think the author’s conclusions are justified. • Is this study useful to you based on the above-described circumstances? Why? List two recommendations you might make based on this study. WEEK 11: Qualitative Readings to be completed before class session: D&JR: Chapter 6, 10.4. Also chapters 9,12 &15 (John’s sections of these three chapters only). Assignment due this week: You are a student doing research for a local agency. This agency has formerly done only marital counseling and youth counseling, but recently got a contract to begin doing court-ordered treatment for sex offenders referred by the courts. Someone else is tracking down what to actually do with these people, so don’t worry about that. The agency director is very concerned that her staff might not be fully prepared to deal with these new clients – that she be able to prepare her workers for the kinds of issues which might arise when working with this population. You find the following article: “Working with Sex Offenders: The Perceptions and Experiences of Professionals and Paraprofessionals” http://ijo.sagepub.com/ cgi/reprint/43/1/103. On a single double-spaced page, answer the following questions: • What is your evaluation of the quality of this study? Address the sample, the design and if you think the author’s conclusions are justified. • Is this study useful to you based on the above-described circumstances? Why? List two particular recommendations you might make based on this study. WEEK 12: Generating Evidence in Clinical Settings Readings to be completed before class session: D&JR: Chapters 7 and 8 We will describe how research skills can be applied to individuals, caseloads or agencies in clinical (micro practice or management - type) settings. Assignment due this week: Each group will present for 10 minutes on their group’s progress. WEEK 13: Implementation Issues Readings to be completed before class session: D&JR: Chapter 11. Also review Chapter 12 again. Classroom component will cover above material. Discussion will center both on examples from the book and on the experiences students are having as they get their own group projects going. We will also revisit the article assigned in week 1 to see if your views of it have changed in any way. evolving the ebp process model-december 1, 2008 -59- the brown school Assignment due this week: Each group will present for 10 minutes on their group’s progress. WEEK 14: Review of Statistical Tools Readings to be completed before class session: D&JR: Chapters 13 through 15. It is OK if Chapter 14 leaves you a bit confused. I just want you to know that those approaches exist. If you get totally lost, chapter 15 will provide you with sort of a guided tour of how this stuff works in practice and you can follow along. Assignment/Classroom Activity: Each team will bring in data from their projects (on a flash drive or otherwise available to the classroom computer) and either (1) show what analyses they have completed and how they did them or (2) show what they have and explain what problems they are encountering. A computer will be set up with a projection screen so that we can manipulate the data live, in class. WEEK 15: Presentation of Final Projects Your team’s final project must include a packet with ALL of the following: • An APA style paper (like Professor Kathy) • A PowerPoint presentation (like Maria) – You will present this in class. • An Agency Report (Like Yuan). Readings to be completed before class session: D&JR: Chapter 16. You will present your PowerPoint in class formally, as if you were at a conference. You will also then engage the class in an informal discussion of what happened, what went wrong, and what you learned. evolving the ebp process model-december 1, 2008 -60- the brown school Appendix I Members of Brown School EBP Steering Committee – 2008-09 Enola Proctor (Chair), Professor & Associate Dean for Research Brett Drake, Associate Professor & Reaffirmation Project Director Tonya Edmond, Associate Professor & Associate Dean for Academic Affairs Curtis McMillen, Professor Betul Ozmat, Assistant Dean for Strategic Initiatives Ramesh Raghavan, Assistant Professor Ellen Rostand, Assistant Dean for Communications Michael Sherraden, Professor Sylvia Toombs, Library Director Cynthia Williams, Director of Field Education evolving the ebp process model-december 1, 2008 -61- the brown school Appendix J Members of Brown School National Council B.A. Bridgewater, Jr. – Chairman CEO (Retired) Brown Shoe Company Washington University Trustee Eugene S. Kahn CEO, Claire’s Stores Jo Ann Arnold Consultant Emerson Electric Company Martha Korman Executive Director - Premier Accounts NGI Digital Carol Ann Barnickol Brown School Graduate Community Volunteer Charles Lowenhaupt Lowenhaupt & Chasnoff, L.L.C. Cynthia Brinkley Senior Vice President - Talent Development & Chief Diversity Officer AT&T Nicole Brueggeman President, Brown School Alumni Assn. Michael Bennett Kaufman Marylen Mann Founder and Chairman The OASIS Institute Sima Needleman Medical Social Worker (retired) Brown School Graduate Carol Duhme Roblee Foundation Michael F. Neidorff Chairman and CEO Centene Corporation Pastora Cafferty Professor Emeriti University of Chicago Tim Noelker Thompson Coburn, LLC Richard F. Ford Bonnie Orkow Brown School Graduate Marc Freedman Founder and CEO Civic Ventures Glenn Sheffield Former Mayor, Webster Groves, Missouri Kathy Friedland Brown School Graduate David Habif Bettie Schroth Johnson Community Volunteer Betty Sims Program Director Steve Spruth Senior Lecturer Carlson School of Management University of Minnesota evolving the ebp process model-december 1, 2008 -62- the brown school Donald M. Suggs President & Publisher St. Louis American Norman J. Tice Chairman Emeritus MasterCard International Barrett A. Toan Chairman and CEO (Retired) Express Scripts Bernarda Wong Founder and President Chinese American Service League Jay T. Youngdahl Roma B. Wittcoff Washington University Trustee evolving the ebp process model-december 1, 2008 -63- the brown school Appendix K Members of Dean’s Professional Advisory Council Susan Stepleton, Chair President & CEO Parents as Teachers National Center Jim Braun President & CEO Youth In Need Kathleen E. Buescher President & CEO Provident, Inc. Lewis Chartock President MERS/Goodwill Beth Griffin Theresa Dunn Jean Neal Barry Rosenberg Executive Vice President Jewish Federation of St. Louis William Siedhoff Director Department of Human Services City of St. Louis Anna Crosslin President & CEO International Institute Gary Dollar President United Way of Greater St. Louis Flint Fowler Executive Director Herbert Hoover Boys & Girls Club Paula Gianino President & CEO Planned Parenthood St. Louis Region Dan Glazier Executive Director Legal Services of Eastern Missouri, Inc. evolving the ebp process model-december 1, 2008 -64-