University of Nevada, Reno School of Social Work Bachelor of Social Work (BSW) Outcome Data 2012-2013 BSW Assessment Report 2012-2013: Page 1 Background The Bachelor of Social Work (BSW) program at UNR engages in ongoing evaluation efforts to assess the program’s attainment of its competencies. In accordance with the Council on Social Work Education’s (CSWE) 2008 Educational Policy and Accreditation Standards (EPAS), the BSW program has adopted a competency-based approach to social work education. This perspective has focused the BSW program’s assessment efforts squarely on the ten (10) core competencies identified by CSWE and the forty-one (41) associated social work practice behaviors. The School’s Program Evaluation Committee is responsible for designing and implementing assessment plans for the both the BSW and MSW programs. The committee consists of the SSW Director, the BSW Program Coordinator, the MSW Program Coordinator, the Field Education Program Coordinator, one faculty member, and 1-2 graduate assistants. The BSW program’s assessment plan, including the evaluation questions, benchmarks and measures are presented below. Assessment Plan Designing the Plan In designing our assessment plan, the first question we asked was “What is the purpose of the BSW program?” The answer to this query took us to our BSW program mission and goals—and the competencies and practice behaviors we have chosen to operationalize them. The next question we raised was “How does the program go about preparing its students to become proficient in carrying out these practice behaviors?” In answering this, we adopted a layered theory of change. As illustrated in Figure 1, this perspective depicts student learning as BSW Assessment Report 2012-2013: Page 2 beginning in the classroom with the lectures, discussions, assignments and skill-building exercises that constitute the first level of our explicit curriculum. Students then build on this layer of knowledge, values and skills in their field placements (Level 2)—where they are given opportunities to “practice” the behaviors that define competent professional social work. The level of proficiency students demonstrate with respect to these behaviors, when aggregated, yields an assessment of how well the program is doing in achieving its competencies. Figure 1 Theory of Change Level 1: Course-Based Outcomes (Explicit Curriculum - Generalist Practice Courses) Inputs = class lecture/discussion, assignments, readings, skill labs, simulations Student Outcomes Level 2: Practice-Based Outcomes (Explicit Curriculum - Field Education) Inputs = shadowing, practice, reflection, critique, and use of supervision. Increase Knowledge Student Outcomes Increase Skills Develop & Clarify Values Demonstrate Generalist Practice Behaviors Program Outcomes (Social Work Generalist Practice) Demonstrated integration and application of social work core competencies in practice with individuals, families, groups, organizations and communities. Achievement of Social Work Competencies Undergirding the layers depicted in Figure 4.1 is the environment in which the explicit curriculum is delivered. Often referred to as the “implicit curriculum,” this program component is viewed as also playing an important role in shaping our students’ character and competency. BSW Assessment Report 2012-2013: Page 3 Deciding on the Questions Program evaluations can serve many purposes. The purposes chosen determine which questions are asked and what data are gathered. Drawing from the work of Stepney and Rostila (2011),1 the UNR School of Social Work developed its program assessment plan around three key purposes: 1) Accountability; 2) Program Development; and 3) Advancing our understanding of “best practices” with respect to social work education (i.e., knowledge development). Figure 2 depicts these foci, along with the key assessment question accompanying each purpose. Figure 2 BSW Program Assessment Purposes and Questions Accountability Purpose is to assess overall program outcomes. •To what extent do students demonstrate the core competencies and practice behaviors? Developmental Purpose is understand where and how the program can be modified and/or enhanced to improve program outcomes. •To what extent did curriculum changes improve student performance on demonstrated practice behaviors? Knowledge Purpose is to strive for a more cirtical understanding of best practices in social work education. •Under what conditions do students learn new skills (i.e. simulation, skills labs, problem-based learning, distance learning, etc.)? 1 The three perspectives and evaluation questions were adapted from Stepney & Rostila (2011). Towards an integrated model of practice evaluation balancing accountability, critical knowledge and developmental perspectives. Health Sociology Review, 20(2). BSW Assessment Report 2012-2013: Page 4 Evaluation Design Given competency-based education’s emphasis on students’ ability to demonstrate the requisite practice behaviors, we began building our assessment plan from the accountability perspective. In so doing, the key assessment question we posed is: To what extent do our BSW students demonstrate mastery of the 10 core competencies and their associated practice behaviors? Table 1 To what extent do students demonstrate the core competencies and practice behaviors? Benchmark Measures Timing 85% of BSW students will 1. Field Instructor Assessment 1. At the completion of achieve an overall mean of Field Practicum Student SW481 – Field Experience of 8 for each Performance in Social Work II competency.* 2. Student Self-Efficacy Survey 2. End of Spring As noted in Table 1, we are using two outcome measures to evaluate the extent to which our BSW students are able to demonstrate attainment of the 10 core competencies. Both instruments use the same 9 point rating scale (1=Lacking Performance through 9=Mastered Performance). In establishing our benchmark, we chose to use a combined (or overall) mean score of 8. We then set our program performance level at 85%. Thus, using data derived from our field instructor and student self-efficacy assessments, we expect at least 85% of our BSW students each year will achieve overall means of eight (8) for each of the 10 competencies. A mean of eight (8) represents an unanchored value falling mid-way between “superior performance” (7) and “mastered performance” (9). BSW Assessment Report 2012-2013: Page 5 While at this stage we are emphasizing the accountability question, we also see the developmental and knowledge questions posed above as essential ingredients for ongoing program renewal. Therefore, at the end of this segment we discuss how we envision incorporating these into our assessment plan. Data Collection Methods Instruments and Procedures Measure One - Field Instructor Assessment of Field Practicum Student Performance Description: This instrument is designed to assess the student’s demonstrated performance of the 41 practice behaviors that comprise the 10 core competencies of generalist social work practice. With this measure, data are gathered from the field instructors’ perspective. Benchmark: As noted above, the benchmark we have set encompasses an overall mean of 8 (computed using both our field instructor and student self-efficacy measures) for at least 85% of our students. This is to say, that we expect 85% of BSW students each year to achieve an overall mean score of 8 (or higher) for each of the 10 core competencies. Protocol: This instrument is administered at the completion of SW 481, Field Experience in Social Work II. This is the 2nd of a 2-course field placement (SW 480 & SW 481), which requires students to complete a minimum of 225 hours each semester in a supervised field setting. Field Instructors are introduced to, and trained on how to use, the assessment form during the fall Field Instructor Orientation and Training. Student performance is rated on the basis of the criteria set forth in the student’s learning BSW Assessment Report 2012-2013: Page 6 agreement and reflects the field instructor’s evaluation of the student’s ability to demonstrate each practice behavior. Data Analysis: Field Instructors submit their completed Field Practicum Student Performance forms to the School of Social Work during the final week of spring semester. Data from these forms are then entered directly to the Statistical Package for Social Sciences (SPSS) for analysis. Utilizing the descriptive statistics feature of this program, mean scores are computed for all 41 practice behaviors. Then, an overall competency mean is calculated by averaging the means of the measures of its associated practice behaviors. Measure Two - Student Self-Efficacy Survey Description: This instrument is designed to assess the student’s “beliefs, expectations and confidence” in their ability –at this moment–to perform each of the 41 practice behaviors (organized around the 10 core competencies). Benchmark: As indicated earlier, the benchmark we have set encompasses an overall mean of 8 (computed using both the field instructor and student self-efficacy measures) for at least 85% of our students. This is to say, that we expect 85% of BSW students each year to achieve an overall mean score of 8 (or higher) for each of the 10 core competencies. Protocol: The student self-efficacy survey is administered during the final weeks of spring semester to students completing SW481 – Field Experience in Social Work II. Data are collected via a web-based interface using “SurveyMonkey.” Students are given BSW Assessment Report 2012-2013: Page 7 time to complete the survey during a field seminar class session, which is scheduled in an on-campus computer lab. Data Analysis: Utilizing tools included in the “SurveyMonkey” program, data are downloaded directly into SPSS. Using the descriptive statistics feature of this program, mean scores are computed for all 41 practice behaviors. Then, an overall competency mean is calculated by averaging the means its associated practice behaviors. Overall Percentage of Students Achieving Benchmark To calculate the percentage of students that meet the benchmark for each competency, the mean scores (of the practice behaviors used to operationalize it) are calculated independently for each instrument (i.e., the Field Instructor Assessment of Field Practicum Student Performance and the Student Self-Efficacy survey). These data are then combined and “averaged” to yield overall mean scores. As the data presented in summary reports appended to this segment of the self-study convey, overall mean scores are generated for each practice behavior, as well as for each core competency. To determine the percentage of students achieving benchmark, the “overall competency mean” is divided by our designated benchmark mean of eight (Holloway, 2012). Feedback Loops An important component of any evaluation plan is the ongoing involvement of the program’s key stakeholders. To ensure strong connections between the BSW program and its’ varied constituencies, the maintenance of ongoing feedback loops with members of the following groups is essential: BSW Assessment Report 2012-2013: Page 8 Administration: SSW Director, BSW, MSW and Field Program Coordinators – as members of the program evaluation committee, the director and program coordinators play an important role in ensuring that assessment data is collected, analyzed, and shared with appropriate constituent groups. Not only do these leaders facilitate discussions with faculty, field, and advisory groups, but they also guide curriculum changes and other program enhancements. Faculty: Findings from the previous spring are presented to the full faculty at the SSW faculty fall retreat. These data are then used throughout the year to inform the work of program level (400, 600,700) and curriculum content (i.e., research, policy, field, practice, HBSE) committees. Students: As data are reviewed and addressed by faculty, they are then shared with student representatives through School Meetings, and special sessions with student constituencies who may be directly impacted by the data (e.g., the Undergraduate Student Social Work Association, parttime students, students new to the program when course requirements change due to assessment findings, and so forth). Employers/Field Supervisors: Findings are shared with field instructors during the fall field instructor orientation. Other stakeholders: Benchmark data are posted each fall to the School’s website. Assessment findings are also presented to the SSW Advisory Council, as they become available. Procedures to Evaluate Outcomes and Implications for Program Renewal As outcome data are gathered, the first layer of review focuses on the identification of competencies that fall below benchmark. The Program Evaluation Committee identifies areas of concern, which are shared by the appropriate program coordinator (i.e., BSW, MSW, or Field) with the faculty of the whole. Following general discussion and review, the matter is referred to the relevant curriculum committee (practice, research, policy, etc.) for deeper assessment and identification of possible actions. After review, the committee shares ideas regarding potential actions (e.g., changes in assignments, activities or readings) with the program coordinator— and, where substantial curriculum changes are involved—the faculty of the whole. The next layer of review involves the identification of practice behaviors that fall below benchmark. Once identified, these data are reviewed by the appropriate Program Coordinator (BSW, MSW, BSW Assessment Report 2012-2013: Page 9 or Field) to determine if it is a programmatic matter; if not, it is referred to the appropriate curriculum committee (policy, practice, research, field, HBSE) for further review and action. Summary of Results by Competency Summary data (i.e., overall mean and benchmark) gathered for each of the core competencies during spring 2013 in the table below. Competency Fall 2012 Spring 2013 Benchmark = 85% of BSW students will achieve an overall mean of 8 for each competency. Overall Mean Benchmark Result Overall Mean Benchmark Result 2.1.1 Identify as a professional social worker and conduct oneself accordingly 5.99 75% 7.27 91% 2.1.2 Apply social work ethical principles to guide professional practice 5.47 68% 7.27 91% 2.1.3 Apply critical thinking to inform and communicate professional judgments 5.50 69% 7.00 88% 2.1.4 Engage in diversity and difference in practice 5.94 78% 7.33 92% 2.1.5 Advance human rights and social and economic justice 5.46 68% 7.11 89% 2.1.6 Engage in research-informed practice and practiceinformed research 4.96 62% 6.74 84% 2.1.7 Apply Knowledge of human behavior and the social environment 5.54 69% 7.14 89% 2.1.8 Engage in policy practice to advance social and economic well-being and to deliver effective social work services 4.88 61% 6.96 87% 2.1.9 Respond to contexts that shape practice 5.00 63% 7.10 89% 2.1.10 Engage, assess, intervene and evaluate individuals, families, groups, organizations and communities 5.36 67% 7.14 89% BSW Assessment Report 2012-2013: Page 10 Appendix A Outcome Data Reports BSW Assessment Report 2012-2013: Page 11 The Council on Social Work Education’s (CSWE) 2008 Educational Policy and Accreditation Standards (EPAS) included a provision for assessment of accredited social work programs. At the BSW and MSW foundation levels, each program must evaluate the extent to which students master the 10 CSWE identified core competencies and the 41 associated practice behaviors. Below, please find performance summaries for the BSW graduates of May 2013. Spring 2013 Competency 2.1.1: Identify as a professional social worker and conduct oneself accordingly. Social workers serve as representatives of the profession, its mission, and its core values. They know the profession’s history. Social workers commit themselves to the profession’s enhancement and to their own professional conduct & growth. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Field Instructor Self-Efficacy Practice Behavior Assessment Survey Overall Mean Assessment F1 Advocate for client access to the 7.02 6.85 6.94 services of social work. F2 Practice personal reflection and selfcorrection to assure continual professional 7.10 7.19 7.15 development. F3 Attend to professional roles and 7.08 7.46 7.27 boundaries. F4 Demonstrate professional demeanor in behavior, appearance, and 7.35 7.50 7.43 communication. F5 Engage in career-long learning. 7.37 7.42 7.40 F6 Use supervision and consultation. 7.35 7.54 7.45 Mean Score Percent of Students Achieving Competency 7.27 91% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 12 Competency 2.1.2: Apply social work ethical principles to guide professional practice. Social workers have an obligation to conduct themselves ethically and to engage in ethical decision making. Social workers are knowledgeable about the value base of the profession, its ethical standards and relevant law. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 7.05 7.19 7.12 7.02 7.15 7.09 6.97 6.88 6.93 7.02 6.81 6.92 F7 Recognize and manage personal values in a way that allows professional values to guide practice. F8 Make ethical decisions by applying standards of NASW Code of Ethics and, as applicable, principles of the International Federation of Social Workers and the Association of Schools of Social Work Ethics in SW. F9 Tolerate ambiguity in resolving ethical conflicts. F10 Apply strategies of ethical reasoning to arrive at principled decisions. Mean Score Percent of Students Achieving Competency 7.02 88% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 13 Competency 2.1.3: Apply critical thinking to inform and communicate professional judgments Social workers are knowledgeable about the principles of logic, scientific inquiry and reasoned discernment. They use critical thinking augmented by creativity and curiosity. Critical thinking also requires synthesis and communication of relevant information. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 6.82 6.88 6.85 6.91 7.00 6.96 7.00 7.38 7.19 F11 Distinguish, appraise, and integrate multiple sources of knowledge, including research-based knowledge, and practice wisdom. F12 Analyze models of assessment, prevention, intervention, and evaluation. F13 Demonstrate effective oral and written communication in working with individuals, families, groups, organizations, communities, and colleagues. Mean Score Percent of Students Achieving Competency 7.00 88% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 14 Competency 2.1.4: Engage diversity and difference in practice Social workers understand how diversity characterizes and shapes the human experience and is critical to the formation of identity. The dimensions of diversity are understood as the intersectionality of multiple factors including age, class, color, culture, disability, ethnicity, gender, gender identity and expression, immigration status, political ideology, race, religion, sex, and sexual orientation. Social workers appreciate that, as a consequence of difference, a person’s life experiences may include oppression, poverty, marginalization and alienation, as well as privilege, power & acclaim. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 7.31 7.31 7.31 7.39 7.38 7.39 7.23 7.23 7.23 7.40 7.38 7.39 F14 Recognize the extent to which a culture’s structures and values may oppress, marginalize, alienate, or create or enhance privilege and power. F15 Gain sufficient self-awareness to recognize the influence of personal biases and values in working with diverse groups. F16 Recognize, value, and communicate their understanding of the importance of difference in shaping life experiences. F17 View themselves as learners and engage those with whom they work as experts of their own circumstances/lives. Mean Score Percent of Students Achieving Competency 7.33 92% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 15 Competency 2.1.5: Advance human rights and social and economic justice. Each person, regardless of position in society, has basic human rights, such as freedom, safety, privacy, an adequate standard of living, health care, and education. Social workers recognize the global interconnections of oppression and are knowledgeable about theories of justice and strategies to promote human and civil rights. Social work incorporates social justice practices in organizations, institutions, and society to ensure that these basic human rights are distributed equitably and without prejudice. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 7.16 7.38 7.27 7.05 6.96 7.01 7.10 7.00 7.05 F18 Understand the forms and mechanisms of oppression and discrimination F19 Advocate for human rights and social and economic justice. F20 Engage in practices that advance social and economic justice. Mean Score Percent of Students Achieving Competency 7.11 89% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 16 Competency 2.1.6: Engage in research-informed practice and practice-informed research. Social workers use practice experience to inform research, employ evidence-based interventions, evaluate their own practice, and use research findings to improve practice, policy, and social service delivery. Social workers comprehend quantitative and qualitative research and understand scientific and ethical approaches to knowledge building. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 6.63 6.58 6.61 6.88 6.84 6.86 F21 Use practice experiences to inform scientific inquiry. F22 Use research evidence to inform practice. Mean Score Percent of Students Achieving Competency 6.74 84% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 17 Competency 2.1.7: Apply knowledge of human behavior and the social environment. Social workers are knowledgeable about human behavior across the life course; the range of social systems in which people live; and the ways social systems promote or deter people in maintaining or achieving health and well-being. Social workers apply theories and knowledge from the liberal arts to understand biological, social, cultural, psychological, and spiritual development. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 7.20 6.92 7.06 7.18 7.23 7.21 F23 Utilize conceptual frameworks to guide the processes of assessment, intervention, and evaluation. F24 Critique and apply knowledge to understand person and environment. Mean Score Percent of Students Achieving Competency 7.14 89% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 18 Competency 2.1.8: Engage in policy practice to advance social and economic well-being and to deliver effective social work services. Social work practitioners understand that policy affects service delivery and they engage in policy practice. Social workers know the history and current structures of social policies Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 6.83 6.92 6.88 6.92 7.15 7.04 F25 Analyze, formulate, and advocate for policies that advance social well-being. F26 Collaborate with colleagues and clients for effective policy action. Mean Score Percent of Students Achieving Competency 6.96 87% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 19 Competency 2.1.9: Respond to contexts that shape practice Social workers are informed, resourceful, and proactive in responding to evolving organizational, community, and societal contexts at all levels of practice. Social workers recognize that the context of practice is dynamic, and use knowledge & skill to respond proactively. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Practice Behavior Field Instructor Assessment Mean Self-Efficacy Survey Assessment Overall 6.90 7.87 7.39 6.75 6.85 6.80 F27 Continuously discover, appraise, and attend to changing locales, populations, scientific and technological developments, and emerging societal trends to provide relevant services. F28 Provide leadership in promoting sustainable changes in service delivery and practice to improve the quality of social services. Mean Score Percent of Students Achieving Competency 7.10 89% Results for Competency: 0 No Opportunity 1 Lacking Performance 2 3 Inadequate Performance 4 5 Competent Performance 6 7 Superior Performance 8 9 Mastered Performance BSW Assessment Report 2012-2013: Page 20 Competency 2.1.10: Engage, assess, intervene, and evaluate with individuals, families, groups, organizations and communities Professional practice involves the dynamic and interactive processes of engagement, assessment, intervention, and evaluation at multiple levels. Social workers have the knowledge and skills to practice with individuals, families, groups, organizations and communities. Practice knowledge includes identifying, analyzing and implementing evidence-based interventions designed to achieve client-goals; using research and technological advances; developing, analyzing, advocating, and providing leadership for policies and services; and promoting social and economic justice. Competency Benchmark: Based upon data gathered during previous years, we expect 85% of our graduates to achieve an overall mean of 8 for the competency-related practice behaviors. Field Instructor Self-Efficacy Practice Behavior Assessment Survey Overall Mean Assessment F29 Substantively and affectively prepare for action with individuals, families, groups, 7.10 7.12 7.11 organizations, and communities F30 Use empathy and other interpersonal 7.52 7.69 7.61 skills F31 Develop a mutually agreed-on focus of 7.10 7.12 7.11 work and desired outcomes F32 Collect, organize and interpret client 7.10 7.15 7.13 data F33 Assess client strengths and limitations 7.25 7.23 7.24 F34 Develop mutually agreed-on intervention 6.98 7.23 7.11 goals and objectives F35 Select appropriate intervention 7.05 6.92 6.99 strategies F36 Initiate actions to achieve 7.10 6.92 7.01 organizational goals F37 Implement prevention interventions that 7.02 6.96 6.99 enhance client capacities F38 Help clients resolve problems 7.16 7.27 7.22 F39 Negotiate, mediate, and advocate for 7.25 7.31 7.28 clients. F40 Facilitate transitions and endings 7.07 7.12 7.10 F41 Critically analyze, monitor, and evaluate 6.96 6.96 6.96 interventions Results for Competency: Mean Score 7.14 Percent of Students Achieving Competency 89% BSW Assessment Report 2012-2013: Page 21