University of Nebraska at Kearney Department of Social Work Fall 2014 and Spring 2015 Assessment Summary Introduction Program assessment in the Department of Social Work at the University of Nebraska-Kearney is completed annually examining the Fall and Spring semesters by utilizing multiple assessment measures as outlined in the department’s assessment plan. Assessment is institutionally mandated by UNK, and all departments must have an assessment plan and submit an annual report to demonstrate evidence of its usage to evaluate curriculum and ensure ongoing evaluation. Additionally, the department of Social Work has designed its assessment plan to coincide with the Educational Policy and Accreditation Standards (EPAS) as outlined by the Council on Social Work Education. The EPAS follow competency-based education, which is an outcome performance approach to curriculum design. The competencies are measurable practice behaviors that are comprised of Knowledge, Values, and Skills. “The goal of the outcome approach is to demonstrate the integration and application of the competencies in practice with individuals, families, groups, organizations, and communities” (CSWE, 2012). The purpose of this report is to show the department’s mission/purpose, goals, methodology, outcomes, and a discussion of the findings from the Fall 2014/Spring 2015 assessment. Mission / Purpose The Social Work Department at the University of Nebraska at Kearney will prepare competent professionals who are equipped with generalist social work knowledge, skills, ethics and values to promote the dignity and wellbeing of all people within a multicultural society. 1 Goals Each of the goals for the UNK Social Work program were designed to coincide with the Council on Social Work Education’s Educational Policy and Accreditation Standards. There are 10 Competencies and total of 41 practice behaviors. The competencies represent the values, knowledge, and skills needed to be competent professional social workers. The table below illustrates which competencies correspond to the department goals. Social Work Department Goal 1) Engage in Generalist Social Work Practice: Equip students with core competencies for beginning generalist social work practice that enables them to work with individuals, families, groups, organizations, and communities. CSWE Competencies EP 2.1.7 Apply knowledge of human behavior and the social environment. EP 2.1.8 Engage in policy practice to advance social and economic well-being and to deliver effective social work services. 2.1.10 (a)-(-d) Engage, assess, intervene, and evaluate with individuals, families, groups, organizations and communities. 2) Critical Thinking: Prepare students for lifelong learning and the application of critical thinking to guide professional decision-making by combining a liberal arts foundation with social work education. EP 2.1.3 Apply critical thinking to inform & communicate professional judgments. 2.1.9. Respond to contexts that shape practice 3) Identity, Knowledge, Values, Skills, Social Justice: Prepare students to develop a professional identity, which will incorporate the knowledge, skills, values, and ethical principles of the social work profession that are crucial for the advancement of human rights, and economic and social justice. 2.1.1 Identify as professional social worker & conduct self accordingly 2.1.2 Apply critical thinking to inform & communicate professional judgment 2.1.5 Advance human rights & social & economic justice 2.1.8 Engage in policy practice to advance social and economic well-being and to deliver effective social work services. 4) Evidence-Based Practice: Prepare students for evidence-based practice with diverse and marginalized populations. EP 2.1.4 Engage diversity and difference in practice. 2.1.6 Engage in research-informed practice and practice-informed research EP 2.1.6 Engage in research-informed practice & practice informed research. 2.1.9. Respond to contexts that shape practice 5) Research: Prepare students as beginning social work generalist practitioners who are engaged in 2 research informed practice and practice informed research. Assessment Methodology The 2014/2015 program assessment utilized four different survey instruments, two of which were developed by the Social Work Education Assessment Project (SWEAP), the department developed Senior Portfolio, and the Department of Social Work’s Student Field Evaluation to assess students in relation to the five departmental goals identified above. A description of each instrument, how it is implemented, and the target achievement for each instrument is provided below: 1. Exit Survey Instrument (SWEAP): This instrument is administered to students in the Senior Social Work Seminar (SOWK 482) at the end of the semester before they graduate. It gathers demographic information, assesses the student’s experience in the program, and professional plans. A section of this instrument addresses the evaluation of curriculum objectives based upon CSWE Accreditation Standards. This instrument uses an 11-point Likert scale and it is expected that students will receive ratings of 5.0 or higher (adequate to very good) on all areas of the survey. 2. Foundation Curriculum Assessment Instrument (SWEAP): The FCAI represents an indirect measure of a student’s knowledge in relation to the core competencies. It is administered to social work students as they enter the program in the Introduction to Social Welfare course (SOWK 170). It is also administered to students at the time of exit to assess the knowledge gained from entry through exit in the program. The instrument consists of 64 multiple-choice questions organized into the following curricular areas: practice, human behavior and social environment (HBSE), policy, research, ethics and values, diversity and social and economic justice. This pre and posttest format enables the department to identify areas needing curriculum improvements. The target achievement expects graduating students to score higher than incoming students in all sections. Additionally, goals 3-5 target achievements expect graduating students to score 60% or higher in the corresponding sections. 3 3. Student Field Evaluation Form is a direct measure of field placement competencies that address Educational Policy 2.1 (Core Competencies) and 2.3 (Signature Pedagogy: Field Education). This instrument is a newly developed instrument that corresponds directly to the competencies and practice behaviors outlined by CSWE. The instrument consists of 41 items scored on a 5-point Likert scale and is used at the end of a field/practicum placement by field supervisors to assess the student's knowledge, skill, and ability in the following areas: professional social work skills and use of supervision, professional communication, social work values and ethics, critical thinking, diversity, human rights, social and economic justice and policy practice, research, human behavior and the social environment, generalist social work practice, and social work practice with individuals, families, groups, organizations, and communities. Additionally, this instrument assists the program in making curriculum improvements, determining the effectiveness of field education and agencies, and tailoring educational processes to be more responsive to students. The achievement target expects graduating students to score a 3.0 or higher in all sections of the survey. This evaluation year is the second year the new field evaluation form was implemented. 4. The Senior Portfolio is a collection of student assignments that demonstrate knowledge, skills, and values for appropriate social work practice. Every graduating senior completes the portfolio by selecting a minimum of five assignments from over the course of the program that demonstrate their competency. The Rationalization for Using Senior Portfolios provides guidance in completing the portfolio and outlines how the portfolios are to be evaluated. Members of the faculty receive de-identified portfolios and evaluate the products based upon a 3-point Likert scale with attention to the department’s program goals. The benchmark established for the Senior Portfolio is that 75% of the portfolios will receive a rating of minimally satisfies for each competency. The Senior Portfolio is a newly developed assessment measure that was first implement during the 2013/2014 evaluation year. 4 Outcomes The program assessment outcomes are presented in the tables below to assist in ease of reporting. Comparisons are made between semesters and incoming and graduating students where appropriate. Table 1 shows the average (mean) scores for the sections that correspond to the department’s goals from the BEAP Exit Survey. The target achievements for the Exit Survey were all met for the 2014-2015 academic year. Table 2 shows the mean scores for the sections that correspond to the department’s goals from the Student Field Evaluation Form for graduating students for the Fall 2014 and Spring 2015 semesters. All achievement targets for the Student Field Evaluation were met for Fall and Spring semesters with the exception of the section related to HBSE, which had a score of 2.03 for the Fall and 2.45 for the Spring. Table 3 shows the percentage scores for the sections that correspond to the department’s goals from the Foundation Curriculum Assessment Instrument (FCAI). All the achievement targets were met for goal one. Goals two through five were partially met. The results for the Senior Portfolio are included in Table 4. For the Fall 2014 semester, students failed to reach the 75% competency level in the following areas: HBSE, Professional Identity, Social Work Values & Ethics, Advance Human Rights, Social & Economic Justice, Engage Diversity. All targets were surpassed for the Spring 2015 semester. A discussion of target achievements is provided under the conclusion. 5 Table 1 Social Work Department Goals 1) Equip students with core competencies for beginning generalist social work practice that enables them to work with individuals, families, groups, organizations, and communities. Exit Survey Fall 2014 Spring 2015 (N=17) (N=20) 8.31 8.04 2) Prepare students for lifelong learning and the application of critical thinking to guide professional decision-making by combining a liberal arts foundation with social work education. 8.26 7.98 3) Prepare students to develop a professional identity, which will incorporate the knowledge, skills, values, and ethical principles of the social work profession that are crucial for the advancement of human rights, and economic and social justice. 4) Prepare students for evidence-based practice with diverse and marginalized populations. 8.29 8.06 8.23 7.95 5) Prepare students as beginning social work generalist practitioners who are engaged in research informed practice and practice informed research. 8.27 7.88 * Target Achievement for the FPPA is a mean score of 5.0 and higher in all sections. 6 Table 2 Social Work Department Goals Sections Student Field Eval. * Fall 2014 (N=17) Sections Student Field Eval.* Spring 2015 (N=19) 2.45 1) Equip students with core competencies for beginning generalist social work practice that enables them to work with individuals, families, groups, organizations, and communities. EPAS 2.1.7 HBSE 2.03 EPAS 2.1.7 HBSE EPAS 2.1.8 Policy Practice EPAS 2.1.10 (A-D) Practice Skills 3.96 EPAS 2.1.8 Policy Practice EPAS 2.1.10 (A-D) Practice Skills 3.38 2) Prepare students for lifelong learning and the application of critical thinking to guide professional decisionmaking by combining a liberal arts foundation with social work education. 3) Prepare students to develop a professional identity, which will incorporate the knowledge, skills, values, and ethical principles of the social work profession that are crucial for the advancement of human rights, and economic and social justice. 4) Prepare students for evidence-based practice with diverse and marginalized populations. EPAS 2.1.3 Critical Thinking 3.94 EPAS 2.1.3 Critical Thinking 3.54 EPAS 2.1.9 Contexts that Shape Practice 3.91 EPAS 2.1.9 Contexts that Shape Practice 3.34 EPAS 2.1.1 Professional Identity EPAS 2.1.2 Social Work Values & Ethics EPAS 2.1.5 Advance Human Rights, Social & Economic Justice 4.16 EPAS 2.1.1 Professional Identity EPAS 2.1.2 Social Work Values & Ethics EPAS 2.1.5 Advance Human Rights, Social & Economic Justice 3.75 EPAS 2.1.8 Policy Practice 3.96 EPAS 2.1.8 Policy Practice 3.38 EPAS 2.1.4 Engage Diversity 4.18 EPAS 2.1.4 Engage Diversity 3.54 EPAS 2.1.6 Research 3.68 EPAS 2.1.6 Research 3.16 5) Prepare students as EPAS 2.1.6 Research 3.68 EPAS 2.1.6 Research beginning social work EPAS 2.1.9 Contexts that 3.91 EPAS 2.1.9 Contexts generalist practitioners Shape Practice that Shape Practice who are engaged in research informed practice and practice informed research. * Target Achievement for the Student Field Evaluation Form is 3.0 or higher in all sections. 3.16 3.34 4.13 4.09 3.96 3.64 3.61 3.48 7 Table 3 Social Work Department Goals Sections 1) Equip students with core competencies for beginning generalist social work practice that enables them to work with individuals, families, groups, organizations, and communities. Practice Skills FCAI Fall 2014 Spring 2015 Grad. Incoming Grad. Incoming % % % % 62.54 45.08 71.15 49.23 HBSE 61.90 48.90 63.00 58.00 2) Prepare students for lifelong learning and the application of critical thinking to guide professional decision-making by combining a liberal arts foundation with social work education. Practice Skills 62.54 45.08 71.15 49.23 HBSE 61.90 48.90 63.00 58.00 Policy 38.22 43.67 45.56 42.22 Research 43.11 38.89 48.33 40.00 3) Prepare students to develop a professional identity, which will incorporate the knowledge, skills, values, and ethical principles of the social work profession that are crucial for the advancement of human rights, and economic and social justice. Ethics & Values** 61.75 51.75 70.63 60 Diversity** 53.13 42.88 56.25 40 Social & Economic Justice** 56.29 51.57 61.43 65.71 4) Prepare students for evidencebased practice with diverse and marginalized populations. Research** 43.11 38.89 48.33 40.00 Diversity** 53.13 42.88 56.25 40 5) Prepare students as beginning social work generalist practitioners who are engaged in research informed practice and practice informed research. Research** 43.11 38.89 48.33 40.00 16 28 20 5 Total N Responding * Target achievement expects graduating students to score higher than incoming students in all sections. ** Target achievement expects graduating students to score 60% or higher. 8 Table 4 Social Work Department Goals Sections Senior Portfolios Fall 2014 % Passing 1) Equip students with core competencies for beginning generalist social work practice that enables them to work with individuals, families, groups, organizations, and communities. 2) Prepare students for lifelong learning and the application of critical thinking to guide professional decision-making by combining a liberal arts foundation with social work education. 3) Prepare students to develop a professional identity, which will incorporate the knowledge, skills, values, and ethical principles of the social work profession that are crucial for the advancement of human rights, and economic and social justice. 4) Prepare students for evidence-based practice with diverse and marginalized populations. 5) Prepare students as beginning social work generalist practitioners who are engaged in research informed practice and practice informed research. EPAS 2.1.7 HBSE 70.59 Spring 2015 % Passing 84.21 EPAS 2.1.8 Policy Practice 82.35 100 EPAS 2.1.10 (A-D) Practice Skills EPAS 2.1.3 Critical Thinking 76.47 84.21 76.47 100 EPAS 2.1.9 Contexts that Shape Practice 82.35 89.47 EPAS 2.1.1 Professional Identity EPAS 2.1.2 Social Work Values & Ethics EPAS 2.1.5 Advance Human Rights, Social & Economic Justice EPAS 2.1.8 Policy Practice EPAS 2.1.4 Engage Diversity EPAS 2.1.6 Research 70.59 100 70.59 94.74 70.5 94.74 82.35 70.59 100 89.47 100 100 EPAS 2.1.6 Research EPAS 2.1.9 Contexts that Shape Practice 100 82.35 100 89.47 N=17 N=19 *Target is 75% of the portfolios will receive a minimally satisfies for each competency. 9 Conclusion The Department of Social Work at UNK is constantly assessing the efficiency, effectiveness, and application of its assessment measures. Through this process, the decision was made to discontinue the use of the FCAI instrument. The FCAI instrument is designed only to measure knowledge and neglects to measure skills and values as part of the competency based approach taken by CSWE. Furthermore, the instrument’s specific questions regarding sections such as Policy and HBSE do not mirror what is taught in the program and therefore may not accurately suggest how well the program is performing. For example, the Policy section of the FCAI contains nine questions, two of which address Elizabethan Poor Laws, and none of the questions assess policy action (2.1.8B). Additionally, students and faculty have grown more comfortable with and confident in the student portfolio and view it as a more valid measure of student learning. The collective use of the assessment instruments in the evaluation of the social work program demonstrates that the program is doing well and is effective in educating and preparing social workers for generalist practice. Examining the outcomes individually illustrates areas for improvement but should not warrant too much concern. For example, the target achievements were only partially met for goals two through five on the Foundation Curriculum Assessment Instrument (FCAI). In regards to goal two, the specific section of “Policy” showed that graduating students in the Fall semester of 2014 had a lower score (38%) than incoming students (44%). However, the other sections of goal 2 demonstrate that students met the target achievement. Results found on the FCAI also suggest that students did not achieve the targeted 60% or higher in the following areas: Diversity, Social and Economic Justice, and Research, with the exception of Social & Economic Justice in Spring 2015. Repeated shortcomings on the 10 FCAI over the previous years as well as examination of the FCAI suggest that the missed targets are more a function of the instrument rather than an indictment of the social work program. The previous paragraph outlines the reasoning behind such sentiment. The Senior portfolio and the Student Field Evaluation Form have been instituted to engage students in assessing their own learning, promote the development of critical thinking, enhance integration of learning across the curriculum, engage in reflective self-evaluation of personal and professional growth, and to prepare students for lifelong learning. The Senior Portfolio continues to increase in its importance as an assessment component for the department to evaluate individual student learning, strengths and weaknesses of the program, curriculum, and pedagogy. The Student Field Evaluation is completed with Field Supervisors at the end of the Field Practicum to assess the student's knowledge, skill, and ability. This instrument corresponds directly to the competencies and practice behaviors outlined by CSWE. The Senior Portfolio and Student Evaluation Form will help to further assess the outcome of the program in relations to all five department goals. In summary, when examining the assessment materials as a whole, the department of Social Work at UNK continues to effectively prepare social workers for generalist social work practice. The curriculum and pedagogy of the program are consistent with the standards set forth by CSWE and results of this assessment are evidence of the hard work and dedication of faculty and staff to ensure that students are prepared, competent, and effective social workers. 11