THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL SCHOOL OF SOCIAL WORK Course Number: SOWO 810 Course Title: Evaluation of Social Work Interventions Semester and Year: Spring 2013 Course Website: http://sakai.unc.edu Time and Location: Mondays 10:30am-11:50am Tuesdays 10:30am-11:50am Instructor: Natasha K. Bowen, PhD, MSW School of Social Work TTK Building, Rm. 421A Phone: 919-843-0063 Email: nbowen@email.unc.edu Office Hours: Tuesdays 12-1:00 pm, and by appointment Course Description: Students gain advanced knowledge of evaluation methods and evidence-based practices that can be applied to the evaluation of social work interventions in their areas of practice. Learning Objectives: Upon completion of this course, students will be able to draw from knowledge of research methods and evidence-based practice, and work collaboratively with supervisors, colleagues, peers, and consumers, to design, implement, and disseminate the results of an evaluation of an intervention, program, service or policy that may be needed by current or prospective consumers or program or policy planners. Specifically, students will demonstrate: 1) Skills for reviewing and presenting information about the importance of a social problem addressed by an agency, including its relation to discrimination based on culture, economic status, gender, religion, age, or sexual orientation; 2) The ability to consider this social problem in the context of current service delivery, and critically determine a matter requiring evaluation related to client need, the process of service delivery, or service outcomes; SOWO 810 Spring 2013 1 3) The ability to design an appropriate evaluation, including identifying a sample, determining a suitable study approach; and selecting appropriate, reliable, and valid measures (including consideration of validity for the targeted population); 4) The ability to plan, conduct, and synthesize appropriate qualitative or quantitative analyses; 5) The ability to apply knowledge of social work ethics and values to the design of a practice intervention or other types of evaluations; 6) The ability to consider the implications of results for practice and policy (including implications related to the agency’s success in working with clients from different groups); 7) The ability to summarize evaluation results and program improvement plans in a concise written document; and 8) The ability to present evaluation plans and program improvement plans to appropriate audiences. SOWO 810 Spring 2013 2 Required Texts: Royse, D., Thyer, B. A., & Padgett, D. K. (2010) Program evaluation: An introduction (5th ed.). Belmont, CA: Brooks-Cole. Rubin, R., & Babbie, E. R. (2010). Essential research methods for social work (2nd ed.). Belmont, CA: Brooks-Cole. Hatry, H., van Houten, T., Plantz, M. C., & Greenway, M. T. (1996). Measuring program outcomes: A practical approach. Alexandria, VA: United Way of America. Supplemental readings are available on Sakai or through UNC libraries databases. Additional supplementary readings may be recommended or assigned at the discretion of the instructor. Accommodations for Students with Disabilities: Students with disabilities, which affect their participation in the course, should notify the instructor if they wish to have special accommodations in instructional format, examination format, etc., considered. Accommodations and services are provided by Disability Services (Voice/TDD 962-8300; 966-4041). Learning Disability Services (9627227) provides supportive services for students with learning disabilities and attentiondeficit/hyperactivity disorders. Policy on Academic Dishonesty: Academic dishonesty is contrary to the ethics of the social work profession, unfair to other students, and will not be tolerated in any form. All written assignments should include the following pledge: “I have neither given nor received unauthorized aid in preparing this written work.” Sign and date your statement. Teaching Methods and Accountability: SOWO 810 is a hybrid course in which the online and text resources used in 510 will be the basis for application assignments and activities related to the analysis and interpretation of evaluation data. Students are expected to be responsible for their own learning of course material. Most in-class sessions focus on team-based application activities. Each team will rely on the expertise, knowledge, presence, and effort of each team member. APA and Written Assignments: The School of Social Work faculty has adopted APA style as the preferred format for papers and publications. The best reference is the Publication Manual of the American Psychological Association, Sixth Edition (2010) that is available at most bookstores. The following web sites provide additional information: http://juno.concordia.ca/help/howto/apa.php and http://www.apastyle.org/apa-stylehelp.aspx . The UNC School of Social Work APA Quick Reference Guide (http://ssw.unc.edu/files/web/pdf/APA_Quick_Reference_Guide.pdf) also contains basic SOWO 810 Spring 2013 3 APA information and a section on how to avoid misuse of the words and ideas of others in your written work. Students are strongly encouraged to review the section on plagiarism carefully. All instances of academic dishonesty will result in disciplinary measures pre-established by the School of Social Work and the University. Course Assignments: 1. Assessments (25%): As in 510, assessments evaluate your mastery of online content and assigned readings. There will be two individual and three team in-class assessments. 2. In-Class Labs (15%): There will be 1 in-class quantitative lab. 3. Qualitative Report (30%) There will be a multi-component qualitative project that will result in a final qualitative report. Students will work on the project individually and with their teams, in and out of class. 4. Expert Lesson (15%) Each student will contribute to a “speed-dating” activity based on his or her new expertise in one evaluation method. 5. Peer Evaluation of Team Participation (15%) Students will evaluate the level and quality of their fellow team members’ participation mid-semester and at the end of the semester. Assignments and Course Performance Assessment: 2 Individual Assessments 3 Team Assessments 1 Quantitative Lab 1 Qualitative Report (team) Expert lesson Peer Evaluations Course Total 10% 15% 15% 30% 15% 15% 100% Grading System: Points <69 70 - 79 80 - 93 94 - 100 Grade F L P H SOWO 810 Spring 2013 4 Attendance and Participation: Attendance at all class sessions is expected; it is important to be on time so as not to disrupt class. We will be covering a great deal of information in each class. If you will not be able to attend a class, let the instructor know as soon as possible. It is your responsibility to obtain handouts, information about class content, and information about announcements, etc., from your classmates if you are unable to attend a class. Students with more than two absences, or those who are late to more than four classes, will receive an “L” unless they have made prior arrangements with the instructor. In order to be prepared for assessments and activities, students should complete required readings/review before coming to class. Policy on Incomplete and Late Assignments: A grade of Incomplete is given on rare occasions when there is sufficient reason to warrant it. It is the student’s responsibility to initiate a conversation with the instructor to request an Incomplete. Late assignments are strongly discouraged. To obtain permission to submit an assignment after the deadline, the student must seek approval from the instructor before the day that the product is due. If permission for late submission is not granted before breaking a deadline, the grade will automatically be reduced 10%, and another 10% reduction will occur each day, including weekends. In case of an emergency, a late paper may be accepted without penalty at the discretion of the instructor. Please note that technical difficulties are not an acceptable excuse for turning in an assignment or quiz late. All technical inquiries should be directed to the staff of the SSW Computing Information and Technology Unit (CITU). Please contact Manuel Garcia at megarcia@email.unc.edu or 919-962-6416. Policy on the Use of Electronic Devices in the Classroom: Please set your cell phones to silent. Use of laptops is permitted only as a tool for small group activities and in-class assignments or as an approved accommodation for students with disabilities. SOWO 810 Spring 2013 5 COURSE OUTLINE AND ASSIGNMENTS Week 1, Jan 15: Introduction to Course Students will have an in-class session this week Goal: Provide the foundation for the current course by reviewing evidence-based practice and types of program evaluation. Assignments to complete after class: As needed, review content on the following SOWO 510 Sakai Lessons Introduction to Evidence-Based Practice Orientation and Strategies for Needs Assessments Formative and Process Evaluation Assessment Outcome and Impact Evaluation Designing and Evaluation Readings to complete after class (review as needed): Rubin & Babbie: Chapter 2 Royse et al.: Chapters 1, 3, 5, 9 Hatry et al.: Step 1 January 22: No class (continue review of readings and Sakai lessons) Week 2, Jan 29 Quantitative Evaluation—Measurement Level and Univariate Statistics (1) Students will have an in-class session this week Goal: By the end of this session, students will be able to interpret univariate statistics and will be familiar with how they can be presented effectively. Students will be familiar with measurement levels and how variables can be coded in different measurement levels. Assignment to complete before class: Review content on the following Sakai Lessons: Measurement and Questionnaire Construction Quantitative Data Analysis (Part I) Readings to complete before class: Rubin & Babbie: Chapter 17 (pp. 290-293) SOWO 810 Spring 2013 6 Royse et al. Chapter 14 (pp. 341-351 & 362-370) Smith, L. A., McCaslin, R., Chang, J., Martinez, P., & McGrew, P. (2010). Assessing the needs of older gay, lesbian, bisexual, and transgender people: A service-learning and agency partnership approach. Journal of Gerontological Social Work, 53, 397401. Recommended: review the measurement level and univariate statistical sections of the statistics review powerpoint at the Sakai website (statsreview.ppt). Class time will include: **Individual Assessment** and **Team Assessment** on measurement level and univariate statistics Team activity on measurement level and interpreting tables with univariate statistics Team activity on developing a clinically useful outcome variable Week 3, Feb 5: Quantitative Evaluation—Bivariate Statistics (2) Students will not have an in-class session this week Goal: Students will become familiar with different types of bivariate analyses Readings to complete this week (focus on sections on bivariate statistics): Rubin & Babbie: Chapter 17 (pp. 293-301) Royse et al.: Chapter 14 (pp. 351-362) Hatry et al.: Step 6 (pp. 113-124) Week 4, Feb 12: Quantitative Evaluation—Multivariate Statistics (3) Students will have an in-class session this week Goal: By the end of this session, students will have had experience interpreting tables and choosing appropriate statistical tests for evaluating practice. Assignments to complete by this week: Review content on Sakai Lesson “Quantitative Data Analysis (Part II)” Readings to complete by this week (focus on sections on multivariate statistics): Rubin & Babbie: Chapter 17 (pp. 293-301) Royse et al. Chapter 14 (pp. 360-362) Hatry et al.: Step 6 (pp. 113-124) Marcynyszyn, L. A., Maher, E. J., & Corwin, T. W. (2011). Getting with the (evidencebased) program: An evaluation of the Incredible Years Parenting Training Program in child welfare. Children and Youth Services Review, 33, 747-757. (Focus on Tables 2 and 4 and the text that refers to them.) SOWO 810 Spring 2013 7 Class time will include: Team assessment on bivariate and multivariate statistics Team application activity on assessment questions Brief presentation on multivariate statistics Week 5, Feb 19: Quantitative Evaluation--Data Analysis (4) Half of the class will have an in-class session this week in the computer lab Goal: By the end of this session, students will be able to run basic statistical analysis with Microsoft Excel, run statistical tests of difference and association, graphically present findings, and interpret the results. Assignment: In class lab using Excel to conduct descriptive and inferential statistical analyses. Email your instructor your completed document by 5:00 PM on Friday Feb 22. Readings to complete before this class: Review readings from Weeks 2-4 as needed. Week 6, Feb 26: Quantitative Evaluation--Data Analysis (5) Half of the class will have an in-class session this week in the computer lab Goal: By the end of this session, students will be able to run basic statistical analysis with Microsoft Excel, run statistical tests of difference and association, graphically present findings, and interpret the results. Assignment: In class lab using Excel to conduct descriptive inferential statistical analyses. Email your instructor your completed document by 5:00 pm on Friday Feb 29. Readings to complete before this class: Review readings from Weeks 2-4 as needed. Week 7, March 5: Qualitative Evaluation—Intro and Overview (1) Students will have an in-class session this week Goal: By the end of this session, students will be able to design and ask qualitative questions for interviews and focus groups, and will be able to design a qualitative evaluation. SOWO 810 Spring 2013 8 Assignment to complete by this class: Review content on Sakai Lesson “Qualitative Evaluation – Methods and Data Collection” Readings to complete by this class: Royse et al.: Chapter 4 (pp. 83-104) Rubin & Babbie: Chapters 7 (pp. 102-110) and 13 (pp. 218-234) Kranke, D. A., Floersch, J., Kranke, B. O., & Munson, M. R. (2011). A qualitative investigation of self-stigma among adolescents taking psychiatric medication. Psychiatric Services, 62, 893-899. National Science Foundation. (1997). Overview of qualitative methods and analytic techniques. User-friendly handbook for mixed method evaluations, Chapter 3. Retrieved from http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm Class time will include: **Individual Assessment** and **Team Assessment** Team activity on planning a qualitative evaluation project (developing appropriate interview questions) March 12: Spring Break No class Week 8, March 19: Qualitative Evaluation—Data Collection (2) Students will have an in-class session this week Goal: Students will gain experience collecting qualitative data and gain intro-level knowledge of qualitative analysis. Readings to complete before class: Rubin & Babbie: Chapter 18 Charles, P., Orthner, D. K., Jones, A., & Mancini, D. (2006). Poverty and couple relationships: Implications for welfare policy. Marriage and Family Review, 39, 27-52 Jackson, K. F. (2012). Living the multiracial experience: Shifting racial expressions, resisting race, and seeking community. Qualitative Social Work: Research and Practice, 11, 42-60. National Science Foundation. (1997). Analyzing qualitative data. User friendly handbook for mixed-methods evaluations. Chapter 4. Retrieved from http://www.nsf.gov/pubs/1997/nsf97153/chap_4.htm Class time will include: Collection of interview data (each student interviews and is interviewed) Team activity related to readings SOWO 810 Spring 2013 9 Week 9, March 26: Qualitative Evaluation--Data Analysis (3) Students will not have an in-class session this week. Goal: Students will gain experience analyzing qualitative data, independently. Assignments to complete by this week: Review content on Sakai Lesson “Qualitative Analysis” Type up your interview notes. Email your interview notes to your team members by March 22 (or according to your own team’s plan). Conduct independent analysis of data of interview notes from each team member. Submit your interview notes and the results of your individual data analysis to the instructor by March 29th at 5 PM. Week 10, April 2: Qualitative Evaluation—Reporting in Writing (4) Students will have an in-class session this week. Goal: Students will gain experience providing evaluation results to stakeholders (based on team analyses) and learn how to write qualitative reports. Assignment to complete before class: Come to class with your field notes analyzed, including potential recommendations. Bring text books to class. Readings: Review readings on qualitative methods as needed. Read sections of Royse et al. chapter 15 Review Rubin & Babbie, Appendix B, Additional considerations when writing a qualitative report. Class time will include: Team-based analysis of data and themes Teams present their findings and recommendations Discuss reporting evaluation results Week 11, April 9: Focus Group Students will not have an in-class session this week. SOWO 810 Spring 2013 10 Goal: Become an expert in one area of program evaluation. Assignments to complete this week: Review content relevant to your assigned topic area Develop handout on topic E-mail your instructor a copy of your 1-2 page handout by 5:00 PM on Tuesday, April 9th – Your team will be provided feedback that will need to be incorporated into your handout. Week 12, April 16: Expert Rotation Students will have an in-class session this week Goal: (1/2 of the class) Practice sharing your expertise in one evaluation topic area. Assignment: Be comfortable in your topic area, revise your handout based on instructor feedback, develop assessment questions, and be prepared to present on your topic. Week 13, April 23: Expert Rotation Students will have an in-class session this week. Goal: (1/2 of the class) Practice sharing your expertise in one evaluation topic area. Assignment to be completed before class: Be comfortable in your topic area, revise your handout based on instructor feedback, develop assessment questions, and be prepared to present on your topic. Final team report due: Saturday May 4th at noon Peer Evaluation of Team Members’ Participation due: Saturday May 4th at noon SOWO 810 Spring 2013 11