THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL
SCHOOL OF SOCIAL WORK
SOWO 810: EVALUATION OF SOCIAL WORK INTERVENTIONS
Section 001
Spring Semester, 2012
Course Number: SOWO 810
Course Title: Evaluation of Social Work Interventions
Semester and Year: Spring 2012
Course Website: http://sakai.unc.edu
Location and Time: TTK #101, Mondays 10:30am-11:50am
Instructor: Johanna K.P. Greeson, PhD, MSS, MLSP
School of Social Work
Phone: 919-613-9878
Email: johanna@johannagreeson.com
Office Hours*: Mondays 12pm-1pm, and by appointment
*I don’t have a physical office in TTK. We will use the classroom.
Course Description:
Students gain advanced knowledge of evaluation methods and evidence-based practices that
can be applied to the evaluation of social work interventions in their areas of practice.
Learning Objectives:
Upon completion of this course, students will be able to draw from knowledge of research
methods and evidence-based practice, and work collaboratively with supervisors, colleagues,
peers, and consumers, to design, implement, and disseminate the results of an evaluation of
an intervention, program, service or policy that may be needed by current or prospective
consumers or program or policy planners. Specifically, students will demonstrate:
1) Skills for reviewing and presenting information about the importance of a social problem
addressed by an agency, including its relation to discrimination based on culture,
economic status, gender, religion, age, or sexual orientation;
2) The ability to consider this social problem in the context of current service delivery, and
critically determine a matter requiring evaluation related to client need, the process of
service delivery, or service outcomes;
3) The ability to design an appropriate evaluation, including identifying a sample,
determining a suitable study approach; and selecting appropriate, reliable, and valid
measures (including consideration of validity for the targeted population);
1
4) The ability to plan, conduct, and synthesize appropriate qualitative or quantitative
analyses;
5) The ability to apply knowledge of social work ethics and values to the design of a
practice intervention or other types of evaluations;
6) The ability to consider the implications of results for practice and policy (including
implications related to the agency’s success in working with clients from different
groups);
7) The ability to summarize evaluation results and program improvement plans in a concise
written document; and
8) The ability to present evaluation plans and program improvement plans to appropriate
audiences.
Available Resources:
1) SOWO 510 on line course materials
2) SOWO 510 texts:
Royse, D., Thyer, B. A., Padgett, D. K., & Logan, T. K. (2010). Program evaluation: An
introduction (5th ed). Belmont, CA: Brooks-Cole.
Rubin, R., & Babbie, E. R. (2010). Essential research methods for social work (2nd ed).
Belmont, CA: Brooks-Cole.
Hatry, H., Van Houten, T., Plantz, M. C., & Greenway, M. T. (1996). Measuring
program outcomes: A practical approach. Alexandria, VA: United Way of
America.
Accommodations for Students with Disabilities:
Students with disabilities, which affect their participation in the course, should notify the
instructor if they wish to have accommodations in instructional format, examination format, etc.,
considered. Accommodations and services are provided by Disability Services (Voice/TDD:
919.962.8300/966.4041). Learning Disability Services (919.962.7227) provides supportive
services for students with learning disabilities and attention-deficit/hyperactivity disorders.
Policy on Academic Dishonesty:
Academic dishonesty is cheating, contrary to the ethics of the social work profession, unfair to
your peers, and generally unacceptable. All written assignments per UNC policy must include
the following signed pledge: “I have neither given nor received unauthorized aid in preparing
this written work.”
The UNC School of Social Work APA Quick Reference Guide, available at
http://ssw.unc.edu/files/web/pdf/APA_Quick_Reference_Guide.pdf , contains basic information
on how to avoid misuse of the words and ideas of others in your written work. Students are
strongly encouraged to review the section on plagiarism carefully. All instances of academic
dishonesty will result in disciplinary measures by the School of Social Work and the University.
Teaching Methods and Accountability:
SOWO 810 is a hybrid class that combines in-person class sessions and on-line lessons. Students
are expected to meet for class as the syllabus outlines as well as complete all online activities.
2
Most in-class sessions will include class discussion of reading materials and critical content
issues/questions as well as skill-building activities.
Assignment Guidelines:
The School of Social Work faculty has adopted APA style as the preferred format for papers and
publications. The best reference is the Publication Manual of the American Psychological
Association, Sixth Edition (2010) that is available at most bookstores. The following web sites
provide additional information:
 http://juno.concordia.ca/help/howto/apa.php
 http://www.apastyle.org/apa-style-help.aspx
 http://owl.english.purdue.edu/owl/
 http://ssw.unc.edu/files/web/pdf/APA_Quick_Reference_Guide.pdf
Course Assignments:
1. Online Quizzes:
Most on-line Sakai lessons include an online quiz. The quizzes will test student knowledge of
online content/materials and the assigned readings for that lesson. There are 2 online quizzes,
one over the quantitative analysis content and the second over the qualitative data collection and
analysis content of the course. These quizzes comprise 20% of your course grade (10% each),
and cannot be made up after the due dates which are listed in the course outline.
 Students will have 1 hour to complete each quiz (10 questions)
 Quiz questions will be randomly assigned from a pool of questions
 You may re-take the quiz one additional time (your highest score will be recorded in
Sakai)
2. In Class Lab Assignments:
There will be 2 in-class labs throughout the semester. These labs are worth 30% of your overall
course grade (15% each). These labs will provide an opportunity for hands on experiences with
multiple aspects/components of the research and evaluation process.
3. Group Assignment Paper and Presentation: From Evidence to Program Improvement
Purpose: This assignment will give you the opportunity to think about how you can use
evaluation findings to improve your practice.
Products: The assignment asks each team for (1) a 7-10 page paper and (2) a 15-minute
presentation. The following is an overview of the components of the assignment, along with an
example for clarification.
Component
1.
Students will divide into teams based
on areas of practice/field placements.
Each team will identify one practice
issue. The issue can be related to
underserved groups (needs
assessment), organizational
challenges (process evaluation), or
the impact of interventions on users
Example
Your team consists of students who are doing their field placements in
agencies working with adults with mental illness. You are aware of the
recent federal lawsuit filed by Disability Rights NC, alleging that the
state is violating the Americans with Disabilities Act by housing
people with mental illness in nursing and long term care homes that are
inappropriately restrictive and inconsistent with best practice
guidelines. You are interested in better understanding the experiences
of your clients who live in nursing homes, and more effectively
3
Component
Example
(outcome evaluation). Evidence that
supports the issue is to be
summarized into an evaluation
statement.
addressing what may be their unmet needs. Evidence of these unmet
needs may be available from agency records or previous research on
the housing needs of older adults with mental illness.
2.
Each team will design a strategy for
studying and understanding this
issue. The strategy will specify the
research question, study design,
sampling, data collection
instruments, and data collection
procedures.
You decide to conduct a needs assessment. The study will be
descriptive in nature, and will include the following quantitative and
qualitative data:
 Demographics and prevalence: Using agency records, determine
the number of clients residing in nursing homes, create a
demographic profile of the group, and identify the homes in
which they live
 Quantitative data: You will find standardized instruments that
measure (a) satisfaction with housing, and (b) overall quality of
life
 Qualitative data: You will conduct qualitative interviews with
(a) a demographically representative sample of agency clients
residing in nursing homes, and (b) care staff in those homes, in
order to better understand issues and challenges from both
resident and provider perspectives
Copies of the draft data and instruments are included in the paper or
attached as appendices.
3.
Teams will generate three possible
alternatives for what might be found
in their research. For each alternative,
they will summarize possible
implications for program
improvement.
Possible alternative findings:
 There is widespread dissatisfaction with the nursing home living
arrangement. Clients report generally poor quality of life,
especially in the areas of social support and access to recreational
resources. Staff report feeling unprepared to effectively address
the needs of residents with mental illness.
Implications: Training with care home staff to improve
responsiveness to residents with mental illness; agency
attention to meeting the social and recreational needs of
clients; legislative advocacy to revise policies around housing
for people with mental illness.
 There is a range of perspectives reported, and findings seem to
vary by care home. Interview responses suggest that care staff and
availability of transportation make the most difference in
determining how a given client experiences his/her life in the
nursing home.
Implications: Targeted work with specific care homes;
possible relocation of clients to homes that seem to provide
superior care to individuals with mental illness.
 There is a range of perspectives reported, and findings seem to
vary by age of client. Younger clients are far more distressed with
living in a nursing home than are older clients. Interview
responses suggest that isolation from peers and the restrictiveness
of rules are the most problematic aspects of nursing home life.
Implications: Increased agency focus on peer support and
interaction for young adults with mental illness; investigation
of supportive housing alternatives for this especially
vulnerable population
4
Attendance and Participation:
Prompt attendance is expected because we have a reduced number of in-class sessions. If you
will not be able to attend a class, let the instructor know as soon as possible. It is your
responsibility to obtain handouts, information about class content, and information about
announcements, etc., from your classmates if you are unable to attend a class. Students with
more than two absences will receive an “L” unless they have made prior arrangements with the
instructor. In order to fully participate in and benefit from each class session, students should
complete required readings before class and come to class prepared to discuss them.
Course Grading and Performance Assessment:
Grading:
Assignment
Quizzes
Labs
Evaluation Paper
Presentation
Course Total
%
20%
30%
40%
10%
100%
Scale:
Points
<69
70 - 79
80 - 93
94 - 100
Grad
F
L
P
H
Late Assignments:
Late assignments are strongly discouraged. In case of a dire, life-threatening emergency, a late
assignment may be accepted at the discretion of the instructor. If permission for late submission
is not granted before breaking a deadline, the grade will be reduced 10%, and another 10%
reduction will occur each day (including weekends) that the assignment is late.
Please note that technical difficulties are not an acceptable excuse for turning in an assignment or
quiz late. All technical inquiries should be directed to the staff of the SSW Computing
Information and Technology Unit (CITU). Please contact Manuel Garcia at 919.962.6416 or
megarcia@email.unc.edu for assistance.
Electronic Devices:
Please set your cell phones to vibrate. Use of laptops is permitted as a tool for small group
activities and in-class assignments or as an approved accommodation for students with
disabilities.
5
COURSE OUTLINE AND ASSIGNMENTS
Week 1 (Jan 9, 2012): Introduction to Course and Assignments (in-class session)
Goal: We will review the procedures for the online course lessons, principles of evidence-based
practice (EBP) and authority-based practice (ABP), and understand the course assignments.
Assignments:
 Review Sakai Lesson “Introduction to Evidence-Based Practice” (no quiz required)
 Group assignments will be made by the instructor
Relevant Readings:
Rubin & Babbie: Chapter 2
Royse et al.: Chapters 1
Hatry et al.: Step 1
Jan. 16, 2012: Martin Luther King, Jr. Day – NO CLASS
Please use this time to meet informally with you group to start discussing your group paper
assignment.
Week 2 (Jan 23, 2012): Evidence Based Practice Methods Review (in-class session)
Goal: By the end of this lesson, students will understand how to conduct a needs, process, and
outcomes assessment of their organization’s target population.
Assignments:
 Review Sakai Lessons:
1. Orientation and Strategies for Needs Assessments
2. Formative and Process Evaluation Assessment
3. Design and Evaluation
4. Outcome and Impact Evaluation
Relevant Readings to Review:
Royse et al.: Chapters 3, 5, 7, 9 and 10
Week 3 (Jan 30, 2012): Quantitative Data Analysis-1 (no in-class session)
Goal: By the end of the lesson students will be able to define how statistical significance impacts
research and practice. Students will be able to code data in a meaningful way, interpret
descriptive statistics, and describe univariate analysis.
Assignment:
 Review Sakai Lesson:
1. Measurement and Questionnaire Design
2. Quantitative Data Analysis (Part I)
6
Readings:
Rubin & Babbie: Chapter 17 (pp. 290-293)
Royse et al. Chapter 14 (pp. 341-351 & 362-370)
Smith, L. A., McCaslin, R., Chang, J., Martinez, P., & McGrew, P. (2010). Assessing the
needs of older gay, lesbian, bisexual, and transgender people: A service-learning
and agency partnership approach. Journal of Gerontological Social Work, 53, 397401.
Week 4 (Feb 6, 2012): Quantitative Data Analysis-2 (in-class session)
Goal: By the end of the session, students will be able to describe multivariate analysis. Students
will also understand how statistical results from research should impact social work practice.
Assignments:
 Review Sakai Lesson:
1. Quantitative Data Analysis (Part II)
 Due Friday, Feb 10 Online Quiz on Saki Lessons “Quantitative Data Analysis (Part I)”
AND “Quantitative Data Analysis (Part II)”
 Submit Evaluation Paper Outline Worksheet # 1: Introduction and Literature Review at the
beginning of class. Please bring a hard copy to class.
Readings:
Rubin & Babbie: Chapter 17 (pp. 293-301)
Royse et al. Chapter 14 (pp. 351-362)
Hatry et al.: Step 6 (pp. 113-124)
Marcynyszyn, L. A., Maher, E. J., & Corwin, T. W. (2011). Getting with the (evidencebased) program: An evaluation of the Incredible Years Parenting Training Program
in child welfare. Children and Youth Services Review, 33, 747-757.
Week 5 (Feb 13, 2012): Quantitative Data Analysis-3 (First half in-class session)
Goal: By the end of the lesson students will be able to run basic statistical tests of difference and
association using Microsoft Excel, graphically present findings, and interpret the results.
Half of the students (Ballance – Herbin) will report to class this week.
Assignment:
 In class lab using Excel to conduct descriptive statistical analyses
The lab is due by 9:00am on Monday, February 2oth. Please submit via e-mail.
Readings:
Review readings from Weeks 3 and 4 as needed.
Excel lab guides to be provided.
7
Week 6 (Feb 20, 2012): Quantitative Data Analysis-4 (Second half in-class session)
Goal: By the end of the lesson students will be able to run basic statistical tests of difference and
association using Microsoft Excel, graphically present findings, and interpret the results.
Half of the students (Hubbard – Walker) will report to class this week.
Assignment:
 In class lab using Excel to conduct inferential statistical analyses
The lab is due by 9:00am on Monday, February 27th. Please submit via e-mail.
Readings:
Review readings from Weeks 3 and 4 as needed.
Excel lab guides to be provided.
Week 7 (Feb 27, 2012): Qualitative Data Collection Methods-1 (no in-class session)
Goal: By the end of this lesson, students will be able to understand how to conduct an evaluation
using qualitative methods, including in-person interviews, focus groups and participant
observations.
Assignment:
 Review Sakai Lesson
1. Qualitative Evaluation – Methods and Data Collection
Readings:
Royse et al.: Chapter 4 (pp. 83-104)
Rubin & Babbie: Chapters 7 (pp. 102-110) and 13 (pp. 218-234)
Kranke, D. A., Floersch, J., Kranke, B. O., & Munson, M. R. (2011). A qualitative
investigation of self-stigma among adolescents taking psychiatric medication.
Psychiatric Services, 62, 893-899.
National Science Foundation. (1997). Overview of qualitative methods and analytic
techniques. User-friendly handbook for mixed method evaluations, Chapter 3. Retrieved
from http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm
Mar 2 – Mar 9, 2012: Spring Break (no class)
Week 8 (Mar 12, 2012): Qualitative Data Analysis-2 (in-class session)
Goal: By the end of this session, students will be able to design, conduct and analyze qualitative
interview data.
Assignments:
 Review Sakai Lesson:
1. Qualitative Analysis
8


Due Friday, March 16 Online Quiz on Sakai lessons “Qualitative Evaluation –
Methods and Data Collection” AND “Qualitative Analysis”
Submit Evaluation Paper Outline Worksheet #2: Research Questions at the beginning of
class. Please bring a hard copy to class.
Readings:
Rubin & Babbie: Chapter 18
Charles, P., Orthner, D. K., Jones, A., & Mancini, D. (2006). Poverty and couple
relationships: Implications for welfare policy. Marriage and Family Review, 39, 27-52
Jackson, K. F. (2012). Living the multiracial experience: Shifting racial expressions,
resisting race, and seeking community. Qualitative Social Work: Research and
Practice, 11, 42-60.
National Science Foundation. (1997). Analyzing qualitative data. User friendly handbook
for mixed-methods evaluations. Chapter 4. Retrieved from
http://www.nsf.gov/pubs/1997/nsf97153/chap_4.htm
Week 9 (Mar 19, 2012): Qualitative Data Analysis-3 (in-class session)
Goal: By the end of this session, students will be able to design questions and collect data using
qualitative interviewing.
Assignment:
 In class lab (Part I) using in-person interviews and recording procedures
The lab Parts I and II are due by 9:00am on Monday, April 2nd. Please submit
via email.
Readings:
None
Week 10 (Mar 26, 2012): Qualitative Analysis-4 (in-class session)
Goal: By the end of the lesson, students will be able to code and interpret qualitative interview
data as well as summarize results for reporting.
Assignment:
 In class lab (Part II) using qualitative interview data for coding and data analysis.
The lab Parts I and II are due by 9:00am on Monday, April 2nd. Please submit
via email.
Readings:
None
9
Week 11 (Apr 2, 2012) : Group Work Sessions (meet w/ group—consultation available)
Goal: By the end of this week, students will have summarized their data collection and analysis
strategies and begun to propose practice solutions based on their alternative analyses.
Assignment:
 Small group work during and outside of class
Readings:
Appropriate to the students papers
Week 12 (Apr 9, 2012): Group Work Sessions (meet w/ group—consultation available)
Goal: By the end of this week, students will proposed practice solutions based on their
alternative analyses, prepared a summary report, and begun work on their class presentations.
Assignment:
 Small group work during and outside of class
Readings:
Appropriate to the students papers
Week 13 (Apr 16, 2012): Student Presentations (in-class session)
Goal: By the end of this session students will effectively present program improvement
proposals to their peers and describe the potential impact on social practice. Students will also
practice providing feedback to peers on ideas for improving evaluation and practice designs.
Assignments:
 Final Due: Evaluation Paper and Presentation
 Student Presentations
 Peer feedback
 Course evaluation
10
Evaluation Paper Outline
Due February 6, 2012
Worksheet #1: Introduction and Literature Review
Introduction
State the problem that to be addressed by your evaluation.
What is the prevalence of the problem (e.g., the number of people in a population who have the
problem)? Identify 1-2 sources that discuss the prevalence of the problem
What are the costs (social and financial) of the problem? Identify 1-2 sources that discuss the
social and financial costs of the problem.
Literature Review
Identify three peer-reviewed journal articles that you will be using in your literature review and
briefly summarize the findings of each article.
Article #1 Reference & Summary:
Article #2 Reference & Summary:
Article #3 Reference & Summary:
11
Evaluation Paper Outline
Due March 12, 2012
Worksheet #2: Research Questions and Methods
Research Question(s)
List the specific and measurable research question(s) that will guide your evaluation. State your
hypothesis for each research question.
Methods
Evaluation Design: Identify the method (e.g., needs assessment, process evaluation, outcome
evaluation) you plan to use and briefly describe the design.
Study sample: Briefly describe the sample population and specific sampling strategy you plan to
use.
Participant recruitment: Briefly describe how you will identify and recruit your sample
participants (or obtain existing records).
Data collection procedures: Briefly describe the strategy you will use to collect the data (e.g.,
focus groups, interviews, record abstraction) and the procedures to be used:
Measures: Briefly describe the measures you plan to use.
12
Evaluation Paper and Presentation
For this paper, students will develop a proposal for an evaluation project related to a specific
social work interest. Each group of students (4 – 5 students per group) will present a summary of
their evaluation to the class in 15 minutes. This 7-10 page double-spaced paper and brief class
presentation is worth 50% of your overall grade.
The paper should include the following seven components:
1. Introduction: (1/2 – 1 page) The introduction should include a statement of the problem
to be addressed by your evaluation and information concerning the prevalence (e.g., the
number of people in a population who have the problem), and costs (social and financial)
of the problem. This section should conclude with a discussion of why the problem is
important to the field of social work.
2. Literature Review: (1 - 2 pages) Empirical research relating to relevant interventions,
evaluations, programs or policies and their outcomes specific to your population and
problem of interest should be reviewed concisely and critically in this section. Also, in
one or two sentences describe your plan for addressing the problem with an evaluation or
intervention.
3. Research Question(s): (1/2 page) List the specific and measurable research question(s)
that will guide your evaluation. In addition, please state your hypothesis for each
question.
4. Methods: (2 - 3 pages) This section will present the plan for how the research question(s)
will be addressed. Please provide separate subheadings for each of the following:





Evaluation design: Identify the method (e.g., needs assessment, process
evaluation, outcome evaluation) and describe the design.
Study sample: Describe the sample population and specific sampling strategy.
Participant recruitment: Describe how you will identify and recruit your sample
participants (or obtain existing records).
Data collection procedures: Describe the strategy you will use to collect the data
(e.g., focus groups, interviews, record abstraction) and the procedures to be used.
Measures: Describe the measures and be sure to include information about the
reliability and validity of each (if available). Please attach the actual survey(s) or
instrument(s) as an appendix if you have access to them, otherwise the types of
questions included.
5. Results and Implications: (2 - 3 pages) Discuss three possible alternatives for what
might be found from your evaluation results. For each alternative, summarize possible
implications for program improvement (see syllabus for examples).
13
6. Conclusion: (1 page) Discuss the strengths and limitations of your proposed evaluation.
Discuss suggestions for further evaluation (e.g., given the limitations of your initial
design, how the next evaluation can be improved).
7. References & Writing: Provide an accurate list of all cited sources in APA format
(minimum of 5 peer-reviewed journal articles). The paper should be clearly and concisely
written and checked for errors prior to submission.
Grading Rubric: Required Components and Points
Possible
Introduction
3
Literature Review
5
Research Question(s)
4
Methods
10
Results and Implications
10
References and Writing
5
Conclusion
3
Class Presentation
10
Total Possible Points
50
14