THE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL
SCHOOL OF SOCIAL WORK
COURSE NUMBER:
COURSE TITLE:
SEMESTER:
DAY/TIME:
INSTRUCTOR:
OFFICE HOURS:
SOWO 810 (SECTION 957)
Evaluation of Social Interventions
Fall 2011
Fridays 10:30 – 11:50 am
Laurie Selz Campbell, MS, CPRP
Tate Turner Kuralt Bldg. Room 354
(919) 843-6394
lauriesc@unc.edu
Mondays 12:30 – 1:30 and by appointment, phone, or email as
needed.
COURSE DESCRIPTION:
Students gain advanced knowledge of evaluation methods and evidence-based
practices that can be applied to the evaluation of social work interventions in their
areas of practice.
LEARNING OBJECTIVES:
Upon completion of this course, students will be able to draw from knowledge of
research methods and evidence-based practice, and work collaboratively with
supervisors, colleagues, peers, and consumers, to design, implement, and disseminate
the results of an evaluation of an intervention, program, service or policy that may be
needed by current or prospective consumers or program or policy planners.
Specifically, students will demonstrate:
1) Skills for reviewing and presenting information about the incidence, prevalence
and social and personal costs of a social problem addressed by an agency,
including its relation to discrimination based on culture, economic status,
gender, religion, age, or sexual orientation;
2) The ability to consider this social problem in the context of current service
delivery, and critically determine a matter requiring evaluation related to client
need, the process of service delivery, or service outcomes;
3) The ability to design an appropriate evaluation, including identifying a sample,
determining a suitable study approach; identifying appropriate data collection
procedures; and selecting appropriate, reliable, and valid measures (including
consideration of validity for the targeted population);
4) The ability to plan, conduct, and synthesize appropriate qualitative or
quantitative analyses;
5) The ability to apply knowledge of social work ethics and values to the design of
a practice intervention or other types of evaluations;
6) The ability to consider the implications of results for practice and policy
(including implications related to the agency’s success in working with clients
from different groups);
1
7) The ability to summarize evaluation results and program improvement plans in
a concise written document; and
8) The ability to present evaluation plans and program improvement plans to
appropriate audiences.
AVAILABLE RESOURCES:
1. Sakai course materials
2. SOWO 510 texts:
Royse, D.; Thyer, B.A.; Padgett, D.K. & Logan, T.K. (2010) Program evaluation: An
introduction (5th Edition). Belmont, CA: Brooks-Cole.
Rubin, R. & Babbie, E.R. (2010). Essential research methods for social work (2nd
Edition). Belmont, CA: Brooks-Cole.
Hatry, H., van Houten, T., Plantz, M.C. & Greenway, M.T. (1996). Measuring
program outcomes: A practical approach. Alexandria, VA: United Way of
America.
TEACHING METHODS AND ACCOUNTABILITY:
SOWO 810 is a hybrid/blended class that includes a combination of in-person and online sessions. Students are expected to meet for class as the syllabus outlines and to
complete all online activities. In-class sessions will include discussion of critical
content issues/questions and skill-building activities.
POLICIES ON THE USE OF ELECTRONIC DEVICES IN THE CLASSROOM:
I expect that we will all be invested in creating a learning environment of respect and
engagement. During class, cell phones and other devices should be turned off or
transferred to voice mail. I welcome the use of laptops in class for taking notes or
completing small group tasks. However, I ask that you use them only for relevant
activities – not for checking email or surfing the Web. If students are observed to be
engaged in these activities, I will need to strictly limit the use of electronics. Your
attention is an important sign of respect to your colleagues, and an important part of
your learning.
COURSE ASSIGNMENTS:
The following is a summary of assignments for the class:
Online Quizzes (2 @ 10 points each):
As in 510, quizzes assess your mastery of online content and assigned readings. There
are 2 quizzes, the 1st covering quantitative methods and the 2nd covering qualitative
methods. Questions are randomly selected from a pool. You have 1 hour to complete
each quiz (10 questions), and may re-take it 1 additional time (the highest score is
recorded in Sakai). Quizzes cannot be made up after the due dates.
In-Class Labs (4 @ 10 points each):
There will be 4 in-class labs throughout the semester. These labs will provide an
opportunity for hands on experiences with multiple aspects/components of the
research and evaluation processes.
2
Group Paper (30 points) and Presentation (10 points): From Evidence to
Program Improvement
This assignment will give you the opportunity to think about how you can use
evaluation findings to improve your practice. Each team of 3 or 4 students will
produce (1) a 7-10 page paper, and (2) a 20-minute presentation to an important
stakeholder audience (with the class serving as a proxy for that audience). The
following is an overview of the assignment, along with an example for clarification.
Assignment Step
Example
You will divide into teams based on
areas of practice. Each team will
identify one practice issue that
warrants study. The issue can be
related to underserved groups
(needs assessment), organizational
challenges (process evaluation), or
the impact of interventions
(outcome evaluation).
Your team consists of students doing their field placements
with adults with severe and persistent mental illness. You
are aware of the recent federal lawsuit filed by Disability
Rights NC, alleging that the state is violating the Americans
with Disabilities Act by housing young adults with mental
illness in nursing and long term care homes. The rationale
for the lawsuit is that this housing arrangement is (a)
inappropriately restrictive and inconsistent with best
practices for young adults with mental illness, and (b) less
than optimal and potentially unsafe for older adults. You
are interested in better understanding the experiences of
your clients who live in such homes, and more effectively
addressing what may be their unmet needs. Evidence of
these needs may be available from agency records or
previous research.
Evidence supporting the
relevance and importance of the
issue will be summarized and
integrated into an Evaluation
Justification Statement (Part 1).
Your team will design a strategy for
studying and understanding the
chosen issue. The strategy will
specify the research question,
study design, sampling and
recruitment process, data
collection instruments, and data
collection procedures.
A description of the above
components will be included in
the Methods Section (Part 2).
Copies or descriptions of the
instruments will be included or
attached as Appendices.
Your team will generate three
possible alternatives for what
might be found in your research.
For each alternative, you will
You decide to conduct a needs assessment. The study will
be descriptive in nature, and will include the following
quantitative and qualitative data:
 Demographics and prevalence: Using agency records,
you will determine the number of mental health clients
residing in nursing homes, create a demographic profile
of the group, and identify the homes in which they live.
 Quantitative data: You will identify standardized
instruments that measure (a) satisfaction with housing,
and (b) overall quality of life.
 Qualitative data: You will conduct qualitative
interviews with (a) a demographically representative
sample of mental health clients residing in nursing
homes, and (b) care staff in those homes, in order to
better understand issues from both perspectives.
Alternative finding 1:
There is widespread dissatisfaction with the nursing home
living arrangement. Clients report poor quality of life,
especially in areas of social support and access to resources.
3
Assignment Step
summarize possible implications
for program improvement.
A description of the alternatives
and implications will be
included in the Alternative
Findings and Implications
Section (Part 3).
Example
Staff report feeling unprepared to effectively address the
needs of residents with mental illness.
Implications: Training with staff to improve
responsiveness to residents with mental illness;
agency attention to meeting social and resource
needs; legislative advocacy to revise policies around
housing for people with mental illness.
Alternative finding 2:
There is a range of perspectives reported, and findings vary
by care home. Interview responses suggest that (a) care
staff and (b) availability of transportation seem to have the
most influence on perceived quality of life.
Implications: Targeted work with specific care homes;
possible relocation of clients to homes that provide
superior care to individuals with mental illness.
Alternative finding 3:
There is a range of perspectives reported; findings vary by
age of client, with younger clients more distressed than
older clients. Interview responses suggest that isolation
from peers and restrictiveness of rules are the most
problematic aspects of nursing home life.
Implications: Increased agency focus on peer support
and interaction for young adults with mental illness;
investigation of supportive housing alternatives.
Teams will identify a plan for
dissemination of their findings.
This plan will be included in the
Dissemination Plan Section
(Part 4).
Your team decides to develop the following:
 A one-page fact sheet for legislators and policy makers
A policy brief suitable for use by mental health advocacy
organizations (e.g. NAMI, Mental Health America)
 A brief training presentation suitable for nursing and
long term care home administrators, focused on how
best to support persons with mental illnesses who may
be residing in their homes;
 A similar training presentation focused on mental
health staff who may be supporting clients in nursing
and long term care homes.
 A presentation proposal to a statewide mental health
conference.
All written assignments must be typed and follow APA format for citations, quotes,
and the like. Students should refer to the Publication Manual of the American
4
Psychological Association (6th Ed.). You can also find a self-paced tutorial for APA
style at http://www.lib.unc.edu/instruct/citations/apa/index.html
The following web sites provide additional information:
 http://www.apastyle.org/elecref.html (APA Style for material in electronic
formats, 6th Ed.)
 http://owl.english.purdue.edu/handouts/research/r_apa.html (general
information about documentation using APA style, 6th Ed.)
SCORING AND GRADING:
Quizzes
Labs
Evaluation Paper
Presentation
Course Total
20%
40%
30%
10%
100%
<69
70 - 79
80 - 93
94 - 100
F
L
P
H
POLICY ON INCOMPLETES AND LATE ASSIGNMENTS:
Assignments are due at the beginning of class on the day noted. You must notify me
at least 3 days before a due date if you would like to be granted an extension. If this
does not happen, you will lose 10% of the assignment’s points per day (including
weekends, and including the date on which the assignment was due, if you submit it
after the beginning of class).
Incompletes may be granted if (a) there are extreme and unforeseeable circumstances
that affect your ability to complete the semester’s work, and (b) you meet with me in
advance to develop a plan and timeline for completing your work.
POLICY ON ACADEMIC DISHONESTY:
I assume that all students follow the UNC Honor Code. Please ensure that the Honor
Code statement “I have neither given nor received any unauthorized assistance in
completing this assignment”, with your signature, is on all assignments. In keeping
with the Honor Code, if reason exists to believe that academic dishonesty has
occurred, a referral will be made to the Office of the Student Attorney General for
investigation and further action as required. Please refer to the APA Style Guide, the
SSW Manual, and the SSW Writing Guide for information on attribution of quotes,
plagiarism, and the appropriate use of assistance in preparing assignments.
ACCOMMODATIONS FOR STUDENTS WITH DISABILITIES:
Students with disabilities, which affect their participation in the course, should notify
the instructor if they wish to have special accommodations in instructional format,
examination format, etc., considered. Accommodations and services are provided by
Disability Services (Voice/TDD 962-8300; 966-4041). Learning Disability Services
(962-7227) provides supportive services for students with learning disabilities and
attention-deficit/hyperactivity disorders.
5
COURSE OUTLINE AND ASSIGNMENTS
Date
Topics & Readings
Due
PART I: REVIEW OF EVIDENCE BASE IN PRACTICE
Aug 26
Sept 2
Weeks 1 & 2
Meet face-t0-face
today
No class meeting – instructor out of town
Topics: Overview of course & review of EBP methods
Sakai materials to review:
Lessons 2, 3, 4, 5, 6 (no quizzes required)
Readings to review:
Rubin & Babbie: Chapter 4 and Appendix A
Royse et al.: Chapters 1, 3, 5, 7, 9, 14
Hatry et al.: Step 1
Nothing due,
but please bring
your final
proposals from
SOWO 510 to
class!
PART II: QUANTITATIVE DATA ANALYSIS
Sept 9
Week 3
Topics: Statistical significance, descriptive statistics, univariate
statistics, coding data
Meet face-to-face
today
Activity: Meet with team members for final project.
Sakai materials: Lessons 8 & 9
Readings to review:
Royse et al. Chapter 14 (pp. 351-359 & 362-370)
Rubin & Babbie: Chapter 17 (pp. 290-300)
Sept 16
Week 4
Topics: Bivariate statistics, interpretation of statistical findings
Sakai materials: Lesson 10
Online – No class
meeting today
Readings to review:
Hatry et al.: Step 6 (pp. 113-124)
Sept 23
Week 5
Topics: Basic statistical analysis with Microsoft Excel, graphical
presentation of findings, interpretation of results.
Meet face-t0-face
today
Activity: In-class Lab #1
Online quiz
covering Weeks
3 & 4 due by
Monday, 9/19 at
5:00 pm
Draft of Part 1
of final paper
(Issue &
Justification)
Readings:
Review readings on coding and quantitative data as needed.
Excel guides to be provided
Sept 30
Week 6
Topics: Statistical tests of difference and association with
Microsoft Excel, interpretation of results.
Meet face-t0-face
today
Activity: In-class Lab #2
In-class Lab #1
writeup
Readings:
Review readings on coding and quantitative data as needed.
Excel guides to be provided.
6
Date
Topics & Readings
Due
PART III: QUALITATIVE DATA COLLECTION AND ANALYSIS
Oct 7
Week 7
Topics: Overview of qualitative evaluation strategies, including
in-person interviews, focus groups and participant observations.
Online – No class
meeting today
Sakai materials: Lesson 11
In-class Lab #2
writeup
Readings to review:
Royse et al.: Chapter 4 (pp. 83-104)
Rubin & Babbie: Chapters 9 (pp. 102-110) and 15 (pp. 218-233)
National Science Foundation. (1997). Overview of qualitative
methods and analytic techniques. User-Friendly
Handbook for Mixed Method Evaluations, Chapter 3.
Available at
http://www.nsf.gov/pubs/1997/nsf97153/chap_3.htm
Oct 14
Week 8
Topics: Designing, conducting, and analyzing qualitative
interview data.
Meet face-t0-face
today
Activity: Meet with team members for final project.
Online quiz
covering Weeks
7 & 8 due by
Monday, 10/17
at 5:00 pm
Sakai materials: Lesson 12
Readings to review:
Rubin & Babbie: Chapter 18 (pp. 304-312)
Smokowski, P., Reynolds, A.J., & Bezruczko, N. (1999).
Resilience and protective factors in adolescence: An
autobiographical perspective from disadvantaged youth,
Journal of School Psychology, 37, 425-448.
Charles, P., Orthner, D.K., Jones, A. & Mancini, D. (2006).
Poverty and couple relationships: Implications for welfare
policy. Marriage and Family Review, 39, 27-52
National Science Foundation (1997). Analyzing qualitative
data. User friendly handbook for mixed-methods
evaluations. Chapter 4. Retrieved November 4, 2009
from,
http://www.nsf.gov/pubs/1997/nsf97153/chap_4.htm
Oct 21
Happy Fall Break!!!
Oct 28
Week 9
Topics: Designing qualitative interview questions and
conducting qualitative interviews
Meet face-t0-face
today
Activity: In-class lab #3
Draft of Part 2
of final paper
(Methods)
Readings:
Review readings on qualitative methods and data as needed.
Nov 4
Week 10
Topics: Coding and interpreting qualitative data
In-class Lab #3
writeup
Activity: In-class lab #4
Meet face-t0-face
today
Readings:
Review readings on qualitative methods and data as needed.
7
Date
Topics & Readings
Due
PART IV: DISSEMINATING EVALUATION FINDINGS
Nov 11
Week 11
Online – No class
meeting today
Topics: From findings to recommendations; disseminating
evaluation findings for reports and publications; components of
summary reports
In-class Lab #4
writeup due by
5:00 pm
Sakai materials: Lesson 13
Readings:
Royse et al., Chapter 15
Thyer, B. (2002). How to write up a social work outcomes
study for publication, Journal of Social Work Research
and Evaluation, 3(2), 215-224.
Nov 18
Week 12
Meet face-t0-face
today
Topics: From findings to recommendations; disseminating
evaluation findings for reports and publications; components of
summary reports
Activity: Meet in teams with instructor consultation for final
papers/presentations
Draft of Part 3
of final paper
(Alternative
Findings &
Implications)
Readings:
None
Nov 25
Happy Thanksgiving!!!
Dec 2
Week 13
Final Presentations
Final Papers &
Presentations
8