The University of North Carolina at Chapel Hill
School of Social Work
SOWO 810 EVALUATION OF SOCIAL WORK INTERVENTIONS
Section 002
Spring Semester, 2010
Instructor
Natasha K. Bowen, Ph.D.
Room 245E
Tate Turner Kuralt Building
CB #3550,
School of Social Work, Chapel Hill, NC 27599-3550
Phone: (919) 843-2434
Email: nbowen@email.unc.edu
Class Meeting Times & Office Hours
Class meets on Monday 10:30-11:50 am in Room 114
Office hours are Monday 12:00 – 1:00 or by appointment
Course Description
Students apply knowledge of research methods and evidence-based practice to the
evaluation of social work interventions by developing and conducting a detailed evaluation
of a specific social work intervention.
Learning Objectives
Upon completion of this course, students will be able to draw from knowledge of research
methods and evidence-based practice, and work effectively with supervisors, colleagues, peers,
and consumers, to design, conduct, and disseminate the results of an evaluation of an
intervention, program, service or policy implemented by their placement agency or needed by
current or prospective consumers or program or policy planners. Specifically, students will
demonstrate:
1) Skills for reviewing and presenting information about the importance of a social problem
addressed by their agency, including its relation to discrimination based on culture, economic
status, gender, religion, age, or sexual orientation;
2) The ability to consider this social problem in the context of current service delivery, and
critically determine a matter requiring evaluation related to client need, the process of service
delivery, or service outcomes;
1
3) The ability to design an appropriate evaluation, including describing a logic model,
identifying a sample, determining a suitable study approach; and selecting appropriate,
reliable, and valid measures (including consideration of validity for the targeted population);
4) The ability to plan, conduct, and synthesize appropriate qualitative or quantitative analyses;
5) The ability to apply knowledge of social work ethics and values to the design of practice
intervention or other types of evaluations;
6) The ability to consider the implications of results for practice and policy (including
implications related to the agency’s success in working with clients from different groups);
and
7) The ability to summarize all of this material in a concise written document.
Available Resources
1) SOWO 510 on line course materials
2) SOWO 510 texts:
Royse, D.; Thayer, B.A.; Padgett, D.K. & Logan, T.K. (2006) Program evaluation: An
introduction. (4th Edition) Belmont, CA: Brooks-Cole.
Rubin, R. & Babbie, E.R. (2007). Essential research methods for social work. Belmont,
CA: Brooks-Cole.
Hatry, H., van Houten, T., Plantz, M.C. & Greenway, M.T. (1996). Measuring program
outcomes: A practical approach. Alexandria, VA: United Way of America.
Course Assignments
The student’s primary assignment in this course is an evaluation of an intervention, program,
service, or policy related to an important social problem that is addressed by the student’s
practice setting. Students are expected to evaluate consumer/program needs or the process and
outcomes of practice in their field placement, in the form of a needs, process, outcome, or other
type of assessment.
This semester-long evaluation will include:
 a focused literature review of the problem being addressed,
 formulation of the practice or policy related question(s),
 development of the evaluation design and sampling strategy, including selection
of measures,
 data collection,
 data analysis,
 writing a final evaluation report, and
 presenting the report.
The final project will be a written evaluation report using the format presented by Royse, Thyer,
& Padgett (Chapter 15); it will include an executive summary, introduction, review of the
literature, methodology, results, discussion, references, and appendices.
2
Criteria for Grading
Component
Proportion
Phase Assignments 1-5
30%
Final Paper (Evaluation Report)
60%
Student Presentation
5%
Class Participation
5%
Description of Course Assignments
Phased Assignments
To ensure that the project is started promptly and can be completed during the semester, students
will complete five phased assignments. These assignments can be used to develop the final
evaluation report.
Phase I – Description of social problem
In no more than one double-spaced page, briefly describe the practice or policy related problem
you will address and its importance. Include at least three citations. In addition, clearly state
your draft evaluation questions.
Length: Maximum 1 page
Due: Week 2
Value: 6%
Phase II –Justification of evaluation
In no more than two double-spaced pages, write your revised evaluation questions and a brief
description and justification of the evaluation you plan to conduct, include ethical considerations
as appropriate, and evidence that suggestions for changes in service delivery are possible if
indicated by the results. Include at least three citations.
Length: Maximum 2 pages
Due: Week 3
Value: 6%
Note for Phases I and II: The final paper will include a literature review with at least six
scholarly citations that are pertinent to the evaluation. Pay special attention to the literature on
relevant evidence-based practices in the area of study.
Phase III – Evaluation design
In no more than one double-spaced page, describe your evaluation design. Indicate the type of
evaluation you will implement (e.g., needs assessment, formative evaluation, process evaluation,
outcome evaluation), your research design, and the strategy you will use to collect the data (e.g.,
focus groups, interviews, chart abstraction). Make sure to include your targeted sample size and
how you will identify and recruit your sample participants (or obtain existing records).
Length: Maximum 1 page
Due: Week 4
Value: 6%
Phase IV - Draft instrument
Obtain/develop a draft of the data collection instrument you propose to use; this may be an
existing instrument, one your create, or a combination of the two.
Length: As needed
Due: Week 5
Value: 6%
3
Phase V- Draft data analysis plan
In no more than two double-spaced pages, create a data analysis plan that includes your research
question(s) and summarizes the data that will be available. Specify your analysis strategy and
which technique(s) you will use to answer your research question(s) (e.g., t-test for independent
groups, ANOVA, correlations, interpretation strategy for qualitative data).
Length: Maximum 2 pages
Due: Week 6
Value: 6%
Final Paper/Program Evaluation Report
The primary written assignment for this course will be an evaluation of an intervention, program,
service, or policy related to an important social problem that is addressed by the student’s
practice setting. It will incorporate the five phased assignments as well as the results and
interpretation of the findings. It will include an executive summary, introduction, review of the
literature, methodology, results, discussion, references, and appendices. The paper is expected to
require 10-12 pages, excluding the executive summary, references, tables/figures, and
appendices. Its organization is as follows, with approximate page allocations indicated in
parentheses:
Executive Summary: (1 page) – This one page summary will incorporate the introduction
through the discussion/interpretation as described below.
Introduction (1-2 pages) – The introduction should include a statement of the problem to
be addressed and its overall impact, such as evidence related to its prevalence (the
number of people in a population who have the problem), incidence (the number of new
cases over a period of time), and/or effect (the importance of the problem). This section
must explain why it is an important issue for social work practitioners to address and
ethical considerations for this evaluation, and end with an explication of the research
questions.
Relevance (Literature Review) (1-2 pages) – Previous empirical research pertaining to
relevant interventions, outcomes, or other matters specific to your population and
problem of interest should be reviewed concisely and critically in this section.
Methods (2 pages) – This section has the following four subsections: (1) Design, (2)
Sample, (3) Measures, and (4) Data analysis. Provide separate subheadings for each
section.
Results (2-3 pages) – This section will describe the results of your evaluation. If
quantitative methods are used, descriptive statistics should be presented first and
inferential statistics presented last. If qualitative methods are used, a description of the
analysis themes is to be provided with illustrative narrative comments included as
examples. Tables and figures should be used as appropriate, and are not included in the
page count.
Discussion/Interpretation (3 pages) – This section should include the following, in this
order: (1) a summary of your findings (e.g., related to the success of treatment, the
discovery of new client needs, or whether staff are operating with a consistent program
model); (2) a discussion of the implications of these findings and related
recommendations (e.g., to change service delivery in a focused way to improve
outcomes, better reach new clients, or assure consistency in service provision); (3)
plausible alternative explanations for your findings; (4) a discussion of the limitations of
4
your evaluation; and (5) suggestions for further evaluation (e.g., given the limitations of
your initial design, how the next evaluation can be improved). This section should
conclude with a discussion of the importance of your findings to social work practice and
research.
References – Provide an accurate list of all cited sources.
Tables, Figures – Tables and figures are suggested to display you results. Typically, the
first table describes the sample.
Appendix – Include copies of any scales or questionnaires used in your program
evaluation and other relevant information.
The complete evaluation report should follow APA formatting, and should follow the structure
and suggestions presented in Royse, Thyer, & Padgett, Chapter 15.
Length: Maximum 10 pages
Due: Week 12
Value: 60%
Oral presentation: Most evaluation findings are presented orally to stakeholders. Throughout
the course, students will present to their classmates and answer their questions. The final
presentation will allow students to present their overall projects as they would to those within
their agencies.
Length: 10 min presentation, Due: Weeks 13 & 14
Value: 5%
5
COURSE OUTLINE AND ASSIGNMENTS
Week 1 – Introduction to Evaluation of Social Work Interventions
January 11
Overview of the course
Review of required assignments for the course
Discussion of how to:
1. Identify issues to evaluate in a practice setting
2. Form evaluation problem statements
Discussion of evaluation possibilities in placement settings
Relevant Readings to Review:
Rubin & Babbie: Chapter 4 and Appendix A
Royse et al.: Chapters 1 and 14
Hatry et al.: Step 1
January 18—Martin Luther King Jr. Day
Week 2 – Development of Evaluation Justification
January 25
Phase 1 paper due
Small group discussion of draft evaluation questions
Discussion of how to develop evaluation justification
Discussion of ethical issues faced in evaluations
Relevant Readings to Review:
Rubin & Babbie: Chapters 6 and 7
Hatry et al.: Step 2
Week 3 – Development of Evaluation Designs
February 1
Phase 2 paper due
Small group discussion of evaluation justification
Discussion of how to develop:
1. Proposal evaluation designs
2. Evaluation logic models
Relevant Readings to Review:
Rubin & Babbie: Chapters 11, 12 and 13
Royse et al.: Chapters 9
Hatry et al.: Step 4
Week 4 – Development of Data Collection Instruments
February 8
Phase 3 paper due
Small group discussion of evaluation designs
Discussion of how to develop and search for appropriate data collection instruments
6
Relevant Readings to Review:
Rubin & Babbie: Chapter 9
Royse et al.: Chapters 11 and 12
Hatry et al.: Step 5 and Appendix
Week 5 – Development of Evaluation Data Analysis Plans
February 15
Phase 4 paper due
Presentation on statistical analysis strategies
Small group discussion of proposal data analysis plans
Relevant Readings to Review:
Rubin & Babbie: Chapters 17 and 18
Royse et al.: Chapter 13
Hatry et al.: Step 6
Week 6 – Development of Proposal Statements of Implications for Agency Practice
February 22
Phase 5 paper due
Small group discussion of evaluation plans
Discussion of how to interpret evaluation findings in a way that is relevant for service
planning and delivery
Relevant Readings to Review:
Royse et al.: Chapter 15
Hatry et al.: Step 8
Week 7 – Consultation on Evaluation Implementation
March 1
Whole class, small group, and individual consultation on evaluation implementation
issues
MARCH 8 -- SPRING BREAK
Week 8 – Consultation on Evaluation Implementation
March 15
Whole class, small group, and individual consultation on evaluation implementation
issues
7
Week 9 – Progress Review Session
March 22
Small group reviews and discussion of evaluation progress
Discussion of how to write evaluation reports and funding proposals
Required Reading:
Rubin & Babbie: Appendix A and B
Royse et al.: Chapter 15
Weeks 10 - 12 – Consultation on Evaluation Implementation
March 29
April 5
April 12
Whole class, small group, and individual consultation on evaluation implementation
issues
Weeks 13 and 14 – Presentations of Student Projects
April 19
April 26
Presentations on evaluation proposals. Presentations may include PowerPoint slides, or
posters.
8