University of North Carolina at Chapel Hill
School of Social Work
Course:
SOWO 810, Evaluation of Social Interventions
Fall 2008, Tuesdays 9:00 – 11:50 A.M.
Professor:
Gary S. Cuddeback, Ph.D., M.S.W., M.P.H.
Assistant Professor, School of Social Work
Faculty Research Fellow, Cecil G. Sheps Center for Health Services Research
University of North Carolina at Chapel Hill
CB#3550, 321 Pittsboro Street, Office #422B
Chapel Hill, NC 27599-7590
Phone: SOWO (919) 962-4363, SHEPS (919) 966-0995
gcuddeba@email.unc.edu or cuddeback@mail.schsr.unc.edu
Office Hours: Before and after class, by appointment
Teaching
Assistant
Jilan Liu
Course Description and Objectives: Students will develop knowledge of the purposes of research and
evaluation and the approaches and methodologies necessary to evaluate social work interventions. Upon
completion of this course students will be able to demonstrate:
1. Skill in developing and implementing social intervention evaluations that promote evidence-based
social work practice and policy;
2. Skill in evaluating social intervention research and applying findings to social work practice and policy;
3. Skill in qualitative and quantitative evaluation design, measurement, data analysis, and knowledge
dissemination;
4. Knowledge of the practical, political, and economic issues related to the evaluation of social
interventions;
5. Skill in designing social intervention research that is sensitive to racial, gender, social, economic, and
other issues of difference; and
6. Ability to apply social work ethics and values to the evaluation of social interventions.
Expanded Course Description: Social workers, whether they are front-line practitioners, program managers,
administrators or policymakers, routinely face complex human situations involving consumers or constituents
who come from diverse backgrounds. The social interventions social workers have at their disposal vary in
their degree of effectiveness with any given individual, family, group, organization or community. In order to
provide the most effective social work policies and interventions, social workers must be able to determine if
what they are doing is beneficial to the individuals, families, groups, organizations, or communities they serve.
To this end, and building on the knowledge gained in the foundation course SOWO 510 (102), Introduction to
Research Methodology, this course provides a results-based accountability framework so students may gain:
(a) skills in accessing and assessing public databases and research literature as a foundation for evidencebased practice, (b) knowledge of evaluation methodologies available to implement results-based evaluation,
and (c) the ability to work within a results-based accountability framework in their social work practice.
Required Texts:
Hatry, H., van Houten, T., Plantz, M. C., & Greenway, M. T. (1996) Measuring Program Outcomes: A Practical
Approach. Alexandria, VA: United Way of America.
SOWO 810 – Spring 2008 - Cuddeback
Royse, D., Thyer, B. A., Padgett, D. K., & Logan, T. K. (2006). Program Evaluation: An Introduction (4th ed.).
Belmont, CA: Brooks-Cole.
The required texts are available in the Health Sciences bookstore. Supplemental readings are available in a
course folder located in the SOSW library on the 5th floor and/or on-line (noted below). Articles noted to be online are available through the UNC Library electronic databases.
Policy on Incomplete or Late Assignments: Students must notify the instructor at least 24 hours before an
assignment is due if an assignment is going to be turned in late. Extensions may be given at the instructor’s
discretion for extenuating circumstances. Students will lose five points (i.e., from an 85 to an 80) for each 24hour period beyond the due date and time (including weekends) for unexcused late assignments.
Policy on Academic Dishonesty: Academic dishonesty is contrary to the ethics of the social work profession,
unfair to other students, and will not be tolerated in any form. All written assignments should include the
following signed pledge: “I have neither given nor received unauthorized aid in preparing this written work.”
Policy on Accommodations for Students with Disabilities: Accommodations will be made for students with
disabilities affecting their participation in this course.
Course Requirements
Quizzes: Each week at the beginning of class a short quiz covering material from the previous week will be
given. Each quiz will be discussed after its completion and returned after grades are recorded. Quiz grades will
be averaged and this final quiz grade will count for 20% of the total grade. The lowest quiz grade will be
dropped. These quizzes will provide ongoing feedback concerning knowledge acquisition and will provide the
information necessary to take corrective measures, by both the professor and students, to ensure that
adequate learning has occurred.
Midterm and Final Exams: The midterm and final exams will be worth 20% each and will mainly cover the
information contained in the weekly quizzes. The final exam will not be cumulative. Exams will consist of
true/false, multiple choice, short answer, and essay questions.
Project: An evaluation project related to each student’s field placement or specific interests will be the primary
written assignment for this course. Students can choose one of two options: (1) students can conduct a real
evaluation in the field (i.e., design an evaluation and collect real data), or (2) students can design a fictitious
evaluation and analyze fictitious data. Students who elect to conduct evaluations in the field should meet with
their professor as soon as possible.
The project will be completed in five parts: (1) Introduction; (2) Literature Review and Research
Question(s); (3) Methods; (4) Results; and (5) Discussion and Conclusion. See the course outline below for
due dates for each of these sections.
Abstract (1 page) – A summary that will include portions of all sections described below.
Introduction (~ 2 pages) – The introduction should include a statement of the problem to be addressed by a
social intervention (i.e., something germane to the student’s field placement or interests) and information
concerning the prevalence (i.e., the number of people in a population who have the problem) and incidence
(i.e., the number of new cases over a period of time) and costs of the problem. This section should
conclude with a discussion of why the problem is important to the field of social work. More information
about this section will be provided in a separate handout.
Literature Review (~ 2 – 3 pages) – Empirical research pertaining to relevant interventions, programs or
policies and their outcomes specific to the student’s population and problem should be reviewed
concisely and critically. Students are encouraged to start their literature reviews early and consult with their
2
SOWO 810 – Spring 2008 - Cuddeback
professor as needed. Also, in one or two sentences the student’s plan for addressing the problem will be
presented. This section should conclude with a research question or hypothesis. More information will be
provided in a separate handout.
Methods (~ 2 - 3 pages) – This section will present the plan for how the research question or hypothesis
will be addressed. Please provide separate subheadings for each of the following: (1) Design, (2) Sample,
(3) Measures, and (4) Data Analysis. More information will be provided in a separate handout.
Results (~ 1 - 2 pages) – This section will present the results of your evaluation.
Discussion and Conclusion (~ 2 pages) – This section should include: (1) a summary of findings; (2)
implications of the findings for social work practice, policy and research; (3) a discussion of the limitations
of the evaluation and any plausible alternative explanations for the findings; (4) recommendations for other
practitioners; and (5) suggestions for further evaluation in the context of the limitations of the current
evaluation.
References – Provide an accurate list of all cited sources in APA format. This will be turned in with the
Literature Review.
Appendix – Provide a copy of the SPSS output related to any and all inferential test statistics that were
created for your project. This will be turned in with your Results portion.
Each part of the project can be resubmitted for re-grading one time. In addition, drafts of all written
assignments can be submitted for feedback as early and often as needed. Each piece will be graded according
to the following criteria and a number grade from 0 – 100 will be assigned:
•
•
•
•
•
Mechanics (grammar, spelling, style, typing)
Organization
Logic
Content
Ability to summarize and draw conclusions
Format for Written Work: APA format should be used for all written assignments. Students should refer to
the Publication Manual of the American Psychological Association (5th ed.) for information on APA format. In
addition, students should refer to the School of Social Work Style Guide, pages 26 – 32, for guidelines on how
to cite work properly and avoid plagiarism.
Evaluation and Grading:
Evaluation Project (Part I)
Evaluation Project (Part II)
Midterm Exam
Quiz Average
Final Exam
20%
20%
20%
20%
20%
100%
Points
94 – 100
80 – 93
70 – 79
< 69
Grade
H
P
L
F
Course Outline and Readings
January 14
Course Overview and Syllabus
Evaluation of Social Interventions
Ethics
Royse et al. Chapter 2
Supplemental Readings:
3
SOWO 810 – Spring 2008 - Cuddeback
Collins, P., Kayser, K., & Tourse, R. C. (1994). Bridging the gaps: an interdependent model for educating
accountable practitioners. Journal of Social Work Education, 30(2), 241-251. [course folder - SOSW
library]
Gambrill, E. (2001). Social work: an authority-based profession. Research on Social Work Practice,
11(2), 166-175. [course folder - SOSW library]
Mancini, J. A., Marek, L. I., Byrne, R. A. W., & Huebner, A. J. (2004). Community-based program
research: context, program readiness, and evaluation usefulness. Journal of Community Practice,
12(1/2), 7 – 21. [on line]
Martin, J. I., & Knox, J. (2000). Methodological and ethical issues in research on lesbians and gay men.
Social Work Research, 24(1), 51-59. [course folder - SOSW library]
Miller, W. R., Sorenson, J. L., Selzer, J. A., & Brigham, G. S. (2006). Disseminating evidence-based
practices in substance abuse treatment: a review with suggestions. Journal of Substance Abuse
Treatment, 31, 25 – 39. [on line]
Millstein, K. H., Dare-Winters, K., & Sullivan, S. (1994). The power of silence: ethical dilemmas of
informed consent in practice evaluation. Clinical Social Work Journal, 22, 317-329. [course folder SOSW library]
Molin, R., & Palmer, S. (2005). Consent and participation: ethical issues in the treatment of children in
out-of-home care. American Journal of Orthopsychiatry, 75(1), 152-157. [on line AND course folder SOSW library]
Rosen, A. (2003). Evidence-based social work practice: challenges and promise. Social Work Research,
27(4), 197-208. [on line AND course folder – SOSW library]
Schoenwald, S. K., Letourneau, E. J., & Halliday-Boykins, C. (2005). Predicting therapist adherence to a
transported family-based treatment for youth. Journal of Clinical Child and Adolescent Psychology,
34(4), 658 – 670. [on line]
Staudt, M. (1997). Pseudoissues in practice evaluation: impediments to responsible practice. Social
Work, 42(1), 99-106. [course folder - SOSW library]
Wakefield, J. C., & Kirk, S. A. (1996). Unscientific thinking about scientific practice: evaluating the
scientist-practitioner model. Social Work Research, 20(2), 83-95. [course folder - SOSW library]
January 21
Martin Luther King, Jr. Day – No Class
January 28
Foundations for Accountable Practice
Hatry et al. pp. 1-9
Royse et al. Chapter 1
Supplemental Readings:
Chronis, A. M., Jones, H. A., & Raggi, V. L. (2006). Evidence-based psychosocial treatments for children
and adolescents with attention-deficit/hyperactivity disorder. Clinical Psychology Review, 26, 486502. [on line]
Cuddeback, G. S. (2004). Kinship family foster care: a methodological and substantive synthesis of
research. Children and Youth Services Review, 26, 623-639. [on line AND course folder - SOSW
library]
Hahn, R. A., Bilukha, O., Lowy, J., Crosby, A., Fullilove, M. T., Liberman, A., Moscicki, E., Snyder, S.,
Tuma, F., Corso, P., & Schofield, A. (2005). The effectiveness of therapeutic foster care for the
prevention of violence: a systematic review. American Journal of Preventive Medicine, 28(2S1), 7290. [on line AND course folder - SOSW library]
National Association of Public Child Welfare Administrators (2005). Guide for child welfare administrators
on evidence-based practice. http://www.aphas.org.
Racusin, R., Maerlender, A. C., Sengupta, A., Isquith, P. K., & Strauss, M. B. (2005). Psychosocial
treatment of children in foster care: a review. Community Mental Health Journal, 41(2), 199 – 221.
[on line]
Wiehe, S. E., Garrison, M. M., Christakis, D. A., Ebel, B. E., & Rivara, F. P. (2005). A systematic review
of school-based smoking prevention trials with long-term follow-up. Journal of Adolescent Health, 36,
162-169. [on line AND course folder - SOSW library]
Wiesz, J. R., & Jensen, P. S. (1999). Efficacy and effectiveness of child and adolescent psychotherapy
and pharmacotherapy. Mental Health Services Research, 1(3), 125-157. [on line AND course folder SOSW library]
Wiesz, J. R., & Jensen, P. S. (2001). Child and adolescent psychotherapy in research and practice
contexts: review of the evidence and suggestions for improving the field. European Child &
Adolescent Psychiatry, 10, 1/12-1/18. [on line AND course folder - SOSW library]
4
SOWO 810 – Spring 2008 - Cuddeback
February 4
Quiz 1
Identifying Outcomes
Baseline Assessments of Needs and Problems
Developing Research Questions and Hypotheses
Hatry et al. pp. 11-80
Royse et al. Chapter 3
Supplemental Readings:
Chen, H., & Marks, M. (1998). Assessing the needs of inner city youth: beyond needs identification and
prioritization. Children and Youth Services Review, 20(9/10), 819-838. [course folder - SOSW
library]
Savaya, R., & Waysman, M. (2005). The logic model: a tool for incorporating theory in development and
evaluation of programs. Administration in Social Work, 29(2), 85 – 103. [on line]
Stewart, D., Law, M., Russell, D., & Hanna, S. (2004). Evaluating children’s rehabilitation services: an
application of a programme logic model. Child: Care, Health & Development, 30(5), 453-462. [on line
AND course folder - SOSW library]
February 11
Quiz 2
Measurement Instruments for Evaluation
Hatry et al. pp. 81-112, 147-163
Royse et al. Chapters 11, 12
Supplemental Readings:
Chorpita, B. F., Moffitt, C. E., & Gray, J. (2005). Psychometric properties of the Revised Child Anxiety
and Depression Scale in a clinical sample. Behaviour Research and Therapy, 43, 309 – 322. [on line
AND SOSW library]
Cuddeback, G. S., Buehler, C., Orme, J. G., & Le Prohn, N. (2007). Measuring foster parent potential: the
psychometric properties of the Casey Foster Applicant Inventory – Worker Version (CFAI-W).
Research in Social Work Practice, 17(1), 93-109. [on line]
Gerkensmeyer, J. E., & Austin, J. K. (2005). Development and testing of a scale measuring parent
satisfaction with staff interactions. Journal of Behavioral Health Services & Research, 32(1), 61-73.
[on line AND course folder - SOSW library]
Orme, J. G., Cuddeback, G. S., Buehler, C., & Le Prohn, N. S. (2007). Measuring foster parent potential:
Casey Foster Parent Inventory – Applicant Version (CFAI-A). Research in Social Work Practice,
17(1), 77-92. [on line]
Rowley, A. A., Roesch, S. C., Jurica, B. J., & Vaughn, A. A. (2005). Developing and validating a stress
appraisal measure for minority adolescents. Journal of Adolescence, 28, 547-557. [on line AND
course folder - SOSW library]
February 18
Quiz 3
Sampling
Internal and External Validity
Royse et al. pp. 184-185, pp. 254-258
February 25
Quiz 4
Client Satisfaction Surveys
Qualitative Research
Royse et al. Chapters 4, 8
Supplemental Readings:
Altshuler, S. J. (1999). Children in kinship foster care speak out: “We think we’re doing fine.” Child and
Adolescent Social Work Journal, 16(3), 215-235. [course folder - SOSW library]
Buehler, C., Cox, M. E., & Cuddeback, G. (2003). Foster parent’s perceptions of factors that promote or
inhibit successful fostering. Qualitative Social Work: Research and Practice, 2(1), 61-84. [course
folder – SOSW library]
Coakley, T. M., Cuddeback, G. S., Buehler, C., & Cox, M. E. (2007). Kinship foster parents' perceptions
of factors that promote or inhibit successful fostering. Children and Youth Services Review, 29(1),
92-109. [on line]
5
SOWO 810 – Spring 2008 - Cuddeback
Shamai, M. (2003). Therapeutic effects of qualitative research: reconstructing the experience of
treatment as a by-product of qualitative evaluation. Social Service Review, 77(3), 455 – 467. [on line]
March 3
Quiz 5 - Midterm Exam and Part I Papers Due
March 10
Spring Break – No Class
March 17
Research and Evaluation Designs
Royse et al. Chapters 6, 9
Supplemental Readings:
Bernstein, G. A., Layne, A. E., Egan, E. A., & Tennison, D. M. (2005). School-based interventions for
anxious children. Journal of the American Academy of Child and Adolescent Psychiatry, 44(11), 1118
– 1127. [on line]
Broner, N., Lattimore, P. K., Cowell, A. J., & Schlenger, W. E. (2004). Effects on diversion on adults with
co-occurring mental illness and substance use: outcomes from a national multi-site study. Behavioral
Sciences and the Law, 22, 519-541. [on line AND course folder - SOSW library]
Clarke, G. N., Herinckx, H. A., Kinney, R. F., Paulson, R. I., Cutler, D. L., Lewis, K., & Oxman, E. (2000).
Psychiatric hospitalizations, arrests, emergency room visits, and homelessness of clients with
serious and persistent mental illness: findings from a randomized controlled trail of two ACT
programs vs. usual care. Mental Health Services Research, 2(3), 155-164. [on line AND in course
folder - SOSW library]
Finkelstein, N., Rechberger, E., Russell, L. A., VanDeMark, N. R., Noether, C. D., O’Keefe, M. O., Gould,
K., Mockus, S., & Rael, M. Building resilience in children of mothers who have co-occurring disorders
and histories of violence. Journal of Behavioral Health Services & Research, 32(2), 141-254. [on line
AND course folder - SOSW library]
Randall, E. (2001). Existential therapy of panic disorder: a single-system study. Clinical Social Work
Journal, 29(3), 259-267. [on-line AND course folder - SOSW library]
Temple, S., & Ho, B. (2005). Cognitive therapy for persistent psychosis in schizophrenia: a casecontrolled clinical trial. Schizophrenia Research, 75, 195-199. [on line AND course folder - SOSW
library]
Ventura, L. A., Cassell, C. A., Jacoby, J. E., & Huang, B. (1998). Case management and recidivism of
mentally ill persons released from jail. Psychiatric Services, 49(10), 1330-1337. [on line AND course
folder - SOSW library]
Zetlin, A. G., Weinberg, L. A., & Kimm, C. (2005). Helping social workers address the educational needs
of foster children. Child Abuse & Neglect, 29, 811-823. [on line AND course folder - SOSW library]
March 24
Quiz 6
Data Analysis
Hatry et al. pp. 113-124
Royse et al. Chapter 13
Supplemental Readings:
Barrett, M. D., & Wolfer, T. A. (2001). Reducing anxiety through a structured writing intervention: a singlesystem evaluation. Families in Society: The Journal of Contemporary Human Services, 82(4), 355362. [course folder - SOSW library]
Cather, C., Penn, D., Otto, M. W., Yovel, I., Mueser, K. T., & Goff, D. C. (2005). A pilot study of functional
Cognitive Behavioral Therapy (fCBT) for schizophrenia. Schizophrenia Research, 74, 201-209. [on
line AND course folder - SOSW library]
Cohen, J. A., Mannarino, A. P., & Knudsen, K. (2005). Treating sexually abused children: 1 year followup of a randomized controlled trial. Child Abuse & Neglect, 29, 135-145. [on line AND course folder SOSW library]
Electronic Statistics Textbook: http://www.statsoft.com/textbook/stathome.html
Ferguson, K. L., & Rodway, M. R. (1994). Cognitive behavioral treatment of perfectionism: initial
evaluation studies. Research on Social Work Practice, 4(3), 283-308. [course folder - SOSW library]
Hourihan, F., & Hoban, D. (2004). Learning, enjoying, growing, support model: an innovative
collaborative approach to the prevention of conduct disorder in preschoolers in hard to reach rural
families. Australian Journal of Rural Health, 12, 269-276. [on line AND course folder – SOSW library]
Reed, V. A., Jernstedt, G. C., Hawley, J. K., Reber, E. S., & DuBois, C. A. (2005). Effects of a smallscale, very short-term service-learning experience on college students. Journal of Adolescence, 28,
6
SOWO 810 – Spring 2008 - Cuddeback
359-368. [on line AND course folder - SOSW library]
Secret, M., & Bloom, M. (1994). Evaluating a self-help approach to helping a phobic child: a profile
analysis. Research on Social Work Practice, 4(3), 338-348. [course folder - SOSW library]
Slonim-Nevo, V., & Vosler, N. R. (1991). The use of single-system design with systemic brief problemsolving therapy. Families in Society: The Journal of Contemporary Human Services, 72(1), 38-44.
[course folder - SOSW library]
Statistical Computing: <http://www.ats.ucla.edu/stat/>
March 31
Quiz 7
Research and Evaluation Designs and Data Analysis - revisited
April 7
Quiz 8
Research and Evaluation Designs and Data Analysis - revisited
April 14
Quantitative Data Analysis – Lab Exercise (Quizzes 9-10)
April 21
Formative Evaluation
Process Evaluation
Cost Analysis and Cost Effectiveness Designs
Royse et al.,: Chapters 5, 10
Supplemental Readings:
Barth, R. P., Lee, C. K., Wildfire, J., & Guo, S. (2006). A comparison of the governmental costs of longterm foster care and adoption. Social Service Review, 80(1), 127-158. [on line]
DeSena, A. D., Murphy, R. A., Douglas-Palumberi, H., Blau, G., Kelly, B., Horwitz, S. M., & Kaufman, J.
(2005). SAFE Homes: is it worth the cost: an evaluation of a group home permanency planning
program for children who first enter out-of-home care. Child Abuse & Neglect, 29, 627 – 643. [on line]
Ettner, S. L., Huang, D., Evans, E., Ash, D. R., Hardy, H., Jourabchi, M., & Hser, Y-I. (2005). Benefit-cost
in the California treatment outcome project: does substance abuse treatment “Pay for Itself”? Health
Services Research, 41(1) 192 – 213. [on line]
Foster, E. M., & Holden, E. W. (2002). Benefit-cost analyses of the child welfare demonstration projects:
understanding the resource implications of the IV-E waivers. Children and Youth Services Review,
24(6/7), 431-453. [on line]
Helitzer, D., Yoon, S., Wallerstein, N., & Garcia-Velarde, L. (2000). The role of process evaluation in the
training of facilitators for an adolescent health education program. Journal of School Health, 70(4),
141 – 147. [on line]
Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance abusing
and dependent delinquents: outcomes, treatment fidelity and transportability. Mental Health Services
Research, 1(3), 171-184. [on line AND course folder - SOSW library]
McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999). Fidelity to assertive community treatment
and client outcomes in the New Hampshire Dual Disorders Study. Psychiatric Services, 50(6), 818824. [on line]
Rosenheck, R. (2000). Cost-effectiveness of service for mentally ill homeless people: the application of
research to policy and practice. American Journal of Psychiatry, 157(10), 1563 – 1570. [on line]
Schweinhart, L. J. (2003). Benefits, costs, and explanation of the High/Scope Perry Preschool program.
Paper presented at the Meeting of the Society for Research in Child Development, Tampa, Florida
http://www.highscope.org/research/PerryProject/Perry-SRCD-2003.pdf. [View slides using
PowerPoint: http://www.highscope.org/research/PerryProject/perrymain.htm]
Story, M., Lytle, L. A., Birnbau, A. S., & Perry, C. L. (2002). Peer-led, school-based nutrition education for
young adolescents: feasibility and process evaluation of the TEENS Study. Journal of School Health,
72(3), 121 – 127. [on line]
Valois, R. F., & Hoyle, T. B. (2000). Formative evaluation results from the Mariner Project: a coordinated
school health pilot program. Journal of School Health, 70(3), 95 – 103. [on line]
May 5
Final Exam / Final Projects Due / All Assignments Due
7