PH7521: Evaluation Research
Monique G. Carry, PhD
Health Promotion & Behavior
Fall Semester 2014
CRN: 88826
Class Day/Time:
Tuesdays/ 4:30 pm -7:00 p.m
Class Location:
Arts & Humanities 320
Course Basics
Prerequisite(s):
Rossi, P., Lipsey M., and Freeman H. (2004). Evaluation: A systematic
approach, 7th Edition. Sage Publications, Inc., Thousand Oaks, CA.
ISBN: 978-0-7619-0894-4
Required Course Materials
Wholey, J., Hatry, H., and Newcomer, K. (2010). Handbook of Practical
Program Evaluation, 3rd Edition. Jossey-Bass., San Francisco, CA.
ISBN: 978-0470522479 (On D2L)
Faculty Accessibility
Centers for Disease Control and Prevention. (1999). Recommended
framework for program evaluation in public health practice. MMWR
Recommendations and Reports, 48(RR-11).
http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm
Instructor(s) of Record:
Monique G. Carry, PhD
Office Location:
843 Urban Life Building
Phone Number(s):
404-585-1508
Email:
mcarry@gsu.edu
Office Hours/Availability:
Before or after class by appointment
Additional Information:
I.
Course Description: Evaluation Research. An examination of the techniques and practice
of program evaluation for effectiveness in program administration. The course contrasts
deductive and inductive approaches. The instructor illustrates the advantages of using
evaluation as a mechanism for program improvement.
II.
Course Objectives / Competency / Assessment of Student Learning:
Students in the Master of Public Health program with a concentration in Health
Promotion and Behavior will be expected to demonstrate competence in the following
areas after completion of this course, as indicated in the GSU School of Public Health
Graduate Student Handbook (see MPH Competencies):

HPMB #1: Identify basic theories, concepts and models from a range of social and
behavioral disciplines that are used in public health research and practice.

HPMB #2: Describe steps and procedures for the planning, implementation and
evaluating public health programs, policies and interventions.

HPMB #4: Develop a logic models for use in program development, implementation,
and evaluation.

HPMB #5: Differentiate among goals, measurable objectives, related activities, and
expected outcomes for a public health program.

HPMB #6: Differentiate the purposes of formative, process, and outcome evaluation,
and explain how findings from each are used.

HPMB #7: Collaboratively assess individual, organizational, and community concerns
and resources for public health programs.

HPMB #8: Assess evaluation reports in relation to their quality, utility, and impact on
public health.
Course Objectives
Differentiate between
basic research, applied
research and program
evaluation.
Program Competency
1, 2
Assessment Method(s)
Class Article Presentation
SMART Questions
Independent Journal Article Review
Apply the following
terms to program
evaluation: formative,
process, and outcome.
1, 2
Class Article Presentation
SMART Questions
Independent Journal Article Review
Design a logic model
from a description of
program operations.
4, 5
Logic Model
Apply a logic model to
2, 4, 5
the identification of
program evaluation
questions, stakeholders,
data collection
methods, and data
interpretation.
Logic Model
Determine the most
appropriate type of
evaluation for a
program given
stakeholder needs for
information and
available resources.
5, 6
Evaluation Matrix
Evaluation Literature Review
Prepare an evaluation
final protocol for a
specific program.
1,2, 4, 5, 6, 7, 8
Evaluation Literature Review
Final Evaluation Protocol
Prepare and present
evaluation protocol in a
manner that maximizes
their effectiveness.
1,2, 4, 5, 6, 7, 8
Evaluation Protocol Presentations
III.
Course Assignments and Requirements
#1) Class Article Presentation (2-3 doubled spaced pages): Each student is responsible for
presenting a summary (oral presentation) and written outline (to be posted on D2L) of one
chapter or article assigned for the week. Students will sign up for their article or chapter during
the first week of class. Students should highlight key points, terms, results, etc. and be able to
lead their classmates into discussion of materials. Written outline summary should be turned in
online on D2L dropbox 24 hours before the day of your presentation. For example if you are
presenting on October 14th, your written summary should be turned into the dropbox no later
than 4:00pm on October 13th to be considered on time. Class members not presenting on a
given day are encouraged to download written summaries so that they may follow along and
ask questions during class. Due 4pm day prior to presentation (Written summary); Assigned
presentation date (Oral presentation)
#2) SMART Questions: Each student will prepare three (3) potential evaluation questions for
final project consideration. These questions will be evaluated based on their adherence to
S.M.A.R.T. (Specific, Measurable, Appropriate, Realistic, Time bound) principles. The purpose of
this exercise is to familiarize students with the processes of developing strong program
evaluation questions. Due September 16, 2014
#3) Logic Model: Each student will individually prepare a logic model for a health promotion
program based on an example program presented in class. Grades will be based upon the
logical organization of the concepts related to the program and the clarity of their presentation.
Assumptions, inputs, activities, outputs, short-term outcomes, intermediate outcomes and
long-term outcomes must be included in the logic model. Due September 23, 2014
* Helpful info on Logic models and templates
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodelworksheets.html
#4) Independent Journal Article Review (3-4 double spaced pages): Search an academic
journal for an article that reports on a program evaluation. Critique the article in written report
format using the criteria below. Please provide a pdf or Word (.doc) copy of the article along
with your report. Be prepared to summarize your article and share your impressions of the
quality of the article with the class. The purpose of this exercise is to familiarize students with
the variety of research designs used in program evaluation and be able to assess the
appropriateness of those designs. Due October 14th, 2014











Describe the program being evaluated in terms of its activities/participants/outcome(s).
Describe the evaluation design.
Describe any problems involved in the evaluation.
State the hypotheses or the evaluation questions.
What information is provided in support of the program in achieving its objectives? i.e.,
what is the program theory or what is the rationale for the way the program was
intended to work?
How were the data analyzed and interpreted?
Does the article refer to any related literature or supporting studies? Discuss how the
related literature (at least 1) supports the current evaluation study. Cite any referenced
studies in the reference section.
What are the conclusions of the evaluation?
What further studies or research is suggested by the authors? If none suggested, what
additional studies would you suggest?
What is your overall impression of what this evaluation was attempting to accomplish?
(Use critical thinking, summaries such as “This was a good read” will not be acceptable).
Provide the citation for the article in the reference section
#5) Evaluation Matrix and Meeting with the Instructor (Group Project): All groups and all
group members are to meet with the instructor to review the progress of the evaluation plan.
During this meeting, group members will present a matrix proposal to the instructor which
includes proposed evaluation questions, corresponding relevant and realistic evaluation
methodologies, including proposed measures and data collection instruments and/or sources.
An example matrix will be provided to students. Attendance at the group meeting is
mandatory. Failure to attend could adversely affect the grades of those not in attendance. This
meeting does not preclude access to the instructor at other times during the semester.
Students are encouraged to meet with the instructor at any time by scheduled appointment.
Due in class October 28th, 2014
#6) Evaluation Literature Review (Group Project): Groups must prepare an evaluation
literature review for their evaluation protocol. The literature review should focus on Sections
3-8 of the evaluation protocol outline with particular emphasis on: a detailed description of the
program, a logic model, and identification of stakeholders, synthesis of relevant literature, case
studies, and evaluation questions. Due November 11th, 2014
#5) Evaluation Protocol Presentations (Group Project): The students will present a summary
of their evaluation plan as a class presentation. The presentation must include an overview of
the logic, methodology, and sources of data. Each group will be given approximately 30 minutes
to present its work. Each group member is required to present during the 30 minute
presentation. An additional 10 minutes will be reserved for Q and A. The grades will be based
on professionalism of the presentation and the ability of the students to respond to questions
regarding the overall utility of the evaluation effort. Special attention will be given to the clarity
of the graphics, the communication attributes of the individual presenters, and the ability of
the students to present a cogent summary of their work. Group presentations will be evaluated
by other class members. Evaluation presentations will be given during class on December 2nd,
2014
#6) Final Evaluation Protocol (Group Project): Groups will prepare an evaluation protocol for
their proposed evaluation question(s). The protocol should follow the proposed outline in the
syllabus. The protocol will be presented under the authorship of the students an addressed
and written for an identified and appropriate stakeholder. The grades will be based upon the
completeness of the protocol, the ability of the protocol to support its questions and the
professional appearance of the document. Due December 9th 2014 on D2L
Final Protocol Outline (20-35 pages plus references and appendices)
1. Title page and table of contents
2. Executive summary (1 page)
3. Introduction (2-3 pages)
4. Literature review of related evaluations (5-6 pages)
5. Program description (3-4 pages)
6. Program objectives, logic and/or theoretical bases (1 page graphic, 2-4 pages total)
7. Program stakeholders (1 page)
8. Evaluation questions (1 page or less)
9. Evaluation methods (5-8 pages)
10. Evaluation data sources (1-2 pages)
11. Contribution to public health (1 page)
12. References
13. Appendices—including sample instrumentation, additional data tables, variable
definitions, and explanatory documents.
#7) Class Participation
Students are expected to be active participants in class discussions, group projects, and
exercises. Class participation is determined into each student’s course grade and will be based
on active class participation, group work, and a participation “quiz” at the end of the semester.
Withdrawals: A student who withdrawals at any time up to the mid-point of the quarter will be
assigned a W or WF depending upon whether he/she is doing satisfactory work at the time of
withdrawal. An average grade of D or F at the time of withdrawal will be assigned a grade of
WF. After the mid-point of the quarter, the Registrar’s Office will assign an automatic WF to any
student who withdraws from the course without a hardship withdrawal. If a student receives
permission to withdraw under hardship, the instructor will assign a W or WF grade depending
upon the student’s work up to the point of time that the student withdrew.
The following is the formal policy at Georgia State University:
“Effective Fall 2001, instructors must on a date after the mid-point of the course to be set by
the Provost (or his designee),
1. Give a WF to all those students who are on their rolls but no longer taking the class and
2. Report the last day the student attended or turned in an assignment.
Students who are withdrawn may petition the department chair for reinstatement into their
classes.”
Incompletes: A student will be given the grade I only if nonacademic circumstances beyond the
student’s control prevent the student from completing a small segment of the course—e.g. the
midterm examination. For a student to receive the grade of I, he/she must be doing satisfactory
work (an average grade of C or better) up to the point that he/she could not continue.
Arrangements must be made with instructor to remove the I within one quarter.
IV.
Grading Policy
The following are the criteria for evaluating student performance:
Assignments
Points
Individual Assignments:
40%
Class Article Presentation
10%
Grade Distribution
94-100% = A
91-93% = A-
SMART Questions
10%
88-90% = B+
Logic Model
Journal Article Review
10%
10%
84-87% = B
80-83% = B-
77-79% = C+
Group Assignments:
50%
Evaluation Protocol Matrix
10%
Evaluation Literature Review
15%
Evaluation Final Protocol
15%
Evaluation Final Protocol Presentation
10%
Meeting with Instructor
~
Participation
Total
74-76% = C
70-73%= C-
10%
100%
Final grades will not be posted or given out over the phone or via email, but will be posted on
GoSolar.
* Writing style and grammar will be taken into account for the grading of all assignments.
Students are expected to write and will be graded on a graduate level. Please be sure to
proofread all assignments. Writing assignments should be prepared as if turning in for a thesis,
practicum, or journal worthy publication. Students should use the following resources to make
sure their writing is on the appropriate level.
Resources:
 GSU Writing Studio http://www.writingstudio.gsu.edu/
 Swales, J. and Feak, C. (2012). Academic Writing for Graduate Students, 3rd Edition:
Essential Tasks and Skills. University Michigan Press ISBN: 978-0472034758
V.
Attendance and Class Participation Policy
This course assumes substantial and informed student participation. General discussion of
theory and practice is encouraged and expected of all students. At a minimum, being informed
requires class attendance, completion of assigned readings and homework. Class attendance
and thoughtful participation are important. Please notify the instructor of an absence before
the class.
VI.
Late Assignments and Make-up Examination Policy
Make-up exams are not provided, except in hardship cases that should be discussed in advance
with the Instructor. Any assignment turned in late may have points deducted up to one letter
grade for each day the assignment is late or fraction thereof. If a student cannot take an exam
or present at the scheduled time, they must consult with the Instructor on scheduling the
make-up. No more than two make-ups will be allowed.
VII.
Syllabus Deviation Policy
The course syllabus provides a general plan for the course; deviations may be necessary.
VIII.
Student Code of Conduct and Policy on Academic Honesty
All students at this University are expected to engage in academic pursuits on their won
with complete honesty and integrity. Any student found guilty of dishonesty in any phase of
academic work will be subject to disciplinary action. The complete Academic Honesty policy
is located in the GSU Graduate Catalog, Section 1350: http://enrollment.gsu.edu/catalogs/.
Students and faculty are expected to review and conform to the university’s policy on
academic honesty. Information on the Student Code of Conduct and related policies and
procedures are available at: http://codeofconduct.gsu.edu/.
Special attention should be paid to the sections on plagiarism and multiple submissions:
Plagiarism. Plagiarism is defined as, “appropriating and putting forth as one’s own
the ideas, language, or designs of another” (The Living Webster, 1975) – and it is
strictly forbidden. Written and oral presentations must be a student’s own work.
Students plagiarizing or cheating in any form will face disciplinary action which could
result in an “F” in this course and suspension or expulsion from the University.
Copying from written materials, presentations, websites, etc. without source
acknowledgement and referencing is plagiarism. Read it, appreciate it, learn from it,
and make sure you source it – and then reflect it with your own thoughts and words!
If you are uncertain about what constitutes plagiarism, please contact the instructor.
Multiple Submissions. It is a violation of academic honesty to submit substantial
portions of the same work for credit more than once without the explicit consent of
the faculty member(s) to whom the material is submitted for additional credit. In
cases in which there is a natural development of research or knowledge in a
sequence of courses, use of prior work may be desirable, even required; however,
the student is responsible for indicating in writing, as a part of such use, that the
current work submitted for credit is cumulative in nature.
IX.
Disability Accommodations Policy
Students who wish to request accommodation for a disability may do so by registering with
the GSU Office of Disability Services. Students may only be accommodated upon issuance
by the Office of Disability Services of a signed Accommodation Plan and are responsible for
providing a copy of that plan to instructors of all classes in which an accommodation is
sought. The Office of Disability Services is located in the GSU Student Center, Suite 230 and
online here: http://disability.gsu.edu/.
X.
Course Evaluations Statement
Your constructive assessment of this course plays an indispensable role in shaping
education at Georgia State. Upon completing this course, please take time to fill out the
online course evaluation.
XI.
Career Services
The School of Public Health provides career services & student leadership opportunities
(student clubs & organizations) to all current SPH students and alumni. SPH Career Services
can help students with resume writing, interviewing, job searching, internship development,
and professional networking. Students are invited to attend our career events and
workshops, and individualized career counseling appointments can be arranged. To see
what career panels, career fairs, and events are available this semester, please visit:
http://publichealth.gsu.edu/students/career-resources/. The SPH Career Services office is
co-located with the Office of Academic Assistance in room 640 at One Park Place.
XII.
Tentative course schedule, topics, and readings
Course Schedule
The course syllabus provides a general plan for the course; deviations may be necessary.
August 26
September 2
Program Evaluation Overview
Readings and Assignments
 Rossi Chapter 1- Overview of Program
Evaluation
Conducting Evaluation Research & Preparing an
Evaluation Plan
Class on: Conducting a proper literature review.
Library Research, databases, citation MGMT, etc
by Sharon Leslie in LibNorth Classroom #2
September 9
Readings and Assignments
 Course Research Guide
http://research.library.gsu.edu/ph7521
 Feasibility Standards [On D2L]
Identifying Stakeholders and Developing
Evaluation Questions
Readings and Assignments
 Rossi Chapter 3- Identifying Issues and
Formulating Questions
 Framework for Program Evaluation in
Public Health Practice [On D2L]
 Bryson, J et al (2011) Working with
evaluation stakeholders: A Rationale,
step wise approach and Tool Kit
Evaluation and Program Planning. 34 (112).
September 16
**SMART Questions Due**
Logic Models
Readings and Assignments
 Rossi Chapter 4: Assessing the Need for a
Program
 Rossi Chapter 5: Expressing and
Assessing Program Theory
September 23
**Logic Model Due**
Exploratory & Formative Evaluations
Readings and Assignments
 Wholey Chapter 4 –Exploratory Evaluation
 NIMH Multisite HIV/STD Prevention Trial for
African American Couples Group: Formative
Study to develop the Eban treatment and
comparison interventions for couples.
(2008). J. Acquir Immune Defic Syndr. 49
(S42-S51).
 Sergott et al (2013). Creating safe places: An
exploratory evaluation of a school-based
emotional support service Pastoral Care in
Education. 31(3). 211-228
September 30
Performance Monitoring & Process Evaluations
Readings and Assignments
 Rossi Chapter 6: Assessing & Monitoring
Program Process
 Baquero, B et al. (2014). Process
evaluation of Food marketing and
Environmental Change Intervention in
Tiendas That Serve Latino Immigrants in
North Carolina. Health Promotion
Practice, (1-10).
 Curran, S. et al. (2005). Process
evaluation of a store-based
environmental obesity intervention on
two American Indian reservations.
Health Education Research. 20(6). 719729.
October 7th
Group Evaluation Project Work Day
October 14
** Journal Article Review Paper Due**
Evaluating Evaluations
Student presentations of Journal Article Reviews
October 21
Designing Outcome Evaluations
Readings and Assignments



October 28
** Group 1-1 Meetings
Evaluation Matrix Due**
Rossi Chapters 7: Measuring and
Monitoring Program Outcomes
El-Bassel, N et al. (2010). NIMH Multisite
Eban HIV/STD Prevention Intervention
for African American HIV SeroDiscordant
Couples A Cluster Randomized Trial Arch
Intern Med., (1-8).
May, P. et al. (2005). Outcome
Evaluation of a Public Health Approach to
Suicide Prevention in an American Indian
Tribal Research and Practice. 95(7). 12381244
Types of Data Collection and Quantitative
Analysis
Readings and Assignments
 Wholey Chapter 20- Using Statistics in
Evaluation
November 4
Types of Data Collection and Quantitative
Analysis
Cont..
Readings and Assignments
 Wholey Chapter 11-Agency Records
 Wholey Chapter 12- Using Surveys

November 11
**Evaluation Literature Review Due
Types of Data Collection and Qualitative
(Sections 3-8)**
November 18 Happy APHA Week!
Analysis
Readings and Assignments
 Wholey Chapter 19- Qualitative Analysis
Participation Quiz
Types of Data Collection and Qualitative
Analysis cont..
Readings and Assignments
 Wholey Chapter 16-Conducting Semi
Structured Interviews
 Wholey Chapter 17-Focus Group
Interviewing
Participation Quiz
November 25
THANKSGIVING BREAK-No Class
December 2
** Group Presentations Due**
Group Presentation of Evaluation Protocols
December 9
** Evaluation Protocols Due**
Evaluation Protocols turn into D2L