College of Arts and Sciences
Political Science Department
Master of Public Administration Program
PA6631 XTIA
Program Evaluation
COURSE SYLLABUS
Term 2 - 2015
October 12 – December 13, 2015
For a course syllabus posted prior to the beginning of the term, the instructor reserves the right to make
minor changes prior to or during the term. The instructor will notify students, via e-mail or Blackboard
announcement, when changes are made in the requirements and/or grading of the course.
eTROY Courses at Troy University
All eTROY courses at Troy University utilize the Blackboard Learning System. In every eTROY course,
students should read all information presented in the Blackboard course site and should periodically check
for updates-at least every 48 hours. To learn how to log in to and use Blackboard, visit the eTROY web
page at www.troy.edu/eTroy and click on the Blackboard link. The eTROY staff at Troy University is here
to serve you and to assist with any questions, problems, or concerns you may have. Current contact
information for all staff members can be found at the web site. Please do not hesitate to contact the eTROY
staff if you need administrative assistance for any reason. Remember: This is not a “correspondence
course” in which a student may work at his/her own pace. Each week there are assignments, online
discussions, online activities and/or exams with due dates. Refer to the schedule in the syllabus for more
information.
PROFESSOR INFORMATION
Instructor: Dr. Robert F. Abbey, Jr., Associate Professor, MPA Program, Department of Political Science
Mailing Address: 51 Lonsdale Street, Geelong, VIC 3220 Australia
Contact Phone: 301-464-3029 (leave voice mail)
Troy Email Address: rabbey@troy.edu
Email is the preferred method of contacting me; I will try to respond within 24 hours. If you prefer, please
telephone anytime (since I am not there) and leave a voice message with your name, telephone number, and
preferred time(s) to call; there is a +15 hour time difference between CDT and Melbourne.
ELECTRONIC OFFICE HOURS
My email is enabled constantly; I will try to respond within 24 hours, usually within the hour.
Instructor Education and Background are posted on Blackboard in Course Documents
MPA eQuad
As an active MPA student you have access to the MPA eQuad located on your eTROY Blackboard page
under "Organizations". The MPA eQuad is your immediate access point to information and links for
suggested course sequencing, course concentrations, registration, MPA and eTROY forms, major program
requirements, professional opportunities, MPA program orientation, faculty advising, "live" chat access and
much more. Visit the MPA eQuad 24/7 for the latest program happenings and to review the available
resources. Please take approximately four minutes to view this video link to find out what the MPA eQuad
can do for you:
http://trojan.troy.edu/artsandsciences/politicalscience/mpa/how-to-access-eQuad-advising.html
COURSE INFORMATION
Prerequisites: Students must successfully complete PA 6601 prior to taking PA 6631.
Course Description:
An overview of the theoretical foundations and techniques of program evaluation including need assessments,
outcome evaluations, surveys, program outcomes and impact evaluations. Prerequisite: PA 6601 Research
Methods in Public Administration
Student Learning Outcomes (SLOs):
Students completing this course should be able to:
1. Discuss the diverse purposes that program evaluation can serve and the wide ranges of environments in
which evaluations are conducted.
2. Develop capabilities to critique and assess program evaluations.
3. Describe the conceptual issues and primary methods, including both quantitative and qualitative, for
conducting evaluations.
Desired Competency:
Students will be able to identify, design and interpret a formative, summative and confirmative evaluation
conducted in a public setting using qualitative and quantitative designs.
Student Expectation Statement
The student is expected to participate in the course via Blackboard postings, email exchange or other
communication with the instructor by reading the assigned readings, submitting comments to the discussion
forums, submitting assignments, and completing exams in a timely fashion. Students are expected to check
their Troy email daily and login to Blackboard at least every 48 hours.
Learner Engagement:
Evaluations of each grading component will be provided in separate emails and/or voicemails to each student.
Grades will be calculated as follows: A = 4.0; A- = 3.7; B+ = 3.3; B = 3.0; B- = 2.7; C+ = 2.3; C = 2.0; C- =
1.7; D = 1.0; F = 0.0. The final grade for the course will be determined by the same method used in calculating
a “grade point average” (GPA), i.e., the grade received for each evaluative component weighted by the
percentages of each. The final grade issued for the course will not reflect the plus and minus system used for
grading course components.
Internet Access:
This is an on-line class. Students must have access to a working computer and access to the Internet. “Not
having a computer” or “computer crashes” are unacceptable excuses for late work. Have a backup plan in
place to insure access should you have computer problems.
Useful Web Sites for This Course:
See the Essential Website Links section in the course Blackboard site.
Other Course Information:
Professor’s Philosophy
Substantive emphasis will be placed on familiarizing students with the concepts of program evaluation (1) by
providing them with experience in examining problems from multiple perspectives, and (2) by analyzing how
modern program evaluation theories have evolved. Quality of participation is more important than quantity.
This course emphasizes breadth in coverage rather than depth. Identification of deficiencies and difficulties is
at least as important as what is already known by each student. Students should think of the time allotted for
each topic as a finite resource and plan accordingly. This course is not designed to make you a program
evaluator, but rather to make you a more effective administrator.
Teaching Methodology
Extensive homework will be required in the form of assigned readings, videos, quizzes, preparation of
essays and critiques, and one proctored exam. There will be posted questions on Blackboard each week;
each student will be required to post answers and responses each week.
Post all questions on Blackboard or email them to all students. This will facilitate information exchange.
A new “Discussion Board” forum will be created for each of the sessions.
REQUIRED TEXTBOOKS AND ARTICLES (articles will be posted on Blackboard)
Handbook in Research and Evaluation, Third Edition, by Stephen Isaac and William B. Michael, EdITS, San
Diego, CA, 1995. [ISBN: 0-912736-32-1]
Program Evaluation: Alternative Approaches and Practical Guidelines, 4th Edition, by Jody L. Fitzpatrick,
James R. Sanders, and Blaine R. Worthen, Pearson, Boston, MA, 2011. [ISBN: 978-0-205-57935-8]
McKean, Kevin, "Decisions, decisions," Discover, 6, 6 (June), 1985, 22-31.
Mitroff, Ian I., and Pondy, Louis R. "On the Organization of Inquiry: A Comparison of Some Radically
Different Approaches to Policy Analysis," Public Administration Review, 34, 5 (September/October), 1974,
471-479.
Paul, Christian F., and Gross, Albert C., “Increasing Productivity and Morale in a Municipality: Effects of
Organization Development,” Journal of Applied Behavioral Science, 17, 1, 1981, 59-78.
The textbook provider for eTROY is Barnes and Noble. For instructions on how to access the bookstore,
please visit: http://troy.bncollege.com Students should have their textbooks from the first week of class.
Not having your textbook(s) will not be an acceptable excuse for late work. Students who add this course
late should refer to the “Late Registration” section of the eTROY Policies and Procedures for further
guidance.
RECOMMENDED FOR FURTHER INFORMATION
Evaluation: A Systematic Approach, Seventh Edition, by Peter H. Rossi, Mark W. Lipsey, and Howard E.
Freeman, Sage, Thousand Oaks, CA, 2004.
Evaluation in Practice: A Methodological Approach, Second Edition, by Richard D. Bingham and Claire L.
Felbinger, Longman, White Plains, NY, 2004.
Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation. E. Jane Davidson, Sage, Thousand
Oaks, CA, 2005. [ISBN: 0-7619-2930-4]
How Effective Are Your Community Services?, by Harry P. Hatry, et al., The Urban Institute, Washington,
D.C., 1992.
Program Analysis for State and Local Governments, by Harry Hatry, et al., The Urban Institute, Washington,
D.C., 1987.
RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints, 2nd Edition by Michael
Bamberger, Jim Rugh, and Linda Mabry, Sage, Thousand Oaks, CA, 2012. [ISBN: 978-1-4129-7962-7]
American Evaluation Association, Task Force on Guiding Principles for Evaluators. (1995).
“Guiding Principles for Evaluators.” New Directions for Program Evaluation, no. 66 (pp. 19-26). San
Francisco: Josey-Bass. Available free online at http:www.eval.org.
Pact, Inc. (2014). Field guide for evaluation: How to develop an effective terms of reference. Available free
online at http://pactworld.org/sites/default/files/Field%20Guide%20for%20Evaluation_Final.pdf
W. K. Kellogg Foundation. (2004). Evaluation handbook. Available free online
http://www.epa.gov/evaluate/pdf/eval-guides/evaluation-handbook.pdf
W. K, Kellogg Foundation. (2004). Logic model development guide. Available free online at
http://www.epa.gov/evaluate/pdf/eval-guides/logic-model-development-guide.pdf
Weiss, C. H. (1972). Evaluation research: Methods of assessing program effectiveness. Englewood Cliffs, NJ:
Prentice Hall.
Wholey, J.S.; Hatry, H. P.; and Newcomer, K. E. (Eds.) (1994). Handbook of practical program evaluation.
San Francisco: Jossey-Bass.
If statistical concepts need to be refreshed, the following are recommended:
Reasoning with Statistics: How to Read Quantitative Research, by Frederick Williams, Holt, Rinehart and
Winston, New York, NY, 1994.
Statistics: A Tool for Social Research, by Joseph F. Healey, Wadsworth, Belmont, CA, 2014.
Retention of Textbooks for Use in PA 6699 Capstone in Public Administration
You are required to apply concepts from the MPA core courses in PA 6699. Accordingly retain your
textbooks from this course to support work in the PA 6699 Capstone in Public Administration course.
CLASS SCHEDULE (indicative, not definitive)
Session
Topic
Assignment: chapters
Week 1
(Oct 12+)
SLO: 1, 3
Introduction to Evaluation
FSW: 1, 2, 3
I&M: 1, 9
M&P: all
Week 2
(Oct 19+)
SLO: 1, 3
Alternative Approaches to Program Evaluation
Expertise- and Consumer-Oriented Approaches
FSW: 4
FSW: 5
I&M: 2, 3
Week 3
(Oct 26+)
SLO: 1, 3
Program-Oriented Evaluation Approaches
Decision-Oriented Evaluation Approaches
FSW: 6
FSW: 7
I&M: 3
McKean: all
Week 4
(Nov 2+)
SLO: 1, 3
Participant-Oriented Evaluation Approaches
Comparative Analysis of Approaches
Cultural Competence and Capacity Building
FSW: 8
FSW: 10
FSW: 9
Week 5
(Nov 9+)
SLO: 2, 3
Clarifying the Evaluation
Setting Boundaries and Analyzing Contexts
Two-hour Exam to be taken November 9 - 12
FSW: 11
FSW: 12
Week 6
(Nov 16+)
SLO: 2, 3
Identifying and Selecting Evaluation Questions and Criteria FSW: 13
Planning How to Conduct the Evaluation
FSW: 14
I&M: 2, 3
Week 7
(Nov 23+)
SLO: 2, 3
Collecting Evaluative Information: Design, Sampling, Cost FSW: 15
Experimental and Quasi-Experimental Designs
I&M: 4
Week 8
(Nov 30+)
SLO: 2, 3
Collecting Evaluating Information: Data Sources, Methods FSW: 16
I&M: 5, 6
Journal Article Critique due 11:59 p.m. CT, Dec 2
P&G: all
Week 9
(Dec 7+)
SLO: 1, 2, 3
Reporting Evaluation Results
FSW: 17
I&M: 9
The Future of Evaluation
FSW: 18
Program Evaluation Proposal due 11:59 p.m. CT, December 9
FSW = Program Evaluation: Alternative Approaches and Practical Guidelines
I&M = Handbook in Research and Evaluation
M&P = "On the Organization of Inquiry: A Comparison of Some Radically
Different Approaches to Policy Analysis"
P&G = “Increasing Productivity and Morale in a Municipality: Effects of Organization Development”
McKean = “Decision, Decisions”
COURSE REQUIREMENTS
Details regarding all assignments and exams are posted in the “Assignments” folder in Blackboard.
COURSE REQUIREMENTS
Blackboard Exercises (24%): Please see schedule of readings and assignments for details. Students are
expected to read all assigned material and participate in class activities. Participation in the various
assignments is an essential component of the course and will enhance understanding of the course material.
Students will earn points for timely and substantive participation in assigned Discussion Board exercises.
Timely means completed within the timeframe the various exercises are scheduled. Timeliness of
participation is considered when assigning grades for interactive exercises and other assigned work.
Substantive means discussion having relevance and importance to the subject at hand. Substantive dialogue
responses demonstrate critical thought and are to focus on understanding concepts and their application to real
world scenarios. To that end, please strive toward effectively using citations (from text and other sources) to
supplement your logical analysis and give extra credibility to your positions. This also helps me evaluate your
answers in terms of (1) your familiarity with the assigned readings, (2) your understanding of the concepts, and
(3) your ability to apply the material in a dialogue with colleagues in relation to discussion board threads.
Journal Article Critique (30%): Critique an assigned article using the criteria that will be provided on
Blackboard. Turn in your critique as scheduled in the syllabus. The purpose of this exercise is to familiarize
students with the variety of research designs used in program evaluation.
Evaluation Research Proposal (30%): Students will apply the concepts of this course by developing an
evaluation research proposal for a program of interest to them. The student will develop the proposal up to the
point of actual data collection. You are expected to be the sole author of original, unpublished work, which
has not been used to fulfill the requirements of another course. The final product will include an introduction
to the program being evaluated, a synthesis of literature related to the program, and a discussion of the
methodology to be used. The length of the paper must be about 20 double spaced pages (5000 words). Details
on the evaluation proposal format will be provided in Blackboard.
Examination (16%): A two-hour, proctored, exam to be taken November 9 – 12; exam must be completed by
5:00 p.m. CT on Thursday, November 12. The exam will cover the material in Chapters 1 – 10 in Program
Evaluation by Fitzpatrick, Sanders and Worthen (2011) and Chapters 1 – 3 in Handbook in Research and
Evaluation by Isaac and Michael (1995).
Examination Schedule & Instructions
Proctored Examination
There is a two-hour timed, proctored, exam in this course. It is to be completed between November 9 – 12, 2015;
the exam must be completed by 5:00 p.m. CT, Thursday, November 12, 2015!!!
You are responsible for choosing an acceptable proctor and submitting the online proctor form to eTROY
by the required deadline. Instructions for doing are under the “Proctor Info” area in Blackboard. All
questions about the proctor form or proctor options should be directed to eTROY. Internet will not be
permitted except to login to Blackboard. The Troy University integrity code will be enforced.
METHOD OF EVALUATION AND ASSIGNMENT OF GRADES
Program Evaluation Proposal [due December 9, 2015]
Discussion Board Weekly Exercises in Blackboard [6 @ 4%]
Exam [2-hour timed, proctored, Nov 9 – 12; completed by 5:00 p.m. CT, Nov 12]
Journal article critique [due December 2, 11:59 p.m.
30%
24%
16%
30%
Every assessment exercise will be evaluated and a letter grade assigned. Grades will be calculated as
follows: A = 4.0; A- = 3.7; B+ = 3.3; B = 3.0; B- = 2.7; C+ = 2.3; C = 2.0; C- = 1.7; D = 1.0; F = 0.0. The
final grade for the course will be determined by the same method used in calculating a “grade point
average” (GPA), i.e., the grade received for each evaluative component weighted by the percentages of
each. The final grade issued for the course will not reflect the plus and minus system used for grading
course components. All course work must be completed to receive a passing grade. I do not use the
“accumulated points” method of grading because it is inherently unfair. If you need an analogy to the
“accumulate points” method, please consider the following:
Program Evaluation Proposal [due December 9, 2015]
Discussion Board Weekly Exercises in Blackboard [6 @ 4%]
Exam [to be taken November 9 – 12; completed by 5:00 p.m. CT, Nov 12]
Journal article critique [due December 2, 11:59 p.m.]
A:
B:
C:
D:
F:
30 points
24 points
16 points
30 points
90 – 100 points
80 – 89 points
70 – 79 points
60 – 69 points
< 60 points
There is an additional grade possible: “FA” indicates the student failed due to attendance. This grade is given
to any student who disappears from the course for three or more weeks. See the Attendance section of this
syllabus for additional information.
COURSE POLICIES
** All assignments, paper, exams are to be accomplished individually with no assistance from others
either in or out of the course. There is to be no collaborating between students on graded materials
in this course unless otherwise directed by the instructor when responding to others discussion board
entries. **
Submitting Assignments
Guidelines for the Discussion Board answers and critiques are found in the Assignments section of
Blackboard. Guidelines for the dimensional analysis paper are found in the Assignments section of
Blackboard; it is to be in Standard English using a 12-point font, 1-inch margins, and double spacing in MS
Word format. The American Psychological Association (APA) guidelines are the standards for writing and
referencing papers in the MPA program. Use the APA Research Style guidelines found on Blackboard or
at: http://www.troy.edu/writingcenter
Details regarding all assignments are posted in the “Assignments” folder in Blackboard.
In terms of course assignments, students are expected to prepare thoroughly for each session, to submit all
written materials on time, to achieve graduate school quality standards, and to participate actively and
constructively in class. APA Style Manual shall be used. Late papers will incur at least a one full-grade
reduction from what would have been the original grade; if more than one full day (24 hours) late, the
grade for that assignment will be an F! No make-up exams will be given! All course requirements must
be completed to receive a passing grade in the course, even if the submission is late. All work must be at
least 80% original.
eTROY Policies and Procedures
Click on the Syllabus tab at the Blackboard site that supports this course for the link to the following
important eTROY Policies and Procedures:
Method of Instruction
Student/Faculty Interaction Expectations
Internet Access Requirements
Technology Requirements
Technical Support Center
Troy E-Mail
Textbooks(s) and/or Other Materials Needed
Library Support
Honesty and Plagiarism Policy
Late Registration
Attendance Policy
Make-Up Work Policy
Incomplete Grade Policy
Non-Harassment Policy
Adaptive Needs (ADA) Policy
Faculty Evaluation
eTROY Contact Information
Standards of Conduct and Plagiarism
The awarding of a university degree attests that an individual has demonstrated mastery of a significant
body of knowledge and skills of substantive value to society. Any type of dishonesty in securing those
credentials therefore invites serious sanctions, up to and including suspension and expulsion (see Standards
of Conduct and Disciplinary Procedures section in The Oracle: The Troy University Student Handbook and
the University Wide Regulations section in the Troy University Graduate Catalog). Examples of
dishonesty include actual or attempted cheating, plagiarism, or knowingly furnishing false information to
any university employee.
Be sure to review the Troy Plagiarism Statement Approved by the Academic Steering Committee
(June, 2011), which may be accessed in Blackboard under the Syllabus Button.
Plagiarism is defined as submitting anything for credit in one course that has already been submitted for
credit in another course, or copying any part of someone else’s intellectual work – their ideas and/or words
– published or unpublished, including that of other students, and portraying it as one’s own. Proper
quoting, using strict APA formatting, is required.
Students must properly cite any quoted material. No assignment may have more than 20% verbatim from the
internet and other sources. Students who need assistance in learning to paraphrase should ask the instructor
for guidance and consult the links at the Troy Writing Center.
This university employs plagiarism-detection software (Turnitin) through which all written student
assignments are processed for comparison with material published in traditional sources, books, journals,
and/or magazines, on the internet to include essays for sale and papers turned in by students in the same
and other classes in this and all previous terms.
If the detection software indicates a high degree of suspected “copying,” the student will receive at least an
F for that assignment and, if severe enough, an F in the course and/or expulsion from the university. All
assignments must be at least 80% original.
Students should carefully study the definitions of cheating and plagiarism:
1. Cheating includes:
a) Copying, or relying upon, another student’s answers or submitting another student’s work as one’s own
or submitting as new work assignments previously completed for another class, while completing any class
assignment, study group assignment, or during in-class or take-home examinations.
b) Providing one’s own answers to another student while completing any class assignment, study group
assignment (except where approved by the instructor due to the nature of the assignment itself), or during
in-class or take-home examinations.
c) Using notes, books, or any other unauthorized aids during an examination, or holding an unauthorized
discussion of answers during in-class examinations.
2. Plagiarism is submitting a paper, other required student course requirement in which the language, ideas,
or thoughts are identical to published or unpublished material from another source, including material
found on the Internet, without correctly giving credit to that source. A good rule of thumb for correctly
crediting a source is found in the citation below:
"Quotation marks should be used to indicate the exact words of another. Summarizing a passage or
rearranging the order of a sentence and changing some of the words is paraphrasing. Each time a source is
paraphrased a credit for the source needs to be included in the text. … The key element of this principle is
that an author does not present the work of another as if it were his or her own work. This can extend to
ideas as well as written words." (Publication Manual of the American Psychological Association, 2009)
COURSE DESIGN MATRIX
This table shows how each Student Learning Outcome will be met – to include the module of instruction,
the specific course content, the learning strategy and the assessment mechanism.
Learning Objective
Module
1 Discuss the diverse
Weeks
purposes that program
1,2,3,4,9
evaluation can serve and
the wide ranges of
environments in which
evaluations are conducted.
Content
Chapters 110; lecture
notes,
exercises
Strategy
Class discussion on
core concepts and
relating them to
program
evaluations
Assessment
Exam, discussion
board assignments
2
Develop capabilities to
Weeks
critique and assess program 1,2,3,4,9
evaluations.
Chapters 1114; lecture
notes,
exercises
3
Describe the conceptual
issues and primary
methods, including both
quantitative and
qualitative, for conducting
evaluations.
Chapters 1517; lectures
notes,
exercises
Class discussion of
approaches to
critiquing studies
and procedures for
developing the
evaluation proposal
Class discussion of
issues
Journal article
critique, program
evaluation
proposal,
discussion board
assignments
Journal article
critique, program
evaluation
proposal,
discussion board
assignments
Weeks
5,6,8,