ECON 7300
Evaluation of Public Programs and Policy
Fall 2012
Time:
Mondays 7:00 – 10:00 pm, (September 10 start)
Location:
Office Hours
Instructor
Contact:
TBA
Monday 5:00 – 7:00 pm (or by appointment)
Greg Mason
Phone ……. 987-2030 (PRA Inc.
e-mail ……. gregory.mason@ad.umanitoba.ca.
Students are encouraged to use
e-mail since this is the fastest way to contact me
Web Site
www.gregorymason.ca (under construction)
Overview
Program evaluation is social science research methodology applied to the development and enhancement of policy,
programs, and organizations. Much of the theoretical base for evaluation draws on education, psychology and other
social sciences. This course public sector programs and policies using principles of economics and policy sciences.
Course objective:
The goal of this course is to provide students with advanced tools of analysis and a critical understanding of the
foundations and assumptions of modern program evaluation. The perspective is evaluation practice as reflected by the
recent Evaluation Policy released by the Treasury Board of Canada, work of the General Accounting Office in the United
States, and requirements of non-governmental organizations, particularly in developing countries.
Course organization:
The course will examine three themes:



rationale for public sector interventions
methods for assessing program design, implementation and impact
application of value for money concepts and measurement to identify superior program and policy
approaches.
The goal is to provide students with advanced tools of analysis and a critical understanding of the foundations and
assumptions of modern program evaluation. Note this is the core course in the graduate specialization in program
evaluation.
Prerequisites:
As a graduate course, students need an understanding of social sciences and statistics to the upper undergraduate level.
While this course will use the principles of micro-economics, the core ideas will be developed throughout the course to
allow those without formal training in economics to participate fully. The course will not be overly technical, but students
should be comfortable with the theory and concepts used in social sciences methods courses.
Course Time and Location
The course will be conducted Mondays from 7 – 10 pm Room 300 Tier. Each session will comprise a general lecture by
the instructor followed by student led discussions of readings and problems.
Texts
The course text will use material will draw from instructor notes and readings made available on the web site. In some
cases students will be required to access on-line materials from other sources (University of Manitoba Library e-journals.)
Materials will be place on the instructor website – www.gregorymason.ca – by the first class.
Mark Allocation:

Interim Papers (30%): Students will prepare element of an evaluation framework and design based on a
program of their choosing (15%) and a detailed methodological plan (10%). Suggested topics appear below.
The interim paper is intended to become a major component of the final paper. In this way the written work
integrates into a unified document central to modern evaluation.

Discussion issues (20%): These appear as questions under each lecture heading on the web site and will be
based on the readings and materials for that day. Students will take turns serving as primary resources and
leading the discussion using the questions. 10 of the 20 marks will derive from a confidential subjective
valuation of each student’s contribution to these discussions based on peer evaluation (conducted on the last
day of class) and 10 marks will be based on instructors equally subjective valuation.

Final Paper and Presentation (50%) (15% presentation comprising written summary and oral presentation
and 35% final paper). The final term paper will consist of the preparation of a full evaluation plan, with special
attention paid to the challenges and mitigation strategies in data collection, analysis and utilization. The two
interim papers will be integrated with this final element to present a full evaluation plan.
DATE
TOPIC (Consult the web site for specific readings and materials)
1. Sep 10
2. Sep 17
3. Sep 24
Rationale for government intervention – policy/program primer
Introduction to Evaluation – origins, evolution and key issues.
Describing the Program – Program theory and logic models
Creating the evaluation matrix/plan
Case example: Evaluation plan for the Chapter Evaluation of Business Risk Management
Programs
4. Oct 1
Oct 8
5. Oct 15
6. Oct 22
7. Oct 29
8. Nov 5
9. Nov 12
10. Nov 19
11. Nov 26
12. Dec 3
Dec 13
Thanksgiving
Attribution - linking inputs, activities, outputs and outcomes
Interim Paper 1 – Program profile and logic
Data collection 1 – Qualitative data collection and analysis – interviews, case studies
and focus groups
Data collection 2 – Quantitative data collection and analysis – surveys
Experimental and quasi-experimental methods for policy evaluation
Case Example: Cost effectiveness of screening programs for three cancers
Evaluating economy, efficiency and effectiveness
Case Example: Evaluation of the Canada-Manitoba Labour Market Agreement for Persons
with a Disability
The Locus of Evaluation – Managing the evaluation function
Case Example: Evaluation of the federal response to the BSE crisis
Interim Paper 2 – Evaluation methodology
Emerging themes and challenges
Case example: Evaluation of Canada role in the 2010 Olympics
Class Presentations (Students may submit their presentations and complete a test run
during office hours on Nov 19 and 26, but not immediately prior to the class on Dec 3.)
Final paper due
Written work
Will only be accepted in Word, using standard academic referencing (APA)
Interim Paper 1: (Due Oct 15, midnight – via e-mail)
The first paper prepares a profile, program theory/logic, and an evaluation matrix. Try to keep the framework to 10
essential questions. (Expected length 10 pages – 2500 words max). Feedback will be provided by return e-mail using
track changes and comments.
Interim Paper 2: (Due November 19, midnight – via e-mail)
Using the program that served as the basis for Interim Paper 1, prepare a methodology plan to collect outcome measures
for the evaluation framework you prepared. You should link the evaluation issues and question to indicators, identify data
sources and then describe the data collection and analysis method. (Expected length 10 pages)
Final Paper (Presentation on Dec 3 and final paper Due Dec 13 – mid-night)
In the final paper, students will prepare a full evaluation framework (25 - 30 pages maximum) that lays out a completed
plan for an evaluation of the selected program. The two interim papers will be integrated in to the final paper, amended as
indicated by comments from the instructor.
Students will make brief (10 minute) presentation in the class (last lecture). This is not much time, so be efficient and
focused. Please distribute a synopsis of your paper (3 - 7 pages) electronically to the rest of the class no later than
Sunday evening Dec 2 or bring the presentation to class for distribution.
If you are unaccustomed to making executive style presentations, please consider completing a test run during the office
hours on Nov 19 or 26.
Topics
Potential topics will be circulated at the first class. Students may also propose their own topic.
Additional Points (Fine Print)
1.
Evaluative feedback (30%) will be provided by the voluntary withdrawal deadline date, which is: November 14, 2012 for Fall
Term Courses
2.
In the interests of equity and fairness, students will generally not be permitted to submit in assignments late, except for ted
medical or compassionate reasons.
3.
Students should acquaint themselves with the University’s policy on plagiarism, cheating, exam impersonation, (“Personation
at Examinations” (Section 5.2.9) and “Plagiarism and Cheating” (Section 8.1)) and duplicate submission by reading
documentation provided at the Arts Student Resources web site at http://www.umanitoba.ca/faculties/arts/student/index.html
Ignorance of the regulations and policies regarding academic integrity is not a valid excuse for violating them
4.
Students appealing any term work whether must do so within 10 working days of receiving their mark.
5.
All submitted work will be returned to students electronically.