“Client-Based Program Evaluation”
UCLA: Honors 105
Spring 2006
5 units
Last updated: April 6, 2006
Instructors:
 Kathy O’Byrne, Center for Community Learning.
kobyrne@college.ucla.edu.
 Neal Richman, Urban Planning. nrichman@ucla.edu.
 Peggy Toy, Center for Health Policy Research. peggytoy@ucla.edu.
Classroom and Building: Royce 362 /Public Policy 5391
Day and time of class sessions: Fridays, 10 a.m. to 12:50 p.m.
Course Description
This is a service learning course designed to teach community based research
skills in program evaluation. It is designed for both undergraduates and
community partners working in public health non-profits. Students will learn the
theory and practice of program evaluation, by designing and implementing a
program evaluation study, which benefits the home agency of the staff persons
enrolled. Students will use technology to achieve the course objectives, including
on-line tutorials, listservs and other data collection and analysis programs.
Course Competencies:
Students enrolled in this course will learn
 Basic concepts of health policy
 The philosophical basis for client-based program evaluation;
 The ethical frameworks that guide evaluation research;
 A general menu of options for the design of program evaluation studies;
 General data collection and analysis techniques;
 Best practices for university/community collaborations;
 Critical thinking and reflection when addressing questions posed by the
client organization or agency;
 Definition of service learning and its best practices as a pedagogy in
higher education.
Enrollment of UCLA Undergraduates and Community Partners
Community partners will enroll through UCLA Extension, and receive course
credit. Executive Directors of partnership agencies and community partners will
review and sign a “Memorandum of Understanding” (or MOU) which outlines the
responsibilities and commitments for the course.
UCLA undergraduates enroll through URSA. Maximum number = 15. Report
difficulties to UCLA Center for Community Learning: A265 Murphy Hall, or call
310.825.7867.
Undergraduates and community partners must all be able to use an EXCEL
spreadsheet, email and internet searches.
Community partners will meet as a group (off-campus) for two sessions
(February 14 and March 14) before the spring quarter class begins to discuss the
programs to be evaluated, the research question that will guide the evaluation
and the data that will used to answer the question. Projects must also be of a
reasonable scope and size to allow for successful completion during the spring
quarter. The team of instructors will closely supervise the program evaluation
design.
Definitions and Guidelines
A. Service Learning: Service learning combines service to the community
with student learning in a way that improves both the student and the
community. According to the National and Community Service Trust Act of
1993:
Service Learning:
1. Is a method whereby students learn and develop through active
participation in thoughtfully organized service that is conducted in
and meets the needs of communities
2. Is coordinated with an elementary school, secondary school,
institution of higher education, or community service program and
the community
3. Helps foster civic responsibility
4. Is integrated into and enhances the academic curriculum of the
students, or the education components of the community service
program in which the participants are enrolled
5. Provides structured time for students or participants to reflect on
the service experience
(Corporation for National Service, Learn and Serve)
B. Community-based research is “research conducted by, for or with the
participation of community partners” (Murphy, Scammel & Sclove, 1997).
Community-based research methods offer theory and technique with which
students, faculty and community partners can investigate questions from the
world of practice. They offer a framework for linking research and scholarship
with teaching and service.
C. Principles of good community-campus partnerships:


Partners have agreed upon mission, values, goals, and measurable
outcomes for the partnership.
The relationship between partners is characterized by mutual trust,
respect, genuineness, and commitment.







The partnership builds upon identified strengths and assets, but also
addresses areas that need improvement.
The partnership balances power among partners and enables
resources among partners to be shared.
There is clear, open and accessible communication between
partners, making it an ongoing priority to listen to each need,
develop a common language, and validate/clarify the meaning of
terms.
Roles, norms, and processes for the partnership are established with
the input and agreement of all partners.
There is feedback to, among, and from all stakeholders in the
partnership, with the goal of continuously improving the partnership
and its outcomes.
Partners share the credit for the partnership's accomplishments.
Partnerships take time to develop and evolve over time.
Required Reading
Patton, M.Q. (1997). Utilization-Focused Evaluation (3rd Edition). Thousand
Oaks, CA.: Sage Publications.
Course Schedule
Week
Topic
1: 4/7
Introductions and Course Overview
Instructor(s)
All
Civic Engagement at UCLA.
O’Byrne
Overview of the syllabus and course requirements.
Richman
Community partners present general plans for program evaluation studies:
the program to be evaluated, what the agency wants to know about that
program, and the data that will be used. Undergrads select projects for the
course, creating research teams for the quarter.
Reading for next class: Patton, Part 1 and Part 2
2: 4/14
NKCA
Richman staff
Meet in computer lab to learn mapping software, and have an “Ask CHIS”
demonstration.
Teams meet to discuss and refine plans for program evaluation projects.
program evaluation projects. Homework assignment due week 3.
Reading for next class: Patton, Part 3
3: 4/21
Program Evaluation powerpoint
O’Byrne
Lecture. Discussion of required readings.
Teams meet to discuss and refine plans for program evaluation projects.
4: 4/28
Students Present Evaluation Plans
All
Each group will present their program evaluation topic, design and
methodology and timeline, for feedback and consultation. What is the
research question proposed by the team? What data will be used? How
will data be analyzed? Is the project of reasonable scope and size? These
presentations will be graded.
5: 5/5
Data Collection: using databases & websites
All
Teams meet to discuss and refine plans for program evaluation projects.
Each group emails a progress report to instructors.
6: 5/12
Data Analysis: shrinking, coding and reporting
Toy, O’Byrne
Discussion of all remaining required reading. Distribute midterm essay
questions.
Teams meet to discuss and refine plans for program evaluation projects.
Each group emails a progress report to instructors.
7: 5/19
Formatting Data: organizing charts & graphs
All
Reading for next time: Patton, Chapter 14
8: 5/26
NO CLASS
9: 6/2
Dress Rehearsals!
All
Each group will give a PowerPoint presentation to the class, for review
and feedback before the Finals Week celebration /recognition event for a
wider audience.
10: 6/9
NO CLASS
Finals Week: 6/12-16
Presentation of Findings
All
Presentation of results and findings to agency staff and directors, faculty,
students, administrators, friends and other invited guests.
Location of the final recognition event (with food!) to be determined.
Graded Assignments:
1. Homework assignment, due week 3.(10% of course grade)
2. Group student presentations, held in class week 4. (25% of course
grade)
3. Dress rehearsal presentations, week 9. (25% of course grade)
4. Final PowerPoint presentation, Finals week. (40% of course grade)
Grading Criteria
1. Homework assignment must be complete, follow the instructions provided
in class, be neat, and turned in week 3.
2. Group student presentations must follow the outline provided in class,
demonstrate the contributions of each team member, reference the
required readings and class materials, provide a design that is reasonable
in size and scope, follow a logical progression, be clear, organized and
respectful of the community partner.
3. The dress rehearsal in week 9 must be a complete presentation (20
minutes) that includes a report on all portions and phases of the project.
Students should expect feedback and recommendations for changes,
which can be incorporated before the final event in Finals Week. All
students in the group must demonstrate their contribution to the group’s
work. The presentation should be free of spelling and grammar errors.
The PowerPoint should be clear, organized and include both findings and
an analysis of those findings.
Evaluation of the Evaluations
Once the course is completed, Executive Directors of each participating agency
will be presented with the final reports. They will be asked to rate them according
to their quality, usefulness and responsiveness to the original question posed by
the agency (and presented by partners in week 1 of the course). These
evaluations will be posted on the Center for Community Learning website along
with recommendations for improvement and suggestions for course
modifications.