DRAFT PROGRAM EVALUATION PROCESS
Peralta Community College District
Program Evaluation Process
DRAFT Common Review Criteria for CSEP Review and Discussion
UPDATED: May 22, 2007
This document presents a program evaluation process for use in the 07-08 educational master
planning process. The Committee for Strategic Educational Planning (CSEP) developed the
process as part of their charge to create a common framework for the colleges to use in
educational master planning in keeping with board policy 5.11, which states: “The goal is to
provide accessible, high quality adult learning opportunities to meet the educational needs of
the multicultural East Bay community.” CSEP used examples from other California
Community College districts and input by Educational Services and college staff. This is a
modification of the basic process approved by the Strategic Management Team (SMT) and
reviewed by Strategic Planning and Policy Advisory Committee (SPPAC).
This document presents a set of criteria and a process that will be implemented at the college
level, not in a district-wide process. CSEP developed this as a tool so that the four colleges
have a consistent procedure that establishes a “level playing field” for the four colleges
regarding program evaluation.
Rationale
CSEP developed this model for two primary reasons. First, it is good academic practice to have
clear procedures and processes for evaluating programs at an institution-wide level to ensure
the full set of offerings meets the changing needs of the community. It is central to the mission
of the Peralta colleges that we regularly evaluate the relevance, quality and productivity of our
offerings. The process below provides a collegial framework for evaluating relevant
information and making systematic planning decisions so that the colleges evolve as the
district’s communities evolve. The CSEP process establishes the implementing procedures for
Board Policy 5.1.1, which describes the need for a program evaluation process.
Secondly, the primary recommendation of the Western Association of Schools and Colleges
accreditation report was that that the four colleges conduct and integrated process of
educational planning: “The team recommends that a district wide plan and implementation
process should be created that is strategic and systematically integrates the educational,
financial, physical and human resources of the district. All planning processes should be
inclusive of the four colleges and the communities served by the district. The plan should
include identified institutional outcomes with criteria for evaluation on a periodic basis. It is
recommended that the district wide plan integrate the educational master plans and program
reviews of the colleges. The chancellor should ensure that the plan and the ongoing planning
processes are communicated throughout the district (Standards 3.B.1, 3.B.3, 3.C.3, 10.C.1,
10.C.6.) The CSEP process is one of the core elements of this integrated strategic educational
planning process for the colleges.
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
1
DRAFT PROGRAM EVALUATION PROCESS
Educational Master Plan Update Process
The colleges will use a continuous program improvement process to support all programs in
achieving quality, relevance, and productivity. The colleges will use a common set of
“horizontal” criteria to assess their program offerings.
Guiding Principles
The purpose of the program improvement process is to ensure responsiveness to our
communities and deliver academically sound programs meeting the mission of California
community colleges as defined in the Education Code. The guiding principles for the process
are as follows:
 Provide high quality education;
 Ensure relevance to student and community needs;
 Meet community expectations for sound financial management of public funding (i.e.,
productivity);
 Emphasize revitalizing programs through staff development, community outreach, use of
best practices, and innovative scheduling and delivery; and
 Respect all contract provisions.
Key Terms and Committees
The following are the key terms and organizations referred to in this document
Committee for Strategic Educational
Planning (CSEP)
Purpose: Develop process to be implemented at the
Colleges to evaluate programs at an institutionallevel to ensure relevance, quality and productivity.
Ad hoc.
Membership: Academic Senate Presidents, Vice
Presidents of Instruction, Vice Chancellor –
Educational Services, President, CoA, Karolyn van
Putten.
Strategic Planning and Policy Advisory
Committee (SPPAC)
Purpose: Shared governance committee to provide
advice to the Chancellor on strategic issues.
Strategic Management Team (SMT)
Purpose: Ensure implementation of strategic
initiatives.
Membership: Chancellor, College Presidents, Vice
Chancellors, Associate Vice Chancellor –
Admissions and Records.
College Educational Planning
Committees (EPCs)
Purpose: Recommend a strategic, communityneeds based educational master plan. EPCs will
review program evaluation results, program
reviews, and environmental scan. These
committees have a specific task in developing
college educational master plans in Fall 2007; there
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
2
DRAFT PROGRAM EVALUATION PROCESS
may also be a periodic role for the EPCs to monitor
innovations and planning trends, and to monitor the
implementation of the program evaluation process.
Membership: These committees are in formation.
Program Evaluation Process (PEP)
Purpose: Conduct an evaluation of program
relevance, quality and productivity as described in
this document. The Program Evaluation Process
(PEP) uses a college-level perspective to identify
three categories: Grow / Maintain / Watch. The
PEP uses a consistent set of criteria across the
colleges and encourages cross-college dialog.
Accelerated Program Review (APR)
Purpose: Engage each instructional, student
service, administrative, and service center unit in a
self-assessment of effectiveness. The colleges’
educational planning committees will assess
instructional program together with the results of
the institution-level program evaluation process.
Unit Plans (UPs)
Purpose: Present a detailed plan of action for
improving quality, relevance, and productivity as
indicated by the APR and the PEP. All units –
departments, programs, disciplines – will complete
unit plans. Unit plans will be reviewed and
synthesized into College Educational Master Plans.
Educational Master Plans (EMPs)
Purpose: Present an College-wide plan that is
strategic and comprehensive and gives direction to
all programs and services. The educational master
plans (EMPs) will be based on the data and
analysis in the APR and PEP and will synthesis the
unit plans.
Roles and Responsibilities



The Vice Presidents of Instruction and Academic Senate Presidents will work together to
establish college educational planning committees (EPCs).
These EPCs will be the primary committees to conduct college-wide educational master
plan update activities.
They will develop recommendations for their respective presidents. District educational
services will provide support and data for the Unit Plans.
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
3
DRAFT PROGRAM EVALUATION PROCESS
OVERALL TIMELINE
Action
Draft Due Date
Initiation of Committee for Strategic Educational Planning
(CSEP)
February 27, 2007
Vice Chancellor Haig and CSEP members present draft process at
the Council for Instruction, Planning and Development
May 7
CSEP prepares criteria and process for Educational Master
Planning review.
May 15
Vice Presidents to work with Academic Senates to establish
Educational Master Planning committees.
May / August
Vice Chancellor, Vice Presidents and CSEP members to present
the program review process to faculty at Flex Day
August
College Educational Master Plan committees to apply criteria and
create lists of Grow / Maintain / Watch programs.
October 15
SMT and SPPAC Review for consistency.
October 18, 22
Colleges prepare draft Educational Master Plan updates.
November 30
College Collegial consultation, SPPAC and SMT review.
December 1 – February 1
Definition of “Program”
Programs are defined as:
 Any series of courses leading to a degree, a completion or achievement certificate, or any
other credential of completion or achievement.
 All vocational programs
 Any discipline offered at two or more colleges
 A set of related courses not leading to a degree of certificate that students take in a
sequence of increasing skill levels (“a set of related courses that students navigate”)
 Multi-disciplinary special topics such as biotechnology.
Steps and Criteria
The Educational Master Planning committees at each college will screen all programs using the
following criteria.
1. Review each program’s enrollment trends for the five preceding Fall terms and code as:
A - Growth in each consecutive year
B - Mixture of growth and decline
C - Declining enrollment in each consecutive term
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
4
DRAFT PROGRAM EVALUATION PROCESS
2. Review FTES/FTEF ratio for the five preceding Fall terms and code as:
A - Over 17.5 for all five years
B - Mixed pattern of under and over 17.5
C - Under 17.5 for five consecutive terms
(Step 2 is not conducted for vocational programs that have mandated class size limitations.)
3. For programs with classes offered at more than one college, review classes for the last five
years for class size and code as:
A – Over 15 students per class for all five years
B - Mixed pattern of over and under 15 students per class
C - Under 15 students for five consecutive terms
4. Vocational Programs Only: Review cost of programs, community needs, and presence of
alternative providers in addition to Peralta. (High cost is defined as twice the average cost
that is embedded in the 17.5 FTES/FTEF standard. **This needs to be quantified by
Educational Services staff.) Code programs as
A - Not high cost, or high cost but also high need and insufficient alternative providers
B - High cost program with low community need and/or multiple alternative providers
5. Review each program’s trends for five terms regarding retention, persistence, student
learning outcome achievement, program completion or student satisfaction. This review is
to provide information on improvement regarding student success. Note any major patterns
or trends.
A – Trends and levels generally positive
B - Some trends/levels positive; some declining or too low.
C - Most trends/levels declining or too low.
6. Relevance to community needs and labor market trends as
documented in state and local labor market scans and
community needs information.
The A, B, C ratings were added to
criteria 5 and 6 at the 5/15 CSEP.
Someone may need to add some
quantification to these definitions to
make them easier to use.
A – Programs clearly aligned to community and labor market needs
B - Programs generally aligned, but could use some refinement or updating
C - Programs need to update or refine offerings
7. Relevance to the college’s strategic plan.
Describe the role of each program within the college’s strategic plan. This can be evaluated
qualitatively but should include evaluation and if possible quantification of how important
the program is to the college’s plan. Relevance can include:
 New program under development
 Program that is integral to the college’s overall strategy
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
5
DRAFT PROGRAM EVALUATION PROCESS



Program that is essential for transfer
Program that serves a community niche.
Programs where student enrollment or success has been demonstrably affected by
extraordinary external factors, such as barriers due to housing, employment, childcare
etc.
Program Evaluation Process
Each college Educational Master Plan committee will implement the program evaluation
process with support from the Vice Presidents of Instruction and District Educational Services.
The goal of the screening process is to consider a range of factors in creating a framework for
the future direction of the college’s educational programs. The EPC will quality, relevance,
productivity and the overall strategic direction of the college. The EPC will also consider
collaborative educational strategies across the four colleges. The end result will be a listing of
the college’s programs into three categories.
Description
Process
Grow
New, existing or modified programs that
the college intends to grow.
Complete a Program Improvement Plan.
Review annually.
Maintain
Existing programs that will be
maintained at current levels.
Complete a Program Improvement Plan.
Review annually.
Watch / Revitalize
Programs that do not meet the
standards set in criteria 2 and/or 4.
Prepare a unit plan; review at key
milestones.
Key Considerations
Link to Community Needs and Visioning. The environmental scan and other dialogs will assist
the colleges in crafting a positive vision for the future of the colleges, including emerging areas
or existing needs not currently served. The process below is only one element of the Colleges’
collaborative educational master planning process.
Colleges Can Move More Rapidly. The process below is proposed as a process that all colleges
will implement consistently. This will provide an “even playing field” in which all programs
receive treatment. The process below regarding “watch/revitalize” program, if adopted, would
be the baseline expectation that all colleges would meet. However, a college can make
decisions regarding program continuance on a faster timeline through its consultation process.
Focus on Programs. The focus of the evaluation process is on programs, not on people. The
faculty contract describes the procedures for identifying reassignment and retraining options for
faculty whose programs are discontinued. For example, faculty can develop new FSAs in an
area of growing community need.
Program Evaluation Builds on Program Review. The EPCs will review program reviews in
developing Grow / Maintain / Watch-Revitalize lists as shown on page 9.
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
6
DRAFT PROGRAM EVALUATION PROCESS
Fall
2007
Each college will screen all programs using the criteria above. All programs complete a Program
Improvement Plan (unit plan) in Fall 2007.
Grow and Maintain Programs
Watch / Revitalize Programs
Programs that meet the standards set in
Programs not meeting criteria 1 through 4 complete
criteria 1 through 4 follow the schedule
the following process. Programs at two more colleges
below
are encouraged to collaborate with their counterpart
programs at the other colleges on district-wide
strategies such as consolidation, enhanced
coordination and specialization.
1. Develop and Implement a Program
Improvement Plan (PIP).
 Curriculum approval
 Marketing and outreach
 Instructional changes
1. Develop and implement a Program Improvement
Plan (PIP) (using existing resources?).
 Curriculum approval
 Marketing and outreach
 Instructional changes
Spring
2008
2. Continue Implementing PIP
2. Implement PIP.
Fall
2008
3. Screen against all criteria and update
PIP.
3. Continue implementing PIP. Adjust as necessary.
Spring
2009
4. PIP Implementation.
4. Continue implementing PIP. Adjust as necessary.
Fall
2009
5. Annual review and planning cycle:
Screen against all criteria and update PIP.
5. Evaluate and adjust PIP. (Remove entry-level
courses from schedule if insufficient progress has
been made.)
Spring
2010
6. PIP Implementation
6. Continue implementing PIP. Adjust as necessary.
Fall
2010
7. Annual review and planning cycle:
Screen against all criteria and update PIP.
7. Make decision on Continuance. Reinstitute entrylevel courses if sufficient progress has been made.
Offer final courses if insufficient progress has been
made.
Implementation Steps
1. Educational Services provides




Description of FTES/FTEF formula with outlier examples to assist faculty in
interpreting data.
Description of how concurrent sections are handled and other methods used to compile
data at the program level.
Break down of data to course and sections – as decided by VPI’s and Deans
Definition of “high cost” for vocational programs in criterion 4.
2. At the August 2007 Flex Day, multi-college disciplines discuss data and develop potential
collaborative strategies.
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
7
DRAFT PROGRAM EVALUATION PROCESS
3. [Educational Services, VPIs, and Academic Senates] set up and facilitate meetings for
multidisciplinary special topics like Biotechnology and Multi-media. This would include
other comparable clusters of topics that fall outside traditional disciplines.
4. College Educational Planning Committees apply ALL criteria to each program using
program reviews as input documents and develop college list of Grow / Maintain / WatchRevitalize.
[The steps below are preliminary and need to be refined by Educational Services in
consultation with VPI’s. The steps as they are now are consistent with the spirit of the CSEP
conversation. They need now to be made more operationally specific and clear so that the work
of the college EPCs is consistent and easily accomplished.]
Step 1: Each college Educational Planning Committee reviews the Program Evaluation
Process (i.e., this document and any supporting documentation).
Step 2: Each college Educational Planning Committee reviews the data sorted by criterion 2
(FTES/FTEF).
Step 3: EPC identifies programs that receive a C score on criterion 2 – i.e., have not met the
17.5 standard for the last five terms and/or receive a C score on criterion 4, i.e., -- are high
cost/low need programs. These are programs that could potentially be on the “watch list”.
Step 4: EPC reviews the “potential watch” programs against the College’s overall
educational strategy according to criterion 7. Also, programs that receive C’s on criteria 5
and 6 should be discussed.
Step 5: The EPC develops a draft Grow / Maintain Watch list and provides it to the
President. With presidential approval, program revitalization process starts for “watch”
programs.
Step 6: Disciplines and departments use criteria and data to prepare unit plans using the
following overall concepts:
 The Unit Plans will include comprehensive productivity plan that addresses the discipline’s
overall productivity trend, i.e., balance high productivity with lower productivity in growth
areas.
 Identify Growth / Maintain / Sunset programs within the disciplines and departments.
 Include contract and non-credit and other innovative options in discipline plans.
Step 7: EPC reviews draft unit plans and compiles the College educational master plan.
Potential Unit Plan Outline Elements
I.
II.
DRAFT
Introduction
Evaluation
A. Criteria Review and Discussion
B. Strategic Issues
C. Opportunities
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
8
DRAFT PROGRAM EVALUATION PROCESS
D. Best Practices
III.
Plan
A. Discipline/Department Growth and Productivity Balancing Plan
Growth Areas
Maintain Areas
Watch/Revitalize Areas
B. Curriculum Changes
C. Community Outreach and Marketing
D. Delivery and Scheduling Changes
IV.
Plan to Evaluate Improvement
Flow Chart
The process includes the overall elements below.
Program
Reviews
Aug 07 Flex Day – Multi
College Discipline
dicussions
College Lists
College
Educational
Planning
Committee
Grow
Maintain
CSEP
Process
and
Criteria 2
Sort
Special Meetings for
Biotech, Media, etc.
Apply all criteria
Watch
Evaluate program
reviews
August – Sep 2007
Miscellaneous Questions to Answer
Are advisory committees meeting and providing the right type of input, especially helping
identify new programs?
Is it feasible to develop an innovation fund to pay for development of new programs?
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
DRAFT
9