University of Colorado
at Boulder
Academic Program Review
Procedures
revised September 9, 2003
D:\98883065.doc_7/12/2016_8:40 AM
TABLE OF CONTENTS
INTRODUCTION AND STATEMENT OF POLICIES ........................................................................... 1
I.
UNIVERSITY OBJECTIVES OF PROGRAM REVIEW ............................................................................... 1
II. PROGRAM REVIEW PANEL MEMBERSHIP ........................................................................................... 1
III. PROGRAM REVIEW PANEL RESPONSIBILITIES .................................................................................... 2
IV. PROGRAM REVIEW REPORTS AND SCHEDULE................................................................................... 2
THE SELF-STUDY ....................................................................................................................................... 3
I.
THE SELF-STUDY TEAM..................................................................................................................... 3
II. GUIDELINES FOR THE SELF-STUDY PROCESS .................................................................................... 3
III. THE SELF-STUDY REPORT ................................................................................................................. 4
SECTION I. DESCRIPTION AND ANALYSIS OF THE UNIT ..................................................4
SECTION II. RESPONSE TO PREVIOUS PRP RECOMMENDATIONS ....................................7
SECTION III. UNIT GOALS AND PLANS ...........................................................................7
SECTION IV. REQUIRED APPENDICES .............................................................................8
INTERNAL REVIEW COMMITTEE (IRC) ........................................................................................... 12
I.
PURPOSE OF THE INTERNAL REVIEW ............................................................................................... 12
II. SELECTING THE INTERNAL REVIEW COMMITTEE............................................................................. 12
III. GUIDELINES FOR THE INTERNAL REVIEW COMMITTEE VISITS AND REVIEW .................................. 12
IV. GUIDELINES FOR THE REPORT OF THE INTERNAL REVIEW COMMITTEE ......................................... 14
V. MEETING WITH THE EXTERNAL REVIEW COMMITTEE .................................................................... 14
THE EXTERNAL REVIEW COMMITTEE (ERC) ............................................................................... 15
I.
COMPOSITION AND SELECTION........................................................................................................ 15
II. EXTERNAL REVIEW COMMITTEE’S VISIT AND REVIEW ................................................................... 15
III. THE EXTERNAL REVIEW REPORT..................................................................................................... 16
FINAL REPORT ......................................................................................................................................... 18
I.
PREPARATION .................................................................................................................................. 18
II. SUBMISSION ..................................................................................................................................... 19
DEANS AND PROGRAM REVIEW ........................................................................................................ 20
PANEL MEMBER/LIAISON RESPONSIBILITIES .............................................................................. 21
UCB - Academic Program Review Procedures
page 1
INTRODUCTION AND STATEMENT OF POLICIES
I.
II.
UNIVERSITY OBJECTIVES OF PROGRAM REVIEW
A.
The University of Colorado Board of Regents and the Colorado
Commission on Higher Education require regular review of all
academic programs. These reviews are conducted on a 7-year cycle
that started in Academic Year 1980-81. Reviews involve systematic
procedures designed to identify strengths and weaknesses of academic
programs. They result in recommendations for program development
and modification. The ultimate goal is promoting and maintaining
high-quality academic programs that are administered efficiently,
consistent with the Institution’s role and mission. The Board receives
periodic reports based on and resulting from the reviews.
B.
The Boulder Campus Pursuit of Excellence Task Force Report (5/94)
reaffirmed the basic premise of the UCB process, namely, “Program
Review facilitates planning by the unit, campus, and university
administration to inform decisions regarding resource allocation,
faculty staffing, program focus, admission standards, curriculum
content, and other academic matters.”
C.
A Program Review Panel (PRP) is appointed to conduct the review
process and to make recommendations for program improvements.
Implementation of PRP recommendations is contingent on availability
of resources and consistency with Campus plans.
PROGRAM REVIEW PANEL MEMBERSHIP
A.
The Associate Vice Chancellor for Faculty Affairs, acting as the
designee of the Provost, serves as PRP Chair, without voting rights.
B.
A number of faculty members equal to at least the number of units to
be reviewed in a given year comprise the voting members of the Panel.
They are chosen from a list jointly arrived at after consultation by the
Provost and the Boulder Faculty Assembly. Each faculty member
serves a three-year term with reappointment possible. The Panel Chair
designates each of the faculty members as a primary liaison for one
unit under review.
C.
Two students, an undergraduate appointed by the University of
Colorado Student Union (UCSU), and a graduate student appointed by
the United Government of Graduate Students (UGGS), are non-voting
members on the Panel.
D.
Non-voting observers are the Provost, the Associate Vice Chancellor
for Diversity and Equity, and the Chair of the Boulder Faculty
Assembly (or designee).
revised 9/2003
UCB - Academic Program Review Procedures
E.
III.
page 2
An Assistant to the Chair coordinates and documents all proceedings.
PROGRAM REVIEW PANEL RESPONSIBILITIES
A.
Consult with units on each stage of the review process, in a timely, fair,
and professional manner.
B.
Select and notify units to be reviewed two years hence.
C.
Select an Internal Review Committee for each review, comprised of
three regular UCB faculty and one graduate student and one
undergraduate student. These individuals may not be directly
associated with the unit they review.
D.
Invite nationally recognized individuals in the same discipline, from
outside institutions, to conduct the external review of each unit.
Normally, two external reviewers are invited for each unit under
review.
E.
Establish guidelines to be followed by these review committees.
F.
Ensure the receipt and distribution of data and written reports
required of each committee and distribute Final Reports.
IV. PROGRAM REVIEW REPORTS AND SCHEDULE
A.
The Panel Chair will request nominations from the unit for the various
review committees well in advance of the review year. These review
committees are assembled by the Program Review Panel, generally in
the semester before the term in which the committee is scheduled to
conduct its work.
B.
A Self-Study Report is conducted internally by the unit, and submitted
to the Chair of the Panel by the first weekday in February (i.e.,
February 1st, 2nd or 3rd, as appropriate).
C.
A campus-level study is conducted by an Internal Review Committee,
and a written report submitted by April 15.
D.
An External Review is conducted by a committee of experts outside
the University of Colorado system early in the fall semester. A written
report of the External Review Committee is submitted within ten days
following the visit to the Boulder Campus.
E.
The PRP Liaison to a unit drafts the Final Report. It is discussed by the
Panel and a final draft of the Report is sent to the unit for comment.
Their response is considered, changes may be made by the PRP, and
the Report is submitted to the Provost.
F.
The Provost then meets with appropriate deans to discuss the Final
revised 9/2003
UCB - Academic Program Review Procedures
page 3
Report. Each may attach comments to the Report, and/or ask the Panel
to reconsider any aspect of it. Upon formal receipt of the Report by the
Provost, it becomes a public document and is submitted to the
Chancellor and to the Regents. The target date for formal receipt of
Final Reports is the last day of the fall semester.
G.
A Program Review may be scheduled to coincide with professional
accreditation reviews as long as the goals and requirements in this
document are addressed.
THE SELF-STUDY
I.
II.
THE SELF-STUDY TEAM
A.
A unit is notified by the Panel Chair two years in advance of the
upcoming review. At least one semester prior to the due date of the
Self-Study Report, Self-Study Team members are invited to meet with
the Chair of the Panel to discuss the Program Review process.
B.
The unit’s administration and faculty are free to develop the exact
composition of the Self-Study Team. At a minimum, the unit must
select a committee consisting of both senior and junior faculty
representing the sub-fields within the department, and at least one
graduate and one undergraduate student. The PRP Chair is informed
of the membership of the committee. (The Office of the Chair of
Program Review maintains a central roster of the membership of all
committees related to Program Review.)
C.
To the extent that aspects of the Self-Study involve personnel matters,
it may be inappropriate for students to participate at certain stages of
the study. The unit's administrative office should provide support to
the Self-Study Team, particularly to the students on the committee as
they organize meetings and other mechanisms to solicit student input
to the self-study process.
D.
Once the Self-Study report is complete, the Self-Study Team is
responsible for providing information and logistical support to the
subsequent IRC and ERC reviews of their unit.
GUIDELINES FOR THE SELF-STUDY PROCESS
A.
The Office of Planning, Budget and Analysis (PBA) provides
information about majors, teaching, degrees, and faculty and staff. See:
http://www.colorado.edu/pba/ia/prp.htm for listings for units
doing PRP self-studies, and for information about other PBA services
revised 9/2003
UCB - Academic Program Review Procedures
page 4
to PRP units, such as gathering comparative information on other
units, both at UCB and elsewhere. At the request of the Self-Study
team, the Office
of Accounting
and Budget
Services
(http://abs.colorado.edu) provides assistance in retrieving multi-year
financial data.
III.
B.
The Self-Study Team may wish to consult with the appropriate dean(s)
early in the process of conducting their work.
C.
Units preparing for Program Review may wish to consider requesting
the services of the Faculty Ombuds Office. The Faculty Ombuds offer
a service to units preparing for Program Review that is designed to
assist units in setting and evaluating goals for strengthening various
aspects of the unit, updating Strategic Plans, or other activities where
mediated discussion and discrete dispute resolution may be of benefit.
D.
The Self-Study Team should submit a draft of their report to the entire
faculty of the unit for comment, and must submit a penultimate draft
to the faculty of the unit for their formal vote of approval. This vote
must be reported in the Self-Study Report.
E.
One, original, single-sided copy of the Self-Study Report is submitted
to the Panel Chair no later than the first working day of February.
THE SELF-STUDY REPORT
The main body of the Self-Study should normally not exceed 20 pages,
double spaced (a page is normally about 350 words). It normally
includes the sections described below, although sections not relevant
to the unit may be omitted, and others may be added.
SECTION I. DESCRIPTION AND ANALYSIS OF THE UNIT
This section of the Self-Study report should describe the major
components and activities of the unit, and an analysis of each of those
activities. In parts B through D of Section I in particular, the following
criteria for analyzing the unit's activities should be explicitly
addressed:
1.
revised 9/2003
Quality and Uniqueness- Indices include the quality and quantity
of teaching and research, funding, reputation and visibility,
leadership, quality of students, placement of graduate students,
and coherence, rigor, and clarity of curriculum. If available, the
unit should make reference to external evaluations from
organizations within its own discipline and/or from federal
funding agencies. The singular or significant roles that the unit
plays at the State, regional, or national level also are indicators of
UCB - Academic Program Review Procedures
page 5
the unit's quality and uniqueness.
2.
Centrality - Indices include the program’s contributions to the role
and mission of the University, the teaching of general education
requirements, productive interdisciplinary research, and
contributions to the aims of the UCB strategic plan. A copy of the
campus Strategic Plan is available online at
http://www.colorado.edu/Chancellor/StrategicPlan/SPbouldercampus.html
A.
3.
Student Demand - Demand is reflected by the amount of student
interest in a unit’s courses, the number of undergraduate majors,
and the number of applicants to the graduate program. It can also
be indicated by a strong service-course role, enrollment and
waiting list trends and by number of degrees awarded.
4.
Budget - It is important to note the 1994 Excellence Task Force’s
statement that, “programs high in quality and centrality would be
maintained regardless of cost.” Academic concerns, not costs, are
the primary focus of Program Review. Further, Program Review
recognizes that there is no single metric for determining cost
versus benefit of a unit, especially across disparate disciplines.
However, an evaluation of costs and budgetary support are
informative in assessing the programs described in Section I and
the goals and plans articulated in Section III. Please provide
budgetary information on Section I A-F as appropriate, and
please provide cost estimates to aspects of your strategic plans
and goals in Section III that are not cost neutral.
Overview of the Unit - Briefly describe the unit. Note its outstanding
characteristics, role and mission within the University, programs
offered, personnel, national reputation, financial status, and placement
of its graduates. If your discipline is ranked in national rankings
(National Research Council, others), please tell us how your unit is
placed in those rankings, and reference your source(s). You may
comment on your rank placement if you feel it appropriate to do so.
Your external reviewers will be asked to do the same. If your
discipline is not ranked by the national ranking systems, you may
ignore this component of your overview. Norlin Libraries Central
Reference Department maintains a web site to provide you easy access
to several ranking systems. It may be found at
http://www-libraries.colorado.edu/ps/ref/rankings.htm
This overview should not exceed two pages. The vote of the unit in
accepting the Self-Study Report should be reported in this section.
revised 9/2003
UCB - Academic Program Review Procedures
B.
C
D.
page 6
Undergraduate Curriculum and Programs - Using the criteria above,
briefly describe and evaluate the undergraduate curriculum for majors,
the service and non-major curriculum offered by the unit, and the
involvement of the unit in providing special or individualized
instructional opportunities for undergraduates. Detailed description of
curricula may be placed into an appendix. Issues such as the following
could be considered.
1.
What is the rationale for the undergraduate curriculum? Is the
unit satisfied with the coherence and overall design of its
undergraduate offerings and programs?
2.
What is the level of the unit’s participation in the Honors
Program? What enrichment activities for undergraduates are
offered outside the classroom? What does the unit do to provide a
sense of community for its majors? How are faculty involved in
academic advising and career counseling for majors?
3.
To what degree does the unit regularly discuss teaching methods
and what support is offered to help faculty members strengthen
their teaching? What methods does the unit use to assess
teaching on an annual basis, and as part of promotion and tenure
decisions?
How does the unit incorporate the Outcomes
Assessment process into its evaluation of its curricular offerings?
What data do you have of the fate of your students after they
graduate?
Graduate Curriculum and Programs - Briefly describe and evaluate the
process used by the unit for graduate student selection and training.
Detailed description of curricula may be placed into an appendix.
Issues such as the following could be considered.
1.
What is the rationale for each graduate degree program and how
is that rationale related to the overall aims of the unit? How does
the unit evaluate the content and rigor of its graduate program?
2
What are the enrollment and graduation trends for each graduate
program over the last several years?
3.
What kinds of placement success do each of the graduate
programs have?
4.
What is the recruitment success into the graduate programs over
the last several years? How do the incoming students perform on
standardized national graduate entrance exams?
Research and Creative Work - Briefly describe and evaluate the research
and/or creative work activities of the unit. What are the primary areas
revised 9/2003
UCB - Academic Program Review Procedures
page 7
of emphases and strength within the unit? In what areas has the unit
recently developed new strengths or collaborations? How does the
unit judge its scholarly productivity relative to its national peers?
What programs does it consider to be its peers? What is the recent
history of research support, fellowships, awards, contracts or
commissions by members of the unit?
How will the faculty
demographics of the unit impact its research and/or creative works
mission over the next several years?
There may be faculty
development issues, as detailed in the description of Appendix F, that
the unit may wish to discuss within this section also.
E.
Staff Support - Program Review recognizes that a key component to
every successful unit is the right mix of support staff for both the
instructional and the research/creative works activities of the unit.
Briefly describe and assess the unit’s investment in and utilization of
State classified, professional exempt, and non-tenure track faculty
personnel within the unit. Non tenure track Faculty titles include
lecturers, instructors, research associates, and professional research
assistants. Describe any professional development and training
opportunities that the unit provides to its staff.
F.
Facilities - Describe the current instructional and research/creative
works facilities of the unit. To what extent do these facilities meet the
needs of the unit? Are there issues related to facilities that are the
responsibility of Academic Affairs? Are there issues of facilities that
you believe should be called to the attention of Facilities Management?
SECTION II -
Response to Previous Program Review Recommendations
This is an integrated, narrative summary, not exceeding one page. A more
complete Appendix, Detailed Response to Previous Program Review
Recommendations, is required as indicated below.
SECTION III - Unit Goals and Plans.
In this final Section of the Self-Study Report narrative, the unit should
summarize its goals for itself over the next seven year period. Each unit has
developed a Diversity Plan document and a Strategic Plan document to
help guide it in attaining its goals. The complete text of both Plans must be
included as appendices. In parts A and B below, summarize the aspects of
these plans as they pertain to the units short- and long-term goals.
A.
Diversity Plan - Each unit contributes to the campus-wide diversity
initiative. A complete version of the unit’s Diversity Plan must be
included as an Appendix. In this section, describe the rationale for the
unit's plan and summarize the progress made in implementing it.
Comment across the range of parameters of diversity noted in the
revised 9/2003
UCB - Academic Program Review Procedures
page 8
official Campus definition--Diversity “may include but is not
necessarily limited to ethnicity, race, gender, age, class, sexual
orientation, religion and physical disabilities.” How is the unit
addressing the issue of climate, given that the Campus document goes
on to say that, “a climate of healthy diversity is one in which everyone
values individual and group differences, respects the perspectives of
others, and communicates openly”?
B.
Strategic Plan -This is one of the most important parts of the Self-Study
Report. The more specific the plan, the more useful it will be. A
Strategic Plan typically addresses the unit’s major short-term and
long-term goals and what the unit wants to be distinguished for in the
next 7 years. It should discuss its curricular, facilities, and faculty
development needs and aspirations during the next review period.
Faculty hiring plans, almost always an important component of a
strategic plan, should be summarized here and defended in terms of
both replacement and in terms of growth and reallocation. A complete
version of the strategic plan must be included as an Appendix. Here,
only the key points of the plan should be presented. Special attention
should be given to how the unit proposes to focus its activities to
achieve or sustain excellence in a few key areas. The resource
implications of the plan must be analyzed, since most new money for a
unit can only be made available through reallocation from some other
campus unit. Provide your estimates of costs for each component of
your plan, as appropriate. The first responsibility of the unit is to plan
improvement by reallocation of existing resources or with minimal
new assistance. Then, the unit should propose ways in which its
contributions would be enhanced if additional resources become
available.
SECTION IV - REQUIRED APPENDICES
A.
Centers & Joint Programs - Describe and provide a specific evaluation of
any formal centers, institutes, or laboratories housed in the unit and of
any joint-degree programs in which the unit participates. Note: this
section is intended to be the equivalent of a Program Review of such
Centers. Each evaluation should be accompanied by an explicit
recommendation to renew or not to renew the center, institute, or
laboratory, along with a rational for that recommendation.
B.
Strategic Plan - The unit’s most recent plan, updated as necessary,
should be included here. Issues that should normally be addressed
include:
1.
revised 9/2003
The most promising prospects for future development and steps
UCB - Academic Program Review Procedures
page 9
that might be taken to fulfill these opportunities, both by internal
reallocations and by the application of new resources.
C.
2.
Enhancements to or changes in current major areas of
research/scholarly emphasis and the development of new ones.
3.
Problems and areas of concern that should receive the attention of
the unit and steps that might be taken to deal with them.
4.
Faculty composition by age and rank and how anticipated
retirements are taken into account in long-range planning.
5.
Plans for faculty recruitment with a rationale developed for the
sub-disciplines to be emphasized in the recruitment.
6.
Procedures and criteria used in hiring new faculty members, and
how diversity goals are addressed: Are women and minority
groups fairly represented in the various faculty ranks? If not,
what measures are being pursued to recruit and retain female and
minority faculty?
7.
Major changes in graduate training programs.
8.
Overall aspects of the undergraduate experience.
9.
Physical plant and other resource issues.
Outcomes Assessment– In partial fulfillment of campus policy regarding
outcomes assessment, include a detailed report describing your unit’s
assessment results, including:
a. Academic assessment actions such as standardized testing, outside
consultant evaluations, etc., that address the skills and knowledge
goals of your unit as delineated in the campus course catalogue;
b. Conclusions reached by your unit with respect to your evaluation of
student performances on those assessment measures;
c. Actions taken or planned in direct response to the results of a. and b.
above.
A full discussion of the outcomes assessment process, and a time table,
may be found at http://www.colorado.edu/pba/outcomes/
D.
Unit Merit – In fulfillment of campus policy regarding unit merit,
include a description that compares the unit to its AAU peers in the
following areas:
revised 9/2003
UCB - Academic Program Review Procedures
1. Salaries:
These
data
are
page 10
available
on
the
web
at
Report the data for your unit
in this section. Provide a brief (one paragraph) narrative of these
data, if necessary.
http://www.colorado.edu/pba/facstaff/facsal/.
A list of the official list of AAU public institutions is at
http://www.colorado.edu/pba/misc/peer.htm. These are the institutions that
the University uses for comparisons. Some disciplines will not be
represented at all the other AAU publics, but a significant sample
of AAU publics should be available for most units.
2. Research/Creative Works National Rankings:
Compare your unit’s research/creative work accomplishment or
reputation to other public AAU units. Use or create whatever
measure or measures that are appropriate for your unit. These
measures should be measures of scholarly reputation
(e.g.,rankings) or productivity (e.g., published refereed articles)
that can be found or compiled from databases or peer websites.
Whichever measures are chosen, provide parallel information for
the other AAU publics where your discipline is represented. Please
note, many common ranking systems are accessible at http://wwwlibraries.colorado.edu/ps/ref/rankings.htm.
3. Non-comparative and Qualitative Measures:
In this section, include other, non-comparative data that help to
describe the scholarly or artistic merit of the unit. Units may
choose to list awards, honors, or special recognition of unit faculty
that cannot easily be compared to the public AAU peers.
4. Unique Indices:
This section can be qualitative or quantitative in nature. This is the
appropriate section to mention other measures which reflect the
quality of the unit, such as service awards or other indicators of
unit merit.
Please note: the PRP external reviewers will be asked to comment on
the comparisons made in this section.
(If the unit belongs to the College of Arts and Sciences, please refer to the
A&S dean’s January 15, 2002 memo of understanding for further directions).
E.
Detailed Response to Previous Program Review Recommendations - Address
each recommendation made by the Program Review Panel in the last
review. Those that have been implemented should be discussed as to
their impact on the unit. If any previous recommendation has not been
revised 9/2003
UCB - Academic Program Review Procedures
page 11
implemented, explain why and whether the unit plans
implementation. This section is most easily approached by
synthesizing the unit’s annual Program Review progress reports.
G.
Diversity Plan - This is the plan previously submitted to the Office of
Academic Affairs.
H.
By-Laws-Existing unit By-Laws and/or related governance documents.
I.
Faculty Development Procedures - Include in this appendix the written
documents defining the standards and procedures for reappointment,
tenure, and promotion within the unit. Describe also any active
mentoring activities that the unit provides to its junior faculty
members. Other issues that the unit may wish to address in this
appendix include its history of reappointment, tenure, and promotion,
its post-tenure review activities, or its policies and record with regards
to applying a policy of differential workloads within the unit.
J.
Brief CVs of unit members - Normally, no more than 2 pages per person.
K.
Internet - Briefly discuss the unit's presence on the Internet. Include a
strategic statement that describes who the unit targets as an audience
for its online resources. Address how the unit plans to maintain the
currency and usability of its website(s).
L.
Information Technology If appropriate, please discuss how information
technology enhances, or will enhance, the research and curricular goals
of the unit. Include a brief strategic statement describing the nature
and objectives of any information technology initiatives, as well as
their implementation and upkeep requirements.
M.
Statistical data - These data are provided to the unit and to the PRP
Office. Thus, that entire package should not be appended here.
However, specific tables that are referenced in the Self-Study should be
copied and included, as should any figures or tables generated by the
unit itself.
N.
Arts and Science Units – Include as an appendix all College Investment
Committee proposals in the last 3 years and their status.
O. Other - Please note that there is normally no need for extensive documentation
beyond what is described in these guidelines. However, optional appendices
may include results of student surveys, annual reports, and other information
that the reviewers (especially the external review team) might find useful.
revised 9/2003
UCB - Academic Program Review Procedures
page 12
INTERNAL REVIEW COMMITTEE (IRC)
I.
II.
PURPOSE OF THE INTERNAL REVIEW
A.
The Internal Review serves as a check of the thoroughness and
usefulness of the Self-Study. It is conducted by a committee familiar
with the overall campus culture. The Internal Review is meant to
complement the Self-Study and the External Review. The IRC
normally presents broad findings, although details are appropriate
when the IRC believes they are not adequately presented in the
Self-Study. The IRC should not attempt to solve problems or mediate;
it is primarily a fact finding group. As such, members should try to
present as complete and balanced a picture of the unit as possible.
B.
The Internal Review Report identifies main themes and issues the IRC
believes should be considered by the external reviewers and in the
final PRP Report.
SELECTING THE INTERNAL REVIEW COMMITTEE
A.
The IRC is comprised of members from outside the unit: three faculty
members (one of whom serves as chair) chosen from cognate
disciplines; a graduate student; and an undergraduate student. (Note:
Privacy and confidentiality of information about individual faculty
members must be protected, thus it may be inappropriate for the
students to participate in some aspects of the work of the committee.)
B.
Nominations for the IRC are requested from the unit, and may also be
solicited from the Boulder Faculty Assembly, United Government of
Graduate Students, University of Colorado Student Union, and PRP
members.
C.
The unit is given an opportunity to review all nominations and may
revised 9/2003
UCB - Academic Program Review Procedures
page 13
strike names from the nominee list for any reason. The IRC
appointments are then finalized by the Panel from the agreed on list.
D.
III.
Members of the IRC must be asked to declare themselves free of any
personal or professional conflicts of interest relative to the unit they are
asked to review.
GUIDELINES FOR THE INTERNAL REVIEW COMMITTEE VISITS AND REVIEW
A.
B.
Procedures
1.
The IRC meets with the Program Review Panel Liaison for
orientation and to plan its review of the unit.
2.
Each faculty member of the Internal Review Committee receives a
copy of the completed Self-Study from the PRP Chair's office.
3.
Each IRC member should review these Procedures, the
Self-Study, and the previous cycle reports. When work is
completed, each member should return the current Self-Study
Report to the Panel Chair's Office.
The Internal Review Committee and Deans
After the IRC receives and reviews the Self-Study, it members should
contact the appropriate dean's office in order to schedule an interview
with the dean. Some deans will prefer to meet with the IRC before
they conduct their review. Others may wish to meet with the IRC to
discuss their draft findings before submitting their report to the
Program Review Panel Chair. In some cases the dean may wish to
meet at both the beginning and end of the Internal Review process.
C.
The Internal Review Committee and Unit Interviews
1.
The Unit is obligated to assist the IRC, including promptly
providing the committee with faculty/staff/student contact
information (email, phone numbers, etc.), arranging for meeting
space, and announcing meetings, if requested by the IRC to do so.
2.
The IRC schedules interviews with representative groups of the
unit’s faculty. In small units, all faculty may be interviewed. One
purpose of these interviews is to check the validity and
thoroughness of the Self-Study Report and to ensure that
dissenting and alternative views are heard. Any faculty member
may request to speak with the IRC privately, as such the IRC
should extend an appropriate invitation to the faculty at the
beginning of their review. The IRC may also seek information
from relevant campus groups beyond the unit.
revised 9/2003
UCB - Academic Program Review Procedures
page 14
3.
The Internal Review Committee should interview representatives
of graduate students and undergraduates in the unit. Students
enrolled in courses considered to be "service" or non-major
courses should also be polled. This has traditionally been a
primary responsibility of the student members of the IRC. The
IRC should discuss how this aspect of the review will be
conducted. Many units have student clubs, organizations, or
email listserves that may be solicited by the IRC. The unit under
review has an obligation to assist the IRC, particularly the student
members of the IRC, in scheduling a thorough review of the
student population.
4.
Unit interviews occur while classes are in session during
February and March and should be concluded by Spring Break.
IV. GUIDELINES FOR THE REPORT OF THE INTERNAL REVIEW COMMITTEE
A.
V.
The Report of the IRC is normally 5-10 pages in length. The primary
audiences for the IRC report are the external review team members,
and the liaison to the Program Review Panel. The report should
consist of the following.
1.
An evaluation of the completeness and adequacy of the SelfStudy.
2.
A general appraisal of the unit.
3.
Brief recommendations that provide the PRP with themes and
issues that the IRC believes should be considered by the external
reviewers and in the Final Report.
B.
The final IRC report is to be delivered to the Chair of the Program
Review Panel by April 15.
C.
Upon receipt of the Internal Review Report, the Panel Chair sends it to
the unit, which then has 7 days to make written comments to the
Panel, for the correction of factual errors only.
MEETING WITH THE EXTERNAL REVIEW COMMITTEE
A.
Upon the arrival of the External Review Committee (ERC), the Chair of
the IRC and the PRP Primary Liaison meet with the ERC for an initial
briefing. Normally, all members of the Internal Review Committee
have a joint meeting with the ERC during the course of their visit.
revised 9/2003
UCB - Academic Program Review Procedures
page 15
THE EXTERNAL REVIEW COMMITTEE (ERC)
I.
II.
COMPOSITION AND SELECTION
A.
The ERC is normally comprised of two persons. The unit under
review is solicited for the names of 2 - 4 Nominators. Nominators
should be national leaders within the discipline of the unit who are
familiar enough with the unit to assist PRP in selecting ERC members.
The unit may instruct the PRP Chair as to the sub-disciplines that it
wishes the two members of the ERC to represent.
B.
The PRP Chair will contact a minimum of two Nominators and ask
each to nominate 10 - 15 candidates, in the sub-discipline area(s)
specified by the unit, who would be appropriate and competent
reviewers of the unit. Nominators will be encouraged to recommend
individuals who are acknowledged experts in the discipline or
profession represented by the unit. Nominees should be interested in,
and broadly knowledgeable of, issues in higher education. If possible,
nominees should be drawn from other AAU institutions and
comparable universities.
C.
The names of the nominated individuals are then presented to the SelfStudy Team, which may strike names from the nominee list for any
reason and rank order the remaining nominees. Any professional or
personal relationship of a nominee with members of the unit must be
noted by the unit.
D.
The PRP makes final selections no later than Spring Break. The
Associate Vice Chancellor extends the initial invitation on behalf of the
institution, followed-up by correspondence from the PRP Chair.
E.
Once the ERC members have accepted the Associate Vice Chancellor's
invitation, the unit is informed of the ERC composition.
F.
External Reviewers are compensated in a manner appropriate with the
Fiscal Rules of the University of Colorado.
G.
The Assistant to the Panel Chair works with reviewers in scheduling
dates, arrangements for transportation, lodging and meals.
EXTERNAL REVIEW COMMITTEE’S VISIT AND REVIEW
A.
Prior to their arrival, external reviewers receive the following: these
Procedures; the unit’s Self-Study Report; the Internal Review Report;
and the Final Report on the unit from the Previous Cycle review.
B.
All members of the External Review Committee must be present at the
visit. If a member cancels, this will require the selection of an
revised 9/2003
UCB - Academic Program Review Procedures
page 16
individual to fill the vacancy and the visit will be rescheduled if
necessary. In extraordinary circumstances, it may be possible to utilize
tele- or video- conferencing for one member of the ERC.
III.
C.
The External Review Committee visits are scheduled while classes are
in session, early in the Fall semester. On arrival, reviewers are
provided complete itineraries of scheduled interviews.
If the
reviewers want to hold additional interviews, the Chair’s Assistant
will make the necessary arrangements.
D.
The Chair of the ERC is selected by the reviewers at their first meeting
in Boulder and becomes the lead writer of the report.
E.
Reviewers typically arrive in Boulder on the afternoon of the first day
and attend a dinner meeting with the Panel Liaison and the IRC Chair.
The Liaison briefs the reviewers on procedures, the planned agenda,
and answers any questions. The IRC Chair discusses IRC findings.
F.
The following day is spent meeting with students, faculty, and staff
from the unit under review. The reviewers (either together or
separately) typically meet with a number of small groups of faculty,
undergraduate students as a group, graduate students as a group, and
staff as a group. Especially in small departments, the external
reviewers meet all of the faculty members at some point during their
visit. Any faculty member from the unit under review may request a
private meeting with an external reviewer. The reviewers may also
meet with members of the IRC or the Self-Study Team. Lunch
meetings can be arranged with members of the unit or members of the
PRP. Following dinner, the reviewers shall have a free evening for
their own discussions.
G.
Interviews continue on the final day with members of the unit, deans
and other administrators. An Exit Interview with the Chancellor or
Provost is scheduled and attended by the Primary and Secondary PRP
Liaisons and the Associate Vice Chancellor.
THE EXTERNAL REVIEW REPORT
A.
The External Review Committee submits a single, joint report, agreed
upon by each of its members, within 10 days after their departure. This
is sent to the Office of the PRP Chair.
B.
The External Review Committee’s report normally includes four
general areas of concern about program quality, as follows.
1.
revised 9/2003
Strengths and Weaknesses of the Scope of the Unit - The unit’s
research and scholarly efforts should reflect appropriate degrees
UCB - Academic Program Review Procedures
page 17
of specialization or comprehensiveness. Very small units may be
justifiably specialized, as their scholarship may suffer if they
attempt to be too general or comprehensive. Regardless of a unit’s
focus, its intellectual goals should be current and creative.
External reviewers will be asked to address the unit’s scope, its
national standing or ranking, and be asked to comment on the
unit’s own assessment of its national standing.
C.
2.
Curricular Offerings - Are the undergraduate and graduate
curricula contemporary and appropriate for the unit? Are the
training opportunities at both the graduate and undergraduate
levels appropriate for an AAU public university?
3.
Personnel - Are the faculty well-trained and competent? Is their
work creative, significant and valuable? Is the unit exercising
appropriate standards in regard to recruitment and retention of
its faculty?
4.
Leadership - Are the administrators of the unit able to maintain a
productive and appropriate climate for work? Is the unit’s
administrative structure appropriate?
5.
Assessment of Unit Plans and Progress Toward Unit Goals - How well
does the unit meet its goals, and the teaching and scholarly goals
of the University? Does the unit appear to be a stronger or more
influential unit on the national scene than it was at the time of its
last review? A unit’s plans are an important reflection of its
strength and ability to progress.
Many ERC members
traditionally feel compelled as members of their discipline to
advocate for the unit's plans for growth. At the same time, the
University relies on the ERC members to provide an independent
assessment of the merits of a unit's plans for faculty growth.
These plans should be evaluated in terms of how realistic they
are, whether they reflect creative insights into the potential of the
unit to contribute to the discipline and to the University. Where
would limited investment in the unit provide the greatest return?
When the Panel Chair receives the report, it is forwarded to the chair
or director of the unit. The unit has 14 days to make written comments
to the Panel, for the correction of factual errors only.
revised 9/2003
UCB - Academic Program Review Procedures
page 18
FINAL REPORT
I.
PREPARATION
A.
The Final Report of the Program Review Panel for each unit is written
by the Primary Liaison. It normally does not exceed 10 single-spaced
pages.
B.
The report contains the following sections, in the order shown. Each
section is headed with the title indicated.
1.
Review Process - The following text is used (modified only as
needed): “The unit prepared a Self-Study following PRP
guidelines. The Internal Review Committee (IRC) made up of
three faculty members, an undergraduate student, and a graduate
student, who were all from outside the unit, held interviews with
the personnel of the Department and analyzed the Self-Study
Report. An External Review Committee (ERC) of two faculty
from other institutions visited the campus, reviewed the SelfStudy and IRC report, and interviewed departmental personnel,
students, and University administrators. The ERC report was
then reviewed by the PRP along with the Self-Study and the IRC
report. This public document reflects the PRP findings and
recommendations.”
2.
Self-Study - This section starts with a general description of the
unit and then summarizes key points raised in the Self-Study that
are important to the context of this final report.
3.
Results of Previous Review - Description of implementation and
results of specific Recommendations from the previous Program
Review [if there was a previous review].
4.
Internal Review Findings - Summary of major findings of Internal
Review Committee.
5.
External Review Findings - Summary of major findings of External
Review Committee.
[NOTE: The previous two sections can be combined into a single section
entitled “Internal and External Review Findings,” if it is preferred to
organize them in an integrated fashion.]
6.
revised 9/2003
Program Review Panel Determinations - General observations and
conclusions of the Program Review Panel, including a summary
of the strengths and weaknesses of the unit under review. This is
UCB - Academic Program Review Procedures
page 19
basically a synthesis of everything that comes before it, that is, of
the Self-Study, Internal Review Report, and External Review
Report. It also reflects the discussions of the Program Review
Panel. [NOTE: “Program Review Panel” may be abbreviated in the
report as “PRP”].
C.
II.
7.
Recommendations- Specific, numbered, recommendations for
program improvement and future program development should
be provided, with timetables stated where relevant. Every
recommendation must be related in some explicit way to a
finding or determination in the body of the report.
8.
Recommendations are generally made to the unit under review.
Occasionally recommendations may be made to deans, or to other
administrators.
9.
The last statement in the recommendations section is the
following, modified as appropriate, standing alone, and
unnumbered: “The unit should report annually to the Dean of [fill
in school/college] and to the Provost on the implementation of
these recommendations.”
The liaison usually informally shares the final draft of the report as a
courtesy to the unit’s Chair (prior to the final PRP discussion of the
unit) in order to receive comments about any significant errors or
omissions.
SUBMISSION TO THE VICE CHANCELLOR FOR ACADEMIC AFFAIRS
After the Program Review Panel has approved the Report and its
recommendations, the chair of the Panel shall submit the document to
the Provost. The Provost shall discuss the content of the Report with
the appropriate deans and the chair of the Panel, and may elect to
make modifications. A copy of the Report signed by the Provost, with
any modifications noted, shall then be distributed to the unit and to
the appropriate deans. This final, signed Report shall be considered a
public document.
revised 9/2003
UCB - Academic Program Review Procedures
page 20
DEANS AND PROGRAM REVIEW
Deans are involved in the Program Review process at the following stages.
1. Self-Study Teams may consult with deans in the process of compiling their
report. Likewise, deans may request to meet with units or Self-Study Teams
at any point in the Self-Study process.
2. After an Internal Review Committee receives the Self-Study Report and has
had sufficient time to examine it, they will schedule an interview with the
appropriate dean(s) in order to discuss the report. All members of the
Internal Review Committee should be present for this evaluation. The dean
may prefer to meet with the IRC after they have conducted their review but
prior to submission of their report.
3. Each Internal Review Committee may schedule a second meeting with the
dean(s) when their final draft is written. The purpose of the meeting is to
discuss their findings with the dean, prior to submitting the Internal Review
Report to the Panel. This second meeting is optional and at the discretion of
the dean.
4. The External Review Committee for each unit under review will interview the
appropriate dean(s).
5. The appropriate dean(s) will receive copies of the Internal and External
Review Reports, the departmental responses, and the first draft of the final
report, and is invited to attend the Panel meeting for their first discussion of
the unit. (If a school/college does not have a departmental structure, the
Provost will take the place of the dean at this meeting.)
6. After the Panel has completed its report and submitted it to the Provost, the
Provost and the PRP Chair meet with the dean(s) to discuss the findings.
7. Appropriate deans will receive copies of all Final Reports.
8. Deans and units will keep the Provost informed, annually each June, of the
progress made by units in regard to recommendations included in the Final
Report.
revised 9/2003
UCB - Academic Program Review Procedures
page 21
PANEL MEMBER/LIAISON RESPONSIBILITIES
MEETINGS
Panel members are advised of meeting schedules, agenda items, and are
provided with materials and reports. Members are asked to inform the PRP
Chair when unable to attend a scheduled meeting.
Meetings begin weekly early in the Fall semester, with agendas covering
planning and organization, Internal Reviewer selection, and scheduling future
reviews. Regular, weekly meetings continue until all Final Reports are
completed and approved by the Panel.
Meetings in the Spring semester are scheduled periodically to consider
unfinished business and for selection of External Reviewers.
LIAISON ASSIGNMENTS
Primary and Secondary Liaison assignments are made by the Panel Chair as soon
as the Panel membership roster is completed each year. Each voting PRP
member is assigned Primary Liaison responsibilities for one unit and Secondary
Liaison responsibilities, as back-up, for another. The Primary Liaison acts as a
link between the unit, the various review committees, and the Panel. As soon as
liaison assignments are made, the Liaison should contact the unit chair/director
to discuss the review process. It is recommended that the Liaison review
previous cycle reports on their assigned unit. These are located in the Office of
Faculty Affairs, Regent 205.
CONFIDENTIALITY
All Panel members are expected to maintain confidentiality throughout the
entire review process. The Self-Study and the IRC and ERC reports are
confidential documents. All Panel meetings and all discussions held regarding
the Review are confidential. After the Panel’s Final Report has been accepted and
approved by the Provost, it becomes a Public Document.
EXTERNAL REVIEW COMMITTEE VISIT
Liaisons will be advised as soon as External Reviewers decide on a date for their
visit. The Panel Chair’s Assistant will prepare the visitors’ agendas, in
conjunction with the unit, coordinating interviews with deans, the Chancellor,
the Provost, and the Dean of the Graduate School. As the agenda nears
completion, the Panel Chair’s Assistant reviews it to make sure that the unit has
met all Program Review procedural requirements. A week before the External
Reviewers are to arrive, the Liaison should contact the Panel Chair’s Assistant to
review all arrangements.
The evening of the day the reviewers arrive, the Primary Liaison and the Chair of
revised 9/2003
UCB - Academic Program Review Procedures
page 22
the Internal Review Committee have a dinner meeting with them to brief them
on procedures and to answer any questions they may have. All dinner and
luncheon arrangements are made by the Chair’s Assistant. Liaisons should try to
make themselves available to see that all reviewer needs are met as they conduct
their reviews. In this role, the Liaison serves as a host to the ERC members.
The Primary and Secondary Liaisons meet for the Exit Interview with the
External Reviewers and the Chancellor. Both Liaisons are advised of the date and
time as soon as it is scheduled
THE PANEL REPORT
The Primary Liaison is responsible for drafting the Panel’s report on the assigned
unit. Examples of previous reports are available in the Office of Faculty Affairs
for reference.
Drafts of the Final Report should be received in the Chair’s Office with ample
time for distribution to the other panel members prior to the meeting at which
the report is discussed. Note that a report is normally revised several times
before final approval by the Panel.
revised 9/2003