Academic Program Review
External Reviewer Packet
(Graduate Programs)
Prepared by the Office of Planning, Evaluation, and Institutional Research (OPEIR)
Introduction
Thank you for your willingness to conduct an external program review for the University of Tennessee
at Chattanooga (UTC). Your commitment to the process (time, input, feedback, etc.) is much
appreciated.
The academic program review process is intended to provide UTC faculty and academic administrators
with information to identify program strengths and weaknesses. This information should play a major
role in helping faculty to define initiatives, improve quality, and justify needed resources. Program
review is perhaps the most essential component in academic planning.
As an external reviewer, you will receive a copy of the program’s self-study for review. You will then
visit the UTC campus for a two-day site visit. During the site visit, you will have the opportunity to
meet with faculty members, students, and key administrators at the university to assess various aspects
of the program under review. Before leaving campus, you will be asked to complete the THEC
Checklist, and within two weeks of the visit, asked to complete a Narrative Report.
This packet contains three documents:
 Letter of Agreement
 THEC Checklist
 Guidelines for Narrative Report
Page 3
Page 4
Page 7
The Letter of Agreement explains your responsibilities as an external reviewer and the compensation
you will receive. Please sign and return this document to [Department Head Name]. The other two
documents are for your use during and after the site visit.
If you have any questions, please contact [Department Head Name], the Department head of [Program
Name] at [phone and/or email address]. We look forward to working with you!
2
[Insert Letter of Agreement from Academic Program Review Packet]
3
THEC Checklist
2010-15 Performance Funding Cycle
Appendix G: Program Review
Graduate Programs
Instructions for External Reviewers:
In accordance with the 2010-15 Performance Funding guidelines of the Tennessee Higher Education Commission (THEC),
each non-accreditable graduate program undergoes either an external peer review or academic audit according to a preapproved review cycle.
The criteria used to evaluate a program appear in the following “Program Review Summary Sheet for Graduate Programs.”
The Summary Sheet consists of 20 criteria grouped into four categories. THEC will use the criteria to assess standards for
graduate programs. All criteria noted with an asterisk are excluded from the performance funding point calculation
For each criterion within a standard, the responsible program has provided evidence in the form of a Self Study document.
Supporting documents will be available as specified in the self study. As the external reviewer, you should evaluate this
evidence and any other evidence observed during the site visit to complete the checklist and prepare the narrative report.
Items on the summary sheet should be rated on a four-point scale ranging from “poor” to “excellent” (or N/A for items
which are not applicable to the program).
This evaluation becomes a part of the record of the academic program review. The summary sheet will be shared with the
department, the college and central administration, as well as the Tennessee Higher Education Commission. When combined
with the written report, prepared by the entire program review committee, the Program Review Summary Sheet will facilitate
development of a program action plan to ensure continuous quality improvement.
Your judgment of the criteria will be used in allocating state funds for the university's budget.
Name, Title, and Institutional Affiliation of Reviewer(s):
___________________________________
____________________________________
Name
Name
___________________________________
____________________________________
Title
Title
___________________________________
____________________________________
Institution
Institution
___________________________________
____________________________________
Signature
Signature
Date
4
Date
Program Review Summary Sheet for Graduate Programs
Institution:
Program Title(s):
Degree Designation(s) and CIP Code:
A. Student Experience
1
2
3
4
5
B.
1
2
3
4
Graduate Faculty Quality
6
Faculty teaching loads are consonant with the
highly individualized nature of graduate
instruction, especially the direction of theses or
dissertations.
2
Minimally
Acceptable
Good
Excellent
N/A
Poor
Minimally
Acceptable
Good
Excellent
N/A
Poor
Minimally
Acceptable
Good
Excellent
Faculty hold terminal degrees in the appropriate
discipline.
Faculty academic credentials correspond to the
concentrations in which they teach.
Faculty scholarly activity is sufficient to serve as
effective mentors for graduate students
Faculty have sufficient
practical/professional/academic experience to serve
as effective mentors for graduate students.
5
1
Poor
There is a critical mass of students to ensure an
appropriate group of peers.
Prudence is exercised in the number and type of
short courses accepted toward the degree.
Programs offered entirely through distance
education technologies are evaluated regularly to
assure outcomes at least equivalent to on-campus
programs.
There are adequate enrichment opportunities,
such as lecture series, to promote a scholarly
environment.
There are adequate professional development
opportunities, such as encouraging membership in
professional associations, participation in
conferences and workshops, and opportunities for
publication.
Faculty have regular opportunities for
professional development, including travel and
participation in professional organizations,
workshops and other learning activities.
C.
N/A
Teaching/Learning Environment
There are ample materials and secretarial support to
encourage research and publication.
There is adequate library support.
5
3
4*
5*
D.
1
2
3
4
There is adequate and accessible computer support.
There are adequate lab facilities.
There is adequate office space.
Program Evaluation
N/A
Follow-up data on graduating students are regularly
and systematically collected
The curriculum is evaluated periodically.
Evaluation of placement of graduates is regular
and systematic.
Completion rates are at an acceptable level.
* Criterion not included in the performance funding calculation.
6
Poor
Minimally
Acceptable
Good
Excellent
Guidelines for Narrative Report
PART 1 – Student Experience
Are admissions standards appropriate to the degree level, stated clearly, and consistently applied?
Is information regarding admissions and retention standards clearly outlined in the institution’s
catalog and available to all applicants and enrolled students?
What efforts are made to recruit and retain students?




Are enrollment levels sufficient to ensure the critical mass of students necessary for appropriate
peer groups?
Are enrolled students properly oriented to the department and institution? Does student support,
such as personalized instruction, advisement, and guidance continue after orientation? Are the
department’s efforts in curricular and career advising adequate to meet student needs? What
changes/improvements are needed to improve the department’s effectiveness in advising? Is
student progress periodically monitored?
Are retention standards clearly stated and consistently applied? How could the department
improve recruitment and retention?
Are there appropriate time limits for degree completion? Are the time limits appropriate for the
degree level to ensure that students’ knowledge is up-to-date when the degree is awarded?
Are there appropriate curricular offerings to enhance student experiences?


Are any short courses accepted toward the degree program? Is prudence exercised in the number
and type of acceptable short courses?
Are distance education programs available? Are programs offered entirely through distance
education technologies evaluated regularly to ensure outcomes at least equivalent to on-campus
programs?
Are there appropriate extracurricular offerings to enhance student experiences?



Does the program provide students with enrichment opportunities, such as lecture series, student
organizations, etc.? Are such opportunities adequate to promote a scholarly environment?
Does the program provide adequate opportunities for student professional development? To
what extent does the program encourage membership in professional organizations, support
participation in conferences and workshops, and/or promote opportunities for student
publication?
Does the program provide adequate opportunities for student internships, practica, and/or field
experiences?
7
PART 2 – Graduate Faculty Quality
Are the faculty competencies/qualifications those needed by the program and by UTC?



Do faculty hold terminal degrees in the appropriate discipline?
Do faculty specialties correspond to program needs and to the concentrations in which they
teach?
If faculty need additional/different competencies/qualifications, how might these needs be
addressed?
Are faculty effectively serving graduate students as mentors?


Do faculty members have sufficient practical, professional, and/or academic experience to serve
as mentors?
Are faculty members actively involved in scholarly activity? To what extent does such scholarly
activity facilitate the mentoring of students?
Do the faculty have regular opportunities for professional development such as travel and
participation in professional organizations, workshops, and other learning experiences?
Are faculty teaching loads sufficiently reasonable and equitable to accommodate the highly
individualized nature of a graduate program, especially the direction of theses or dissertations?
PART 3 – Teaching and Learning Environment
Is the current curriculum adequate to enable students to develop the skills and attain the outcomes
needed for graduates of this program?






Is the curriculum content what it needs to be? Is the core curriculum appropriate to the
discipline? Is there a provision for planned disciplinary specialization beyond the core including
courses characterized by advanced disciplinary content and academic rigor? As the master’s
level, is degree credit from dual-listed courses limited to 1/3 of the total hours required?
Are there appropriate opportunities to learn the tools, techniques, or methodologies that are
needed to understand the discipline and actually practice the education acquired?
Are courses offered frequently enough to ensure the timely completion of the degree?
Do students have adequate opportunities to participate in extra-disciplinary courses or
experiences without increasing the total number of hours required toward the degree?
Will the curriculum need to be updated in the near future? If so, please explain and advise.
Should the school consider the development of any new or alternative offerings?
Are adequate resources available to students?

Are there adequate materials and secretarial support to encourage research and publication?
8




Is there adequate library support?
Is there adequate and accessible computer support?
Are there adequate laboratory facilities?
Is there adequate office space?
Considering current budget constraints, what are the most pressing resource needs of the program?
Could these needs be met in ways without requiring additional budgetary resources, such as savings
from current program operations?

How should needs of the program be prioritized? Could savings be realized from current
program operations in order to fund any new budgetary needs?
PART 4 – Program Evaluation
What are the competencies/educational outcomes that are expected of students who satisfactorily
complete the program?



Has the department defined the skills/student outcomes desired for its students? Does the
curriculum ensure the development of appropriate skills in written and oral communications,
critical thinking, and computer and technology-related skills? Does the school have adequate
information to know when skills have been adequately developed/outcomes have been
accomplished? If not, define what kind of data and/or analyses is needed.
Does the program require an oral or written comprehensive exam or culminating experience? To
what extent does the comprehensive exam or culminating experience demonstrate breadth of
knowledge across the discipline, depth of knowledge in specific areas, and the ability to integrate
and apply knowledge independently?
How do program mechanisms ensure that students demonstrate mastery of research in the
discipline (e.g., through theses, research papers, case studies, etc.)?
Are completion rates at an acceptable level?


What are the current trends in the number of degrees awarded and completion rates?
How does the department evaluate completion rates? What evidence is available toward
acceptable completion rates? If unacceptable, what are possible contributing factors? How is
this information used toward program revision?
Does the program systematically track its graduates?


What mechanisms or procedures are in place for the collection of information on graduates? Is
follow-up data collection regular and systematic?
What mechanisms or procedures exist for placing graduates in positions related to the discipline?
Is placement regular and systematic? How are placement activities evaluated?
9
What mechanisms or procedures are in place for curriculum evaluation? How often is the
curriculum evaluation? How often is the curriculum evaluated and how are proposed changes
implemented?
PART 5 – Summary Recommendations
Overall, what are your impressions of the program?


What are the major strengths of the program?
What are the major weaknesses of the program?
What goals would you suggest the program set for the next five years? Please list goals in order of
priority (i.e., the most important goal first, followed by the second most important goal, etc.)
How can the program work to achieve these goals over the next five years?


Considering current budget constraints, what are the most realistic strategies the program can use
to achieve the highest priority goals?
What goals would require additional resources? What level of resources would these goals
require? How might the program secure these resources?
10