Academic Program Review
Handbook
2010-17
East Tennessee State University
Revised September 16, 2013
Contact Us:
Office of Assessment & Teaching
Center for Academic Achievement
assessment@etsu.edu
423-439-7484
136 Sherrod Library
PO Box 70278
Cheri Clavier
Director of Assessment & Teaching
clavier@etsu.edu
Reedena Newlon
Executive Aide, Assessment & Teaching
newlon@etsu.edu
ETSU 2010-17 Program Review Handbook, Page 2 of 26
Contents
What is Academic Program Review? ............................................................................................... 4
Why is Program Review important?................................................................................................ 4
What is the Timeline for the Program Review Process? ................................................................. 5
How do we identify potential reviewers? ....................................................................................... 6
What goes into in the self-study?.................................................................................................... 7
What additional information might the reviewers need? ............................................................ 14
How do the reviewers assess the program? ................................................................................. 15
How is the site visit scheduled? .................................................................................................... 15
What needs to be done by whom? ............................................................................................... 15
Appendix A: Sample Program Review Schedule …………………………………………………………………….. 17
Appendix B: Baccalaureate Program Checklist ………………………………………………………………………. 20
Appendix C: Graduate Program Checklist ……………………………………………………………………………… 23
Appendix D: Sample Faculty Data Chart ………………………………………………………………………………… 26
ETSU 2010-17 Program Review Handbook, Page 3 of 26
What is Academic Program Review?
Academic program review is a peer review process designed to improve the quality of the
university’s academic programs. Program reviews provide a systematic method to evaluate
quality, productivity, and need, not only in the university but across the state and region.
The program review process takes place on a seven year cycle. During the year scheduled for
the formal review, academic units collaborate with Assessment and Teaching (A&T), their
college, internal consultants and external reviewers to engage in the self-study process. The
review team conducts a site visit during spring semester and compiles a narrative report that
includes recommendations for improvement.
Departments create action plans based on these recommendations and present the outcomes
of the program review to the Provost and others involved in implementing changes. Ongoing
tracking of recommendations and outcomes continues throughout the cycle, until the next peer
review. The follow-up portion of the program review cycle provides the vital link that enacts
improvements brought to light in self-study and peer review processes.
The Office of Assessment and Teaching (A&T) coordinates all reviews. We act as a resource as
you work on your self-study and attempt to make the review process as smooth as possible for
all involved. If you have any questions about the process, feel free to get in touch with us and
we will help in any way we can.
Why is Program Review important?
Program review is important, primarily, to help you identify the strengths and weaknesses of
your program and to help you improve your program, with administrative support. The followup process after program review makes sure that planned changes are implemented in a
systematic manner.
Program review is also an important part of the Performance Funding formula that constitutes a
significant part of ETSU’s income stream. Performance Funding is based on a 100 point scale,
and 20 points are allotted to the two types of programmatic evaluation, Program Review and
Academic Audit. Depending on our degree of success, ETSU may earn over $3.5 million per year
through Performance Funding. Your work to build a better program and to write a thorough
review helps us receive this incentive funding.
ETSU 2010-17 Program Review Handbook, Page 4 of 26
What is the Timeline for the Program Review Process?
Activity
When
1. Decide who will coordinate the review and/or self-study. Inform
A&T.
Summer before review*
2. Begin self-study (See: What goes into the self-study?)
Fall Semester
3. Recommend and rate potential reviewers; recommend site visit
dates (See: How do we identify potential reviewers?)
Fall Semester
4. Submit draft of self-study to deans and A&T
January 15
5. Edit self-study draft based on recommendations of dean and A&T.
Return to A&T.
2 weeks from the time edits are
received
6. Finalize site visit schedule (See: Appendix A of this handbook)
2 months before site visit
7. A&T distributes self-studies to reviewers, campus administrators,
and program faculty.
4 weeks prior to site visit
8. Review team submits narrative report.
1 month after site visit
9. A&T enters into TracDat: recommendations proposed in unit’s selfstudy, reviewers’ recommendations, and any unmet or partially met
standards not addressed in recommendations.
1 week after receipt of narrative
report
10. A&T convenes department faculty and staff to discuss
recommendations and to develop a plan of action. A&T records
department responses in TracDat.
2 weeks after receipt of narrative
report
11. Department and dean meet with VPAA/VPHA and formally present
plan for responding to recommendations. Provost’s office enters
agreement or disagreement, plus comments, in TracDat
Summer/Fall after site visit
12. Department and others (deans and VPAA/VPHA as appropriate)
report implementation of plan in TracDat
Annually, same deadline as PIE
(Oct. 15)
13. Provost, Dean of College, Chair of Department and others, as
appropriate, meet to review progress in implementing followthrough on recommendations
Not later than Aug. 15, 3 years after
site visit.
*Departments may elect to begin the self-study process during Spring and Summer of the year
prior to their scheduled review in order for site visits to be scheduled in Fall of that year.
ETSU 2010-17 Program Review Handbook, Page 5 of 26
How do we identify potential reviewers?
Identify potential reviewers: 2 to 3 in-state, 2 to 3 out-of-state, and at least 3 internal
consultants. Provide a list of these reviewers Assessment and Teaching as early in the academic
year as possible, but no later than January 15. All reviewers must meet the qualifications
delineated below. The Director of Assessment and Teaching will select from your list the two
external reviewers (one in-state and one out-of-state) that will actually conduct the review,
based on their credentials and availability. The external reviewers must be professionals in the
field under review and have no personal or professional affiliation with members of ETSU’s
faculty in the program involved in the self-study. The VPAA/VPHA and TBR will have final
approval of the review team.
When you provide the list of potential reviewers to A&T, please include your reasons for
selecting each individual (addressing the criteria below), contact information for the potential
reviewers (including email address), and a copy of each external reviewer’s vita (or links to web
versions).
External Reviewers must:
1.
Be professionals in the field under review;
2.
Hold a terminal degree;
3.
Hold an academic position, preferably at a regional public university comparable to
ETSU; in some cases a practicing professional in the field is an appropriate substitute;
4.
NOT be ETSU graduates;
5.
NOT have an active or previous professional or personal affiliation with faculty in
department to be reviewed, or with other reviewers (co-author, classmate,
professor/student, former colleague, etc.);
It is recommended that each of the following qualifications is held by at least one reviewer:
6.
Chair of Department or coordinator experience;
7.
Training/experience as a program reviewer.
Internal Consultants
Three internal consultants are included on every review team. One should be from within the
same college as the program under review; one should be from outside that college. We
request that a third candidate come from a program that will be reviewed in coming years, in
order to mentor those future reviews and provide a fresh perspective. The internal consultants
can provide important campus-related information to external reviewers, but they are still key
members of the reviewing team, providing insight from within the university but outside of the
program.
Internal consultants must:
1.
Be ETSU Graduate Faculty (if graduate program is reviewed);
2.
Be faculty members outside program being reviewed (one within college; one outside
college; one in a program to be reviewed in near future);
3.
Not be co-author or co-creator with department faculty.
ETSU 2010-17 Program Review Handbook, Page 6 of 26
Tips for identifying potential reviewers:




Ask appropriate professional associations for help in identifying potential reviewers.
Many disciplinary organizations provide training for program reviewers and can provide
names of experienced/trained individuals.
Ask department faculty for suggestions.
Contact comparable programs at other regional universities to learn who successfully
reviewed their programs.
Ask potential reviewers if they are “willing to be considered” as a reviewer and to send
their vitae. Be sure to let them know that this query does not constitute a commitment.
What goes into in the self-study?
The narrative of the self-study should be constructed after an open and frank discussion by the
program faculty and staff members as they prepare for the review. Individual faculty or
committees could be appointed to write the self-study, but the program chair is responsible for
the final product.
Use the following outline to organize the self-study. Unless otherwise noted, all sections must
be included and use boldfaced headings as given. Feel free to include additional sections or
information in any part of the document. Sections VII-XIII correspond directly to the THEC
checklist for academic programs that your reviewers will use to evaluate your program. If your
program does not include a graduate component, then ignore any items labeled “graduate
only.” Some items may not pertain to your program; under such headings, indicate that the
item does not apply. In each section, provide a description of that aspect of the program
(focusing on the period since your last self-study), any data available to describe or measure
that item, and a discussion of the findings of that aspect of the self-study. The outline should
assist you in describing the program thoroughly.
I.
II.
III.
IV.
Title page
a. Title
b. Program name
c. CIP Code
d. URL of department website
e. Year of review
Table of contents
Executive summary
a. Highlight key features of the review throughout
b. Purpose of self-study and/or description of self-study process
c. Mission/goals of program
d. Summary of conclusions (strengths, weaknesses, costs, benefits)
Introduction
a. History of the program
i. 3-4 major points
ETSU 2010-17 Program Review Handbook, Page 7 of 26
V.
VI.
VII.
ii. Any critical events since last self-study
b. General description of program, including distance education/online elements
c. Positive features: What makes this program innovative, outstanding, or unique?
Mission
a. What is the program’s mission?
b. How does the mission fit with that of the department (as applicable), college,
and university?
Goals
a. Teaching
b. Research
c. Service
Learning objectives (Undergraduate Standard 1)
Much of the information on learning objectives should come directly from PIE. Including
complete PIE reports in Appendix A and referring to them in the narrative is a
recommended approach.
a. Program and learning outcomes (1.1)
i. What do you want graduates of your program to know and to be able to
do?
ii. How were intended learning outcomes developed?
iii. How satisfied are your faculty members with these objectives? Are
there issues that need to be addressed?
VIII.
b. Outcome measures (1.2)
i. How do you determine if students have achieved desired learning
outcomes?
1. Examples: observational data, surveys, alumni placement,
program testing, capstone experiences
2. Because the checklist specifically mentions them, be sure to
include student/alumni/employer survey information and other
university research; see A&T for more information
ii. Discuss any other data you gather to improve your program, including
satisfaction, retention, market research, etc.
iii. Discuss the methods described in i. and ii. Are the measures adequate?
How can they be improved?
c. Results of outcome measures
i. Generally describe how your students perform on the outcome
measures you use(include appendices if you wish to provide detailed
information)
ii. How satisfied are you with these outcomes?
d. Use of outcome data (1.3)
i. Describe how you use data from the sources above to improve
strengthen program effectiveness
Graduate Student Experience (if applicable)
a. Peer relationships (Graduate Standard A-1)
i. Provide the number of majors and graduates for the past 5 years
ETSU 2010-17 Program Review Handbook, Page 8 of 26
IX.
ii. How many assistantships are allocated to the program and is this
number adequate?
iii. Are other funding sources available?
iv. Discuss how the numbers of students and delivery format of courses
relates to the peer relationships among students
b. Short courses (Graduate Standard A-2)
“ETSU does not permit short courses at the graduate level.”
c. Distance education (Graduate Standard A-3)
i. Describe DE course offerings in the program or plans to implement such
offerings
ii. Discuss ongoing evaluation of courses delivered through technological
means (if no such offerings, skip to VIII-c)
iii. Provide data comparing online (or other technologies) with on ground
outcomes
d. Enrichment opportunities (Graduate Standard A-4)
i. Describe and list enrichment opportunities (e.g., lecture series) in the
program and university
ii. Discuss how these opportunities promote a scholarly environment for
your students
e. Professional development opportunities (Graduate Standard A-5)
i. Describe professional development opportunities available to students
ii. Identify any student research, publications, presentations, and/or grants
that are the result of faculty collaboration with students in the program
iii. Discuss findings
Curriculum
a. Program requirements (2.1)
Describe program requirements and credit hours needed for
graduation; link to university catalog (optional: link to departmental
website)
b. General education components (2.3)
Describe the ETSU general education program and general education
requirements specific to your program
c. Program core (2.4)
List core requirements of the program, including capstone experiences
d. Coursework outside major (2.5)
i. Describe any program requirements outside department
ii. Describe any opportunities for students to take courses outside the
major
iii. Describe any collaborations with other programs at ETSU or elsewhere
e. Service Courses (2.11)
i. List and describe service courses offered
ii. Describe how you evaluate student learning outcomes in these courses
iii. Provide data related to these outcomes
f. Curriculum revision and currency (2.2, 2.6; Graduate standard D-2)
i. Describe efforts to keep curriculum and practices current, in relation to
standards, practices, and issues in the discipline (include description of
curricular review process)
ETSU 2010-17 Program Review Handbook, Page 9 of 26
X.
ii. What changes have been made in the curriculum in the last 5-7 years?
g. Critical thinking (2.7)
i. Specifically describe where and how the curriculum encourages critical
thinking
ii. Describe engagement of faculty in critical thinking initiatives/trainings
on campus
iii. Provide data related to critical thinking measures (CCTST) in your
program
h. Research in the curriculum (2.8)
i. Describe where and how students are exposed to research methods and
strategies of the discipline within the program
ii. Describe any opportunities for students to participate in research
i. Application of learning (2.9)
i. Describe opportunities for students to apply their knowledge outside
the classroom (e.g., internships, service learning, practica)
ii. Describe any connections with organizations or corporations that
facilitate creative or research interaction, internships, etc.
j. Professional and career opportunities (2.10)
Describe how you convey appropriate professional and career
opportunities to students
k. Service courses (2.11)
i. Describe indicators used to evaluate achievement in services courses
l. Schedule of offerings (2.12)
i. Describe the frequency of course offerings
ii. Describe the process for arriving at this schedule
iii. Put in Appendix B a table of course offerings for 3 years in the past,
current year, and whatever projections are available for the future; if
your schedule is standardized, provide one complete cycle instead
iv. Discuss how frequency of student offerings impacts success for students
in your program
Teaching and learning environment
a. Instructional practices (3.1)
i. The program’s instructional practices are consistent with the standards
of the discipline
1. Describe the importance of teaching practice within the
program and efforts to keep methods current and effective
2. Describe opportunities available for teaching improvement,
including continuing education, campus initiatives (e.g., FTL,
instructional development grants), etc.
3. Provide a summary of teaching improvement activities and
major innovations during the last 5 years; details may be
included in an appendix or on a linked website
4. Discuss findings related to teaching improvement
b. Interaction (3.2)
i. Describe opportunities for students to interact with each other, faculty,
and professionals in the field (e.g., group assignments; student
organizations related to major; cohort activities; internships; seminars;
ETSU 2010-17 Program Review Handbook, Page 10 of 26
XI.
co-curricular programs; faculty collaboration with students in research,
publication, creative endeavors, presentations, grants, or service)
ii. Discuss findings related to interaction
c. Advising (3.3)
i. Describe the advisors and advisement process
ii. Report data related to advising
iii. Discuss findings related to advising
d. Library holdings and services (3.4; Graduate Standard C-2)
Discuss library support services (note: the Office of Assessment and
Teaching will supply the department with a library adequacy report to
include as Appendix X of the report – what is requested here is a
departmental faculty assessment of library support materials and
services)
e. Support for faculty research and publication (Graduate Standard C1)(Graduate programs only)
i. Describe materials specific to the program
ii. Identify number of support staff for program
iii. Evaluate sufficiency of available materials and staffing
f. Diversity (3.5)
i. Describe efforts to expose students to diverse perspectives and
experiences (e.g., recruiting, faculty awareness, curricular content,
extracurricular activities, etc.)
ii. Include a chart in Appendix D describing the diversity of students
including, but not limited to, gender and race (gender/race data
provided by A&T)
iii. Discuss findings
g. Faculty evaluations (3.6)
i. Describe the SAI process at ETSU and any other student evaluation of
faculty in the department
ii. Describe how the department uses SAI (and any other measures) to
improve classroom teaching and/or programmatic elements
iii. If available, provide aggregated SAI information
iv. Discuss findings
Faculty (Undergraduate Standard 4; Graduate Standard B)
a. Teaching load (4.1, Graduate Standard B-6)
i. Quantitative information (differentiate graduate and undergraduate,
where applicable)
1. Average teaching load
2. FTE
3. Number of majors and minors
4. Class size
ii. Provide a table in Appendix Ea complete listing all faculty names (with
degree suffix—PhD, EdD, etc.), specifying tenured, tenure-track, nontenured, full and part time faculty; identify which faculty members have
graduate faculty status
iii. Compare loads and SCH/FTE with peer institutions
iv. Discuss findings
ETSU 2010-17 Program Review Handbook, Page 11 of 26
b. Faculty diversity (4.2)
i. Describe faculty diversity including, but not limited to, demographics of
gender, ethnicity, and academic background
ii. As needed, include a chart in Appendix E detailing aspects of faculty
diversity
iii. Compare these demographics with those of the discipline nationally
iv. Discuss findings
c. Faculty preparation (4.3, Graduate Standard B1-2)
i. In relation to the following SACS criteria, summarize the preparation of
faculty for teaching in your discipline
SACS 3.7 Faculty
3.7.1 The institution employs competent faculty members qualified to accomplish the mission and goals of the institution. When determining
acceptable qualifications of its faculty, an institution gives primary consideration to the highest earned degree in the discipline in accordance with
the guidelines listed below. The institution also considers competence, effectiveness, and capacity, including, as appropriate, undergraduate and
graduate degrees, related work experiences in the field, professional licensure and certifications, honors and awards, continuous documented
excellence in teaching, or other demonstrated competencies and achievements that contribute to effective teaching and student learning outcomes.
For all cases, the institution is responsible for justifying and documenting the qualifications of its faculty.
Credential Guidelines:
a. Faculty teaching general education courses at the undergraduate level: doctor’s or master’s degree in the teaching discipline or master’s degree with a concentration
in the teaching discipline (a minimum of 18 graduate semester hours in the teaching discipline).
b. Faculty teaching associate degree courses designed for transfer to a baccalaureate degree: doctor’s or master’s degree in the teaching discipline or master’s degree
with a concentration in the teaching discipline (a minimum of 18 graduate semester hours in the teaching discipline).
c. Faculty teaching associate degree courses not designed for transfer to the baccalaureate degree: bachelor’s degree in the teaching discipline, or associate’s degree
and demonstrated competencies in the teaching discipline.
d. Faculty teaching baccalaureate courses: doctor’s or master’s degree in the teaching discipline or master’s degree with a concentration in the teaching discipline
(minimum of 18 graduate semester hours in the teaching discipline). At least 25 percent of the discipline course hours in each undergraduate major are taught by faculty
members holding the terminal degree—usually the earned doctorate—in the discipline.
e. Faculty teaching graduate and post-baccalaureate course work: earned doctorate/ terminal degree in the teaching discipline or a related discipline.
f. Graduate teaching assistants: master’s in the teaching discipline or 18 graduate semester hours in the teaching discipline, direct supervision by a faculty member
experienced in the teaching discipline, regular in-service training, and planned and periodicevaluations.
ii. Refer to Appendix F, which should contain faculty vitae or links to online
versions; note that complete vitae are not necessary, but all must
include degrees earned and specializations, courses typically taught,
publication, research, and/or creative endeavors, service, professional
association activities, and awards/honors for the last five years
iii. Describe the periodic review of graduate faculty status (graduate
programs only)
iv. Provide a summary evaluation of full-time faculty credentials as related
to courses/concentrations taught (Refer to Appendix E or F)
d. Faculty scholarly, creative, professional association, grant, and service
activities. (4.4; Graduate Standards B-3, B-4, B-5)
ETSU 2010-17 Program Review Handbook, Page 12 of 26
XII.
i. Describe professional development opportunities available to faculty
(e.g., travel, professional organization memberships, participation in
conferences and workshops, and opportunities for publication)
ii. Describe departmental and university support provided for these
opportunities (note that “support” is not limited to monetary
assistance)
iii. Summarize activities in which faculty participated, referring to Appendix
F
iv. Discuss any external funding generated in the program and how it
relates to program mission
v. Discuss how these activities enhance instructional expertise and other
aspects of the program
vi. Graduate faculty as mentors for students(graduate programs only)
1. Discuss faculty scholarly activity and how it helps in mentoring
graduate students
2. Discuss faculty experience (practical, academic, professional) in
relationship to mentoring graduate students
e. Adjunct faculty (4.5)
i. Discuss use of adjunct faculty in the program
ii. Discuss adjunct faculty in relation to SACS guidelines quoted above
f. Faculty evaluation system (4.6)
i. Describe FAS and any other faculty evaluation processes
ii. Discuss how evaluation of faculty is used to improve teaching, scholarly
and creative endeavors, and service
Support (Undergraduate Standard 5; Graduate Standard C)
a. Equipment and facilities (5.1, Graduate Standard C-3;C-4;C-5)
i. Describe the equipment, technology (include available support),
computers, classrooms, offices and laboratories that are required and
available to deliver the program
ii. Describe changes made in the last five years in equipment and facilities
iii. Discuss any plans for improvement of these facilities (include
information on faculty computer replacement policy)
iv. Discuss strengths and weaknesses of equipment and facilities
b. Budget(5.2)
i. Evaluate the functionality of the operating budget. Is it sufficient to
support the program’s essential operations?
ii. Describe the program’s success at attracting private support
iii. Describe any unmet budgetary needs, prioritizing needs based on
program mission and essential functions
c. Enrollment and graduation rates (5.3)
i. Discuss enrollment and graduation rates for the last 3-5 years. Provide a
data table(s) in Appendix G (information provided by A&T)
ii. Discuss any plans or strategies related to these rates
ETSU 2010-17 Program Review Handbook, Page 13 of 26
XIII.
XIV.
XV.
XVI.
XVII.
XVIII.
XIX.
XX.
XXI.
XXII.
Program Evaluation (Graduate Standard D) (Graduate only)
a. Graduate Alumni (Graduate Standards D-1; D-3)
i. Describe process for follow-up on graduating students (e.g., exit
interview, newsletters, and requests for updates); include systematic
follow-up conducted by Alumni Office
ii. Describe evaluation of graduate placement
1. Office of Career Placement and Internship Services
2. Alumni Office
3. Within your program
b. Completion rates (Graduate Standard D-4)
i. Describe graduation rated for past 3 years
ii. Interpret findings
Conclusions and Recommendations
a. Draw conclusions and summarize the most salient strengths and major
improvements that have taken place since the last program review
b. Summarize the areas with the greatest need for improvement in the next 5-7
years
c. Describe action planned for improving the program in the next 5-7 years
Appendix A:PIE Report
Appendix B:Table of course offerings
Appendix C:Library Holdings & Services Report
Appendix D:Gender/race data (provided by A&T)
Appendix E:Faculty listing with status/rank, courses taught (see sample chart Appendix
D of this handbook)
Appendix F: Faculty vita (either link(s) to online versions or complete vita
Appendix G: Enrollment and graduation rates (data provided by A&T)
Any additional appendices you wish to include to house data too cumbersome to be
included in text.
What additional information might the reviewers need?
If any of the data requested in the outline above is too unwieldy to include in full in an appendix,
then summarize and include either a) a website where the exhibits are posted; or, b) a note that
the exhibit will be available at the site visit. Physical exhibits should be gathered in a convenient
location in case they are requested by the review team at the time of the site visit.
Among those materials that should be available:




Graduate student written exams or results of oral exams for past 5 years;
Graduate or undergraduate student capstone or culminating experiences (e.g., theses,
practicum/internship reports) for past 3-5 years;
Previous program review narrative report and summary document;
Syllabi for all courses in the program.
ETSU 2010-17 Program Review Handbook, Page 14 of 26
How do the reviewers assess the program?
The review team reads the self-study and related materials before the visit and notes questions
and concerns to be addressed during the review. During the site visit, they observe, question,
and assess the program in light of the self-study. They may also examine additional information
that you will prepare for their perusal.
Before adjourning on the final day, the external reviewers complete the checklist(s) provided by
THEC (see Appendices B & C). The outline given above for the self-study addresses every
checklist point; using this outline will make it easier for reviewers and ensure compliance with
performance funding guidelines.
How is the site visit scheduled?
After reviewers are selected and approved, the Office of Assessment and Teaching will
correspond with them to confirm their participation and send them all of the necessary
materials regarding the self-study, including reviewer guidelines, university catalogs, and selfstudy documents. A&T will reimburse external reviewers for travel expenses. Once the
reviewers arrive in Johnson City, you are their hosts and therefore responsible for
transportation and information.
The faculty and staff of the program under review are responsible for:





Scheduling rooms for departmental sessions (see Appendix A for details) and
collaborating with the Executive Aide, A&T to create complete schedule;
Schedule participation of departmental faculty and students;
Distributing schedule to departmental participants (A&T will distribute to
administrators and reviewers);
Arrange meal events and refreshments;
Provide local transportation for review team.
ETSU 2010-17 Program Review Handbook, Page 15 of 26
What needs to be done by whom?
Use the following checklists to keep track of the activities required for the performance review
process.
Academic Unit
____
Prepare and submit prioritized list and vitae of recommended external reviewers to
Assessment & Teaching Director
____
Prepare and submit draft of self-study to Assessment & Teaching and Dean(s)
____
Revise self-study as needed
_____ Schedule rooms for departmental sessions
_____ Distribute final schedule to review team and participants
_____ Schedule participation in site visit by departmental faculty and students
_____ Arrange all local transportation and communicate plans to external reviewers
_____ Arrange meal events and refreshments
Dean of College
_____ Recommend internal consultant to represent college
_____ Review self-study draft and final version
_____ Meet with Review Team
_____ Participate in follow-up meetings after review
Review Team
_____ Review documents
_____ External reviewers complete and submit checklist for programs
_____ Internal Consultants review self-study and participate in site visit
ETSU 2010-17 Program Review Handbook, Page 16 of 26
APPENDIX A
SAMPLE SCHEDULE FOR PROGRAM REVIEWS
DEPARTMENT OF __________
PROGRAM REVIEW SCHEDULE
DATE
External Reviewers
Name:
Institution:
Name:
Institution:
Internal Reviewers
Name:
Department:
Name:
Department:
Name:
Department:
Day 1 Date
8:15 a.m.
Orientation and Review Team Meeting (A&T Schedules at TBD Location)*
Review Team (External and Internal)
Chair of Department
Dean of College
Vice Provost of Undergraduate Education
Dean of Graduate Studies (Graduate Programs Only)
Director of Assessment and Teaching
9:15 a.m.
Tour and Overview of Department (Program Schedules at TBD Location)
Review Team (External and Internal)
Chair of Department
9:45 a.m.
Faculty Interviews (Program Schedules)
Review Team (External and Internal)
Faculty
12:00 p.m.
Lunch (Program Schedules)
Review Team (External and Internal)
Chair of Department
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
1:30 p.m.
Faculty Interviews (Program Schedules)
Review Team (External and Internal)
Faculty
2:15 p.m.
Dean of College (Program Schedules at Dean’s Office) *
Review Team (External and Internal)
Dean of College
ETSU 2010-17 Program Review Handbook, Page 17 of 26
3:00 p.m.
Break
3:30 p.m.
Undergraduate Students (Program Schedules)
Review Team (External and Internal)
Undergraduate Students (Majors and Students from Service Courses)
4:00 p.m.
Graduate Students (Graduate Programs Only; Program Schedules)
Review Team (External and Internal)
Graduate Students
4:30 p.m.
Summary Session (Program Schedules)
Review Team (External and Internal)
7:00 p.m.
Dinner (Program Schedules)
Review Team (External and Internal)
Chair of Department
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
Day 2 Date
8:15 a. m.
Dean of Graduate Studies (Graduate Programs Only; A&T Schedules)*
Review Team (External and Internal)
Dean of Graduate Studies
Or
8:15 a. m.
Work Time Meeting (A&T Schedules) *
Review Team (External and Internal)
8:45 a. m.
Central Administration (A&T Schedules)*
Review Team (External and Internal)
Dean of College
Vice President of Health Affairs (Academic Health Science Center programs
only)
Vice Provost of Academic Technology Support
Vice Provost of Undergraduate Education
Dean of Graduate Studies (Graduate Programs Only)
Dean of University Libraries
Director of Assessment and Teaching
9:30 a.m.
Review Team Work Session (Program Schedules)
Reviewers begin outline of summary report
External Reviewers complete forms:
Checklist for Assessment of Baccalaureate Programs
Checklist for Assessment of Graduate Programs
ETSU 2010-17 Program Review Handbook, Page 18 of 26
12:00 p.m.
Lunch (Program Schedules)
Review Team (External and Internal)
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
1:30 p.m.
Chair of Department Exit Interview (Program Schedules)
Review Team (External and Internal)
Chair of Department
2:30 p.m.
Concluding Session (A&T Schedules)*
(Evaluation forms are collected from external reviewers)
Review Team (External and Internal)
Chair of Department
Dean of College
Available Department Faculty
Vice President of Academic Affairs or Health Affairs (Academic Health
Science Center Programs Only as Appropriate)
Vice Provost of Academic Technology Support
Vice Provost of Undergraduate Education
Dean of Graduate Studies (Graduate Programs Only)
Dean of University Libraries
Director of Assessment and Teaching
(Addresses and Phone Numbers for Reference)
Chair of Department
Dean of College
* Denotes fixed times. All other meeting times are flexible.
Attendees listed in italics are suggested, but not required. Participants may choose to send
designees to any required meetings.
ETSU 2010-17 Program Review Handbook, Page 19 of 26
2010-15 Performance Funding Cycle
Appendix B: Program Review
Baccalaureate Programs
Institution: ___________________________________________________________________
Program Title:_________________________________________________________________
CIP Code:_____________________________________________________________________
Instructions for External Reviewer(s):
In accordance with the 2010-15 Performance Funding guidelines of the Tennessee Higher Education Commission
(THEC), each non-accreditable baccalaureate program undergoes either an academic audit or external peer review
according to a pre-approved review cycle.
The criteria used to evaluate a program appear in the following “Program Review Summary Sheet for Baccalaureate
Programs.” The Summary Sheet lists 30 criteria grouped into five categories. THEC will use the criteria to assess
standards in the baccalaureate programs. All criteria noted with an asterisk are excluded from the performance
funding point calculation. The Support category will be used by the institution, but will not be included in the overall
assessment reported to THEC.
For each criterion within a standard, the responsible program has provided evidence in the form of a self study
document. Supporting documents will be available as specified in the self study. As the external reviewer, you
should evaluate this evidence and any other evidence observed during the site visit to determine whether each
criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether
you believe that a program has “met” or “not met” each criterion in the checklist. If a particular criterion should be
inappropriate or not applicable to the program under review, the item should be marked “NA”.
This evaluation becomes a part of the record of the academic program review. The summary sheet will be shared
with the department, the college and central administration, as well as the Tennessee Higher Education Commission.
When combined with the written report, prepared by the entire program review committee, the Program Review
Summary Sheet will facilitate development of a program action plan to ensure continuous quality improvement.
Your judgment of the criteria will be used in allocating state funds for the university's budget.
Name, Title, and Institutional Affiliation of Reviewer(s):
_______________________________
____________________________________
Name
Name
___________________________________
Title
___________________________________
Institution
____________________________________
Institution
___________________________________
Signature
____________________________________
Title
Date
____________________________________
Signature
Date
ETSU 2010-17 Program Review Handbook, Page 20 of 26
Program Review Summary Sheet for Baccalaureate Programs
Institution:
Program Title:
CIP Code:
Evaluation
Results
1. LEARNING OBJECTIVES
1.1
Intended program and learning outcomes are clearly identified.
1.2
The program uses appropriate indicators to evaluate appropriate and sufficient achievement of
program outcomes.
1.3
The unit makes use of information from its evaluation of program outcome attainment; student,
alumni, and employer surveys; and university research to strengthen the program's effectiveness.
2. CURRICULUM
2.1
The curriculum is appropriate to the level and purpose of the program.
2.2
The curriculum content and organization is reviewed regularly.
2.3
Program requirements include a strong general education component.
2.4
The curriculum includes a required core of appropriate courses in the discipline.
2.5 *
Met
Not
Met
Met
Not
Met
Met
Not
Met
An appropriate balance is maintained between courses inside the major and outside the major.
2.6
Curricular content reflects current standards, practices, and issues in the discipline.
2.7
The curriculum encourages the development of critical thinking.
2.8
The curriculum exposes students to appropriate research strategies from the program area and
students have the opportunity to participate in research.
2.9
Students have opportunities to apply what they have learned to situations outside the classroom.
2.10
Students are exposed to professional and career opportunities appropriate to the field.
2.11
The program uses appropriate indicators to evaluate appropriate and sufficient achievement in
service courses.
2.12
Courses are offered regularly to ensure that students can make timely progress.
3. TEACHING AND LEARNING ENVIRONMENT
3.1
The program's instructional practices are consistent with the standards of the discipline.
3.2
As appropriate to the discipline, the program provides students with the opportunity for interaction
with one another, faculty, and professionals in the field.
3.3
Effective advising is provided by well-informed faculty and/or professional staff.
ETSU 2010-17 Program Review Handbook, Page 21 of 26
3.4
Library holdings are current and adequate to meet students' needs.
3.5
The program seeks to include the perspectives and experiences of underrepresented groups through
curricular and extracurricular activities.
3.6
Students have the opportunity to regularly evaluate faculty relative to the quality of their teaching
effectiveness.
4. FACULTY
4.1
Not
Met
Met
Not
Met
Yes
No
The faculty is adequate in number to meet the needs of the program with appropriate teaching
loads.
4.2*
As appropriate to the demographics of the discipline, the faculty are diverse with respect to gender,
ethnicity, and academic background.
4.3
Faculty are appropriately prepared for the level of the program, at least meeting SACS requirements
for faculty preparation.
4.4
Met
Faculty are engaged in scholarly, creative, professional association, and service activities that
enhance instructional expertise in their areas of specialty.
4.5*
Adjunct faculty meet the high standards set by the program and expected SACS qualifications and
credentials.
4.6
The unit uses a faculty evaluation system to improve teaching, scholarly and creative activities, and
service.
5. SUPPORT (Note: The Support category is NOT included in the Performance Funding calculation. If the Program
Review process did not address these criteria, they should be marked “NA.”)
5.1 *
The unit regularly evaluates its equipment and facilities, encouraging necessary improvements
within the context of overall college resources.
5.2 *
The program's operating budget is consistent with the needs of the program.
5.3 *
The program has a history of enrollment and graduation rates sufficient to sustain high quality and
cost-effectiveness.
SUMMARY EVALUATION
The program meets or exceeds the minimum standards of good practice.
* Criterion not included in the performance funding calculation.
ETSU 2010-17 Program Review Handbook, Page 22 of 26
2010-15 Performance Funding Cycle
Appendix C: Program Review
Graduate Programs
Instructions for External Reviewers:
In accordance with the 2010-15 Performance Funding guidelines of the Tennessee Higher Education
Commission (THEC), each non-accreditable graduate program undergoes either an external peer review or
academic audit according to a pre-approved review cycle.
The criteria used to evaluate a program appear in the following “Program Review Summary Sheet for
Graduate Programs.” The Summary Sheet consists of 20 criteria grouped into four categories. THEC will
use the criteria to assess standards for graduate programs. All criteria noted with an asterisk are excluded
from the performance funding point calculation
For each criterion within a standard, the responsible program has provided evidence in the form of a Self
Study document. Supporting documents will be available as specified in the self study. As the external
reviewer, you should evaluate this evidence and any other evidence observed during the site visit to
complete the checklist and prepare the narrative report. Items on the summary sheet should be rated on
a four-point scale ranging from “poor” to “excellent” (or N/A for items which are not applicable to the
program).
This evaluation becomes a part of the record of the academic program review. The summary sheet will be
shared with the department, the college and central administration, as well as the Tennessee Higher
Education Commission. When combined with the written report, prepared by the entire program review
committee, the Program Review Summary Sheet will facilitate development of a program action plan to
ensure continuous quality improvement.
Your judgment of the criteria will be used in allocating state funds for the university's budget.
Name, Title, and Institutional Affiliation of Reviewer(s):
_______________________________
Name
____________________________________
Name
___________________________________
Title
___________________________________
Institution
___________________________________
Institution
___________________________________
Signature
____________________________________
Title
Date
___________________________________
Signature
Date
ETSU 2010-17 Program Review Handbook, Page 23 of 26
Program Review Summary Sheet for Graduate Programs
Institution:
Program Title(s):
Degree Designation(s) and CIP Code:
A. Student Experience
1
There is a critical mass of students to ensure an
appropriate group of peers.
2
Prudence is exercised in the number and type of
short courses accepted toward the degree.
3
Programs offered entirely through distance
education technologies are evaluated regularly to
assure outcomes at least equivalent to oncampus programs.
4
There are adequate enrichment
opportunities, such as lecture series, to
promote a scholarly environment.
5
There are adequate professional development
opportunities, such as encouraging membership
in professional associations, participation in
conferences and workshops, and opportunities
for publication.
B. Graduate Faculty Quality
1
Faculty hold terminal degrees in the appropriate
discipline.
2
Faculty academic credentials correspond to the
concentrations in which they teach.
3
Faculty scholarly activity is sufficient to serve as
effective mentors for graduate students
4
Faculty have sufficient
N/A
Poor
Minimally
Acceptable
Good
Excellent
N/A
Poor
Minimally
Acceptable
Good
Excellent
ETSU 2010-17 Program Review Handbook, Page 24 of 26
practical/professional/academic experience to
serve as effective mentors for graduate students.
5
Faculty have regular opportunities for
professional development, including travel
and participation in professional
organizations, workshops and other learning
activities.
6
Faculty teaching loads are consonant with
the highly individualized nature of graduate
instruction, especially the direction of theses
or dissertations.
C. Teaching/Learning Environment
1
There are ample materials and secretarial
support to encourage research and publication.
2
There is adequate library support.
3
There is adequate and accessible computer
support.
4*
There are adequate lab facilities.
5*
There is adequate office space.
D. Program Evaluation
1
Follow-up data on graduating students are
regularly and systematically collected
2
The curriculum is evaluated periodically.
3
Evaluation of placement of graduates is
regular and systematic.
4
Completion rates are at an acceptable level.
N/A
Poor
Minimally
Acceptable
Good
Excellent
N/A
Poor
Minimally
Acceptable
Good
Excellent
* Criterion not included in the performance funding calculation.
ETSU 2010-17 Program Review Handbook, Page 25 of 26
Appendix D: Sample Faculty Data Chart
Name
John M. Doe, PhD
Status
Professor, tenured, graduate
faculty
Fred X. Expert, MNS
Part-time instructor, non-tenure
track
Assistant Professor, tenure track
Jane F. Smith, PhD
Courses Taught
NOTH1410 Introduction to
Nothingness
NOTH3500 Nothingness in
Context
NOTH4000 Nothingness Seminar
NOTH1400 Before Nothingness,
there is Nothing
NOTH2000 Nothingness as a
Profession
NOTH3200 Nothingness and
Philosophy
NOTH3600 Making Something
Out of Nothing
NOTH4000 Nothingness Seminar
ETSU 2010-17 Program Review Handbook, Page 26 of 26