Academic Program Review
Handbook
2015-2020
East Tennessee State University
Revised August 31, 2015
Contact Us:
Office of Assessment
423-439-4236
Center for Academic Achievement,
1st floor Sherrod Library
PO Box 70278
Leigh Lewis
Assistant Director
lewill01@etsu.edu
Kim Lanham
Executive Aide
lanham@etsu.edu
ETSU 2010-17 Program Review Handbook, Page 2 of 26
Contents
What is Academic Program Review? ....................................................................................................................... 4
Why is Program Review important?......................................................................................................................... 4
What is the Timeline for the Program Review Process?................................................................................... 5
How do we identify potential reviewers?............................................................................................................... 6
What goes into in the self-study?............................................................................................................................. 7
What additional information might the reviewers need? ............................................................................ 17
How is the site visit scheduled?.............................................................................................................................. 17
Appendix A: Sample Program Review Schedule ……………………………………………..…………………………18
Appendix B: Baccalaureate Program Checklist …………………………………………………………………………..21
Appendix C: Graduate Program Checklist ………………………………………………………………………………….25
ETSU 2010-17 Program Review Handbook, Page 3 of 26
What is Academic Program Review?
Academic program review is a peer review process designed to improve the quality of the university’s
academic programs. Program reviews provide a systematic method to evaluate quality, productivity, and
need, both in the university and across the state and region.
The program review process takes place on a seven year cycle. During the year of review, an academic
program will collaborate with the Office of Assessment to complete a self-study report and site visit.
After reading the report and participating in the site visit, reviewers of the program will compile a
narrative report that includes recommendations for improvement.
Programs create action plans based on these recommendations and present the outcomes of the
program review to the Provost and others involved in implementing changes. Ongoing tracking of
recommendations and outcomes continues throughout the cycle, until the next review. The follow-up
portion of the program review cycle provides the vital link that enacts improvements brought to light in
self-study and peer review processes.
The Office of Assessment coordinates all reviews. We act as a resource as you work on your self-study
and attempt to make the review process as smooth as possible for all involved. If you have any questions
about the process, please contact our office and we will help in any way we can.
Why is Program Review important?
Program review is important, primarily, to help you identify the strengths and weaknesses of your
program and to help you improve your program, with administrative support. The follow-up process
after program review makes sure that planned changes are implemented in a systematic manner.
Program review is also an important part of the Quality Assurance Funding (QAF) formula that
constitutes a significant part of ETSU’s income stream. QAF is based on an annual 100 point scale, of
which 25 points are allotted to the three types of programmatic evaluations: accreditation, academic
program review, and academic audit. Depending on our degree of success, ETSU can earn over $4 million
per year through QAF. Your work to build a better program and to write a thorough review helps us
receive this incentive funding.
ETSU 2010-17 Program Review Handbook, Page 4 of 26
What is the Timeline for the Program Review Process?
Activity
1. Meet with the Office of Assessment to go over handbook and
process.
2. Recommend and rank potential reviewers; recommend site visit
date. Send to Office of Assesment.
3. Receive data from Office of Assessment.
4. Begin writing self-study.
5. Submit self-study draft to Office of Assessment.
6. Edit self-study draft based on recommendations from the
Office of Assessment. Return to Office of Assessment.
7. Finalize site visit schedule (See: Appendix A of this handbook)
8. Office of Assessment distributes self-study to reviewers and
campus administrators; department distributes to program
faculty and staff.
9. Site visit occurs.
10. Review team submits narrative report
11. Office of Assessment distributes narrative report to Department
chair, Dean, and others involved in site visit; department
distributes to program faculty and staff and initiates
discussions regarding improvement
12. Office of Assessment enters into TracDat: recommendations
proposed in unit’s self-study, reviewers’ recommendations, and
any unmet or partially met standards not addressed in
recommendations
13. Department faculty and staff to discuss recommendations and
to develop a plan of action. Department responses are
recorded in TracDat
14. College dean enters response to the recommendations.
15. Department and Dean meet with VPAA/VPHA and formally
present plan for responding to recommendations; Provost’s
office indicates agreement or disagreement, plus comments;
Office of Assessment enters in TracDat
16. Department and others (Deans and VPAA/VPHA as
appropriate) report implementation of plan in TracDat
Deadline
September 15th (year of review)
October 15th
October 15th
Fall semester
End of fall semester
January and February
1 months before site visit
3 weeks prior to site visit
Spring semester, typically on a
Thursday and Friday during midFebruary to late-April
1 month after site visit
1 week after receipt of narrative
report
1 month after receipt of
narrative report
September 1st (year following
review)
September 15th
October/November (year
following site visit)
July 1st (annually)
ETSU 2010-17 Program Review Handbook, Page 5 of 26
How do we identify potential reviewers?
Identify potential reviewers: 2 to 3 external reviewers (at least one from outside the state of TN), and 2 to
3 internal consultants. Please contact potential reviewers to ask if they are willing to be considered and
able to serve as a reviewer for your program. Once you have established those that can serve, provide a
list of these reviewers to the Office of Assessment as early in the academic year as possible, but no later
than October 15th. All reviewers must meet the qualifications delineated below. The Office of
Assessment will select from your list the two external reviewers (at least one out-of-state) that will
conduct the review, based on their credentials and availability. The external reviewers must be
professionals in the field under review and have no personal or professional affiliation with members of
ETSU’s faculty in the program involved in the self-study. The VPAA/VPHA and TBR will have final
approval of the review team.
When you provide the list of potential reviewers to the Office of Assessment, please include contact
information for the potential reviewers (including email address), and a copy of each external reviewer’s
vita (or links to web versions).
External Reviewers must:
1.
Be professionals in the field under review;
2.
Hold a terminal degree;
3.
Hold an academic position, preferably at a regional public university comparable to ETSU; in
some cases a practicing professional in the field or a retiree is an appropriate substitute;
4.
NOT be ETSU graduates;
5.
NOT have active or previous professional or personal affiliations with faculty in department to be
reviewed, or with other reviewers (co-author, classmate, professor/student, former colleague,
etc.);
It is recommended that each of the following qualifications is held by at least one reviewer:
6.
Chair of Department or coordinator experience;
7.
Training/experience as a program reviewer.
Internal Consultants
Two to three internal consultants are included on every review team. One should be from within the
same college as the program under review; one should be from outside that college. We give preference
to consultants from programs that will be reviewed in coming years, in order to mentor those future
reviews and provide a fresh perspective. The internal consultants can provide important campus-related
information to external reviewers, but they are still key members of the reviewing team, providing insight
from within the university but outside of the program.
Internal consultants must:
1.
Be ETSU Graduate Faculty (if graduate program is reviewed);
2.
Be faculty members outside program being reviewed;
3.
Not be co-author or co-creator with department faculty.
Tips for identifying potential reviewers:

Ask appropriate professional associations for help in identifying potential reviewers. Many
ETSU 2010-17 Program Review Handbook, Page 6 of 26


disciplinary organizations provide training for program reviewers and can provide names of
experienced/trained individuals.
Ask department faculty for suggestions.
Contact comparable programs at other regional universities to learn who successfully reviewed
their programs.
The Office of Assessment does not pay reviewers or consultants for their services. However, we will
reimburse external reviewers for travel costs and provide the per diem rate for meals and incidentals.
Departments may choose to offer honoraria, although this is not an expectation.
What goes into in the self-study?
The narrative of the self-study should be constructed after an open and frank discussion by the program
faculty and staff members as they prepare for the review. Individual faculty or committees could be
appointed to write the self-study, but the program chair is responsible for the final product.
Use the following outline to organize the self-study. Unless otherwise noted, all sections must be
included and use boldfaced headings as given (except for title page). Feel free to include additional
sections or information in any part of the document. Sections V-X correspond directly to the THEC
checklist that your reviewers will use to evaluate your program. If your program does not include a
graduate component, then ignore any items labeled only GR. In each section, provide a description of
that aspect of the program (focusing on the last 5-7 years), any data available to describe or measure
that item, and a discussion of the findings of that aspect of the self-study. The outline should assist you
in describing the program thoroughly.
ETSU 2010-17 Program Review Handbook, Page 7 of 26
I.
Title page
 Title
 Program name
 CIP Code (available from the Office of Assessment)
 URL of department website
 Year of review
II.
Table of contents
III.
Executive summary
 Purpose of self-study and/or description of self-study process.
 Highlight key features of the review or things that will help clarify the rest of the
document.
IV.
Introduction
 Provide a brief history of the program, including
o Any critical events that have taken place since last self-study.
o Major improvements that have taken place since the last program review
 Positive features: What makes this program innovative, outstanding, or unique?
 Summary of conclusions (strengths, weaknesses, costs, benefits)
V.
Learning Outcomes (UG Standard 1; GR Standard 1)
Much of the information on learning outcomes should come directly from IE (or former PIE).
IE/PIE reports may be included in Appendix A and referred to in this section.
 Mission (UG 1.4; GR 1.4)
o What is the programs mission?
o What is the university’s mission?
o How does the programs mission align with the university
 Program and learning outcomes (UG 1.1; GR 1.4)
o What should graduates of this program know and be able to do?
o How were intended learning outcomes developed?
o How are these outcomes measured (i.e. tests, capstone experience, paper)?
 Results of outcome measures (UG 1.2; GR 1.2)
o Summarize how students performed on the outcome measures used for the
program and learning outcomes (include an additional appendices item if you wish
to provide detailed information).
o How satisfied are you with these outcomes?
 Use of outcome data (UG 1.3; GR 1.3)
o Describe how you use data from the sources above to improve and strengthen
program effectiveness.
VI.
Curriculum (UG Standard 2; GR Standard 2)
a. Program overview
ETSU 2010-17 Program Review Handbook, Page 8 of 26
b.
c.
d.
e.
f.
g.
h.
i.
Describe program requirements and credit hours needed for graduation; link to university
catalog (optional: link to departmental website). Include information about different
course options (online, hybrid courses, cohorts, etc.).
Program core (UG 2.7)
List core requirements of the program, including capstone experiences; how do these
courses provide students with a solid foundation in the discipline?
Curricular Mapping (UG 2.4; UG 2.8; GR 2.4)
o How do courses map to the program and student learning outcomes described in
1.1?
o Describe how the curriculum contributes to the mastery of program and student
learning outcomes
o Are courses organized so that knowledge and skill levels are developed from
introductory to advanced?
o How does the curriculum prepare students for careers or advanced study?
Advanced Content (GR 2.3; GR 2.5)
o Describe how the graduate program reflects progressively more advanced content
than its related undergraduate program.
o How does the graduate program include knowledge of the literature of the
discipline throughout the curriculum?
Distance Education (GR 2.7)
o Describe distance education course offerings in the program or plans to
implement such offerings.
o Discuss the evaluation of program and learning outcomes in courses delivered
through technological means, if these types of courses are available to students.
o Compare online (or other technologies) with on ground outcomes. Provide data if
available.
Communication Proficiency (UG 2.9)
o Describe how the curriculum contributes to the development and verbal
presentation of results and ideas.
o Describe how the curriculum contributes to the development and written
presentation of results and ideas.
Technology Proficiency (UG 2.3; GR 2.8)
o Describe how the latest technology is used to enhance student learning.
o What other innovative teaching techniques are used throughout the curriculum?
o Describe efforts to keep teaching methods current and effective.
Research in the curriculum (UG 2.10; GR 2.6)
o Describe where and how students are exposed to research methods and
strategies of the discipline within the program.
o Describe any opportunities for undergraduate students to participate in research.
o How are graduate students encouraged to engage in research?
o What opportunities are available for graduate students to engage in professional
practice and training experiences?
Critical thinking (UG 2.6)
ETSU 2010-17 Program Review Handbook, Page 9 of 26
o Specifically describe where and how the curriculum encourages critical thinking.
o Describe engagement of faculty in critical thinking initiatives/trainings on campus.
o Provide data related to critical thinking measures (CCTST) from the last 5 years in
Appendix B.
j. Curriculum revision and currency (UG 2.1, UG 2.5; GR 2.1)
o Describe the curriculum review process for this program (including how often the
curriculum is reviewed).
o Describe efforts to keep curriculum and practices current, in relation to standards,
practices, and issues in the discipline.
o What changes have been made to the curriculum in the last 5-7 years?
k. Schedule of offerings (UG 2.2; GR 2.2)
o Describe the frequency of course offerings.
o Describe the process for arriving at this schedule.
o In Appendix C, provide a table of course offerings for 3 years (past two years and
current year); if your schedule is standardized, provide one complete cycle
instead.
o Discuss how frequency of student offerings impacts success for students in this
program.
VII.
Student Experience (UG Standard 3; GR Standard 3)
a. Critical Mass (GR 3.1)
o Explain how to the number of students enrolled in the program maintains the
academic integrity of the program.
o Discuss how the numbers of students and delivery format of courses relates to the
peer relationships among students.
b. Professional and career opportunities (UG 3.2; GR 3.3)
o Describe appropriate professional and career opportunities available to students.
o How are these opportunities conveyed to students?
o How are graduate students encouraged to participate in conferences and
workshops, obtain membership in professional organizations, and find
opportunities for publication?
o Identify any student research, publications, presentations, and/or grants that are
the result of faculty collaboration with students in the program (possible appendix
item).
c. Application of learning (UG 3.3; GR 3.4)
o Describe opportunities for students to apply their knowledge outside the
classroom (e.g., internships, service learning, practica).
o Describe any connections with organizations or corporations that facilitate
creative or research interaction, internships, etc.
o Describe and list enrichment opportunities (e.g., lecture series) in the program
and university.
o Discuss how these opportunities promote a scholarly environment for your
students.
ETSU 2010-17 Program Review Handbook, Page 10 of 26
d. Diversity (UG 3.4; GR 3.5)
o Describe efforts to expose students to diverse perspectives and experiences
(curricular content, extracurricular activities, service learning, etc.).
o Include a chart in Appendix D describing the diversity of students including, but
not limited to, gender and race (gender/race data provided by Office of
Assessment).
o Discuss findings.
e. Student Assessment of Instruction (SAI) (UG 3.1; GR 3.2)
o Describe the SAI process at ETSU and any other student evaluation of faculty or
courses in the department.
o Describe how the department uses SAI (and any other measures) to improve
classroom teaching and/or programmatic elements.
f. Academic Support Services (UG 3.5; GR 3.6)
o Library holdings and services
Discuss library support services related to students (note: the Office of
Assessment will supply the department with a library adequacy report to
include as Appendix E of the report – what is requested here is a departmental
faculty assessment of library support materials and services).
o Tutoring (CFAA)
Discuss tutoring services at the university; include a description of program
specific tutoring opportunities.
o Testing (CFAA)
Discuss testing services at the university; include an explanation of how this
program/department utilizes this service for students.
o Advising
1. Describe the advisors and advisement process.
2. Report data related to advising.
3. Discuss findings related to advising.
VIII.
Faculty (UG Standard 4; GR Standard 4)
a. Teaching load (UG 4.2, GR 4.2)
o Compare the number of instructors (both part-time and full-time) with the
number of students (data on number of enrolled students, majors, and minors, as
well as average class size, from the last 5 years will be provided by the Office of
Assessment to include in Appendix F).
o Provide data on average teaching loads.
o Based on the data provided, discuss whether the number of faculty is adequate to
meet the needs of the program.
ETSU 2010-17 Program Review Handbook, Page 11 of 26
o Explain how faculty teaching loads for graduate courses align with the faculty
members’ area of expertise; include information on these and dissertation
mentoring.
o Compare teaching loads and faculty to student ratios with peer institutions
(Delaware Study provided by Institutional Research for Appendix G).
b. Faculty preparation (UG 4.1, GR 4.1)
o Provide a table in Appendix H listing all faculty members; include names, degree
(PhD, EdD, etc.) and specialization, specify tenured, tenure-track, non-tenured, full
and part time faculty; identify which faculty members have graduate faculty
status.
o In relation to the following SACS criteria, summarize the preparation of faculty for
teaching in this discipline.
o Describe the process to obtain and periodic review of graduate faculty status.
ETSU 2010-17 Program Review Handbook, Page 12 of 26
SACS 3.7 Faculty
3.7.1 The institution employs competent faculty members qualified to accomplish the mission and goals
of the institution. When determining acceptable qualifications of its faculty, an institution gives primary
consideration to the highest earned degree in the discipline in accordance with the guidelines listed
below. The institution also considers competence, effectiveness, and capacity, including, as appropriate,
undergraduate and graduate degrees, related work experiences in the field, professional licensure and
certifications, honors and awards, continuous documented excellence in teaching, or other
demonstrated competencies and achievements that contribute to effective teaching and student
learning outcomes. For all cases, the institution is responsible for justifying and documenting the
qualifications of its faculty.
Credential Guidelines:
a. Faculty teaching general education courses at the undergraduate level: doctor’s or master’s degree in
the teaching discipline or master’s degree with a concentration in the teaching discipline (a minimum
of 18 graduate semester hours in the teaching discipline).
b. Faculty teaching associate degree courses designed for transfer to a baccalaureate degree: doctor’s or
master’s degree in the teaching discipline or master’s degree with a concentration in the teaching
discipline (a minimum of 18 graduate semester hours in the teaching discipline).
c. Faculty teaching associate degree courses not designed for transfer to the baccalaureate degree:
bachelor’s degree in the teaching discipline, or associate’s degree and demonstrated competencies in
the teaching discipline.
d. Faculty teaching baccalaureate courses: doctor’s or master’s degree in the teaching discipline or
master’s degree with a concentration in the teaching discipline (minimum of 18 graduate semester
hours in the teaching discipline). At least 25 percent of the discipline course hours in each
undergraduate major are taught by faculty members holding the terminal degree—usually the earned
doctorate—in the discipline.
e. Faculty teaching graduate and post-baccalaureate course work: earned doctorate/ terminal degree in
the teaching discipline or a related discipline.
f. Graduate teaching assistants: master’s in the teaching discipline or 18 graduate semester hours in the
teaching discipline, direct supervision by a faculty member experienced in the teaching discipline,
regular in-service training, and planned and periodic evaluations.
ETSU 2010-17 Program Review Handbook, Page 13 of 26
c. Faculty diversity (UG 4.3; GR 4.3)
o Describe faculty diversity including, but not limited to, demographics of gender,
ethnicity, and academic background.
o As needed, include a chart in Appendix I detailing aspects of faculty diversity.
o Compare these demographics with those of the discipline nationally.
d. Faculty evaluations (UG 4.4; GR 4.6)
o Describe peer reviews, FAS, and any other faculty evaluation processes
o Discuss how evaluation of faculty is used to improve teaching, scholarly and
creative endeavors, and service.
e. Faculty development (UG 4.5; GR 4.4)
o Describe professional development opportunities available to faculty (e.g., travel,
professional organization memberships, participation in conferences and
workshops, and opportunities for publication)
o Describe departmental and university support provided for these opportunities
(note that “support” is not limited to monetary assistance)
o Summarize activities (research, publications, grants, etc.) by faculty member for
the last 5 years in Appendix J.
o Discuss how these activities enhance instructional expertise and other aspects of
the program.
o Graduate faculty as mentors for students:
1. Discuss faculty scholarly activity and how it helps in mentoring graduate
students.
2. Discuss faculty experience (practical, academic, professional) in
relationship to mentoring graduate students.
f. Faculty involvement (UG 4.6, GR 4.5)
o Describe faculty’s involvement in planning, evaluation, and improvement
processes related to the program and student learning outcomes listed in 1.1.
o How satisfied are faculty members with the outcomes and measurements?
o Are faculty members satisfied with the improvements made based on the
measurement results?
IX.
Learning Resources (UG Standard 5; GR Standard 5)
a. Equipment and facilities (UG 5.1, GR 5.1)
o Describe the equipment, technology (include available support), computers,
classrooms, offices and laboratories that are required and available to deliver the
program
o Describe changes made in the last five years in equipment and facilities
o Discuss any plans for improvement of these facilities (include information on
faculty computer replacement policy)
o Discuss strengths and weaknesses of equipment and facilities
b. Learning resources (UG 5.2; GR 5.2)
o Discuss library support services related to faculty members (note: the Office of
Assessment will supply the department with a library adequacy report to include
ETSU 2010-17 Program Review Handbook, Page 14 of 26
as Appendix E of the report – what is requested here is a departmental faculty
assessment of library support materials and services).
o Describe any other resources that support teaching and learning (i.e. D2L).
c. Support for faculty research and publication (GR 5.3)
o Describe materials directly related to the encouragement and support of research
and publication.
o Describe the staff support available for research and publication.
o Evaluate the sufficiency of available materials and staffing.
X.
Support (UG Standard 6; GR Standard 6)
a. Budget(UG 6.1; GR 6.1)
o Summarize the program’s/department’s operating budget. Is it sufficient to
support the program’s essential operations?
o Describe any unmet budgetary needs, prioritizing needs based on program
mission and essential functions.
b. Enrollment and graduation rates (UG 6.2; GR 6.2)
o Discuss enrollment and graduation rates for the last 5 years. Data will be provided
by the Office of Assessment for Appendix F. Are these rates sufficient to sustain a
high quality program?
o Are enrollment and graduation rates cost-effective for this department/program?
o Discuss any plans or strategies related to these rates.
c. Responsiveness to needs (UG 6.3; GR 6.3)
o Provide data on local, state, regional, and/or national needs related to this
program.
o Explain how this program responds to those needs (volunteer work, job market,
etc.)
d. Graduate alumni (GR 6.4)
o Describe the process for follow-up on graduating students (e.g., exit interview,
newsletters, and requests for updates); include systematic follow-up conducted by
Alumni Office.
o Describe evaluation of graduate placement, consider data collected by the:
1. Office of Career Placement and Internship Services.
2. Alumni Office.
3. Program or department.
e. Policies and procedures (GR 6.5)
o Describe the policies and procedures regarding graduate student enrollment and
services (may link to the graduate student handbook).
o How to the policies and procedures in the program/department align with the
School of Graduate Studies and the university?
o How often are these policies and procedure reviewed?
ETSU 2010-17 Program Review Handbook, Page 15 of 26
XI.
Conclusions and Recommendations
a. Summarize the areas with the greatest need for improvement in the next 5-7 years.
b. Describe actions planned for improving the program in the next 5-7 years.
XII.
Appendix Items
a. Appendix A: IE Report
b. Appendix B: CCTST Results
c. Appendix C: Table of Course Offerings
d. Appendix D: Student Diversity Data (provided by Office of Assessment)
e. Appendix E: Library Holdings & Services Report (provided by the Office of Assessment)
f. Appendix F: Enrollment, Graduate Rates, and Class Size Report (provided by the office of
Assessment)
g. Appendix G: Delaware Study (provided by Office of Assessment)
h. Appendix H: Faculty Report
i. Appendix I: Faculty Diversity Data
j. Appendix J: Faculty Activities Report
k. Any additional appendices you wish to include to house data too cumbersome to be
included in text.
ETSU 2010-17 Program Review Handbook, Page 16 of 26
What additional information might the reviewers need?
If any of the data requested in the outline above is too unwieldy to include in full in an appendix, then
summarize and include either a) a website where the exhibits are posted; or, b) a note that the exhibit
will be available at the site visit. Physical exhibits should be gathered in a convenient location in case
they are requested by the review team at the time of the site visit.
Among those materials that could be available:






Written exams, reports, projects, etc. used for IE over the past 5 years;
Previous program review narrative report and summary document;
Syllabi for all courses in the program;
Journal articles from students or student/faculty collaborations;
Research presentations from students.
How do the reviewers assess the program?
The review team reads the self-study and related materials before the visit and notes questions and
concerns to be addressed during the review. During the site visit, they observe, question, and assess the
program in light of the self-study. They may also examine additional information that you will prepare
for their perusal.
Before adjourning on the final day, the external reviewers complete the checklist(s) provided by THEC
(see Appendices B & C). The outline given above for the self-study addresses every checklist point;
using this outline will make it easier for reviewers and ensure compliance with performance funding
guidelines.
How is the site visit scheduled?
After reviewers are selected and approved, the Office of Assessment will correspond with them to
confirm their participation and send them all of the necessary materials regarding the self-study,
including reviewer guidelines, university catalogs, and self-study documents. The Office of Assessment
will reimburse external reviewers for travel expenses. Once the reviewers arrive in Johnson City, the
program will act as their host and therefore be responsible for transportation and information.
The faculty and staff of the program under review are responsible for:


Scheduling rooms for departmental sessions (see Appendix A for details) and collaborating with
the Office of Assessment to create complete schedule;
Schedule participation of departmental faculty, students, and stakeholders;
Distributing schedule to departmental participants (Office of Assessment will distribute to
administrators and reviewers);
Arrange meal events and refreshments;

Provide local transportation for review team.


ETSU 2010-17 Program Review Handbook, Page 17 of 26
APPENDIX A
PROGRAM REVIEW SCHEDULE
Academic Department Name
Date of Visit
External Reviewers
Name, Institution
Name, Institution
Internal Reviewers
Name, Department
Name, Department
Name, Department
Day 1 Date (Program schedules travel to and from hotel.)
8:30 a.m.
Orientation and Review Team Meeting (Assessment schedules at TBD location.)*
Review Team (External and Internal)
Chair of Department
Assistant Director of Assessment
9:15 a.m.
Tour and Overview of Department (Program schedules location.)
Review Team (External and Internal)
Chair of Department
9:45 a.m.
Faculty Interviews (Program schedules location and attendees.)
Review Team (External and Internal)
Faculty
12:00 p.m.
Lunch (Program schedules location, attendees, and transportation.)
Review Team (External and Internal)
Chair of Department
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
1:30 p.m.
Faculty Interviews (Program schedules location and attendees.)
Review Team (External and Internal)
Faculty
2:15 p.m.
Dean of College (Assessment schedules in Dean’s office.) *
Review Team (External and Internal)
Dean of College
ETSU 2010-17 Program Review Handbook, Page 18 of 26
3:00 p.m.
Break
3:30 p.m.
Undergraduate Students (Program Schedules)
Review Team (External and Internal)
Undergraduate Students (majors only)
4:00 p.m.
Graduate Students (Graduate programs only; program schedules location and attendees.)
Review Team (External and Internal)
Graduate Students
4:30 p.m.
Summary Session (Program schedules location.)
Review Team (External and Internal)
7:00 p.m.
Dinner (Program schedules location and attendees.)
Review Team (External and Internal)
Chair of Department
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
Day 2 Date (Program schedules travel to and from hotel.)
8:15 a. m.
office.) *
Dean of Graduate Studies (Graduate Programs Only; Assessment schedules in Dean’s
Review Team (External and Internal)
Dean of Graduate Studies
8:45 a. m.
Central Administration (Assessment schedules location and attendees.) *
Review Team (External and Internal)
Dean of College
Vice President of Academic or Health Affairs (depending on program)
Vice Provost of Academic Technology Support
Dean of University Libraries
Assistant to the Provost for Institutional Research
Director of the Center for Academic Achievement
Director of Institutional Effectiveness
Assistant Director of Assessment
9:30 a.m.
Review Team Work Session (Program schedules location.)
Reviewers begin outline of summary report
External Reviewers complete forms:
Checklist for Assessment of Baccalaureate Programs
ETSU 2010-17 Program Review Handbook, Page 19 of 26
Checklist for Assessment of Graduate Programs
12:00 p.m.
Lunch (Program schedules location and attendees.)
Review Team (External and Internal)
Available Faculty
Others invited by program (e.g., students, alumni, etc.)
1:30 p.m.
Chair of Department Exit Interview (Program schedules location.)
Review Team (External and Internal)
Chair of Department
2:30 p.m.
Concluding Session (Assessment schedules location and attendees.)*
(Evaluation forms are collected from external reviewers.)
Review Team (External and Internal)
Chair of Department
Dean of College
Available Department Faculty
Vice President of Academic Affairs or Health Affairs (depending on program)
Dean of Graduate Studies (Graduate programs only)
Assistant Director of Assessment
(Addresses and Phone Numbers for Reference)
Chair of Department
Office of Assessment (423) 439-4236
* Denotes fixed times. All other meeting times are flexible.
ETSU 2010-17 Program Review Handbook, Page 20 of 26
APPENDIX B
2015-20 Quality Assurance Funding
Program Review: Baccalaureate Programs
Institution:
Program Title:
CIP Code:
Instruction for External Reviewer(s)
In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education
Commission (THEC), each non-accreditable baccalaureate program undergoes either an academic audit or external
peer review according to a pre-approved review cycle.
The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review
Rubric lists 30 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute
points in to baccalaureate programs. The four criteria noted with an asterisk are excluded from the point calculation
but will be used by the institution in their overall assessment.
For each criterion within a standard, the responsible program has provided evidence in the form of a Self Study.
Supporting documents will be available for review as specified in the Self Study. As the external reviewer, you
should evaluate this evidence and any other evidence observed during the site visit to determine whether each
criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether
the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is
inappropriate or not applicable to the program under review, the item should be marked NA.
This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the
department, college and central administration, as well as the Tennessee Higher Education Commission. When
combined with the written report, prepared by the entire program review committee, the Program Review Rubric
will facilitate development of a program action plan to ensure continuous quality improvement.
Your judgment of the criteria will be used in allocating state funds for the university's budget.
Name, Title and Institutional Affiliation of Reviewer(s)
Name
Name
Title
Title
Institution
Institution
Signature
Signature
Date
Date
ETSU 2010-17 Program Review Handbook, Page 21 of 26
Program Review Rubric
Baccalaureate Programs
Directions: Please rate the quality of the academic program by placing a checkmark in the appropriate box to indicate
whether the program currently exhibits poor, fair, good or excellent evidence of meeting the criterion.
1. Learning Outcomes
1.2
1.3
1.4
Curriculum
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
3.2
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
The curriculum content and organization are reviewed regularly
and results are used for curricular improvement.
The program has developed a process to ensure courses are
offered regularly and that students can make timely progress
towards their degree.
The program incorporates appropriate pedagogical and/or
technological innovations that enhance student learning into the
curriculum.
The curriculum is aligned with and contributes to mastery of
program and student learning outcomes identified in 1.1.
The curricular content of the program reflects current standards,
practices, and issues in the discipline.
The curriculum fosters analytical and critical thinking and
problem-solving.
The design of degree program specific courses provides students
with a solid foundation.
The curriculum reflects a progressive challenge to students and
that depth and rigor effectively prepares students for careers or
advanced study.
The curriculum encourages the development of and the
presentation of results and ideas effectively and clearly in both
written and oral discourse.
The curriculum exposes students to discipline-specific research
strategies from the program area.
3. Student Experience
3.1
Poor
Program and student learning outcomes are clearly identified and
measurable.
The program uses appropriate evidence to evaluate achievement
of program and student learning outcomes.
The program makes use of information from its evaluation of
program and student learning outcomes and uses the results for
continuous improvement.
The program directly aligns with the institution's mission.
1.1
2.
N/A
The program provides students with opportunities to regularly
evaluate the curriculum and faculty relative to the quality of their
teaching effectiveness.
The program ensures students are exposed to professional and
career opportunities appropriate to the field.
ETSU 2010-17 Program Review Handbook, Page 22 of 26
The program provides students with the opportunity to apply
what they have learned to situations outside the classroom.
The program seeks to include diverse perspectives and
experiences through curricular and extracurricular activities.
Students have access to appropriate academic support services.
3.3
3.4
3.5
4. Faculty (Full-time and Part-time)
4.1
4.2
4.3*
4.4
4.5
4.6
5.
5.2
6.
6.2*
6.3
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
The program regularly evaluates its equipment and facilities,
encouraging necessary improvements within the context of
overall institutional resources.
The program has access to learning and information resources
that are appropriate to support teaching and learning.
Support
6.1*
Poor
All faculty, full time and part-time, meet the high standards set
by the program and expected SACSCOC guidelines for
credentials.
The faculty are adequate in number to meet the needs of the
program with appropriate teaching loads.
The faculty strives to cultivate diversity with respect to gender,
ethnicity, and academic background, as appropriate to the
demographics of the discipline.
The program uses an appropriate process to incorporate the
faculty evaluation system to improve teaching, scholarly and
creative activities, and service.
The faculty engages in regular professional development that
enhances their teaching, scholarship and practice.
The faculty is actively engaged in planning, evaluation and
improvement processes that measure and advance student
success.
Learning Resources
5.1*
N/A
The program's operating budget is consistent with the needs of
the program.
The program has a history of enrollment and/or graduation rates
sufficient to sustain high quality and cost-effectiveness.
The program is responsive to local, state, regional, and national
needs.
*Criteria not scored as part of Quality Assurance Funding.
ETSU 2010-17 Program Review Handbook, Page 23 of 26
APPENDIX C
2015-20 Quality Assurance Funding
Program Review: Graduate Programs
Institution:
Program Title:
CIP Code:
Degree Designation:
Instruction for External Reviewer(s)
In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education
Commission (THEC), each non-accreditable graduate program undergoes either an academic audit or external peer
review according to a pre-approved review cycle.
The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review
Rubric lists 32 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute
points in to graduate programs. The four criteria noted with an asterisk are excluded from the point calculation but
will be used by the institution in their overall assessment.
For each criterion within a standard, the responsible program has provided evidence in the form of a Self Study.
Supporting documents will be available for review as specified in the Self Study. As the external reviewer, you
should evaluate this evidence and any other evidence observed during the site visit to determine whether each
criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether
the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is
inappropriate or not applicable to the program under review, the item should be marked NA.
This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the
department, college and central administration, as well as the Tennessee Higher Education Commission. When
combined with the written report, prepared by the entire program review committee, the Program Review Rubric
will facilitate development of a program action plan to ensure continuous quality improvement.
Your judgment of the criteria will be used in allocating state funds for the university's budget.
Name, Title and Institutional Affiliation of Reviewer(s)
Name
Name
Title
Title
Institution
Institution
Signature
Signature
Date
Date
ETSU 2010-17 Program Review Handbook, Page 24 of 26
Program Review Rubric
Graduate Programs
Directions: Please rate the quality of the academic program by placing a checkmark in the appropriate box to
indicate whether the program currently exhibits poor, fair, good or excellent evidence of meeting the criterion.
1. Learning Outcomes
1.1
1.2
1.3
1.4
2.
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
3.2
3.3
Poor
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
Program and student learning outcomes are clearly identified
and measurable.
The program uses appropriate evidence to evaluate
achievement of program and student learning outcomes.
The program makes use of information from its evaluation of
program and student learning outcomes and uses the results
for continuous improvement.
The program directly aligns with the institution's mission.
Curriculum
The curriculum content and organization is reviewed
regularly and the results are used for curricular improvement.
The program has developed a process to ensure courses are
offered regularly and that students can make timely progress
towards their degree.
The program reflects progressively more advanced in
academic content than its related undergraduate programs.
The curriculum is aligned with and contributes to mastery of
program and student learning outcomes identified in 1.1.
The curriculum is structured to include knowledge of the
literature of the discipline.
The curriculum strives to offer ongoing student engagement
in research and/or appropriate professional practice and
training experiences.
Programs offered entirely through distance education
technologies are evaluated regularly to assure achievement of
program outcomes at least equivalent to on-campus
programs.
The program incorporates appropriate pedagogical and/or
technological innovations that advance student learning into
the curriculum.
3. Student Experience
3.1
N/A
The program ensures a critical mass of students to ensure an
appropriate group of peers.
The program provides students with the opportunities to
regularly evaluate the curriculum and faculty relative to the
quality of their teaching effectiveness.
The program provides adequate professional development
opportunities, such as encouraging membership in
professional associations, participation in conferences and
workshops, and opportunities for publication.
ETSU 2010-17 Program Review Handbook, Page 25 of 26
3.4
The program provides adequate enrichment opportunities,
such as lecture series, to promote a scholarly environment.
3.5
The program seeks to include diverse perspectives and
experiences through curricular and extracurricular activities.
Students have access to appropriate academic support
services.
3.6
4. Faculty
4.1
4.2
4.3*
4.4
4.5
4.6
5.
5.2
5.3
6.
6.2*
6.3
6.4
6.5
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
N/A
Poor
Fair
Good
Excellent
The program regularly evaluates its equipment and facilities,
encouraging necessary improvements within the context of
overall institutional resources.
The program has access to learning and information
resources that are appropriate to support teaching and
learning.
The program provides adequate materials and support staff to
encourage research and publication.
Support
6.1*
Poor
All faculty, full time and part-time, meet the high standards
set by the program and expected SACSCOC guidelines for
credentials.
The faculty teaching loads are aligned with the highly
individualized nature of graduate instruction, especially the
direction of theses or dissertations.
The faculty strives to cultivate diversity with respect to
gender, ethnicity, and academic background, as appropriate
to the demographics of the discipline.
The faculty engages in regular professional development that
enhances their teaching, scholarship and practice.
The faculty is actively engaged in planning, evaluation and
improvement processes that measure and advance student
success.
The program uses an appropriate process to incorporate the
faculty evaluation system to improve teaching, scholarly and
creative activities, and service.
Learning Resources
5.1*
N/A
The program's operating budget is consistent with the needs
of the program.
The program has a history of enrollment and/or graduation
rates sufficient to sustain high quality and cost-effectiveness.
The program is responsive to local, state, regional, and
national needs.
The program regularly and systematically collects data on
graduating students and evaluates placement of graduates.
The program's procedures are regularly reviewed to ensure
alignment to institutional policies and mission.
*Criteria not scored as part of Quality Assurance Funding.
ETSU 2010-17 Program Review Handbook, Page 26 of 26