Office of Academic Assessment and Program Review, 2014

Program review ensures the effectiveness and currency of established degrees. The review process helps strengthen the quality of UVU’s degree programs and ensure that they are cost-effective and address regional work force needs. Program review and possible redesign of degrees is essential in maintaining vibrant and responsive academic programs. According to USHE policy 411, all programs are reviewed every 7 years. See http://higheredutah.org/wpcontent/uploads/2013/08/R411.pdf.
Departments review all of their degree programs every 7 years based on established criteria.
Departments prepare report in Insight by
October 31.
AEC assists with report preparation &
Insight support.
Dean reviews report & provides feedback by November 30.
Report is revised in Insight as needed, approved by dean, and submitted for AEC review by January 15.
AEC reviews report and scores with rubric.
AEC inputs scores, explanation of scores, & summary of strengths and weaknesses in
Insight by March 15.
Dean and SVPAA review department and AEC reports by April 15.
Dean & department chair discuss results & determine action steps/timeline by April 30.
Dean & SVPAA discuss action steps/timeline by May 31.
Dean & department chair discuss progress annually; update action plan/timeline; summarize progress in Insight by October 31.
1 | P a g e

 Program reviews are submitted in Insight – go to the planning module and then the program review tab.
 Criterion 1, 8, and 9 should be addressed at the department level and reflect all degree programs – bachelor, associate, minors, emphases, etc. Criteria 4-7 should be completed for each degree program within the department.
 Write a succinct response for each of the criterion. Responses should not be more than 250-
300 words.
 Use the guiding questions in the i-buttons for each criterion to direct your response. Be sure to respond specifically to the questions. If there is a reason you are unable to respond to an area within a criterion, please explain why.
 Program reviews cover a time period of 5 years.
 Note the rubric by which each criterion will be evaluated.
 Specific data sources are indicated for some criterion. In other cases, a variety of choices for supporting evidence exist. These data sources are listed by criterion at http://www.uvu.edu/iri/academicprograms/program_review.html
 Additional sources can be used to support responses as appropriate. IRI is available to help with additional data.
 In cases in which data tables are used, the written response should summarize and analyze the data.
 Avoid making claims for which no evidence is provided.
 Keep in mind that peers outside of your department will be reviewing this report. As such, be clear and specific, and remember that you are communicating to those outside of your field.
 Associate deans and Academic Effectiveness Committee members are available for training and guidance.
2 | P a g e

The checklist below provides specifics for each of the criterion.
General:
 Insight – include Criterion 1, 8, 9 at the department level; complete fields for criterion 4-7 for each degree program.
 Responses generally concise
 Responses summarize and analyze data
 Data referenced with links (tables not included in the text of the report)
Criterion 1: History, Development, and Expectations
 Reference to program history, original intent; no numerical data included unless briefly to demonstrate program growth or development
 Reference to adaptation to change, evolution of program, anticipation of change/future
Criterion 2 – External Demand
 2-3 external data sources used/referenced – highlight main/important points
 Summary/analysis of evidence - what does it mean? For example – trends for the degree, not just the subskills of the degree
Criterion 3 – Internal Demand
 Reference to connection of program to other degrees
 Summary/analysis of these connections in terms of service to other areas in the institution
Criterion 4: Degree Profile
 Description of department dashboard statistics referred to/relevant points summarized o Student credit hours and FTE o Number of majors by class level o Number of awards granted o Faculty headcount and composition o Faculty teaching load (FTE, SCH, ICHE, sections by full-time, overload/adjunct) o Graduation rates
 Analysis/interpretation of what this means – course and program/degree strengths and weaknesses, etc.
Criterion 5: Quality of Degree Program Outcomes
 Reference to student learning outcomes for department, summarize main points
 Reference student placement/alumni data
 Analyze the above per findings and resulting actions (acknowledge significance of findings based on number of respondents, etc.)
 Reference faculty output/productivity (% of faculty publishing/presenting, etc.), summarize highlights
 If areas requiring data are missing or the department is unable to provide, a statement of how this is in process/plan to get should be given
Criterion 6: External Revenue and Resources Generated
3 | P a g e

 Reference to highlights of grants, fundraising, etc., include totals, external relationships providing benefits
 Analyze these resources, describing significance to department/school /institution, and resulting actions/plans
Criterion 7: Costs and Other Expenses
 Reference to finance dashboard populated by the Budget Office
 Summarize/analyze the data given (comment on efficiencies, investments provided and needed, how the program is supported in terms of the resources – students, degrees, labs, faculty, etc.); Refer to DFTE data.
Criterion 8: Internal Impact, Justification & Overall Essentiality
 Summation of essentiality of program – impact, benefits, connection to institutional mission, relation to internal factors
 Summation of how the degree program should continue, be strengthened/altered
Criterion 9: External Opportunity Analysis
 Analyze opportunities for improvement, strengthening, change, etc. – refer to how these could be capitalized on/benefitted from, factors that affect the program, etc.
4 | P a g e

The following guidelines are available in the i-buttons in Insight and are included here for your reference.
Criterion 1 – History, Development, and Expectations (Department Level)
Guiding Questions: What was the original intent of the department’s degree programs? How have the degree programs evolved over the years? How have they adapted to meet change?
Criterion 2 – External Demand
Guiding Questions: What external indicators show the need for and attractiveness of the degree program? Consider national and local statistics and trends over time. Consider employer demand for broad educational outcomes.
Criterion 3 – Internal Demand
Guiding Questions: What is the relationship of the degree program to other degrees or curriculum?
How does the degree program serve other degrees or institutional needs? How would alterations in the degree program affect other degrees?
Criterion 4 – Degree Profile
Guiding Questions: What are the strengths, weaknesses, efficiencies, and needs of the degree program?
How effective is the program in graduating students? What issues might need to be addressed in terms of the viability, health, or size of the programs? What trends are indicated by the data?
Criterion 5 - Quality of Degree Program Outcomes
Guiding Questions: What evidence exists for congruence between intended and actual student learning outcomes? How does placement data provide evidence for program effectiveness? How does faculty research/creative work contribute to the program?
Criterion 6: External Revenue and Other Resources Generated
Guiding Questions: What external sources of data (e.g., research grants, fundraising, equipment grants, etc.) does the program have? Indicate the source, amount, and timeframe for the funding as well as total amounts. What potential revenue sources exist? What external relationships or joint partnerships with other educational institutions, corporations, businesses, and governments exist of might be nurtured?
Criterion 7: Costs and Other Expenses
Guiding Questions: What demonstrable efficiencies are associated with the degree program? What investments are needed to bring the degree programs to a higher level of quality?
Criterion 8: Internal Impact, Justification, and Overall Essentiality (Department Level)
Guiding Questions: What impact have the degree programs had or are likely to have? What are the benefits to the institution? How do the degree programs help achieve the institutional mission? How are these degrees related to the success of other degrees? This is a summative measure of why the degree programs should continue or be strengthened.
Criterion 9: External Opportunity Analysis (Department Level)
5 | P a g e

Guiding Questions: What opportunities exist for improvement and strengthening the degree programs
(cf. curriculum, student achievement, competitiveness, innovation)? How might these be capitalized upon? What external environmental factors affect the degree programs? Would a change in degree program formats be beneficial?
6 | P a g e

Program Prioritization Scoring Rubric
Adapted from Robert C. Dickeson
The purpose of the scoring rubric is to ensure inter-rater reliability. AEC members should rate degree programs based on the information provided. The scores 1, 3, and 9 are intended to force differentiation among program review results.
Criterion 1 - Minimal/Limited 3 - Moderate 9 - Exceptional/Significant
History, Development, and Expectations
External Demand
Internal Demand
Degree Profile
The program meets the original expectations of the
University
Demand for the program is limited; trends are flat or declining, which raises questions about its efficacy
The program provides minimal or no service to other programs
The program meets the original expectations of the
University, and has demonstrated the ability to adapt to the changing needs of the
University and its internal and external stakeholders
Demand for the program is moderate
The program provides moderate service to other programs
The program meets the original expectations of the University, has demonstrated the ability to adapt to the changing needs of the University and its internal and external stakeholders, and demonstrates exceptional ability to anticipate change and build for the future
Demand for the program is exceptional; it enjoys a positive trend; it meets a variety of external expectations, and is seen as central to the University’s future
The program provides exceptional service to other programs; such programs could not flourish without the service provided by this program
Enrollments and awards granted are exceptional given the dedicated resources (faculty, sections, class size, etc.)
Quality Outcomes
Revenue
Costs
Enrollments and awards granted are minimal compared to dedicated resources (faculty, sections, class size, etc.)
Evidence of exemplary student performance and results of student learning outcomes assessment and related actions are limited; placement data is weak; faculty productivity is minimal
Program generates little or no revenue on its own
[Less than $_______]
Program costs are not commensurate with quality outcomes such as enrollments, awards granted, student performance, etc.
Enrollments and awards granted are satisfactory compared to dedicated program resources (faculty, sections, class size, etc.)
Evidence of exemplary student performance and results of student learning outcomes are satisfactory; placement data is acceptable; faculty productivity is adequate
Program generates moderate revenue on its own
[From $_____ to
$_____]
Program costs are appropriate for quality outcomes such as enrollments, awards granted, student performance, etc.
Evidence of exemplary student performance and results of student learning outcomes assessment and related actions are exceptional; faculty productivity is commendable
Program generates exceptional revenue, sufficient to sustain the program without draining institutional resources
Program costs are minimal compared to quality outcomes such as enrollments, awards granted, student performance, etc.
7 | P a g e

Impact There is minimal evidence that the program is missioncritical
There is moderate evidence that the program is missioncritical
There is an exceptional relationship between the program/service to the
University’s mission. The program is integral to the University’s future
Opportunity Analysis Projections for the future of this program are unknown or tenuous; additional resources may be needed to maintain this program
Projections for the future of this program indicate a moderate potential for improvement and enhancement of the University’s overall portfolio
Projections for the future of this program are exciting and hold great promise
8 | P a g e