Methodology for Quintiling Non-Academic Programs
The first step towards non-academic program prioritization by way of the attached questionnaire will be
to have each non-academic area identify its respective programs to be reviewed. Once the programs for
evaluation are identified and reviewed, the following criteria and weights will be used to rank nonacademic programs into quintiles:





25% - Cost-Effectiveness (budget vs. actual, productivity, performance)
20% - Importance to the institution (mission, vision, strategic plan, core themes, mandates)
20% - Demand (internal, external)
20% - Quality (input, outcomes, how well delivered)
15% - Opportunity (collaboration, resource sharing, savings, improvements)
Each program will be scored from 1 to 5 for each of the above criteria according to the following scale:
1.
2.
3.
4.
5.
Does not meet criteria
Slightly meets criteria
Meets criteria
Slightly exceeds criteria
Exceeds criteria
The score (1-5) for each criteria will be converted to a weighted score according to the importance of
the criteria. Under this methodology, the maximum score a program could receive is 5 and the lowest is
1. For instance, a program receiving a 5 (exceeds criteria) for each of the five criteria at their respective
weights, would receive a total weighted score of 5 because the weights total 100% (100% x 5 = 5). Or, a
program receiving a 1 (does not meet criteria) for each of the five criteria at their respective weights,
would receive a total weighted score of 1 (100% x 1 = 1). To better illustrate this scoring methodology,
following is an example of how an intermediate non-academic program could be scored:
Criteria
Cost-Effectiveness
Institutional Importance
Demand
Quality
Opportunity
Weight
Score
0.25
0.20
0.20
0.20
0.15
1
4
4
2
3
Total Weighted Score
1
Weighted Score
0.25
0.80
0.80
0.40
0.45
2.70
Each program will sum to a total weighted score that will enable categorization into quintiles with the
following rankings:
1.
2.
3.
4.
5.
Programs needing enrichment.
Programs needing a higher level of continued support.
Programs needing a neutral or similar level of continued support.
Programs needing a lower level of continued support.
Programs requiring further review, assessment, and action plan to determine what options will
be taken.
The following scores will determine which quintiles the program will fall into:
Score
4.21 – 5.00
3.41 – 4.20
2.61 – 3.40
1.81 – 2.60
1.00 – 1.80
1.
2.
3.
4.
5.
Quintile
Programs needing enrichment
Programs needing a higher level of continued support
Programs needing a neutral or similar level of continued support
Programs needing a lower level of continued support
Programs requiring further review, assessment, and action plan to
determine what options will be taken
In the previous example illustrating how an intermediate program could be scored, the example
program received a weighted score of 2.70. This score places the program in quintile three, a program
needing a neutral or similar level of continued support.
2
Idaho State University
Administrative (Non-Academic) Program Review
for University Program Prioritization1
Program Name:
Administrator:
Department/Unit:
Program Definition: any activity or collection of activities that consumes resources
(i.e. dollars, people, time, space, equipment)
For your responsible area, please identify the major, significant activities that consume resources and
complete this questionnaire for each of these programs. For example, in the Controller’s Office there is
the general accounting department that has major, significant activities such as accounts payable,
collections, financial reporting, travel, and cash management that will qualify as separate programs
within the single area of general accounting. In some cases, these activities themselves may be
subdivided further into additional programs. The major, significant activities are the programs that will
be reviewed through program prioritization by use of this questionnaire. Identify your programs to be
reviewed as those that are major activities consuming significant resources. Please keep in mind that on
our first pass with this new program prioritization process, areas are encouraged to keep programs
broadly defined, so as not to produce more programs that can be reasonably evaluated.
1. Key Goals and Objectives:
- Organizational structure and performance
- Institutional objectives primarily driven by this organization
1.a. Is this program mandated federally?
Yes No Provide Details
1.b. Is this program mandated by the state?
Yes No Provide Details
1
Adapted from materials obtained from Seattle Central Community College, Washington State University, College
of Micronesia, and the University of Central Oklahoma.
3
1.c. Is this program required because of obligations other than federal or state?
If the program is required by previously signed contracts or agreements, identify the party or parties involved in the agreement,
the date and history of the agreement, and the date of expiration of the agreement.
Yes
No
Provide Details
1.d. Is this program essential to the operation of the university?
Yes
No
Provide Details
ISU Mission Statement
The mission of Idaho State University is to advance scholarly and creative endeavors through the creation of new
knowledge, cutting-edge research, innovative artistic pursuits and high-quality academic instruction; to use these
qualities to enhance technical, undergraduate, graduate, and professional education, health care, and other
services provided to the people of Idaho, the Nation, and the World; and to develop citizens who will learn from the
past, think critically about the present, and provide leadership to enrich the future in a diverse, global society.
Core Themes
Learning and Discovery, Access and Opportunity, Leadership in the Health Sciences, and Community Engagement
and Impact.
1.e. How does this program support the mission, vision, strategic plan, core themes, and institutional
objectives of the university?
Provide Details
1.f. How is the program’s performance reviewed? By whom? How often?
Methods of Reviewing Performance
Person/Group Reviewing
Method
4
How Often is Success
Reviewed
2. Key Services Provided to Customers (Internal/External)
2.a. Please provide a brief description of each service/function your program provides, its primary
customers, and the nature of demand.
Description of
Service/Function
Example:
i.e. Planning
Demand:
1. # of people served/unit time (e.g. 10/day)
2. Increasing/Stable/Decreasing
Internal
External
Primary
Customers
List the primary customers
of the service
Insert the number of
people this function
services per unit of time
(day, week or month) and
if that number is
increasing, stable, or
decreasing for internal
users
Insert the number of
people this function
services per unit of time
(day, week or month) and
if that number is
increasing, stable, or
decreasing for external
users
2.b. How many positions were assigned to the program over the last two years?
Position Type
FT
2011 – 2012 (FY12)
Headcount
PT
Total
FTE
Professional
Classified
Student
Exempt
Other
Total
5
FT
2012 – 2013 (FY13)
Headcount
PT
Total
FTE
2.c. How does the size and scope compare with similar/same programs at peer institutions?
Provide Details
2.d. Are there any current or proposed state, regional, or local mandates, or new policies or laws that
may impact external demand for the program’s services?
Yes
No
Provide Details (describe the expected impacts)
2.e. Are there any current or proposed state or regional mandates or new policies that may impact
internal demand for the program’s services?
Yes
No
Provide Details (describe the expected impacts)
3. Key Processes or Implementation Elements:
- Review to reduce bureaucracy and/or streamline operations
- Sunset clauses (where applicable)
- Cost-effective technological improvements
3.a. What are the key processes or implementation elements for this program? How many personnel
are assigned to the key processes or implementation elements? Can any of these key processes or
implementation elements be streamlined or eliminated?
1. Key Implementation Element Identified
Personnel Assigned to Key Element
Can This Element Be Streamlined or Eliminated?
Explain how this implementation element can be streamlined or eliminated, or why this
Yes No
implementation element cannot be streamlined or eliminated.
6
2. Key Implementation Element Identified
Personnel Assigned to Key Element
Can This Element Be Streamlined or Eliminated?
Explain how this implementation element can be streamlined or eliminated, or why this
Yes No
implementation element cannot be streamlined or eliminated.
3. Key Implementation Element Identified
Personnel Assigned to Key Element
Can This Element Be Streamlined or Eliminated?
Explain how this implementation element can be streamlined or eliminated, or why this
Yes No
implementation element cannot be streamlined or eliminated.
4. Key Implementation Element Identified
Personnel Assigned to Key Element
Can This Element Be Streamlined or Eliminated?
Explain how this implementation element can be streamlined or eliminated, or why this
Yes No
implementation element cannot be streamlined or eliminated.
5. Key Implementation Element Identified
Personnel Assigned to Key Element
Can This Element Be Streamlined or Eliminated?
Explain how this implementation element can be streamlined or eliminated, or why this
Yes No
implementation element cannot be streamlined or eliminated.
3.b. Does the program have unmet equipment/software needs?
If yes, complete the chart below.
7
Yes
No
Description of What is Needed
Describe equipment/software needs
Role of the Needed Item in
Fulfilling Program Mission
How will the equipment/software
enable your office to work
effectively and efficiently
Approximate Cost
Estimate the cost
3.c. What technologies are available to the program? Are there technological improvements that could
be made to save on labor, or to improve the product/service offered? How does the program get
technological support?
Technologies Available
Savings Expected on
Labor/Products/Service
Currently Offered
Improvements Could
Be Made For Savings
Technological Support
Currently Receiving
4. Organization Review
- Detailed organization chart for program area
• Position analysis
• Span of control
- Opportunities for:
• Collaboration and restructuring
• Sharing skill sets and resources
• Cross-training
4.a. Display, or attach, the program organization chart with the number of positions in each program
area. Name the key positions identified in span of control.
8
4.b. Designate the key positions in each program area. Which individuals are cross-trained and in what
areas?
Position/Title
Cross-Trained
Yes / No
If Yes, List Areas
4.c. Are there other campus programs providing similar services? If yes, how are this program’s services
different than those of other programs?
Yes
No
Provide Details
4.d. Are there any external or outsourced programs/services providing similar services? If yes, how do
this program’s services differ from theirs?
Yes
No
Name of External Services
Provide Details
9
4.e. Does this program have any external collaborations? If yes, how do these external collaborations
benefit the university? Examples include, but are not limited to, cooperative agreements, articulation
agreements, outreach efforts, corporate partnerships, economic relationships, etc.
Yes
No
Provide Details
5. Budgeting/Planning
- Organization structure and performance metrics comparisons to:
• Similar institutions
• National standards
• “Best of Breed”
5.a. Does the program have any operations that generate revenue?
Yes
No
If yes, provide a list of financial resources generated by the program.
Revenue Source
Amount $
List the source of any revenue collected, i.e. sales, consultancy, etc.
List the amount per program/year
5.b. What were the budgeted costs of the program, actual expenditures, and difference between the
two for fiscal year 2012?
Program Expenditures
Budget
Expenditures
Salaries
Benefits
Services
Supplies
Travel
Equipment
Professional Development
Other
TOTAL
10
Difference
5.c. What metrics are used to evaluate the program’s performance? How does this program’s
performance metric compare to those from peer institutions, national standards, and industry
benchmarks? Click on Peer Institutions to see a list of Institutional Research identified peers.
Name/Description of Performance
Metric
Program
Metric Value
Peer Institution
Metric Value
Industry
Benchmark Metric
Value
“N/A” if not applicable or
metric does not exist
“N/A” if not applicable
or metric does not exist
“N/A” if not applicable or
metric does not exist
6. Opportunities for Savings or Additional Investments
6.a. Have you identified any opportunities for savings or additional investments? If yes, please describe.
Yes
No
Provide Details
Program Review Signature and Date
Program Director:
Date:
Vice President:
Date:
11