Evolution of Program Review at UCF
Using SAS Business Intelligence Tools to move
Towards Optimization
Patricia Ramsey & Carlos Piemonti
2013 FAIR Conference
January 31, 2013
Program Review Process
Task: Examine the quality and productivity of programs
- Mandated by state statute
- 7 year cycle
- Approximately 30 programs per year
Goal is to determine which programs to:
Enhance, Reduce, Maintain, Eliminate, or Study Further
Process requires a self-study, external consultant visit, and
committee recommendations to the Provost, BOT & BOG
Program Review
Review Data
Data Value
Value
Program
Wide range of data is required to support the process
during all steps
 Departments and programs - self-study prep
 External Consultants - better understand the
program prior to site visit
 Program Review Committee members and the
Provost - to identify opportunities or weaknesses
Program Review Data Requirements
Each program review data document includes the
following:
 Series of data points
 Trend information
 Comparison of program to overall department
and college
Program Review History
Cycle Data
Generation
2000 Manual
Data Output
Data Delivery
MS Word Tables
Emailed
2003 Programmatic
MS Access summary
tables
Web display via
.asp
2006 Programmatic
SAS® Enterprise Guide
(EG) to MS Excel pivot
tables
EG – developed report
format to .pdf
Emailed & Web
(.pdf)
EG and WRS – report
surfaced on web
Web
2008 Programmatic
2012 Programmatic
Web (.pdf)
Progress Through the Years
2000
MS Word
2006
SAS EG
2012
SAS BI
Recent Advances
• Surface the data on the web via
SAS BI tools – Web Report Studio (WRS)
(IR waited for cascading parameters to be available!)
• Provide data for all programs, not just programs
under review
• Auto-populate the self-study templates

Graphs and charts are now developed in Web Report
Studio not in EG
Old Way
New Way
 Update filters to reflect new reporting cycle
The Magic Behind the Scenes
Official data is loaded to Data
Warehouse each term
SAS Information Maps create a view
of the data designed for a purpose
SAS Web Report Studio graphically
surfaces the data on the web
New Maintenance Routine
1. Data prep: run an EG project once a year
and Roll forward one cohort in each
report by updating filters (Feb)
2. Request refreshed data to UAPS
(Projections – May), UAPS does it by
replacing table
3. Generate reports: preliminary (Apr);
final (Aug) for programs under review
New Program Review Cycle
Preparation
Feb
Update ‘Year’ filters
for next cycle – in
each tab in each
report
SAS Web Report
Studio
1H
Feb
Update source data
to include for next
reporting year –
Automatically Info
Maps are refreshed
SAS Enterprise
Guide
University Community has access now!
New Program Review Cycle
PDF Report Generation
Day 4
Day 3
Days 1-2
Respond to
prompts and
print reports as
PDFs (PMP)
Edit PDFs
manually
Combine PDFs
into 1 report per
‘program area’
and update
bookmarks
Program Review 2012-2013:
12 department reports were created
twice (preliminary and final versions)
in less than 10 workdays.
Pegasus Mine Portal
https://my.ucf.edu/
Report Contents
College
Department
Undergraduate
Graduate
Sample
Prompts Page on the Portal
Dynamically generate values
&
Cascading parameters
Cascading Parameters
Prompts Sample
Report Display in the Portal
Update Filters
External Access
web
Program Review 2012-2013
by Department/Program Area
1
Communication
Dean's Office UGST
Health Mgmt. & Informatics
Health Professions
FL Interactive Entertainment Academy
Office of Interdisciplinary Studies
Legal Studies
Modern Languages
Optics
Public Administration
Political Science
12
Physical Therapy
Document Layout Key
For a total of 26 programs!
Limitations
 Security -
Program Review in the portal is
limited to select users. Consultants must have
access to data outside the portal. IR must
make .pdf versions and post to a location
outside this firewall so they have easy access
 Design -
SAS has not yet designed prompts
that apply to the entire report. The user must
respond to prompts on each tab