Integrating Performance Measures
into University Endeavor
Victor M. H. Borden, Ph.D.
Associate Vice President
University Planning, Institutional Research, and Accountability (IU)
Associate Professor of Psychology (IUPUI)
Or
Becoming an Evidence-Driven
Learning Organization
Victor M. H. Borden, Ph.D.
Associate Vice President
University Planning, Institutional Research, and Accountability (IU)
Associate Professor of Psychology (IUPUI)
Or
How I Learned to Stop Worrying
and Love Performance Measures
Victor M. H. Borden, Ph.D.
Associate Vice President
University Planning, Institutional Research, and Accountability (IU)
Associate Professor of Psychology (IUPUI)
Integrating Performance Measures into University Endeavor
If this were a simple matter, you
would have figured it out long ago and
I wouldn’t be here. Do not expect my
explanations to be simple nor my
advice to be straightforward.
This will be more like a graduate-level
seminar than an introductory course
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
The Institutional Research Credo
I realize that I will not succeed in answering all of
your questions. Indeed, I will not answer any of
them completely. The answers I provide will only
serve to raise a whole new set of questions that lead
to more problems, some of which you weren’t aware
of in the first place. When my work is complete, you
will be as confused as ever, but hopefully, you will be
confused on a higher level and about more important
things
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Why Not “Data-Driven?”
 Data,
per se, are not what we need
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
NMPSID
AS0022
AS0023
AS0024
AS0201
AS0202
AS0203
AS0204
AS0205
AS0206
AS0207
AS0208
AS0209
AS0210
AS0211
AS0212
AS0213
AS0214
AS0215
AS0216
AS0217
AS0218
AS0219
AS0220
AS0221
AS031282
AS031284
AS031288
AS031310
AS031327
AS031328
Tracked
1
0
1
1
1
1
1
1
1
1
1
0
0
1
0
0
0
1
0
0
0
1
0
1
0
0
0
0
0
0
DBIRTH
05/24/1962
09/09/1954
07/02/1982
8/9/76
8/17/83
11/9/70
8/26/74
3/26/68
8/31/83
9/8/76
11/6/55
1/7/1985
3/19/1977
11/15/1971
6/29/1971
7/5/1956
10/21/1960
7/23/1975
7/6/1972
2/23/1977
2/5/1983
8/15/1967
11/21/1961
7/21/1962
3/17/1982
9/2/1974
5/30/1975
7/30/1968
Age
43.3
51.0
23.2
26
19
32
28
34
19
26
47
17.6
25.5
30.8
31.2
-9
46.2
41.9
-9
27.1
30.2
25.5
19.6
35
41.8
41.1
21.5
29
28.3
35.1
GENDER
1
1
0
1
0
1
1
0
1
1
1
0
1
1
1
-9
1
1
-9
1
1
1
1
1
1
1
0
1
1
1
RACE
1
3
6
6
6
6
1
6
6
6
6
3
6
3
1
7
6
1
7
3
6
4
6
2
-9
-9
-9
3
3
3
MARSTAT
2
1
1
-9
1
-9
1
-9
1
-9
-9
-9
1
-9
-9
-9
-9
4
-9
1
1
1
-9
-9
-9
-9
-9
-9
-9
-9
SATV
-9
-9
530
-9
-9
-9
-9
520
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
SATM
-9
-9
530
-9
-9
-9
-9
460
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
Victor M. H. Borden – February 9, 2006 – Ohio University
ACTC HSCTYP
-9
0.00
-9
-9.00
20
1.00
-9
-9.00
-9
1.00
-9
1.00
20
1.00
-9
1.00
-9
1.00
-9
-9.00
-9
0
-9
-9.00
20
1
-9
-9
-9
0
-9
-9
-9
-9.00
-9
-9.00
-9
-9.00
-9
-9.00
-9
-9
-9
1
-9
1
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
1
HSGPA
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
-9
2.47
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
If Not Data-Driven, Then What?
 Evidence-based
practice to decide…
 What
to do
 How best to do it
 If it is working as desired
 So
that we can learn from what we do and
improve
 We want to be part of a Learning Organization
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Learning Organizations
 …organizations
where people continually
expand their capacity to create the results
they truly desire, where new and
expansive patterns of thinking are
nurtured, where collective aspiration is set
free, and where people are continually
learning to see the whole together.
(Senge, 1990)
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Learning Organizations
 …are
characterized by total employee
involvement in a process of collaboratively
conducted, collectively accountable
change directed towards shared values or
principles. (Watkins and Marsick 1992)
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Overview
 Lessons
I’ve learned (the hard way) about
developing university performance
measures
 Performance measures as the “tip of the
evidence-based iceberg”
 Going
below the surface
 Applying
an organizational learning lens
 Some implications and related thoughts
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Lessons Learned
 Early
lessons on measurement theory
 1994 NDIR Volume
 Measuring Institutional Performance
Outcomes (APQC-MIPO)
 Developing campus PIs to link planning,
budgeting, evaluation and improvement
 Taking it to the next level
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Measurement Theory
Inductive – Deductive Cycle
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Measurement Theory

Validity


Warranted assertion (Dewey)
Degree to which the measure accurately represents the concept
(what you are attempting to measure)
• Size of a person (weight, height, circumference, body mass)
• Quality of instruction (course ratings, peer review, student learning)?

Reliability

Degree to which measure faithfully represents the concept
• Course ratings taken mid-term/end-term

Unless very careful attention is paid to one’s theoretical
assumptions and conceptual apparatus, no array of
statistical techniques will suffice – Blalock, 1982
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
1994 NDIR Volume
 Using
Performance Indicators to Guide
Strategic Decision Making (Borden and
Banta, Eds.)
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Lessons

Borden and Bottrill: Where you stand on PIs
depends on where you sit
 Ewell and Jones: Think before you count
 Joengblood and Westerheijden (Europe): PIs
out, Quality Assurance in
 Dorris and Teeter (TQM): PIs are fine, if P
stands for Process
 Dolence and Norris: KPIs are the fuel of a
strategic decision engine
 DeHayes and Lovrinic (ABC): Show me the
money…and what you use it for doing.
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Lessons (continued)
 Banta
and Borden Criteria for Effective PIs
 Start
with purpose
 Align throughout organization
 Align across input, process, output
 Coordinate a variety of methods
 Use in decision making
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Measuring Institutional
Performance Outcomes
 An American
Productivity and Quality
Center (APQC) benchmarking study
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
APQC MIPO Findings

The best institutional performance measures
communicate the institution’s core values
 Good institutional performance measures are carefully
chosen, reviewed frequently, and point to action to be
taken on results
 External requirements and pressures can be extremely
useful as starting points for developing institutional
performance measurement systems
 Performance measures are best used as “problem
detectors” to identify areas for management attention
and further exploration
 Clear linkages between performance measures and
resource allocation are critical, but the best linkages are
indirect (and non-punitive)
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
MIPO Cont.

Performance measures must be publicly available,
visible, and consistent across the organization
 Performance measures are best considered in the
context of a wider transformation of organizational
culture
 Organizational cultures supportive of performance
measures take time to develop, require considerable
“socialization” of the organization’s members, and are
enhanced by stable leadership
 Performance measures change the role of managers
and the ways in which they manage
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
MIPO – Boiling it Down
 You
cannot ‘lead’ with performance
measures
 Performance measures emerge from a
broader culture of evidence, that is, they
are part of something bigger
Victor M. H. Borden – February 9, 2006 – Ohio University
www.iport.iupui.edu
Integrating Performance
Measures into University Endeavor
E.G.: PIs@IUPUI
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Taking it to the Next Level:
Accountability at Indiana
University
Articulating and Attaining Strategic
Goals and Objectives
Integrating Performance Measures into University Endeavor
Audiences
 Board
 Most
of Trustees
comprehensive, University-wide view
 Campus
accreditors and (prospective)
partners
 Campus-specific
 Targeted
objectives and indicators
packaging for…
 Media;
legislators; alumni; current and
prospective students and their parents;
research agencies and collaborators
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Purposes


Position IU strategically
Improve the effectiveness and quality of
programs and services
 Provide a common framework to align efforts
across campuses
 Communicate a clear and consistent message
about IU’s broad goals
 Enhance IU’s image


Define and document IU’s contributions to the state,
students, and communities
Demonstrate integrity in accounting for the use of
public and private resources
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Principles
 Mission-centered
 Research-driven
 Transparency
 Inclusive
dimensions of excellence and
quality
 Empowerment and responsibility
 Influenced by “best practices”
 National
Commission on Accountability in
Higher Education
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Framework
 University-wide
strategic goals and core
performance indicators
 Campus performance objectives and
indicators derived from mission, aligned to
university goals and core indicators
 Explicit link to administrative area goals
and objectives
 Annual performance reports and reviews
 University
and campuses
Victor M. H. Borden – February 9, 2006 – Ohio University
Advance University
Distinction and
Distinctiveness
Enhance Academic
Program Quality
Rankings and recognitions
Expand the Scope
Funding
Focused areas of distinction
Research and
Creative Activities
Research collaborations
Integrating Performance Measures
University
andinto
Impact
of Endeavor
Centers of Excellence
Faculty
participation/productivity
Overall campus quality
Space and equipment
Quality of faculty
Academic Impact
Program accreditation and
review
Practical Impact
Advancing Indiana
Teaching and learning
development
Improve Student
Achievement and
Success
Economic development and
impact
Cultural development and
impact
Information/technology
resources
Educational development
Physical resources
Indiana professional practice:
Preparation and service
Program demand and
delivery
Civic engagement
Preparation and support
Access and affordability
Increase
Operational
Efficiency and
Effectiveness
Finances and budgeting
Enrollment
Leadership development
Student engagement
Administrative overhead
Progress
Quality of administrative
services to
Outcomes
Victor M. H. Borden – February 9, 2006 – Ohio University
Faculty/staff/student
Integrating Performance Measures into University Endeavor
Limitations of Measures/Metrics
 Inherently
imperfect
 Overly simplistic
 Not everything that counts can be
counted, and not everything that can be
counted counts – Albert Einstein
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Accommodating the Limitations
 An
imprecise answer to the right question
is much better than a precise answer to
the wrong question (paraphrasing John Tukey)
 Triangulation
 Using
multiple, convergent measures to better
reflect the underlying
 Performance
measures as the tip of the
evidence-based iceberg
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Vertical (hierarchical) alignment
Performance Measures as the Tip of the
Evidence-Based Practice Iceberg
Performance
measures
Evidence
Based Practice
Plan
Improve
Implement
Assess
VictorHorizontal
M. H. Borden –(cross-unit)
February 9, 2006alignment
– Ohio University
Integrating Performance Measures into University Endeavor
Evidence-Based Practice
 Commonly
used in clinical domain
 Validity
derived from rigorous research
conducted by others and believed to
generalize to other settings
 For
university endeavor there are limits to
generalizability across settings
 Focus shifts to more continuous use of
process-generated data using less
rigorous methods to monitor, reflect, and
adjust
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Methods of Evidence-Based Practice
 The
many faces of evidence-based
practice
 Student
learning outcomes assessment
 Program evaluation
 Program review
 Quality improvement
 Balanced score card
 Benchmarking
 The
role of collaborative inquiry
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
The Evaluation Cycle
Back to the drawing board
On to something else
1. THINK ABOUT
ISSUES
2. ENGAGE WITH THE
PROBLEM
6. PLAN TO
IMPROVE
5. EVALUATE IMPACT
* did it work as I intended?
* how did people respond?
* what were the results?
3. DEVELOP
RESOURCES/
STRATEGIES
TO IMPROVE
4. IMPLEMENT
INTERVENTIONS
* experiment
Adapted
from Norman
Victor M. H. Borden – February 9, 2006
– Ohio University
Jackson
Integrating Performance Measures into University Endeavor
The Assessment Matrix
1.
What
general
outcome
are you
seeking?
2.
3.
4.
How
How are How could
would you you trying
you
know it if to achieve measure
you saw
the
what you
it?
outcome?
hope to
see?
5.
What
have you
found?
Victor M. H. Borden – February 9, 2006 – Ohio University
6.
What
changes
will you
make?
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
The Support Unit Matrix
1.
What
objective
are you
trying to
achieve?
3.
2.
What are
Why is it
important you doing
to achieve
to the
it?
University
?
6.
5.
4.
Based on
How do
How will
you know you know evidence
from #4
you are
when you
and #5
doing it
are
achieving efficiently? what can
you do to
it
(effectiveness)
improve?
?
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Quality Improvement Models
 Advantages
 Focus
on process provides best chances for
identifying points of improvement
 Collaborative teams empower staff and help
improve communication across units
 Formulaic method and external staff support
help guide and keep on track
 Sample
methods
 Penn
State’s Fast Track
 U of Wisconsin Accelerated Improvement
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
PSU Fast Track
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
UWisc Accelerated Improvement
http://www.wisc.edu/improve/improvement/accel.html
Define
Goals and measures of success
Document process
Understand customer needs
Check/refine goals
Design
Develop potential solutions
Analyze solutions/options
Finalize solution develop implementation plan
Implement
Inform affected people
Conduct training, if needed
Execute action plans w/timeline
Follow-up
Collect data to track improvement
Review and refine process changes
Issue final report with results
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Program Review
self-study, site visit by “peers”
 Common method for academic programs
 Program
 Increasing
use for administrative programs
 Fits
well with accreditation framework
 Guidelines shape tone and tenor
 Content
standards
 Review team composition
 Flexibility
accommodates range of inquiry
orientations
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Limits of Program Review
 Expensive
and time-consuming
 Can be done with little participation
 Or
with a lot
 Results
not always directly useful for
change
 Memorandum
of understanding helpful
 Episodic
nature not responsive to
changing environment
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Balanced Score Card (BSC)
 Kaplan
& Norton propose business model
 Financial
performance
 Customer service and satisfaction
 Process effectiveness and efficiency
 Organizational learning
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
BSC in Higher Education

Ruben (1999)

Teaching/Learning
• Programs/Courses, Student Outcomes

Service/Outreach
• University, profession, alumns, state, prospective students,
families employers

Scholarship/Research
• Productivity/Impact

Workplace satisfaction
• Faculty/staff

Financial
• Revenues/expenditures
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Benchmarking

Best practices in organizations sharing similar
internal work procedures
 HE focus often on peer or aspirational
institutions
 NACUBO study searched for measures
 APQC introduces qualitative benchmarking to
higher education


Measuring institutional performance outcomes
Electronically supported student services
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
More Complex Models
 The
Evaluation Center
Stufflebeam, Eastern Michigan University
http://www.wmich.edu/evalctr/checklists/
 CIPP Model
 Constructivist Evaluation
 Deliberative Democratic Evaluation
 Key Evaluation Checklist
 Qualitative Evaluation
 Utilization-Focused Evaluation
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Limits of Complex Models
 Too
complex and expensive to be practical
 They require an…
 “evaluation
unit as a staff operation at a high
level of the organization in order to help
insulate the unit from inappropriate internal
influences and enhance its influence on
decision making .”
Daniel J. Stufflebeam
http://www.wmich.edu/evalctr/checklists/institutionalizingeval.htm
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Collaborative Action Inquiry
Continuous cycle of data collection  data
analysis  data feedback  action plans 
data collection
 Stakeholder empowerment through active and
on-going participation
 Data feedback meetings promote collaboration,
dialogue, and collective analysis
 Active learning and discovery fostered by critical
reflection process
 Data-driven action plans developed = research
linked to action

Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Linking Research and Action
 Who
does what?
 Decides
what actions are taken?
 Is responsible for effective implementation?
 Can devise appropriate evaluation protocols?
 Has access to or can collect appropriate
evidence?
 Reviews the results and decides what to do?
 What
can be done to get these people to
work together and in concert?
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
A Learning Paradigm
 Typical
data-driven focus supposes
rational world
 Learning incorporates uncertainty,
ambiguity, and multiple styles
 Individual learning and organizational
learning are compatible concepts
 Evidence-based practice is compatible
with learning approach
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Single- and Double-Loop Learning


Argyris and Schön
Learning is the detection and correction of error
(unintended consequences)
 “Governing Variables” are those things what we
feel are important to keep within limits
 “Action Strategy” is what we do or plan to do to
keep the governing variables within limits
 “Consequences” are the intended and
unintended outputs and outcomes


Intended: confirm our theory in use
Unintended: suggests error in our theory in use
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Single-Loop Learning
 Governing
variables not called into
question
 Adjustments made to action strategies at
best
 Defense mechanisms can readily arise to
maintain single-loop learning
Governing
Variables
Action
Strategies
Consequences
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Double-Loop Learning

Questioning the role of the framing and learning
systems which underlie actual goals and
strategies
 Reflection is fundamental




Basic assumptions are confronted
Hypotheses publicly tested
Falsification is sought
Ego is laid aside
Governing
Variables
Action
Strategies
Consequences
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Model I and II Org Learning
 Single-
and double-loop learning at the
organizational level
 Model I: Organizational members
prescribe to a common theory in use
 Organizational
policies and practices inhibit
change
 Model
II: Governing values, policies, and
practices promote double-loop learning
Victor M. H. Borden – February 9, 2006 – Ohio University
John Seely Brown & Paul Duguid
The Social Life of Information
(2000) Harvard Business School Press
Organizational Learning and Communities-of-Practice:
Toward a Unified View of Working, Learning, and
Innovation. (1991) Organization Science, 2(1), 40-57.
Integrating Performance Measures into University Endeavor
Learning To Be / Know How
 Based
on collaborative practice
 Communities
 Knowledge
of practice
as inseparable from the
knower
 Evidence from a variety of sources,
including practitioner experience
 Sharing interpretations as process
 Common priorities and strategies as
output
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Learning is Good
 We
promote (lifelong) learning for students
 We seek to contribute to the creation of
knowledge within our disciplines and
professions
 What about in our practice as...
 Classroom
teachers
 Conferrers of degree credentials
 Managers and administrators
 Support staff
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
The Learning/Performance
Measure Conundrum

If our general objective is to collectively learn
how to do our work better, then we must accept
that our current thinking, practices, structures,
etc., need to change
 Our current best thinking about what measures
reflect progress toward desired changes may
change through the learning process
 We should not be rigid about our performance
measures but rather allow our evidence-based
collaborative learning efforts to guide their
evolution
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Implications for Faculty/Staff/Org Development
 There
are many viable ways to integrate
inquiry into organizational practices
 Administrative support focus may need to
shift from information provision and toward
collaborative inquiry
 Someone needs to focus on how this all
fits together
 The
institutional portfolio provides one such
mechanism
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Implications for Information Use
 Data
sources
 Types of needs
 Types of users
 Sources of information
 Tools for user needs
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Data Sources

Sources of evidence





Derived from Institution’s operational information
systems



Student, Human Resource, Finances
Space, program inventory, courseware
Surveys


Documented
Provider/practitioner experience
User/client experience
Contextual
Students, faculty, staff, prospects, community
External data sources


Federal and State (K-16) education data, national efforts (CDS,
rewards and recognitions, media)
Census, labor, workforce development, licensing boards
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Types of Information Needs

Operational




Tactical





Directly support the ongoing operation of a system
Formatted presentations of transactional
Often use data from a single operational domain
Monitor and respond quickly to a variety of short-term situations
Typically more aggregate (less granular) than operational reports
Includes both recurrent and ad hoc information needs
Often requires merging data from multiple operational domains
as well as data from non-operational sources
Strategic


Focuses on higher level policy and practice issues, often with
longer timeframes
Often requires more significant analysis of institutional, survey,
and external data sources
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
User Roles

Casual


Recurrent


more frequent use but modest technical expertise OR
insufficient time to employ technical skills
Power


occasional use that demands relatively little technical
expertise
modest to frequent use with capacity for using more
complex technical systems
Individuals may occupy different roles at
different times
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Information Needs and Users
Type of User
Type of Use
Casual
Operational
Pre-packaged
Operational
Reports
Recurrent
Power
Report Modules ODBC Access to
With Parameter Data Warehouse
Choices
Tables
Tactical
Strategic
Research Briefs
And Analyses
Web-based
Report
Generators
Victor M. H. Borden – February 9, 2006 – Ohio University
OLAP
Tools
Integrating Performance Measures into University Endeavor
Implications for IT


Analytic data warehouse is essential, but
Think more broadly about data sources



Not just enterprise system as we now know it
Data from courseware platform
Mechanisms for collecting “droppings” from other
important activities
• Faculty vitae and annual reports
• Portfolios of faculty and student work
• Civic engagement inventory

Access/reporting technology should focus on
enabling “value-added resellers” to deliver to
broad range of users
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Responsibility-Centered Budgeting
 Similar
to Churchill’s opinion of democracy
 It
has been said that democracy is the worst
form of government except all the others that
have been tried
 Concerns
about changing to RCB
 It
changes everything and yet nothing really
changes
 [I] have known a great many troubles, but
most of them never happened ~Mark Twain
Victor M. H. Borden – February 9, 2006 – Ohio University
Integrating Performance Measures into University Endeavor
Parting Thought
 It
is good to have an end to journey
towards; but it is the journey that
matters in the end - Ursula Le Guin
Victor M. H. Borden – February 9, 2006 – Ohio University