Dashboards as a Data
Management Tool
Dr. David C. Oehler
Director of Assessment, Information and Analysis
Northwest Missouri State University
Presentation Overview



Facilitating decision-making and driving
improvement
Measurement impact on focus of
institutional activities and processes
How we developed our system and
integrated it into information systems
06/21/2006
2
Building a House of Quality
MANAGEMENT BY FACT
TRUST
SHARED VALUES AND VISION
ASSESSMENT &
FEEDBACK
INVOLVEMENT
HIGH EXPECTATIONS
CUSTOMER FOCUS
CONTINUOUS
IMPROVEMENT
Facilitating Decision-making
and Driving Improvement
Importance of institutional culture
 Importance of internal communication
 Alignment with mission and vision
 Alignment with strategic planning

06/21/2006
4
Institutional Culture
CQI Philosophy
“All work is process.”
“Every process is perfectly designed to
create the outcomes it produces.”
“If you can’t define it, you can’t measure
it; if you don’t measure it, you won’t
improve it.”
06/21/2006
5
Communication –
Northwest’s Planning Model


Since 1993, all units at Northwest use a
Seven-Step Planning Process (the
Hubbard Cycle) to focus and define
their work.
This process is based on five underlying
questions:
06/21/2006
6
CQI Underlying Questions
1. Who are your customers/clients?
2. What are their expectations?
3. How do you “deliver” quality?
4. How do you know how well you are
doing?
5. What performance improvement
targets are reasonable?
06/21/2006
7
Key Quality Indicators



“Key” - identifying a few very important
things to watch.
“Quality” - a perception held by
students, stakeholders, and each other.
“Indicators” - evaluative tools that
indirectly measure aspects of quality;
multiple indicators are needed where
direct measures are not available.
06/21/2006
8
Focusing on Peoples’
Information Needs
1.
2.
3.
4.
5.
What are your key decision areas? (as
defined in your job description)
What kinds of decisions do you make?
What information do you need to make
good decisions?
What data do you need to create good
information?
What is the best way to obtain that data?
06/21/2006
9
Communication and
Systems Alignment
Annual Report Format
 Five-year Academic Department
Review (Coordinating Board for
Higher Education)
 Annual departmental visits (Deans’
executive summaries)

06/21/2006
10
NORTHWEST QUALITY SYSTEMS MODEL
Phase 1:
STRATEGIC
CONTEXT
MISSION/
VISION/
VALUES
STRATEGIC
KQIs
Phase 2:
STRATEGY
IDENTIFICATION
& REFINEMENT
STRATEGIC
INITIATIVES
SI
CHAMPIONS
ENVIRONMENTAL
SCAN
DECISION
DRIVERS
DEPLOYMENT
PLAN
Phase 3:
7-STEP TACTICAL
PLAN
DEVELOPMENT &
DEPLOYMENT
TACTICAL KQIs
LEADERSHIP
OBJECTIVES
PLANNING
ASSESSMENT
STUDENT
FOCUS
ACTION PLANS
INFORMATION
& ANALYSIS
TRENDS
FACULTY/
STAFF FOCUS
IMPROVEMENTS
PROCESS
MANAGEMENT
STRETCH GOALS
RESULTS
MEASUREMENT
PLAN
INTERNAL
ANALYSIS
COMMUNICATIONS
PLAN
Phase 4:
BALDRIGE
SELFASSESSMENT
Presentation Overview



Facilitating decision-making and driving
improvement
Measurement impact on focus of
institutional activities and processes
How we developed our system and
integrated it into information systems
06/21/2006
13
Phenomenon of Measurement
This and the following slide were
adapted from a Dee W. Hook
presentation.
What
is acted on
gets improved
What is noticed
gets acted on
What is measured
gets noticed
What is a Dashboard?
 Definition/use:
 Both a process and a tool
 Looking for unfavorable trends or patterns and
focusing energy on improving priority areas
 A (diagnostic) means for monitoring performance
to ascertain what is working well and where
additional attention is needed
 A few (4-6) sets of indicators, representing the
most central areas related to high performance
06/21/2006
15
How Does a Dashboard Focus
Activities and Processes?
 Requires clear definition of outcomes
 Focuses on a manageable (small) set of key
outcomes (results)
 Encourages cross-functional communication
 Requires fact-based decision processes
Data reporting structures
Process improvement orientation
Layering of detail (summative vs. formative)
06/21/2006
16
Time is Increasingly a
Precious Resource



Dashboards help you know what’s
important
Dashboards focus on actions that make
a difference
Collect data to create information you
can use, then use it
06/21/2006
17
Data for Decision-making


Assessment needs to answer questions
Systems to collect, analyze, and report
information need to be developed to
support the specific information
requirements
06/21/2006
18
Assessment System Design
1. Data Collection


Centralized measures
Decentralized measures
2. Data Processing



3. Reporting Systems


Summative Information
Formative Information
4. Analysis/Decision-making


06/21/2006
Disaggregation system
Aggregation system
No transformation
Cabinet, Deans, Directors,
Department Chairs
Department Chairs, Faculty
19
No Transformation
Summative
Information
Disaggregation
System
(Dashboards,
Profiles)
Cabinet, Governing
Board, External
Audiences
Aggregation
System
Deans, Directors,
Department Chairs
Formative
Information
(Operational)
No Transformation
Detail
Analysis and
Decision-making
Detail
Decentralized
Reporting
Systems
Little
Centralized
Data Processing
Much
Data
Collection
Directors,
Department Chairs
Directors,
Department Chairs,
Faculty, Staff
What Metrics are in a
Dashboard?
 The Dashboard includes a balanced view of an
organization
 Learning and Growth (student academic progress; faculty
and staff development, scholarship and research productivity)
 Customer Relations (students, faculty, staff, alumni,
parent satisfaction and involvement)
 Internal Processes (functional area performance –
accuracy, timeliness, friendliness)
 Financial Measures (fiscal health and viability)
06/21/2006
21
Sample Traditional Balanced Scorecard
7
6
5
4
3
2
1
Wgt
Learning &
Growth
8
Index 1 - Student Academic Progress
Customer
Relations
9
Index 4 - Student Satisfaction & Involvement
Internal
Processes
10
Index 7 - Internal offices process accuracy
Financial
Measures
Measure/Index title
Index 10 - Fund balance
10%
Index 11 - Endowment
10%
20%
Index 2 - Faculty & Staff Development
10%
Index 3 - Scholarship & Research Productivity
5%
12%
Index 5 - Faculty & Staff Satisfaction & Involvement
4%
Index 6 - Alumni & Parent Satisfaction & Involvement
4%
10%
Index 8 - Internal offices process timeliness
5%
Index 9 - Internal offices process friendliness
5%
Index 12 - Expenditures
5%
Northwest’s Dashboard Model
Our model includes several types of
information/report presentations
Dashboards – single screen current status
Trend charts – key data element trends over
time
Data tables –key data detail trends over time
Majors, minors, advisees, degrees, SCH, financials
Special interest charts/tables
06/21/2006
26
Features of the Northwest
Balanced Scorecard System








Dashboard “lights” to indicate current status
Hyperlinks to navigate through workbooks
Hyperlinks to “drill down” to detail
Comparative data links for setting targets
Real-time data updates
Accommodates various data sources
Modular design to facilitate upgrading
Automated updating of modules
06/21/2006
27
President’s Dashboard
Provost’s Dashboard
VP Student Affairs’ Dashboard
Additional Detail
Comparative Data
In order to judge how good your
performance is, results should be put
into some context
Trends over time
Comparisons to other internal units
Comparisons with peer groups
Comparisons outside of the education
sector
06/21/2006
32
Student Satisfaction
1.0
1.5
2.0
2.5
3.0
3.5
4.0
93-94
94-95
95-96
96-97
97-98
98-99
Instructor made objectives clear
1.5
1.5
1.5
1.5
Instructor helped me achieve them
1.8
1.7
1.7
1.7
Instructor provided regular feedback
1.8
1.8
1.8
1.8
Instructor accessible during office hrs
1.7
1.6
1.6
1.6
1.8
1.8
1.6
Alumni - overall satisfaction rating
99-00
00-01
01-02
1.5
1.5
1.4
1.4
1.5
1.7
1.6
1.6
1.6
1.6
1.8
1.7
1.6
1.6
1.6
1.5
1.4
1.4
1.4
1.4
1.7
02-03
03-04
1.4
Alumni 'N'
Academic Workload - SCH Production
150%
140%
130%
120%
110%
100%
90%
80%
70%
60%
50%
SCH P r oduct i on (SU T ot al ) %
SCH P r oduct i on (FA T ot al ) %
SCH P r oduct i on (SP T ot al ) %
SCH P r oduct i on (T ot al ) #
0
0
0
Su 94
F 94
Sp 95
100. 0%
Su 95
F 95
Sp 96
96. 0%
100. 0%
100. 0%
100. 0%
Su 96
F 96
Sp 97
97. 9%
100. 1%
101. 2%
Su 97
F 97
Sp 98
105. 2%
101. 6%
101. 7%
Su 98
F 98
Sp 99
103. 9%
100. 3%
103. 8%
Su 99
F 99
Sp 00
131. 7%
102. 3%
102. 6%
Su 00
F 00
Sp 01
142. 9%
104. 3%
106. 2%
Su 01
F 01
Sp 02
123. 3%
106. 2%
104. 3%
Su 02
F 02
Sp 03
122. 5%
104. 9%
107. 7%
Su 03
F 03
Sp 04
121. 6%
103. 3%
104. 2%
101. 5%
12, 602 73, 708 67, 404 12, 095 73, 704 68, 193 12, 339 73, 770 68, 564 13, 259 74, 908 69, 945 13, 089 73, 906 69, 143 16, 601 75, 386 71, 599 18, 011 76, 857 70, 282 15, 533 78, 312 72, 580 15, 442 77, 356 70, 213 15, 328 76, 177 68, 421
70%
60%
50%
40%
30%
sp 95 dat a
sp 96 dat a
Nor t hwest
sp 97 dat a
sp 98 dat a
sp 99 dat a
sp 00 dat a
sp 01 dat a
sp 02 dat a
sp 03 dat a
44. 0%
44. 0%
43. 0%
48. 0%
53. 0%
57. 3%
59. 0%
45. 0%
53. 0%
54. 0%
55. 2%
56. 0%
B est compet i t or
Positive Referrals - Alumni Outcomes Survey
Percent Positive (0-100)
percent (0-100)
Graduation Rate (CBHE)
100%
90%
80%
70%
2000
2001
2002
81.7%
choose national
77.8%
81.2%
choose cohort
75.7%
80.3%
79.1%
recommend national
94.3%
97.0%
97.0%
recommend cohort
94.6%
98.0%
97.5%
if beginning again, choose this school NW
85.0%
92.9%
94.1%
recommend to others NW
97.0%
99.5%
99.4%
sp 04 dat a
sp 05 dat a
Interpreting Dashboard
Indicators
To follow up on indicators of interest,
use hyperlinks to access increasing
levels of detail
Student satisfaction as an example
President’s dashboard
to
Provost’s dashboard
to
Noel-Levitz Student Satisfaction Inventory
data trends
06/21/2006
40
Provost’s Dashboard Metrics Architecture
Department ‘A’
Profile and
Dashboard
Department ‘etc.’
Profile and
Dashboard
College ‘A’
Profile and
Dashboard
College ‘etc.’
Profile and
Dashboard
Provost’s
Profile and
Dashboard
Undergrad
majors and
minors,
Graduate
majors,
Degrees,
Advisees
module
Academic
Profile
results
module
Major
Field test
results
module
General
Education
local
module
Financial
data
module
Program
Praxis and
SCH
C-BASE
generation
module
results
module
Major field
local/senior
capstone
module
Student
satisfaction
module
Alumni
satisfaction
module
Comparative
EMSAS
module
(freshman
success)
Student
opinionnaires
of teaching
module
Placement
data
module
President’s
Dashboard
Service unit ‘A’
Profile and Dashboard
Service unit ‘B’
Profile and Dashboard
Service unit ‘C’
Profile and Dashboard
Service unit ‘etc.’
Profile and Dashboard
data for
targets
July 28, 2002
Satisfaction
Students
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Fr.
Item
Noel-Levitz Academic Advising
Noel-Levitz Safety and Security
N-L Instructional Effectiveness
Noel-Levitz Student Centeredness
Noel-Levitz Campus Climate
Noel-Levitz Concern for Individual
N-L Registration Effectiveness
N-L Campus Support Services
Noel-Levitz Recruitment/Fin. Aid
Noel-Levitz Service Excellence
Noel-Levitz Campus Life
1.160
1.074
0.756
Perf
0.350
1.360
0.640
0.490
0.530
0.580
0.700
0.420
0.690
0.590
0.600
highest
overall
lowest
Wgt
1.8%
1.4%
5.0%
2.2%
6.1%
2.2%
1.8%
2.5%
2.2%
2.9%
5.4%
Ratio
Calculation
1.160
0.756
1.044
1.104
1.088
1.068
1.020
1.132
1.024
1.064
1.060
Current
Status
1.160
1.074
0.756
Link to
run chart
1.160
0.756
1.044
1.104
1.088
1.068
1.020
1.132
1.024
1.064
1.060
annual
show me!
show me!
show me!
show me!
show me!
show me!
show me!
show me!
show me!
show me!
show me!
Presentation Overview



Facilitating decision-making and driving
improvement
Measurement impact on focus of
institutional activities and processes
How we developed our system and
integrated it into information systems
06/21/2006
46
President’s Dashboard
College Profiles (3)
Department Profiles (11)
Provost’s Dashboard
Unit Profiles (8)
Department Profiles (5)
VP Student Affairs Dashboard
Unit Profiles (8)
VP Information Systems Dashboard
Unit Profiles (6)
VP Finance and Support Services Dashboard
Unit Profiles (6)
Department Profiles (5)
Primary Source Modules
VP Communication and Marketing Dashboard
Unit Profiles (3)
VP Institutional Advancement Dashboard
Unit Profiles (3)
Director of Athletics Dashboard
Unit Profiles (4)
Exec. Assistant to the President Dashboard
Alignment of Dashboards and Profiles
NORTHWEST QUALITY SYSTEMS MODEL
Phase 1:
STRATEGIC
CONTEXT
MISSION/
VALUES
STRATEGIC
KQIs
DECISION
DRIVERS
ENVIRONMENTAL
SCAN
VISION
Phase 2:
STRATEGY
IDENTIFICATION
& REFINEMENT
STRATEGIC
INITIATIVES
CHAMPIONS
DEPLOYMENT
PLAN
Phase 3:
7-STEP PLANNING
PROCESS
DEVELOPMENT &
DEPLOYMENT
TACTICAL KQIs
LEADERSHIP
GOALS AND
OBJECTIVES
PLANNING
ASSESSMENT
STUDENT
FOCUS
ACTION PLAN
INFORMATION
& ANALYSIS
TRENDS
FACULTY/
STAFF FOCUS
IMPROVEMENTS
PROCESS
MANAGEMENT
STRETCH GOALS
RESULTS
HR
PLANS
INTERNAL
ANALYSIS
MEASUREMENT PLANS
COMMUNICATIONS
PLAN
Phase 4:
BALDRIGE
SELFASSESSMENT
Northwest Dashboard Information
and Sharing Days
Contact information:
Dave Oehler
Northwest Missouri State University
800 University Drive
660.562.1527 (voice)
oehler@nwmissouri.edu
06/21/2006
51