USING A QUALITY IMPROVEMENT PRIORITY MATRIX
IN A UNIVERSITY DEPARTMENT
By Stuart Umpleby and Oleksandr Melnychenko
with assistance from Nino Okribelashvili, Nadya Krylova, and Mohamad Lazkani
Research Program in Social and Organizational Learning
The George Washington University
Washington, DC 20052 USA
Email: umpleby@gwu.edu
URL: http://www.gwu.edu/~rpsol
September 17, 2001
ABSTRACT
A Quality Improvement Priority Matrix is a useful method for achieving data-driven decision-making.
Regular information from employees and customers about the features of the organization that most need
improvement allows managers to focus attention and resources where they can contribute most to improving
employee and customer satisfaction. In May 2001 the members of the Department of Management Science at
The George Washington University used a Quality Improvement Priority Matrix to identify those features of
the Department that they felt were high on importance but low on performance. This quadrant of the matrix
contained 17 of the 51 features listed. Several questions were also asked concerning whether the members of
the Department found the exercise to be useful and whether they thought it would be helpful to other
departments in the University. A large majority thought the results were useful and that similar exercises in
other departments would be helpful to them as well.
INTRODUCTION
This study was undertaken to achieve several purposes:
1. To introduce the method of a quality improvement priority matrix to the members of the GW
Department of Management Science,
2. To generate data that would be helpful in focusing department improvement efforts during the coming
academic year,
3. To test the utility and acceptance of a quality improvement priority matrix in an academic department
before proceeding to encourage its adoption by other departments within The George Washington
University, and
4. To test its utility and acceptance within an American university before encouraging other universities
in other countries to use it in their planning and decision-making activities.
Hence, this exercise not only generated data for use by the members of the Department of Management
Science. It also asked for their assessment of the method and its utility. In addition the paper makes some
suggestions on how the method might be improved in the future.
A QUALITY IMPROVEMENT PRIORITY MATRIX
A Quality Improvement Priority Matrix was described by the people from GTE Directories in their
presentation in February 1995 describing how they won the Baldrige Award (Carlson, 1995). A similar
matrix, called a “strategic improvement matrix,” was used by the people from Armstrong Building Products
Operations in their presentation to the February 1996 Baldrige Award conference (Wellendorf, 1996). Stuart
Umpleby’s students have used similar matrices in several group projects (). A matrix was used by visiting
scholars at GW in December 2000 to identify how the US Department of State’s Junior Faculty Development
Program might be improved (Naoumova and Umpleby, 2001).
A Quality Improvement Priority Matrix asks customers or employees to rate several features of an
organization on two scales – importance and performance. That is, how important to them is that particular
feature, and how effectively is the organization currently performing on that feature. For this exercise we
asked the faculty in the Department of Management Science at GW to evaluate various features of the
Department and the School of Business and Public Management. Although the Department is functioning
2
very well, the quality improvement literature claims that improvement is always possible. If so, where is
improvement most needed? With this method one looks at the quadrant that corresponds to high importance
and low performance. What features of the organization fall into this quadrant? Those are the features where
improvement will lead to the greatest increase in customer and/or employee satisfaction.
The quality improvement priority matrix is a simple but powerful management tool. It makes possible
data-driven decision-making. Often in organizations resources are allocated to favored groups. For example,
a dean who is a chemist might choose to allocate more funds to the chemistry department. Or additional
resources are allocated to the group that complains the loudest. This matrix presents an alternative. By
gathering data from customers and/or all employees a manager can find out where improvement is most
needed and where additional attention will produce the greatest returns in customer and employee satisfaction.
The matrix can also be used to explain why funds are allocated as they are. The matrix thereby tends to
silence critics of the way funds are allocated.
RESULTS FOR THE DEPARTMENT OF MANAGEMENT SCIENCE
Table 1 is the questionnaire that was distributed to members of the Department of Management
Science at their annual retreat at The Woods, a conference center in Hedgesville, WV. Eighteen
questionnaires were returned eventually, but one came late. So only seventeen returns were used for the data
in this report. Table 2 lists the mean responses for importance and performance. In some cases people did not
rate all features. In these cases the mean is based on the number who replied, not seventeen. Figure 1 is the
Quality Improvement Priority Matrix that presents graphically the data in Table 2. On the second day of the
retreat the Quality Improvement Priority Matrix was presented to the members of the Department and the
results were discussed.
If one looks at the Quality Improvement Priority Matrix (QIPM) for the Department of Management
Science, one sees that 16 of 51 features lie in the lower right quadrant. This quadrant contains the features
that were rated high in importance but low in performance. Table 3 lists the features in this quadrant that are
most important in descending order, that is with the most important at the top and less important items below.
The five items in the lower right quadrant with the highest importance are salaries, classroom facilities, office
space for faculty, building/ physical environment, and parking for faculty and staff. Table 4 lists the features
in the lower right quadrant that have the lowest performance ratings. The five items with the lowest
performance ratings are parking for students, SBPM working papers series, help with writing research
proposals, conference room and other space, and use of continuous improvement methods in the Department.
Following the discussion, a second questionnaire was distributed (see Table 5). This second
questionnaire asked the members of the Department to evaluate the usefulness of the Matrix. The distribution
of responses to the five questions in the second questionnaire are presented in Figures 2, 3, 4, 5, and 6.
Eighteen of the second questionnaires were returned.
A METHODOLOGICAL ISSUE
One member of the Department of Management Science noted that it is incorrect to average ordinal
data. This is a valid methodological point. Our reply is that we find the matrix useful. An issue that we
believe is at least as important is the fact that human beings have difficulty interpreting large amounts of
quantitative information. We believe the Matrix helps people to find informative patterns in a large amount of
3
data. Somewhat like a road map, the Matrix helps people find the points of interest and then plan more
detailed exploration.
CONCLUSIONS AND SUGGESTIONS
We believe that this brief report is useful for the Department of Management Science not only because
of the results but also because it illustrates the utility of a simple management method in helping a group of
people to focus attention where improvement efforts can be most productive. We believe that if such
management methods were routinely used by departments at GW, faculty, staff, and student satisfaction and
productivity would rise, and the university would continue to improve at a faster rate.
Using this method has led us to suggest some possible improvements for the future.
1. To increase feedback, some blank lines could be left at the bottom of the original questionnaire.
Respondents could be asked to suggest additional features to be included in future studies using the
matrix. In addition respondents could be asked for their comments – either questions about the
meaning of some features or other comments on the method.
2. For those items that are of particular interest, a follow-up questionnaire could ask for additional
information on what aspects of that feature are of concern.
REFERENCES
Carlson, Marilyn. “GTE Directories: Customer Focus and Satisfaction,” The Quest for Excellence VII, The
Official Conference of the Malcolm Baldrige National Quality
Award, February 6-8, 1995, Washington, DC.
Wellendorf, James A. “Armstrong Building Products Operations: Information and Analysis,” The Quest for
Excellence VIII, The Official Conference of the Malcolm Baldrige National Quality Award, February 5-7,
1995, Washington, DC.
Naoumova, Irina and Stuart Umpleby, “Two Methods Useful for Starting A Quality Improvement Program,”
Research Program in Social and Organizational Learning, The George Washington University, Washington,
DC, April 15, 2001, 14 pages.
4
Table 1
QUESTIONNAIRE FOR A QUALITY IMPROVEMENT PRIORITY MATRIX
Below is a list of features of the Department of Management Science or SBPM. We would like to create a
Quality Improvement Priority Matrix using these features. So, please rate each feature on a scale from 0 to
10. That is, on the importance scale, 0 would mean the feature is not important at all. 5 would mean the
feature is moderately important. 10 would mean the feature is very, very important. On the performance
scale, 0 would mean that current performance is very, very poor. 5 would mean that current performance is
neither bad nor good. 10 would mean that current performance is excellent.
FEATURE
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Importance
Computer hardware
Computer software
Office space for faculty
Conference room and other space
Computer labs
Copiers
Fax machines
Office security
Secretarial support
Teaching assistants
Annual retreat
Social activities
Recreational activities
Building/ physical environment
Accounts payable
Classroom scheduling
Classroom facilities
Projection equipment
Course catalogue
Faculty websites
Dept. websites
SBPM websites
Campus grounds
Parking for faculty and staff
Parking for students
Library journal collection
Library book collection
Interlibrary loan
Coordination with other depts.
A supportive climate in the dept.
Dept. head protects faculty from admin.
interference
Transparency of APT process
Travel support
Funds to support research
SBPM working papers series
5
PERFORMANCE
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Help with writing research proposals
English skills of students
General ability of students
Course evaluations
Faculty annual reports
Salaries
Health care benefits
Retirement benefits
Opportunities for academic work with Dept.
faculty
Opportunities for academic work with other GW
faculty
Assistance with learning IT, e.g., Prometheus
Dept. strategic plan
Dept. organization to implement its strategic plan
Use of continuous improvement methods in the
Dept.
Consulting opportunities in DC area
Opportunities to meet local businessmen and govt
managers
6
Table 2
DATA FOR A QUALITY IMPROVEMENT PRIORITY MATRIX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
FEATURE
Computer hardware
Computer software
Office space for faculty
Conference room and other space
Computer labs
Copiers
Fax machines
Office security
Secretarial support
Teaching assistants
Annual retreat
Social activities
Recreational activities
Building/ physical environment
Accounts payable
Classroom scheduling
Classroom facilities
Projection equipment
Course catalogue
Faculty websites
Dept. websites
SBPM websites
Campus grounds
Parking for faculty and staff
Parking for students
Library journal collection
Library book collection
Interlibrary loan
Coordination with other depts.
A supportive climate in the dept.
Dept. head protects faculty from admin.
interference
Transparency of APT process
Travel support
Funds to support research
SBPM working papers series
Help with writing research proposals
English skills of students
General ability of students
Course evaluations
Faculty annual reports
Salaries
Importance PERFORMANCE
9.4375
6.4375
9.466667
6
9
4.055556
7.611111
3.176471
8.9375
5.0625
8.235294
5.764706
7.4375
6
8.875
5.3125
7.5
4.1875
8.5
4.75
7
6.941176
5.1875
5.266667
4.375
4.333333
8.6875
3.75
8
3.583333
8.2
5.466667
9
4.0625
8.647059
5.882353
7.125
6.375
6.375
5.125
7
5.5
6.9375
5.5
7.2
6
8.125
4.625
7
2.923077
8.588235
5.235294
8.5625
5.5
8.4375
6.357143
7.266667
4.428571
8.882353
7.764706
8.25
8.625
8.928571
8.533333
8.625
6.1875
6.764706
8.529412
8.764706
7
6.375
9.444444
7
6.785714
8.333333
5.071429
3
3.071429
5.058824
6.882353
5
5.8125
4.277778
42
43
44
45
46
47
48
49
50
51
Health care benefits
Retirement benefits
Opportunities for academic work with Dept.
faculty
Opportunities for academic work with other GW
faculty
Assistance with learning IT, e.g., Prometheus
Dept. strategic plan
Dept. organization to implement its strategic plan
Use of continuous improvement methods in the
Dept.
Consulting opportunities in DC area
Opportunities to meet local businessmen and govt
managers
8
9.5
9.5
8.533333
5.266667
6.4375
5.6
8.357143
5.214286
7.75
7.3125
7.058824
5.75
7.266667
4.4375
3.625
3.4375
6.6875
6.875
4.5625
4.5625
9
Table 3
SORTED BY IMPORTANCE
Feature
41 Salaries
3 Office space for faculty
17 Classroom facilities
14 Building/physical environment
10 Teaching assistants
24 Parking for faculty and staff
15 Accounts payable
4 Conference room and other space
9 Secretarial support
47 Dept. strategic plan
29 Coordination with other depts.
48 Dept. organization to implement its strategic plan
25 Parking for students
51 Opportunities to meet local businessmen and govt.
managers
36 Help with writing research proposals
50 Consulting opportunities in DC area
35 SBPM working papers series
49 Use of continuous improvement methods in the Dept.
10
Importance
9.44
9.00
9.00
8.69
8.50
8.13
8.00
7.61
7.50
7.31
7.27
7.06
7.00
6.88
6.76
6.69
6.19
5.75
Performance
4.28
4.06
4.06
3.75
4.75
4.63
3.58
3.18
4.19
4.44
4.43
3.63
2.92
4.56
3.07
4.56
3.00
3.44
Table 4
SORTED BY PERFORMANCE
Feature
25 Parking for students
35 SBPM working papers series
36 Help with writing research proposals
4 Conference room and other space
49 Use of continuous improvement methods in the Dept.
15 Accounts payable
48 Dept. organization to implement its strategic plan
14 Building/physical environment
3 Office space for faculty
17 Classroom facilities
9 Secretarial support
41 Salaries
29 Coordination with other depts.
47 Dept. strategic plan
51 Opportunities to meet local businessmen and govt.
managers
50 Consulting opportunities in DC area
24 Parking for faculty and staff
10 Teaching assistants
11
Importance Performance
7.00
2.92
6.19
3.00
6.76
3.07
7.61
3.18
5.75
3.44
8.00
3.58
7.06
3.63
8.69
3.75
9.00
4.06
9.00
4.06
7.50
4.19
9.44
4.28
7.27
4.43
7.31
4.44
6.88
4.56
6.69
8.13
8.50
4.56
4.63
4.75
Table 5
EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX
Please circle one response to each statement.
1. The results of the Quality Improvement Priority Matrix for the Department of Management Science were
interesting.
strongly
agree
agree
slightly
agree
neutral
slightly
disagree
disagree
strongly
disagree
2. The results will be useful in planning activities in the coming academic year.
strongly
agree
agree
slightly
agree
neutral
slightly
disagree
disagree
strongly
disagree
slightly
disagree
disagree
strongly
disagree
disagree
strongly
disagree
3. We should do this exercise again next year.
strongly
agree
agree
slightly
agree
neutral
4. Other departments could benefit by using such matrices.
strongly
agree
agree
slightly
agree
neutral
slightly
disagree
5. Regular use of such matrices through out the university would help the university move forward more
rapidly.
strongly
agree
agree
slightly
agree
neutral
slightly
disagree
12
disagree
strongly
disagree
Table 6
RESULTS OF THE EVALUATION OF THE QUALITY IMPROVEMENT PRIORITY MATRIX
1. The results of the Quality Improvement Priority Matrix for the Dept. of Management Science were
interesting.
strongly agree
agree
slightly agree
neutral
slightly disagree
disagree
strongly disagree
4
8
5
0
0
1
0
2. The results will be useful in planning activities in the coming academic year.
strongly agree
agree
slightly agree
neutral
slightly disagree
disagree
strongly disagree
2
8
3
2
1
2
0
3. We should do this exercise again next year.
strongly agree
agree
slightly agree
neutral
slightly disagree
disagree
strongly disagree
3
7
3
3
1
1
0
4. Other departments could benefit by using such matrices.
strongly agree
agree
slightly agree
neutral
slightly disagree
disagree
strongly disagree
2
8
1
3
1
1
0
13
5. Regular use of such matrices throughout the university would help the university move forward more
rapidly.
strongly agree
agree
slightly agree
neutral
slightly disagree
disagree
strongly disagree
2
7
2
3
1
2
0
14
Figure 2
THE RESULTS OF THE QUALITY IMPROVEMENT PRIORITY MATRIX FOR THE DEPARTMENT
OF MANAGEMENT SCIENCE WERE INTERESTING
Figure 3
THE RESULTS WILL BE USEFUL IN PLANNING ACTIVITIES IN THE COMING ACADEMIC YEAR
15
Figure 4
WE SHOULD DO THIS EXERCISE AGAIN NEXT YEAR
Figure 5
OTHER DEPARTMENTS COULD BENEFITY BY USING SUCH MATRICES
16
Figure 6
REGULAR USE OF SUCH MATRICES THROUGHOUT THE UNIVERSITY WOULD HELP THE
UNIVERSITY MOVE FORWARD MORE RAPIDLY
17