Unit Evaluation summary report semester 2 2009
Table 1 below shows the average value of the median for the overall satisfaction question for each faculty for each of the six Australian and two international campuses. It is
pleasing to note in general, the 2954 units offered on a campus are regarded as meeting aspirations. Exceptions are Information Technology units at Gippsland, and Medicine
Nursing and Health Sciences units at Peninsula and South Africa which are rated on average as needing improvement.
Table 1: Average UE median & number of units evaluated by faculty and campus – “overall satisfaction” question semester 2 2009
Average median
Number of units
Average median
Arts
Number of units
Average median
BusEco
Number of units
Average median
Edu
Number of units
Average median
Eng
Number of units
Average median
InfoTech
Number of units
Average median
Law
Number of units
Average median
MNHS
Number of units
Average median
Pharm
Number of units
Average median
Sci
Number of units
Average location median
Total number of units
ArtDes
Berwick
Caulfield
Clayton
Gippsland
Parkville
Peninsula
South Africa Sunway
Average
4.00
3.90
4.08
3.81
3.90
1
113
1
11
126
3.64
4.01
4.16
3.82
3.67
3.97
3.94
4.06
33
176
637
131
1
53
40
1,071
3.96
4.01
4.01
3.67
4.00
4.05
4.05
3.97
214
86
70
25
34
49
520
42
3.94
3.73
3.81
3.87
38
56
213
119
3.75
3.78
3.77
3.76
15
63
217
139
3.83
3.94
3.73
3.40
3.72
3.60
3.75
15
85
49
31
14
25
219
4.11
4.31
4.11
37
1
38
3.97
3.99
3.80
3.58
3.30
3.80
3.85
70
127
35
77
5
11
325
3.82
3.69
3.81
4
57
53
4.06
3.93
3.68
3.97
39
25
168
104
3.82
3.98
4.04
3.75
3.82
3.73
3.93
3.84
3.95
93
658
1299
370
53
159
106
218
2954
1
Table 2 below shows the average value of the median for the overall satisfaction question for each faculty offering units at locations other than a campus. In general, the 174
units of this type are seen to meet aspirations. Exceptions are outstanding results for three Art and Design units at Prato, but poor results for Information Technology units in
Hong Kong and Singapore Arts units (which was also the case for Arts units at Singapore in Semester 1). Arts units offered at the City location, the two Business and
Economics units at “other Australian locations”, and Medicine Nursing and Health Sciences units in Malaysia and Singapore, are rated on average as needing improvement.
Note that Hong Kong and Singapore units are all offered off-campus.
Table 2: Average UE medians & no. of units evaluated for other locations by faculty – “overall satisfaction” question semester 2 2009
City
ArtDes
Arts
BusEco
Edu
Eng
InfoTech
Law
MNHS
Leongatha
Mildura
Alfred
Other Aus
Prato
Hong Kong
Malaysia
Singapore
US
Other OS
Average
Average median
4.90
4.90
Number of units
3
3
Average median
3.56
4.67
3.72
3.00
3.88
3.53
Number of units
16
3
8
13
4
44
Average median
3.53
3.88
3.76
Number of units
2
4
6
Average median
3.97
4.13
4.09
Number of units
3
8
11
Average median
4.44
4.44
Number of units
4
4
Average median
2.50
3.89
3.54
Number of units
3
9
12
Average median
4.32
4.55
4.35
Number of units
40
6
46
Average median
3.94
4.00
3.94
3.75
3.25
3.33
3.77
Number of units
2
1
30
2
1
12
48
4.10
3.94
4.00
3.94
4.21
4.22
3.37
3.25
3.53
4.44
3.88
3.90
56
2
1
30
5
17
8
1
46
4
4
174
Average location median
Total number of units
2
Table 3 below shows the average value of the median for the “overall satisfaction” question for units offered in on-campus, off-campus and composite mode by each faculty.
The table omits the Law results as these are on-campus and their results are shown in Table 1 above, and the Hong Kong and Singapore units as these are all offered in offcampus mode and their results are shown in Table 2 above. In general, the units rated as needing improvement were composite and off-campus units. Exceptions are oncampus Engineering and Information Technology units at Gippsland, and on-campus MNHS units at South Africa, also rated on average as needing improvement. Apart from
these, on-campus units are generally rated as meeting aspirations.
Table 3: Average UE median & (number of units evaluated) by faculty and mode – “overall satisfaction” question semester 2 2009
Faculty
Mode
Off
ArtDes
On
Composite
Arts
Off
On
Off
BusEco
On
Composite
Edu
Off
On
Off
Eng
On
Off
InfoTech
On
Composite
MNHS
Off
On
Off
Pharm
On
Off
Sci
On
Average median
Total no. of units
Berwick
Caulfield
Clayton
Gippsland
Parkville
Peninsula
South Africa Sunway
Average
3.63 (2)
3.15 (4)
3.31 (6)
4.00 (1)
3.91 (111)
4.08 (1)
4.19 (7)
3.93 (120)
2.75 (2)
4.04 (6)
3.72 (8)
4.24 (11)
4.39 (25)
3.78 (77)
4.60 (4)
3.99 (117)
3.64 (33)
4.01 (163)
4.15 (606)
3.86 (54)
3.67 (1)
3.92 (49)
3.94 (40)
4.07 (946)
3.55 (18)
3.86 (2)
3.47 (44)
3.51 (64)
3.96 (42)
4.05 (196)
4.01 (84)
4.00 (26)
4.00 (25)
4.05 (34)
4.05 (49)
4.03 (456)
3.05 (2)
3.05 (2)
3.82 (42)
3.83 (18)
3.18 (4)
3.78 (64)
4.03 (75)
3.64 (20)
3.85 (52)
3.91 (147)
3.65 (11)
4.17 (7)
3.86 (18)
3.76 (128)
3.43 (8)
3.77 (63)
3.75 (199)
4.00 (7)
3.33 (22)
3.49 (29)
3.83 (15)
3.93 (78)
3.73 (49)
3.58 (9)
3.72 (14)
3.60 (25)
3.80 (190)
4.00 (1)
3.19 (3)
3.89 (6)
3.69 (10)
3.93 (36)
3.94 (49)
3.73 (13)
3.20 (14)
3.82 (112)
4.01 (34)
4.02 (77)
3.95 (19)
3.64 (57)
3.30 (5)
3.80 (11)
3.88 (203)
3.91 (19)
3.91 (19)
3.78 (34)
3.69 (4)
3.77 (38)
3.88 (18)
3.88 (18)
4.06 (104)
3.97 (21)
3.68 (25)
3.98 (150)
3.82
3.98
4.04
3.75
3.82
3.73
3.93
3.84
3.95
91
658
1299
370
53
159
106
218
2954
3
Table 4 below shows the average value of the median for the “overall satisfaction” question for each faculty by unit level (L designates a unit level at later year in an
undergraduate program and M a combined unit). Faculties should note that there is still a handful of units that do not appear to comply with the Unit Coding Policy requiring
the first numeric character to indicate course level. Units at all levels are evaluated as meeting aspirations in first semester 2009, with the exception of Education Level 2
units and Pharmacy Level 3 units which are rated as needing improvement. The Faculty of Business and Economics has one stand-out Level 6 unit rated as outstanding.
Table 4: Average UE median & (no. of units evaluated) by faculty and unit level – “overall satisfaction” question semester 2 2009
1
Average median
Number of units
Average median
Arts
Number of units
Average median
BusEco
Number of units
Average median
Edu
Number of units
Average median
Eng
Number of units
Average median
InfoTech
Number of units
Average median
Law
Number of units
Average median
MNHS
Number of units
Average median
Pharm
Number of units
Average median
Sci
Number of units
Total Average median
Total Number of units
ArtDes
2
3.90
38
4.04
156
3.90
72
3.81
20
3.76
23
3.67
42
3.72
2
3.90
47
3.71
12
3.82
36
3.90
448
3
4.04
44
4.04
255
3.88
133
3.79
18
3.66
47
3.62
54
4.06
2
3.66
55
3.63
10
3.98
54
3.90
672
4
3.73
25
3.99
267
3.98
142
3.59
9
3.73
51
3.61
62
4.20
5
3.87
58
3.96
12
4.01
74
3.91
705
5
3.97
16
4.18
152
4.38
9
3.87
103
3.86
75
3.64
17
4.20
35
3.89
122
3.71
4
4.48
2
3.99
535
6
3.69
4
3.96
65
4.05
86
3.94
66
3.81
25
4.06
41
4.32
40
3.83
86
3.39
4
3.98
417
9
4.88
1
L
3.93
70
4.07
6
M
4.29
2
4.03
220
4.19
13
4.15
2
4.19
15
3.00
1
4.05
15
4.88
1
3.98
107
4.14
4
4.14
6
4.48
2
4.04
237
Average
3.92
129
4.04
1,115
3.96
526
3.88
224
3.77
221
3.74
231
4.24
84
3.84
373
3.81
57
3.97
168
3.94
3,128
Table 5 below provides a time series comparison of average medians for the 2146 units with two previous evaluations in addition to the current evaluation for Semester 2
2009 (these data have been extracted back to 2006; as there was a total of 3128 units evaluated this semester, this means that there are 982 units for which there are no
previous evaluation data in the last four years – however note that this figure includes 210 previous “exceptions” units and an unknown number of new units). UW5-1
represents data for Semester 2 2009; UW5-2 depicts medians for the previous evaluation; and UW5-3 values are for the evaluation prior to UW5-2. In general the perception
of units does not show much change over the course of time with the great majority of units showing stability in being regarded as meeting aspirations. The faculties may
need to keep a watching brief on units that have improved out of the red or orange zone to ensure they don’t slide back, and Info Tech and MNHS units with decreased
satisfaction warrant investigation.
4
Table 5: Time series data for average UE median by faculty and location – “overall satisfaction” question semester 2 2009
Berwick
ArtDes
Arts
BusEco
Edu
Eng
InfoTech
Law
MNHS
Pharm
Sci
UW5-1
UW5-2
UW5-3
UW5-1
UW5-2
UW5-3
UW5-1
UW5-2
UW5-3
4.00
4.10
4.19
3.62
3.98
3.93
3.94
3.89
3.93
Caulfield
3.88
3.92
3.91
3.98
3.92
3.86
3.99
3.97
3.93
City
3.56
3.81
3.95
UW5-1
UW5-2
UW5-3
UW5-1
UW5-2
UW5-3
UW5-1
UW5-2
UW5-3
3.81
3.60
3.98
3.83
3.99
4.00
4.22
4.27
4.09
UW5-1
UW5-2
UW5-3
3.87
3.90
3.89
UW5-1
UW5-2
UW5-3
Clayton
4.08
4.64
4.13
4.15
4.11
4.01
3.96
3.96
3.90
3.93
3.92
3.82
3.75
3.72
3.60
3.81
3.63
3.61
4.10
4.13
4.15
4.02
3.97
3.94
Gippsland
MMS
3.87
4.25
4.11
3.91
3.98
3.97
3.62
3.79
3.62
3.84
4.02
3.78
3.67
3.38
3.71
3.42
3.72
3.37
Hong
Kong
4.17
3.00
4.00
Singapore
3.00
3.67
3.71
3.83
3.42
3.67
4.22
4.25
3.72
Other
Aus
4.50
4.00
4.00
Other
OS
3.75
4.25
4.00
Average of UW5-3
Prato
3.67
3.83
3.25
4.03
3.85
3.75
3.90
3.85
3.88
South
Africa
Sunway
4.05
4.00
3.81
3.99
3.91
3.96
3.70
3.75
3.93
4.01
3.86
3.91
3.96
3.83
4.02
3.80
3.95
3.83
3.67
3.52
3.52
3.30
3.78
4.03
3.96
3.93
3.74
4.19
4.05
3.85
3.66
3.86
3.80
3.95
3.93
3.91
4.19
4.05
3.85
3.75
3.65
3.45
4.05
3.95
3.93
3.50
3.80
3.58
3.11
4.02
3.74
3.62
3.75
3.71
3.97
3.90
3.86
UW5-1
UW5-2
UW5-3
Average of UW5-2
Peninsula
4.00
3.67
3.87
UW5-1
UW5-2
UW5-3
Average of UW5-1
Parkville
3.83
3.87
3.94
3.83
3.87
3.94
3.94
3.94
3.92
4.00
3.92
3.95
3.95
4.08
4.04
3.97
4.03
3.72
4.03
3.99
3.92
3.79
3.89
3.76
4.05
3.95
3.93
3.84
3.40
3.79
3.38
3.77
3.75
4.50
4.00
4.00
3.75
4.25
4.00
3.97
3.90
3.86
3.81
3.80
3.77
Average
3.88
3.96
3.94
4.03
4.04
3.97
3.93
3.91
3.87
3.92
3.91
3.83
3.76
3.78
3.67
3.76
3.74
3.73
4.15
4.18
4.11
3.85
3.88
3.84
3.97
3.90
3.86
3.94
3.93
3.87
3.81
3.85
3.83
5
Table 6, 7 and 8 below show the distribution of medians for the “overall satisfaction” question for each faculty.
Because units that would have previously been omitted as “exceptions” have now been flagged and their results
included in the pivot tables, the distributions are presented separately for:
•
•
•
all units, including those which would have been omitted as “exceptions” in Semester 1, ie with fewer than
five responses, or more than 100% response rate (Table 6);
the subset of units which have not been flagged as exceptions (71% of all units) - it is only this latter group
that can be strictly compared with Semester 1 results (Table 7);
the units that would have been previously omitted as “exceptions” (Table 8).
There are substantial differences in the median distributions for these different types of unit. In Table 6 which is
based on all units with non-zero responses, even when there may have been only one student in the unit responding,
the overall percentage of units classed as outstanding is considerably higher than the percentage of units falling into
this category in Semester 1 2009, but so is the percentage classed as needing critical attention. The percentage of
units rated as needing improvement is roughly comparable; as a result of these movements, the proportion rated as
meeting aspirations is lower.
However when the units flagged as “exceptions” and “non-exceptions” are separated out, there are striking
differences; the “non-exceptions” behave roughly similarly to the median distributional patterns observed in semester
1 (Table 7), but the “exceptions” units are far more likely to be given ratings at either extreme (very dissatisfied or
very satisfied) with less than half of them seen as meeting aspirations (Table 8). It is proposed that the Learning and
Teaching Committee consider whether any of the “exceptions” units with medians at or above 4.70 should receive
commendations (the letters from the DVCE). Some of these units have small enrolments and responses but healthy
response rates, with students giving extremely positive ratings for the unit. Given that there is a close to zero
correlation between overall satisfaction and number of responses (-0.04 this semester), it may unduly penalise staff
with small units to exclude them from the unit commendations.
Table 6: Percentage of all units falling into each “traffic light” category semester 2 2009
≤3.00
Number of units
ArtDes
Percentage of units
Number of units
Arts
Percentage of units
Number of units
BusEco
Percentage of units
Number of units
Edu
Percentage of units
Number of units
Eng
Percentage of units
Number of units
InfoTech
Percentage of units
Number of units
Law
Percentage of units
Number of units
MNHS
Percentage of units
Number of units
Pharm
Percentage of units
Number of units
Sci
Percentage of units
Number of units
Percentage of units
Semester 1 comparison
16
12.4%
116
10.4%
25
4.8%
36
16.1%
29
13.1%
40
17.3%
0
0.0%
45
12.1%
4
7.0%
9
5.4%
320
10.2%
5.7%
3.01-3.59
18
14.0%
91
8.2%
63
12.0%
17
7.6%
38
17.2%
34
14.7%
5
6.0%
46
12.3%
12
21.1%
23
13.7%
347
11.1%
12.6%
3.60-4.69
79
61.2%
695
62.3%
399
75.9%
143
63.8%
138
62.4%
140
60.6%
66
78.6%
255
68.4%
39
68.4%
124
73.8%
2078
66.4%
75.9%
≥4.70
16
12.4%
213
19.1%
39
7.4%
28
12.5%
16
7.2%
17
7.4%
13
15.5%
27
7.2%
2
3.5%
12
7.1%
383
12.2%
5.7%
Total
129
100.0%
1115
100.0%
526
100.0%
224
100.0%
221
100.0%
231
100.0%
84
100.0%
373
100.0%
57
100.0%
168
100.0%
3128
100.0%
100.00%
6
Table 7: Percentage falling into each category excluding previous “exceptions” semester 2 2009
≤3.00
Number of units
ArtDes
Percentage of units
Number of units
Arts
Percentage of units
Number of units
BusEco
Percentage of units
Number of units
Edu
Percentage of units
Number of units
Eng
Percentage of units
Number of units
InfoTech
Percentage of units
Number of units
Law
Percentage of units
Number of units
MNHS
Percentage of units
Number of units
Pharm
Percentage of units
Number of units
Sci
Percentage of units
Number of units
Percentage of units
Semester 1 comparison
10
9.2%
44
7.0%
15
3.2%
30
17.0%
23
11.7%
18
12.8%
0
0.0%
18
7.4%
4
8.3%
4
2.9%
166
7.5%
5.7%
3.01-3.59
14
12.8%
46
7.4%
54
11.4%
13
7.4%
38
19.3%
20
14.2%
5
6.5%
30
12.3%
10
20.8%
19
13.9%
249
11.2%
12.6%
3.60-4.69
74
67.9%
470
75.2%
372
78.6%
115
65.3%
123
62.4%
99
70.2%
64
83.1%
190
77.9%
33
68.8%
107
78.1%
1647
74.0%
75.9%
≥4.70
11
10.1%
65
10.4%
32
6.8%
18
10.2%
13
6.6%
4
2.8%
8
10.4%
6
2.5%
1
2.1%
7
5.1%
165
7.4%
5.7%
Total
109
100.0%
625
100.0%
473
100.0%
176
100.0%
197
100.0%
141
100.0%
77
100.0%
244
100.0%
48
100.0%
137
100.0%
2227
100.0%
100.00%
Table 8: Percentage of previous “exceptions” units falling into each category for semester 2 2009
≤3.00
Number of units
ArtDes
Percentage of units
Number of units
Arts
Percentage of units
Number of units
BusEco
Percentage of units
Number of units
Edu
Percentage of units
Number of units
Eng
Percentage of units
Number of units
InfoTech
Percentage of units
Number of units
Law
Percentage of units
Number of units
MNHS
Percentage of units
Number of units
Pharm
Percentage of units
Number of units
Sci
Percentage of units
Number of units
Percentage of units
Semester 1 comparison
6
30.0%
72
14.7%
10
18.9%
6
12.5%
6
25.0%
22
24.4%
0
0.0%
27
20.9%
0
0.0%
5
16.1%
154
17.1%
5.7%
3.01-3.59
4
20.0%
45
9.2%
9
17.0%
4
8.3%
0
0.0%
14
15.6%
0
0.0%
16
12.4%
2
22.2%
4
12.9%
98
10.9%
12.6%
3.60-4.69
5
25.0%
225
45.9%
27
50.9%
28
58.3%
15
62.5%
41
45.6%
2
28.6%
65
50.4%
6
66.7%
17
54.8%
431
47.8%
75.9%
≥4.70
5
25.0%
148
30.2%
7
13.2%
10
20.8%
3
12.5%
13
14.4%
5
71.4%
21
16.3%
1
11.1%
5
16.1%
218
24.2%
5.7%
Total
20
100.0%
490
100.0%
53
100.0%
48
100.0%
24
100.0%
90
100.0%
7
100.0%
129
100.0%
9
100.0%
31
100.0%
901
100.0%
100.00%
7