STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT OF UNIT EVALUATION RESULTS SEMESTER 2, 2015 10 December 2015 INTRODUCTION This report summarises the semester 2 2015 responses to SETU’s1 ‘overall satisfaction’ item. The overall item required students to rate the statement; ‘Overall I was satisfied with the quality of this unit’, with either ‘Strongly Agree’ (5), ‘Agree’ (4), ‘Neutral’ (3), ‘Disagree’ (2)’ or ‘Strongly Disagree’ (1). From these data, a ‘median’2 score was calculated for each unit offering3. Unit offerings were then classified into one of the four groups using the following median score cut‐offs4:  Outstanding: ≥4.70  Meeting aspirations: 3.80 – 4.69  Needing improvement: 3.01 – 3.79  Needing critical attention: ≤3.00 FINDINGS Overall satisfaction data was collected for 2,534 degree level5 unit offerings in Semester 2, 2015. Note that this figure only includes offerings with overall satisfaction responses. 1. FACULTY VARIATION Table 1 and Figure 1 both show that the vast majority of unit offerings were categorised as ‘meeting aspirations’ (61.6%) and the highest performing faculty was Law (33.6% of units ‘outstanding’). Notwithstanding high overall satisfaction levels, 6.7% of unit offerings were classified as ‘needing critical attention’. Among all faculties, Law (9.2%) and Medicine Nursing & Health Sciences (9.8%) had a relatively high percentage of units falling into this category. Table 2 and Figure 2 show the results for unit offerings with 5 or more responses. Similarly to Table 1 and Figure 1, the majority of unit offerings were in ‘meeting aspirations’ (67.1%) where Art, Design and Architecture (18.8%) and Law (26.5%) had a high percentage of ‘outstanding’ units. Results for unit offerings with less than 5 responses are shown in Table 3 and Figure 3. 1
Further detail on the SETU can be found at: http://www.opq.monash.edu.au/us/surveys/setu/index.html Further detail on how the median is calculated can be found at: http://www.opq.monash.edu.au/us/surveys/setu/setu_median_calculation.pdf 3
A unit offering is defined here as a unit at a specific location, in a specific mode (e.g. on campus/off campus). It does not take into account instances where the unit offering may run in different calendar types within the same survey period or different unit classes within the same mode i.e. DAY, EVENING (both ON campus mode). Also on‐shore non‐campus locations are grouped into ‘Other Australian Locations’ and off‐shore non‐campus locations are grouped into ‘Other Overseas locations’. 4
SETU response bands have changed to reflect the new standards impacting the Meeting Aspirations and Needing Improvement range. 5
Evaluations were also undertaken for Monash College diploma, Malaysia National Subject units and Sth Africa Foundation Program units but these are not included in this report. 2
1
University Planning and Statistics
Table 1: No. of unit offerings falling into each "traffic light" category by faculty, Semester 2 2015 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
Total
116
6
13
74
23
Arts
28
41
340
97
506
Business & Economics
26
79
356
79
540
Education
20
54
149
41
264
Engineering
18
50
130
25
223
Information Technology
12
30
106
18
166
Law
11
11
57
40
119
Med Nursing & Health Sci
37
70
216
54
377
Pharmacy & Pharm Science
4
18
44
13
79
Science
8
29
90
17
144
170
395
1,562
407
2,534
Grand Total
Table 2: No. of unit offerings with 5 or more responses falling into each "traffic Light" category by faculty, Semester 2 2015 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
Total
4
11
63
18
96
Arts
12
32
295
57
396
Business & Economics
12
69
313
40
434
Education
10
46
118
25
199
Engineering
11
44
113
12
180
Information Technology
4
28
92
9
133
Law
6
10
45
22
83
20
58
177
24
279
3
15
35
5
58
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
4
28
80
13
125
86
341
1,331
225
1,983
Table 3: No. of unit offerings with less than 5 responses falling into each "traffic Light" category by faculty, Semester 2 2015 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
Total
2
2
11
5
20
Arts
16
9
45
40
110
Business & Economics
14
10
43
39
106
Education
10
8
31
16
65
Engineering
7
6
17
13
43
Information Technology
8
2
14
9
33
Law
5
1
12
18
36
17
12
39
30
98
Pharmacy & Pharm Science
1
3
9
8
21
Science
4
1
10
4
19
84
54
231
182
551
Med Nursing & Health Sci
Grand Total
2
Figure 1: Proportion of unit offerings falling into each ‘traffic light category’ by faculty, Semester 2 2015 3
Figure 2: Proportion of unit offerings with 5 or more responses falling into each ‘traffic light category’ by faculty, Semester 2 2015 4
Figure 3: Proportion of unit offerings with less than 5 responses falling into each ‘traffic light category’ by faculty, Semester 2 2015 5
2. CAMPUS/FACULTY VARIATION Table 4 displays average median satisfaction (overall) scores across unit offerings, with data separated by campus and faculty. Art, Design and Architecture at Clayton (1 unit offering with response rate of 27%), Arts at Other Australian locations (1 unit offering with response rate of 67%), Business and Economics at Prato (1 unit offering with response rate of 18%), Business and Economics at China South East University (3 unit offerings with response rate ranging from 16% to 17%) and Information Technology at China South East University (2 unit offerings with response rate ranging from 61% to 97%) were in the ‘Outstanding’ range. The Faculty of Medicine, Nursing & Health Sciences at the Gippsland campus (2.99) had an average median in the ‘needing critical attention’ range. This has 8 unit offerings with response rate ranging from 10% to 46%. Examination of campuses alone (far right column) indicates a high percentage of units with the critical attention flag at Gippsland (20.5%). Peninsula (10.8%), Prato (10.5%) and Other Australian locations (10.5%) percentage of units with the critical attention flag is relatively high with the percentage ranging above 10%. For all other campuses, less than 10% of units fell into the critical range. Examining Faculty alone (third bottom row of Table 4), shows the average median for all faculties is in the ‘meeting aspirations’ range. 3. UNIT MODE VARIATION Table 5 displays average median satisfaction broken down by faculty and mode (off‐campus, on‐campus and on/off‐campus). Most of the faculty/mode groups fell into the ‘meeting aspirations’ category with a few off‐
campus in the ‘needing improvement’ category. Collapsing across faculty groups (bottom 4 rows of Table 5) shows that the average median scores from all three modes (on/off, off and on‐campus) are in the ‘meeting aspirations’ range. However, 15.9% of ‘off‐campus’ unit offerings fell into the ‘needing critical attention’ range. 4. UNIT LEVEL VARIATION Table 6 shows average median satisfaction split by faculty and unit level. The faculty of Art, Design and Architecture with one level 6 unit (5.00) achieved the ‘Outstanding’ classification for their average median score. No faculty had unit offerings falling into the ‘needing critical attention’ range. Collapsing results across faculty (bottom 4 rows of Table 6) shows that the average median score from each unit level is in the ‘Meeting Aspirations’ range. The percentage of units falling in the ‘needing critical attention’ range was under 10% for all levels. 6
Table 4: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and location, Semester 2 2015 Location
Berwick
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Caulfield
Av. Median
No. of unit offerings
No. needing critical attention
Clayton
Av. Median
No. of unit offerings
No. needing critical attention
Gippsland Av. Median
No. of unit offerings
No. needing critical attention
Parkville
Av. Median
No. of unit offerings
No. needing critical attention
Peninsula Av. Median
No. of unit offerings
No. needing critical attention
South Africa Av. Median
No. of unit offerings
No. needing critical attention
Malaysia
Av. Median
No. of unit offerings
No. needing critical attention
Prato
Av. Median
No. of unit offerings
No. needing critical attention
Monash Av. Median
Online
No. of unit offerings
No. needing critical attention
Os China
Av. Median
South East No. of unit offerings
University No. needing critical attention
Other Av. Median
Australian No. of unit offerings
Locations No. needing critical attention
Other Av. Median
Offshore No. of unit offerings
Locations No. needing critical attention
Overall Av. Median
Total no. of unit offerings
Total no. needing critical attention
Art, Des & Arch
4.19
106
4
5.00
1
3.42
3
1
4.21
6
1
Arts
3.57
7
1
4.22
95
2
4.29
251
7
3.83
56
13
4.08
60
2
4.12
23
2
4.65
12
1
Bus Eco Educ
4.29 3.95
23
37
2
4.18
218
6
4.12 4.04
104
112
6
10
3.98 4.01
41
8
6
4.13
13
3
4.06
42
3
4.04
85
Eng
4.01
144
12
3.61
14
4
Info Tech
Law
4.09
1
4.05
86
7
3.79
33
2
3.63
4
1
4.31
50
2
4.04
70
6
3.92
62
2
3.99
15
2
4.20
24
4.15
5
1
4.74
3
3.50
1
4.18
116
6
4.19
506
28
Sci
4.06
99
5
4.08
17
3
3.92
28
4.75
1
4.08
1
4.83
1
Pharm & Pharm MNHS
Sci
3.70
10
1
3.91
47
7
4.00
134
10
2.99
8
3
4.12
62
3
3.97
65
7
3.86
13
2
4.14
3.86
18
17
1
4.05
5
1
4.20
5
1
4.13
540
26
4.31
3
4.82
2
4.38
10
4.07
27
2
4.04
264
20
3.97
223
18
3.94
10
4.01
166
12
4.22
64
8
4.07
72
7
4.25
119
11
3.97
377
37
4.07
79
4
4.03
144
8
% needing critical Total attention
3.98
78
4
5.1%
4.15
552
26
4.7%
4.12
928
54
5.8%
3.83
151
31
20.5%
4.12
62
3
4.8%
4.02
148
16
10.8%
4.04
130
9
6.9%
4.02
262
6
2.3%
4.52
19
2
10.5%
3.96
11
0.0%
4.60
8
0.0%
4.16
152
16
10.5%
4.07
33
3
9.1%
4.09
2534
170
6.7% 7
Table 5: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and Mode, Semester 2 2015 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Off Campus On Campus
4.13
2
3.89
117
16
4.26
21
1
3.67
3
1
3.93
358
4.18
114
6
4.23
419
16
4.16
491
12
4.03
176
14
3.99
210
14
4.06
145
6
4.25
119
11
4.00
241
19
3.99
58
3
4.04
141
7
4.11
2114
57
15.9%
108
5.1%
4.01
79
11
3.79
49
14
4.09
55
4
3.63
13
4
3.69
19
6
On/Off Campus
Grand Total
4.05
62
4.18
116
6
4.19
506
28
4.13
540
26
4.04
264
20
3.97
223
18
4.01
166
12
4.25
119
11
3.97
377
37
4.07
79
4
4.03
144
8
4.09
2534
5
8.1%
170
6.7%
4.21
8
1
4.04
33
2
3.83
2
4.03
19
2
8
Table 6: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and unit level, Semester 2 2015 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
1
4.16
30
2
4.14
106
3
4.09
65
2
3.88
23
4
3.66
15
2
4.03
22
4.04
4
2
4.07
33
4
4.16
156
9
3.91
110
6
3.94
39
4
3.90
50
2
3.90
28
1
4.10
6
3.92
44
1
3.87
12
3.55
46
12
4.02
16
3.97
25
1
4.04
346
15
4.3%
4.06
52
2
3.98
536
40
7.5%
3
4.16
21
4.22
175
12
4.13
152
5
4.14
33
1
3.94
56
3
3.92
36
4
4.00
6
1
3.97
68
6
4.10
22
2
4.06
64
4
4.10
633
38
6.0%
4
4.25
21
0
4.15
20
2
4.42
27
2
4.02
60
2
4.06
65
5
3.92
6
1
4.44
35
1
4.16
83
4
3.86
8
1
3.42
3
1
4.15
328
19
5.8%
5
4.40
9
6
5.00
1
4.33
48
2
4.21
180
11
4.09
109
9
4.00
27
5
4.05
60
6
4.21
68
9
3.93
112
13
4.29
20
1
3.60
1
4.13
633
56
8.8%
9
4.35
1
L
4.63
6
4.24
10
1
3.86
3
4.38
16
1
4.48
11
4.47
4
4.17
4
3.83
1
4.34
37
2
5.4%
4.43
17
4.17
4
Grand Total
4.18
116
6
4.19
506
28
4.13
540
26
4.04
264
20
3.97
223
18
4.01
166
12
4.25
119
11
3.97
377
37
4.07
79
4
4.03
144
8
4.09
2534
170
6.7%
9
5. CHANGES OVER TIME Figures 4 and 5 display the faculty‐level trends in unit evaluations between 2011 and 2015 with regard to the percentage of units classified as ‘outstanding’ (Figure 4) or as ‘needing critical attention’ (Figure 5). The percentage of unit offerings classified as ‘outstanding’ (all faculties combined) increased from 12.9% in 2014 to 15.6% in 2015. Increases in the percentage of unit offerings classified as ‘outstanding’ from 2014 to 2015 are seen in all faculties. In decreasing magnitude, the faculties with the largest increase are Pharmacy & Pharmaceutical Sciences (7.4%), Art, Design and Architecture (5.7%) and Engineering (3.6%). The percentage of unit offerings classified as ‘needing critical attention’ (all faculties combined) remained the same (6.5% in 2014 and 6.5% in 2015). At the faculty level, Information Technology showed the largest decline in the percentage of units in the ‘needing critical attention’ range from 2014 to 2015 (‐2.2%). Other signs of improvement from 2014 to 2015 are observed across all faculties with three exceptions: Law had the largest percentage increase in the number of unit offerings classified as ‘needing critical attention’ from 2014 to 2015 (5.7%) followed by Science (1.9%). 10
Figure 4: Percentage of evaluated unit offerings classified as 'outstanding' by faculty, 2011‐20156, 7 6
7
Non‐standard unit offerings originated from ‘Malaysia National Subjects’ and ‘Enhancement Studies Program’ were excluded. 2012, 2013, 2014 and 2015 figures include full year data, i.e. Semester 1, Semester 2 and Summer Semester data. 11
Figure 5: Proportion of evaluated unit offerings classified as 'needing critical attention' by faculty, 2011‐20158,9 8
9
Non‐standard unit offerings originated from ‘Malaysia National Subjects’ and ‘Enhancement Studies Program’ were excluded. 2012, 2013, 2014 and 2015 figures include full year data, i.e. Semester 1, Semester 2 and Summer Semester data. 12