STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT
OF UNIT EVALUATION RESULTS
SEMESTER 1, 2014
24 July 2014
INTRODUCTION
This report summarises the semester 1 2014 responses to SETU’s1 ‘overall satisfaction’ item. The overall item
required students to rate the statement; ‘Overall I was satisfied with the quality of this unit’, with either
‘Strongly Agree’ (5), ‘Agree’ (4), ‘Neutral’ (3), ‘Disagree’ (2)’ or ‘Strongly Disagree’ (1).
From these data, a ‘median’2 score was calculated for each unit offering3. Unit offerings were then classified into
one of the four groups using the following median score cut-offs:
• Outstanding: ≥4.70
• Meeting aspirations: 3.60 – 4.69
• Needing improvement: 3.01 – 3.59
• Needing critical attention: ≤3.00
FINDINGS
Overall satisfaction data was collected for 2,484 degree level4 unit offerings in Semester 1, 2014. Note that this
figure only includes offerings with overall satisfaction responses.
1. FACULTY VARIATION
Table 1 and Figure 1 both show that the vast majority of unit offerings were categorised as either ‘meeting
aspirations’ (73.2%) or ‘outstanding’ (12.0%) and the highest performing faculty was Law (26.8% of units
‘outstanding’).
Notwithstanding high overall satisfaction levels, 6.5% of unit offerings were classified as ‘needing critical
attention’. Among all faculties, Medicine Nursing & Health Sciences (10.2%) and Information Technology (7.9%)
had a relatively high percentage of units falling into this category.
Table 2 and Figure 2 show the results for unit offerings with 5 or more responses. Results for unit offerings with
less than 5 responses are shown in Table 3 and Figure 3.
1
Further detail on the SETU can be found at: http://www.opq.monash.edu.au/us/surveys/setu/index.html
Further detail on how the median is calculated can be found at:
http://www.opq.monash.edu.au/us/surveys/setu/setu_median_calculation.pdf
3
A unit offering is defined here as a unit at a specific location, in a specific mode (e.g. on campus/off campus). It does not
take into account instances where the unit offering may run in different calendar types within the same survey period or
different unit classes within the same mode ie. DAY, EVENING (both ON campus mode). Also on-shore non-campus
locations are grouped into ‘Other Australian Locations’ and off-shore non-campus locations are grouped into ‘Other
Overseas locations’.
4
Evaluations were also undertaken for Monash College diploma, Malaysia National Subject units and Sth Africa Foundation
Program units but these are not included in this report.
2
1
Office of Planning and Quality – University Planning and Statistics
Table 1: No. of unit offerings falling into each "traffic light" category by faculty, Semester 1 2014
Median Response Band for Overall Satisfaction Item
Needing Critical
Attention
(≤3.0)
Needing
Improvement
(3.01-3.59)
Meeting Aspirations
(3.6-4.69)
Outstanding
(≥4.7)
3
6
58
8
75
Arts
37
41
454
104
636
Business & Economics
25
36
410
59
530
Education
17
24
171
24
236
Engineering
14
16
147
15
192
Information Technology
13
20
118
14
165
Owning Faculty
Art Des & Architecture
Law
Total
7
4
60
26
97
36
42
240
35
353
Pharmacy & Pharm Science
2
9
54
Science
8
9
106
12
135
162
207
1,818
297
2,484
Med Nursing & Health Sci
Grand Total
65
Table 2: No. of unit offerings with 5 or more responses falling into each "traffic Light" category by faculty,
Semester 1 2014
Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical
Attention
(≤3.0)
Needing
Improvement
(3.01-3.59)
Meeting Aspirations
(3.6-4.69)
Outstanding
(≥4.7)
2
15
12
11
11
5
1
23
2
6
88
6
31
28
20
14
13
3
34
8
9
166
56
378
362
152
137
103
52
204
48
96
1,588
5
51
25
21
10
8
14
20
7
161
Total
69
475
427
204
172
129
70
281
58
118
2,003
Table 3: No. of unit offerings with less than 5 responses falling into each "traffic Light" category by faculty,
Semester 1 2014
Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Needing Critical
Attention
(≤3.0)
1
22
13
6
3
8
6
13
Pharmacy & Pharm Science
Science
Grand Total
2
74
Needing
Improvement
(3.01-3.59)
10
8
4
2
7
1
8
1
41
Meeting Aspirations
(3.6-4.69)
Outstanding
(≥4.7)
2
76
48
19
10
15
8
36
6
10
230
3
53
34
3
5
6
12
15
5
136
Total
6
161
103
32
20
36
27
72
7
17
481
2
Figure 1: Proportion of unit offerings falling into each ‘traffic light category’ by faculty, Semester 1 2014
100%
90%
80%
% of unit offerings
70%
60%
50%
40%
30%
20%
10%
0%
Art Des &
Architecture
(n = 75)
Arts
(n = 636)
Business &
Economics
(n = 530)
Needing Critical Attention
4.0%
5.8%
Needing Improvement
8.0%
6.4%
Meeting Aspirations
77.3%
Outstanding
10.7%
Pharmacy &
Pharm
Science
(n = 65)
Science
(n = 135)
Grand Total
(n = 2484)
10.2%
3.1%
5.9%
6.5%
11.9%
13.8%
6.7%
8.3%
61.9%
68.0%
83.1%
78.5%
73.2%
26.8%
9.9%
0.0%
8.9%
12.0%
Information
Technology
(n = 165)
Law
(n = 97)
Med Nursing
& Health Sci
(n = 353)
7.3%
7.9%
7.2%
8.3%
12.1%
4.1%
72.5%
76.6%
71.5%
10.2%
7.8%
8.5%
Education
(n = 236)
Engineering
(n = 192)
4.7%
7.2%
6.8%
10.2%
71.4%
77.4%
16.4%
11.1%
3
Figure 2: Proportion of unit offerings with 5 or more responses falling into each ‘traffic light category’ by faculty, Semester 1 2014
100%
90%
80%
% of unit offerings
70%
60%
50%
40%
30%
20%
10%
0%
Art Des &
Architecture
(n = 69)
Arts
(n = 475)
Business &
Economics
(n = 427)
Education
(n = 204)
Engineering
(n = 172)
Information
Technology
(n = 129)
Law
(n = 70)
Needing Critical Attention
2.9%
3.2%
2.8%
5.4%
Needing Improvement
8.7%
6.5%
6.6%
9.8%
Meeting Aspirations
81.2%
79.6%
84.8%
Outstanding
7.2%
10.7%
5.9%
6.4%
3.9%
1.4%
8.2%
8.1%
10.1%
4.3%
12.1%
74.5%
79.7%
79.8%
74.3%
72.6%
10.3%
5.8%
6.2%
20.0%
7.1%
4
Med Nursing & Pharmacy &
Health Sci
Pharm Science
(n = 58)
(n = 281)
Science
(n = 118)
Grand Total
(n = 2003)
3.4%
5.1%
4.4%
13.8%
7.6%
8.3%
82.8%
81.4%
79.3%
0.0%
5.9%
8.0%
Figure 3: Proportion of unit offerings with less than 5 responses falling into each ‘traffic light category’ by faculty, Semester 1 2014
100%
90%
80%
% of unit offerings
70%
60%
50%
40%
30%
20%
10%
0%
Art Des &
Architecture
(n = 6)
Arts
(n = 161)
Business &
Economics
(n = 103)
Needing Critical Attention
16.7%
13.7%
12.6%
18.8%
Needing Improvement
0.0%
6.2%
7.8%
12.5%
Meeting Aspirations
33.3%
47.2%
46.6%
59.4%
Outstanding
50.0%
32.9%
33.0%
9.4%
Education
(n = 32)
Information
Technology
(n = 36)
Law
(n = 27)
15.0%
22.2%
22.2%
18.1%
10.0%
19.4%
3.7%
11.1%
50.0%
41.7%
29.6%
50.0%
25.0%
16.7%
44.4%
20.8%
Engineering
(n = 20)
5
Med Nursing & Pharmacy &
Health Sci
Pharm Science
(n = 7)
(n = 72)
Science
(n = 17)
Grand Total
(n = 481)
0.0%
11.8%
15.4%
14.3%
0.0%
8.5%
85.7%
58.8%
47.8%
0.0%
29.4%
28.3%
2. CAMPUS/FACULTY VARIATION
Table 4 displays average median satisfaction (overall) scores across unit offerings, with data separated by
campus and faculty. There were no campus/faculty groups which had average median scores in the ‘needing
critical attention’ range. Three campus/faculty groups achieved the ‘Outstanding’ classification for their
average median score these being Business and Economics (3 unit offering 1 respondent for each) and
Engineering (one unit offering with one respondent) at ‘Other Offshore Locations’, and Information Technology
at China South East University with four unit offerings and over 30 responses for each. One location, China
South East University, received a total average median score classified as ‘Outstanding’ (4.72), based upon 221
responses across 10 unit offerings.
Two campus/faculty groups achieved average median scores placing them into the ‘needing improvement’
range. These were the Faculty of Education at the Malaysia campus (3.25) and the Faculty of Information
Technology at the Gippsland campus (3.51). The number of unit offerings within these faculties needing critical
attention was one (3 respondents) and four units (6 respondents in total across the units), respectively.
Examination of campuses alone (far right column) indicates a relatively high percentage of units with the critical
attention flag Gippsland (25%) and at Parkville (15.7%). For all other campuses, less than 10% of units fell into
the critical range. Examining Faculty alone (third bottom row of Table 4), shows the average median for all
faculties is in the ‘meeting aspirations’ range.
3. UNIT MODE VARIATION
Table 5 displays average median satisfaction broken down by faculty and mode (off-campus, on-campus and
on/off-campus). Almost all faculty/mode groups fell into the ‘meeting aspirations’ category. The ‘on/off-campus’
Arts faculty (3.59) was the exception, which fell into the ‘needing improvement’ category.
Collapsing across faculty groups (bottom 4 rows of Table 5) shows that the average median scores from all three
modes (on/off, off and on-campus) are in the ‘meeting aspirations’ range. However, a proportion of units did fall
into the ‘needing critical attention’ range; 12.3% for ‘off-campus’ and 10.2% for ‘on/off-campus’ modes.
4. UNIT LEVEL VARIATION
Table 6 shows average median satisfaction split by faculty and unit level. The average median satisfaction with
level 9 units offered by the faculty of Engineering was the only Faculty to achieve an average median score in
the ‘outstanding’ range (4.83). However, responses from only one unit and four respondents formed this result.
Unit offerings falling into the ‘needing attention’ range include level 4 units in the faculty of Information
Technology (3.50) and, level 5 and 9 units in the faculty of Art, Design and Architecture (3.42 and 3.50,
respectively).
Collapsing results across faculty (bottom 4 rows of Table 6) shows that the average median score from each unit
level is in the ‘Meeting Aspirations’ range. The percentage of units falling in the ‘needing critical attention’ range
was under 8% for all levels.
6
Table 4: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and location, Semester 1 2014
Location
Berwick
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Caulfield
Av. Median
No. of unit offerings
No. needing critical attention
Clayton
Av. Median
No. of unit offerings
No. needing critical attention
Gippsland Av. Median
No. of unit offerings
No. needing critical attention
Parkville
Av. Median
No. of unit offerings
No. needing critical attention
Peninsula Av. Median
No. of unit offerings
No. needing critical attention
South Africa Av. Median
No. of unit offerings
No. needing critical attention
Malaysia
Av. Median
No. of unit offerings
No. needing critical attention
Prato
Av. Median
No. of unit offerings
No. needing critical attention
Os China
Av. Median
South East No. of unit offerings
University No. needing critical attention
Other
Av. Median
Australian No. of unit offerings
Locations No. needing critical attention
Other
Av. Median
Offshore
No. of unit offerings
Locations No. needing critical attention
Overall Av. Median
Total no. of unit offerings
Total no. needing critical attention
Art,
Des &
Arch
Arts Bus Eco Educ Eng
4.37
4.08 4.02
26
28
32
1
1
2
4.05 4.17
4.13
75 134
221
3
9
9
4.21
3.91 4.07 4.00
303
85 118 116
9
8
7
7
3.81
4.15 3.89 3.61
84
58
18
13
14
3
3
3
4.04
54
3
4.11
27
1
4.07
15
1
4.01
34
1
3.99
83
2
4.00
53
4
3.25
2
1
3.95
62
4
Pharm &
Pharm
Law MNHS
Sci
4.01
20
3
4.05
3.87
79
54
4
6
3.99 4.26 3.99
28
31 116
1
6
3.51
3.76
16
19
4
6
4.03
50
1
3.84
51
6
3.81
4.05
14
13
Info
Tech
3.67
24
4
3.72
12
2
Sci
4.15
82
1
4.07
28
4
3.77
15
1
3.89
25
3
3.97
65
2
4.09
135
8
3.90
20
5
4.60
3
4.65
3
4.45
5
4.05
75
3
4.14
636
37
4.87
4
4.53
5
5.00
3
4.16
8
5.00
1
4.07
530
25
4.04
236
17
3.96
192
14
3.93
165
13
4.31
46
2
3.97
68
7
4.21
97
7
3.93
353
36
% needing
critical
Total attention
4.12
106
7
6.6%
4.09
563
31
5.5%
4.10
879
39
4.4%
3.90
236
37
15.7%
4.03
50
1
2.0%
3.94
119
11
9.2%
4.00
115
4
3.5%
3.92
250
18
7.2%
3.90
20
5
25.0%
4.72
10
0.0%
4.14
124
9
7.3%
4.44
12
0.0%
4.05
2484
162
6.5%
7
Table 5: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and Mode, Semester 1 2014
Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Off Campus On Campus
3.75
4.06
1
74
3
3.86
4.20
96
536
17
19
4.10
4.07
53
477
4
21
3.94
4.04
50
157
5
10
3.98
3.96
14
178
1
13
3.64
3.99
26
139
6
7
4.21
97
7
3.94
3.92
106
231
11
23
4.18
3.89
18
47
2
4.09
4.08
10
125
2
6
3.94
4.07
374
2061
46
12.3%
111
5.4%
On/Off
Campus
4.11
49
Grand Total
4.05
75
3
4.14
636
37
4.07
530
25
4.04
236
17
3.96
192
14
3.93
165
13
4.21
97
7
3.93
353
36
3.97
65
2
4.09
135
8
4.05
2484
5
10.2%
162
6.5%
3.59
4
1
4.22
29
2
4.05
16
2
8
Table 6: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and unit level, Semester 1 2014
Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
1
4.20
13
2
4.19
23
4.16
113
1
3.95
64
4.12
194
12
3.95
114
6
4.08
26
4.11
23
1
3.89
23
2
3.99
24
3.78
1
3.82
50
4
3.81
12
1
4.04
32
1
4.02
355
10
2.8%
3.83
44
5
3.87
31
1
4.28
2
3.80
54
8
3.88
12
4.10
40
2
4.01
540
34
6.3%
3
3.98
18
1
4.17
197
10
4.07
145
8
4.09
17
2
3.96
52
2
3.85
39
5
3.89
10
2
3.96
70
8
3.97
16
1
4.10
63
5
4.06
627
44
7.0%
4
4.04
15
1
4.12
87
10
4.14
38
4
4.06
100
6
4.10
61
4
3.50
6
1
4.25
30
1
3.90
61
5
3.84
7
5
3.42
5
1
4.12
41
4
4.17
164
7
3.96
70
8
3.84
11
1
4.03
60
5
4.24
54
4
4.00
97
9
4.20
17
6
4.06
405
32
7.9%
4.09 4.21
519
23
39
2
7.5% 8.7%
9
3.50
1
L
4.28
4
4.50
5
4.83
1
4.05
5
1
4.08
14
2
4.66
5
3.90
2
3.92
1
4.29
13
1
7.7%
3.90
2
Grand
Total
4.05
75
7
4.14
636
43
4.07
530
27
4.04
236
19
3.96
192
11
3.93
165
17
4.21
97
5
3.93
353
35
3.97
65
6
4.09
135
13
4.05
2484
183
7.4%
9
5. CHANGES OVER TIME
Figures 4 and 5 display the faculty-level trends in unit evaluations between 2011 and 2014, with regard to the
percentage of units classified as ‘outstanding’ (Figure 4) or as ‘needing critical attention’ (Figure 5).
The following commentary summarises the broad trends in SETU overall satisfaction results across the years.
However, 2014 results need to be interpreted with a degree of caution as they represent only ‘part year’ of the
year with semester 2 results yet to be obtained.
The percentage of unit offerings classified as ‘outstanding’ (all faculties combined) decreased from 13.4% in
2013 to 12.6% in 2014. However, the 2014 result is more comparable with those achieved in 2011 and 2012
(12.4% and 12.3%, respectively). Increases in the percentage of unit offerings classified as ‘outstanding’ from
2013 to 2014 are seen in four faculties. In decreasing magnitude they are Law (2.6%), Business & Economics
(1.0%), Engineering (0.7%) and Science (0.2%). The largest decreases from 2013 to 2014 were seen in the Art
Design & Architecture and Education faculties (7.7% and 3.9%, respectively).
The percentage of unit offerings classified as ‘needing critical attention’ (all faculties combined) continues to
decrease (8.2% in 2011; 7.7% in 2012; 6.9% in 2013; 6.7% in 2014). At the faculty level, Pharmacy &
Pharmaceutical Sciences showed the largest decline in the percentage of units in the ‘needing critical attention’
range since 2011 (11.2% in 2011 to 3.0% in 2014). This faculty also experienced the largest decrease from 2013
to 2014 (-3.5%) followed by the Art, Design and Architecture faculty (-2.3%). Other, signs of improvement from
2013 to 2014 are observed at the Education and Information Technology faculties where the percentage of units
in the ‘needing critical attention’ range decreased by 0.7% and 0.6%, respectively. Law experienced the largest
percentage increase in the number of unit offerings classified as ‘needing critical attention’ from 2013 to 2014
3.5% in 2013 to 6.4% in 2014). The percentage of units classified as ‘needing critical attention’ in the faculty of
Medicine, Nursing and Health Sciences continues to exceed 10% annually.
10
Figure 4: Percentage of evaluated unit offerings classified as 'outstanding' by faculty, 2011-20145, 6
30%
% of unit offerings 'outstanding'
25%
20%
15%
10%
5%
0%
Art, Design &
Architecture
Arts
Business &
Economics
Education
Engineering
Information
Technology
Law
Medicine,
Nursing &
Health Sciences
Pharmacy &
Pharmaceutical
Sciences
Science
Grand Total
2011-FullYear
12.1%
19.2%
8.6%
12.4%
6.6%
6.8%
18.3%
8.6%
8.2%
5.6%
12.4%
2012-FullYear
13.0%
18.2%
9.1%
11.8%
4.4%
9.3%
22.8%
9.9%
9.8%
5.5%
12.3%
2013-FullYear
18.4%
17.3%
11.0%
14.9%
7.5%
10.0%
24.7%
11.2%
7.2%
8.4%
13.4%
2014-HalfYear
10.7%
17.0%
12.0%
11.0%
8.2%
9.4%
27.3%
10.4%
0.0%
8.6%
12.6%
5
Adjustment 1: 2011 data has been adjusted to include Summer Semester data.
Adjustment 2: Non-standard unit offerings originated from ‘Malaysia National Subjects’ and ‘Enhancement Studies Program’ were excluded.
6
2012 and 2013 figures include full year data, i.e. Semester 1, Semester 2 and Summer Semester data. 2014 ‘half year’ figures include summer semester and Semester 1 data.
11
Figure 5: Proportion of evaluated unit offerings classified as 'needing critical attention' by faculty, 2011-20145, 6
25%
% of unit offerings 'needing critical attention'
20%
15%
10%
5%
0%
Art, Design &
Architecture
Arts
Business &
Economics
Education
Engineering
Information
Technology
Law
Medicine,
Nursing &
Health Sciences
Pharmacy &
Pharmaceutical
Sciences
2011-FullYear
6.2%
8.6%
5.2%
10.5%
8.8%
8.9%
4.1%
11.4%
11.2%
6.9%
8.2%
2012-FullYear
8.3%
7.9%
6.0%
11.3%
5.7%
7.4%
4.7%
10.7%
8.4%
5.2%
7.7%
2013-FullYear
6.3%
6.7%
4.8%
7.8%
7.3%
8.3%
3.5%
11.1%
6.5%
6.1%
6.9%
2014-HalfYear
4.0%
6.8%
4.7%
7.1%
7.1%
7.7%
6.4%
10.1%
3.0%
5.8%
6.7%
12
Science
Grand Total