STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT OF UNIT EVALUATION RESULTS SEMESTER 1, 2015 3 August 2015 INTRODUCTION This report summarises the semester 1 2015 responses to SETU’s1 ‘overall satisfaction’ item. The overall item required students to rate the statement; ‘Overall I was satisfied with the quality of this unit’, with either ‘Strongly Agree’ (5), ‘Agree’ (4), ‘Neutral’ (3), ‘Disagree’ (2)’ or ‘Strongly Disagree’ (1). From these data, a ‘median’2 score was calculated for each unit offering3. Unit offerings were then classified into one of the four groups using the following median score cut‐offs4:  Outstanding: ≥4.70  Meeting aspirations: 3.80 – 4.69  Needing improvement: 3.01 – 3.79  Needing critical attention: ≤3.00 FINDINGS Overall satisfaction data was collected for 2,377 degree level5 unit offerings in Semester 1, 2015. Note that this figure only includes offerings with overall satisfaction responses. 1. FACULTY VARIATION Table 1 and Figure 1 both show that the vast majority of unit offerings were categorised as ‘meeting aspirations’ (64.5%) and the highest performing faculty was Law (22.7% of units ‘outstanding’). Notwithstanding high overall satisfaction levels, 6.2% of unit offerings were classified as ‘needing critical attention’. Among all faculties, Law (11.4%) and Medicine Nursing & Health Sciences (9.1%) had a relatively high percentage of units falling into this category. Table 2 and Figure 2 show the results for unit offerings with 5 or more responses. Similarly to Table 1 and Figure 1, the majority of unit offerings were in ‘meeting aspirations’ (69.4%) where Arts (17.1%) and Law (15.6%) had a high percentage of ‘outstanding’ units. Results for unit offerings with less than 5 responses are shown in Table 3 and Figure 3. 1
Further detail on the SETU can be found at: http://www.opq.monash.edu.au/us/surveys/setu/index.html Further detail on how the median is calculated can be found at: http://www.opq.monash.edu.au/us/surveys/setu/setu_median_calculation.pdf 3
A unit offering is defined here as a unit at a specific location, in a specific mode (e.g. on campus/off campus). It does not take into account instances where the unit offering may run in different calendar types within the same survey period or different unit classes within the same mode ie. DAY, EVENING (both ON campus mode). Also on‐shore non‐campus locations are grouped into ‘Other Australian Locations’ and off‐shore non‐campus locations are grouped into ‘Other Overseas locations’. 4
SETU response bands have changed to reflect the new standards impacting the Meeting Aspirations and Needing Improvement range. 5
Evaluations were also undertaken for Monash College diploma, Malaysia National Subject units and Sth Africa Foundation Program units but these are not included in this report. 2
1
University Planning and Statistics
Table 1: No. of unit offerings falling into each "traffic light" category by faculty, Semester 1 2015 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
Total
5
18
65
23
111
Arts
29
51
335
96
511
Business & Economics
25
81
348
77
531
Education
12
35
156
34
237
Engineering
16
38
132
21
207
151
Information Technology
7
24
106
14
Law
10
4
54
20
88
Med Nursing & Health Sci
31
65
205
39
340
Pharmacy & Pharm Science
3
14
43
7
67
Science
10
23
88
13
134
Grand Total
148
353
1,532
344
2,377
Table 2: No. of unit offerings with 5 or more responses falling into each "traffic Light" category by faculty, Semester 1 2015 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
Total
2
15
57
10
84
Arts
9
37
279
67
392
Business & Economics
5
69
314
36
424
Education
10
29
131
27
197
Engineering
10
36
106
13
165
2
22
95
8
127
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
6
4
44
10
64
23
55
174
24
276
2
13
40
2
57
5
20
80
11
116
74
300
1,320
208
1,902
Table 3: No. of unit offerings with less than 5 responses falling into each "traffic Light" category by faculty, Semester 1 2015 Median Response Band for Overall Satisfaction Item
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.79)
Meeting Aspirations (3.8‐4.69)
Outstanding (≥4.7)
3
3
8
13
27
Arts
20
14
56
29
119
Business & Economics
Owning Faculty
Art Des & Architecture
Total
20
12
34
41
107
Education
2
6
25
7
40
Engineering
6
2
26
8
42
Information Technology
5
2
11
6
24
Law
4
0
10
10
24
Med Nursing & Health Sci
8
10
31
15
64
Pharmacy & Pharm Science
1
1
3
5
10
Science
5
3
8
2
18
74
53
212
136
475
Grand Total
2
Figurre 1: Proportion o
of unit offerings faalling into each ‘trraffic light catego
ory’ by faculty, Sem
mester 1 2015 3
Figurre 2: Proportion o
of unit offerings w
with 5 or more ressponses falling intto each ‘traffic ligh
ht category’ by faculty, Semester 1
1 2015 4
Figurre 3: Proportion o
of unit offerings w
with less than 5 reesponses falling into each ‘traffic lig
ght category’ by faaculty, Semester 1 2015 5
2. CAMPUS/FACULTY VARIATION Table 4 displays average median satisfaction (overall) scores across unit offerings, with data separated by campus and faculty. Engineering at China South East University (3 unit offerings with response rate ranging from 22% to 48%) and Information Technology at China South East University (4 unit offerings with response rate of 100%) were in the ‘Outstanding’ range. The Faculty of Information Technology at the Gippsland campus (3.00) had an average median in the ‘needing critical attention’ range. This instance is an exception as two unit offerings (with 1 respondent for each) were in the ‘needing critical attention’ range. Examination of campuses alone (far right column) indicates a high percentage of units with the critical attention flag at Gippsland (23.7%). Peninsula (11.5%), Prato (12.5%) and Monash Online (12.5%) percentage of units with the critical attention flag is relatively high with the percentage ranging above 10%. For all other campuses, less than 10% of units fell into the critical range. Examining Faculty alone (third bottom row of Table 4), shows the average median for all faculties is in the ‘meeting aspirations’ range. 3. UNIT MODE VARIATION Table 5 displays average median satisfaction broken down by faculty and mode (off‐campus, on‐campus and on/off‐campus). Most of the faculty/mode groups fell into the ‘meeting aspirations’ category with a few off‐
campus and on/off‐campus in the ‘needing improvement’ category. Collapsing across faculty groups (bottom 4 rows of Table 5) shows that the average median scores from all three modes (on/off, off and on‐campus) are in the ‘meeting aspirations’ range. However, 15.2% of ‘off‐campus’ unit offerings fell into the ‘needing critical attention’ range. 4. UNIT LEVEL VARIATION Table 6 shows average median satisfaction split by faculty and unit level. The faculty of Arts with one level 6 unit (4.75) and the faculty of Business and Economics with four level 6 units (4.81) achieved the ‘Outstanding’ classification for their average median score. Unit offerings falling into the ‘needing critical attention’ range include level 4 unit in the faculty of Science (3.00). Collapsing results across faculty (bottom 4 rows of Table 6) shows that the average median score from each unit level is in the ‘Meeting Aspirations’ range. The percentage of units falling in the ‘needing critical attention’ range was under 10% for all levels with the exception of level 6 (11.1%). 6
Table 4: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and location, Semester 1 2015 Location
Berwick
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Caulfield
Av. Median
No. of unit offerings
No. needing critical attention
Clayton
Av. Median
No. of unit offerings
No. needing critical attention
Gippsland Av. Median
No. of unit offerings
No. needing critical attention
Parkville
Av. Median
No. of unit offerings
No. needing critical attention
Peninsula Av. Median
No. of unit offerings
No. needing critical attention
South Africa Av. Median
No. of unit offerings
No. needing critical attention
Malaysia
Av. Median
No. of unit offerings
No. needing critical attention
Prato
Av. Median
No. of unit offerings
No. needing critical attention
Monash Av. Median
Online
No. of unit offerings
No. needing critical attention
Os China
Av. Median
South East No. of unit offerings
University No. needing critical attention
Av. Median
Other Australian No. of unit offerings
Locations No. needing critical attention
Av. Median
Other No. of unit offerings
Offshore Locations No. needing critical attention
Overall Av. Median
Total no. of unit offerings
Total no. needing critical attention
Art, Des & Arch
Arts
4.01
9
1
4.16 4.27
111
102
5
2
4.23
273
7
3.50
47
15
4.22
56
1
3.97
22
3
Bus Eco Educ
4.33 4.08
21
35
4.18
233
9
4.07
81
1
3.96
43
6
4.01
15
2
3.89
37
3
3.98
89
4
4.18
117
4
3.71
7
2
Eng
Info Tech
3.97
128
12
3.93
11
1
4.18
80
1
3.94
29
1
3.00
2
2
4.02
65
3
3.65
14
2
4.01
22
1
4.89
3
4.71
4
3.97
64
6
3.75
2
4.06
1
4.18
3
4.63
2
4.16
111
5
4.16
511
29
3.75
4
4.67
5
4.63
4
4.35
7
4.09
531
25
4.11
237
12
3.99
207
16
4.06
151
7
Pharm & Pharm Law MNHS
Sci
3.66
11
1
4.02
54
4
4.13 3.99
31
116
5
8
3.95
9
2
4.06
51
3
3.95
52
7
3.95
13
1
3.80
3.95
12
16
1
4.16
16
2
3.79
7
1
4.21
41
3
4.10
66
6
4.17
88
10
3.99
340
31
4.04
67
3
Sci
4.05
89
3
3.81
20
5
3.86
25
2
3.98
134
10
% needing critical Total attention
4.08
76
2
2.6%
4.18
580
21
3.6%
4.11
864
41
4.7%
3.75
139
33
23.7%
4.06
51
3
5.9%
3.97
131
15
11.5%
4.02
120
7
5.8%
3.97
253
14
5.5%
4.16
16
2
12.5%
3.83
8
1
12.5%
4.60
10
0.0%
4.16
118
9
7.6%
4.45
11
0.0%
4.08
2377
148
6.2% 7
Table 5: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and Mode, Semester 1 2015 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Off Campus On Campus
On/Off Campus
Grand Total
4.00
116
12
4.19
19
2
3.57
7
2
3.91
328
4.16
111
5
4.25
439
10
4.11
480
17
4.12
173
9
4.01
195
14
4.04
134
5
4.17
88
10
4.00
205
16
3.97
48
1
4.01
125
7
4.11
1998
3.38
2
1
3.97
51
4.16
111
5
4.16
511
29
4.09
531
25
4.11
237
12
3.99
207
16
4.06
151
7
4.17
88
10
3.99
340
31
4.04
67
3
3.98
134
10
4.08
2377
50
15.2%
94
4.7%
4
7.8%
148
6.2%
3.61
70
19
3.95
51
8
4.01
36
3
3.78
12
2
4.17
17
2
4.13
2
4.17
28
0
3.73
19
3
8
Table 6: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and unit level, Semester 1 2015 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
1
4.22
17
4.16
105
4
3.98
67
3
4.07
22
1
3.78
16
1
3.93
23
2
4.10
34
2
4.13
168
10
3.94
111
5
4.03
35
3
3.92
46
1
3.99
26
4.02
3
4.14
4
3.88
42
4
3.81
12
3.82
49
7
4.01
12
1
4.07
44
1
4.02
529
30
5.7%
4.02
23
4.03
330
13
3.9%
3
4.05
24
3
4.14
158
14
4.10
137
2
4.16
27
1
4.01
51
3
3.98
30
3
4.06
7
1
3.87
63
5
4.03
16
4
4.30
25
0
4.23
32
1
4.03
30
6
3.96
54
3
4.12
57
4
3.33
5
1
4.16
29
6
3.97
73
8
4.04
8
3.91
65
8
4.05
578
40
6.9%
3.00
1
1
4.06
314
30
9.6%
5
4.27
9
6
4.50
1
4.28
47
4.75
1
4.22
182
9
4.21
99
4
4.09
28
5
4.17
54
3
4.21
45
3
4.15
93
6
4.19
18
2
4.25
1
4.81
4
3.52
8
2
9
3.33
1
L
4.17
1
4.37
13
4.23
13
1
4.20
4.14
576
27
32
3
5.6% 11.1%
4.40
5
4.35
2
4.20
1
4.31
21
4.35
2
Grand Total
4.16
111
5
4.16
511
29
4.09
531
25
4.11
237
12
3.99
207
16
4.06
151
7
4.17
88
10
3.99
340
31
4.04
67
3
3.98
134
10
4.08
2377
148
6.2%
9
5. CHANGES OVER TIME Figures 4 and 5 display the faculty‐level trends in unit evaluations between 2011 and 2015 with regard to the percentage of units classified as ‘outstanding’ (Figure 4) or as ‘needing critical attention’ (Figure 5). 2015 data is comprised of Semester 1, 2015 data only at this stage. The percentage of unit offerings classified as ‘outstanding’ (all faculties combined) increased from 12.9% in 2014 to 14.5% in 2015. Increases in the percentage of unit offerings classified as ‘outstanding’ from 2014 to 2015 are seen in three faculties. In decreasing magnitude they are Art, Design and Architecture (6.2%), Pharmacy & Pharmaceutical Sciences (4.2%) and Engineering (3.2%). The largest decreases from 2014 to 2015 were seen in Law (4.7%). The percentage of unit offerings classified as ‘needing critical attention’ (all faculties combined) continues to decrease (6.5% in 2014 and 6.2% in 2015). At the faculty level, Information Technology showed the largest decline in the percentage of units in the ‘needing critical attention’ range from 2014 to 2015 (‐3.5%). Other, signs of improvement from 2014 to 2015 are observed across all faculties with three exceptions: Law had the largest percentage increase in the number of unit offerings classified as ‘needing critical attention’ from 2014 to 2015 (6.7%) followed by Science (3.0%). 10
Figurre 4: Percentage o
of evaluated unit offerings classifieed as 'outstanding
g' by faculty, 2011
1‐20156, 7
6
7
Non‐standard unit offferings originated frrom ‘Malaysia Natio
onal Subjects’ and ‘‘Enhancement Stud
dies Program’ were excluded. 201
12, 2013 and 2014 ffigures include full yyear data, i.e. Semeester 1, Semester 2 and Summer Seme
ester data, 2015 figures include only Semester 1 data. 1
11
Figurre 5: Proportion o
of evaluated unit o
offerings classified as 'needing crittical attention' by faculty, 2011‐201
158,9 8
9
Non
n‐standard unit offeerings originated from ‘Malaysia Natio
onal Subjects’ and ‘Enhancement Studies Program’ were excluded. 201
12, 2013 and 2014 ffigures include full yyear data, i.e. Semeester 1, Semester 2 and Summer Seme
ester data, 2015 figures include only Semester 1 data. 1
12