STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT OF UNIT EVALUATION RESULTS SEMESTER 2, 2014 21 January 2015 INTRODUCTION This report summarises the semester 2 2014 responses to SETU’s1 ‘overall satisfaction’ item. The overall item required students to rate the statement; ‘Overall I was satisfied with the quality of this unit’, with either ‘Strongly Agree’ (5), ‘Agree’ (4), ‘Neutral’ (3), ‘Disagree’ (2)’ or ‘Strongly Disagree’ (1). From these data, a ‘median’2 score was calculated for each unit offering3. Unit offerings were then classified into one of the four groups using the following median score cut‐offs:  Outstanding: ≥4.70  Meeting aspirations: 3.60 – 4.69  Needing improvement: 3.01 – 3.59  Needing critical attention: ≤3.00 FINDINGS Overall satisfaction data was collected for 2,648 degree level4 unit offerings in Semester 2, 2014. Note that this figure only includes offerings with overall satisfaction responses. 1. FACULTY VARIATION Table 1 and Figure 1 both show that the vast majority of unit offerings were categorised as either ‘meeting aspirations’ (72.3%) or ‘outstanding’ (13.1%) and the highest performing faculty was Law (27.6% of units ‘outstanding’). Notwithstanding high overall satisfaction levels, 6.3% of unit offerings were classified as ‘needing critical attention’. Among all faculties, Information Technology (8.6%) and Medicine Nursing & Health Sciences (8.6%) had a relatively high percentage of units falling into this category. Table 2 and Figure 2 show the results for unit offerings with 5 or more responses. Results for unit offerings with less than 5 responses are shown in Table 3 and Figure 3. 1
Further detail on the SETU can be found at: http://www.opq.monash.edu.au/us/surveys/setu/index.html Further detail on how the median is calculated can be found at: http://www.opq.monash.edu.au/us/surveys/setu/setu_median_calculation.pdf 3
A unit offering is defined here as a unit at a specific location, in a specific mode (e.g. on campus/off campus). It does not take into account instances where the unit offering may run in different calendar types within the same survey period or different unit classes within the same mode ie. DAY, EVENING (both ON campus mode). Also on‐shore non‐campus locations are grouped into ‘Other Australian Locations’ and off‐shore non‐campus locations are grouped into ‘Other Overseas locations’. 4
Evaluations were also undertaken for Monash College diploma, Malaysia National Subject units and Sth Africa Foundation Program units but these are not included in this report. 2
1
University Planning and Statistics
Table 1: No. of unit offerings falling into each "traffic light" category by faculty, Semester 2 2014 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.59)
Meeting Aspirations (3.6‐4.69)
Outstanding (≥4.7)
Total
6
14
89
22
131
Arts
45
36
440
106
627
Business & Economics
29
50
388
64
531
Education
17
21
195
39
272
Engineering
13
20
161
12
206
Information Technology
15
15
129
16
175
Law
3
6
67
29
105
32
32
270
38
372
Pharmacy & Pharm Science
3
13
53
9
78
Science
5
12
122
12
151
168
219
1,914
347
2,648
Med Nursing & Health Sci
Grand Total
Table 2: No. of unit offerings with 5 or more responses falling into each "traffic Light" category by faculty, Semester 2 2014 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.59)
Meeting Aspirations (3.6‐4.69)
Outstanding (≥4.7)
Total
1
15
15
10
8
6
3
22
2
3
85
11
23
41
19
18
12
5
26
13
10
178
75
358
340
171
151
110
53
228
47
110
1,643
11
65
37
22
9
7
19
17
5
7
199
98
461
433
222
186
135
80
293
67
130
2,105
Table 3: No. of unit offerings with less than 5 responses falling into each "traffic Light" category by faculty, Semester 2 2014 Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical Attention
(≤3.0)
Needing Improvement
(3.01‐3.59)
Meeting Aspirations (3.6‐4.69)
Outstanding (≥4.7)
Total
5
30
14
7
5
9
0
10
1
2
83
3
13
9
2
2
3
1
6
0
2
41
14
82
48
24
10
19
14
42
6
12
271
11
41
27
17
3
9
10
21
4
5
148
33
166
98
50
20
40
25
79
11
21
543
2
Figurre 1: Proportion o
of unit offerings faalling into each ‘trraffic light catego
ory’ by faculty, Sem
mester 2 2014 3
Figurre 2: Proportion o
of unit offerings w
with 5 or more ressponses falling intto each ‘traffic ligh
ht category’ by faculty, Semester 2
2 2014 4
Figurre 3: Proportion o
of unit offerings w
with less than 5 reesponses falling into each ‘traffic lig
ght category’ by faaculty, Semester 2
2 2014 5
2. CAMPUS/FACULTY VARIATION Table 4 displays average median satisfaction (overall) scores across unit offerings, with data separated by campus and faculty. No campus/faculty groups were in the ‘needing critical attention’ range. Art, Design and Architecture at Clayton (2 unit offering with 8 and 1 respondents), Science at ‘Other Australian Locations’ (1 unit offering with 1 respondent), and Information Technology at China South East University (2 unit offerings and over 30 responses for each) were in the ‘Outstanding’ range. The Faculty of Engineering at the Gippsland campus (3.56) had an average median in the ‘needing improvement’ range. Four unit offerings (with 2, 4, 10 and 13 respondents) were in the ‘needing critical attention’ range. Examination of campuses alone (far right column) indicates a relatively high percentage of units with the critical attention flag at Gippsland (15.7%). For all other campuses, less than 10% of units fell into the critical range. Examining Faculty alone (third bottom row of Table 4), shows the average median for all faculties is in the ‘meeting aspirations’ range. 3. UNIT MODE VARIATION Table 5 displays average median satisfaction broken down by faculty and mode (off‐campus, on‐campus and on/off‐campus). All faculty/mode groups fell into the ‘meeting aspirations’ category. Collapsing across faculty groups (bottom 4 rows of Table 5) shows that the average median scores from all three modes (on/off, off and on‐campus) are in the ‘meeting aspirations’ range. However, 12.3% of ‘off‐campus’ and ‘on/off‐campus’ unit offerings fell into the ‘needing critical attention’ range. 4. UNIT LEVEL VARIATION Table 6 shows average median satisfaction split by faculty and unit level. The faculty of Information Technology with level 9 units (4.78) and the faculty of Science with level 4 units (4.88) achieved the ‘Outstanding’ classification for their average median score. 36 responses formed the result for Information Technology and 5 responses formed the result for Science. Unit offerings falling into the ‘needing attention’ range include level 4 units in the faculty of Pharmacy and Pharmaceutical Sciences (3.59). Collapsing results across faculty (bottom 4 rows of Table 6) shows that the average median score from each unit level is in the ‘Meeting Aspirations’ range. The percentage of units falling in the ‘needing critical attention’ range was under 10% for all levels. 6
Table 4: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and location, Semester 2 2014 Location
Berwick
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Caulfield
Av. Median
No. of unit offerings
No. needing critical attention
Clayton
Av. Median
No. of unit offerings
No. needing critical attention
Gippsland Av. Median
No. of unit offerings
No. needing critical attention
Parkville
Av. Median
No. of unit offerings
No. needing critical attention
Peninsula Av. Median
No. of unit offerings
No. needing critical attention
South Africa Av. Median
No. of unit offerings
No. needing critical attention
Malaysia
Av. Median
No. of unit offerings
No. needing critical attention
Prato
Av. Median
No. of unit offerings
No. needing critical attention
Online
Av. Median
No. of unit offerings
No. needing critical attention
Os China
Av. Median
South East No. of unit offerings
University No. needing critical attention
Other Av. Median
Australian No. of unit offerings
Locations No. needing critical attention
Other Av. Median
Offshore No. of unit offerings
Locations No. needing critical attention
Overall Av. Median
Total no. of unit offerings
Total no. needing critical attention
Art, Des & Arch
4.12
119
5
4.85
2
4.13
10
1
Arts Bus Eco Educ Eng
4.08
4.14 4.04
18
28
33
1
2
4.18
4.06
123
217
8
14
4.26
4.03 4.07 3.97
296
95 132 130
8
3
9
8
3.81
4.02 3.95 3.56
84
56
19
15
17
5
3
4
3.92
62
5
3.85
27
5
4.40
15
1
3.86
14
3
4.14
39
2
4.03
79
2
4.07
59
2
3.98
61
1
Pharm & Info Pharm Tech
Law MNHS
Sci
4.28
3.66
1
16
4
4.08
3.99
84
51
4
6
3.86 4.20 4.05
34
40 121
1
1
5
3.95
3.80
14
13
3
2
4.04
61
2
3.93
61
5
3.94
3.80
16
14
2
1
3.71 4.25 3.91
3.85
24
1
20
17
5
1
1
Sci
4.08
92
3
4.13
29
2
3.89
29
4.13
2
4.00
3
4.38
2
4.14
131
6
4.13
627
45
4.82
2
4.35
10
1
4.33
19
4.05
531
29
4.09
272
17
3.94
206
13
3.97
175
15
4.40
64
2
4.03
74
8
4.32
105
3
3.98
372
32
5.00
1
4.00
78
3
4.06
151
5
% needing critical Total attention
4.02
96
7
7.3%
4.09
594
37
6.2%
4.11
942
38
4.0%
3.90
230
36
15.7%
4.04
61
2
3.3%
3.99
134
10
7.5%
3.98
131
10
7.6%
3.93
258
15
5.8%
4.30
25
2
8.0%
4.13
2
0.0%
4.33
5
0.0%
4.22
151
11
7.3%
4.33
19
0.0%
4.06
2648
168
6.3% 7
Table 5: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and Mode, Semester 2 2014 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Off Campus On Campus
4.02
103
10
4.16
22
1
4.03
10
1
3.98
405
4.14
131
6
4.16
513
31
4.07
479
19
4.09
164
7
3.97
191
11
4.02
152
9
4.32
105
3
3.96
254
19
3.94
56
2
4.06
141
4
4.08
2186
50
12.3%
111
5.1%
3.98
106
13
3.87
52
10
4.09
74
7
3.64
15
2
3.66
23
6
On/Off Campus
Grand Total
4.03
57
4.14
131
6
4.13
627
45
4.05
531
29
4.09
272
17
3.94
206
13
3.97
175
15
4.32
105
3
3.98
372
32
4.00
78
3
4.06
151
5
4.06
2648
7
12.3%
168
6.3%
3.95
8
1
4.09
34
3
3.94
15
3
8
Table 6: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical attention’ by faculty and unit level, Semester 2 2014 Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
1
4.05
28
1
4.14
116
1
3.97
66
1
4.11
30
2
4.25
36
4.01
22
3.90
22
1
3.78
2
3.96
50
2
4.03
35
2
4.16
4
3
4.17
33
3
4.14
214
19
4.07
151
4
4.23
18
1
3.87
53
6
3.93
43
5
4.18
9
3.95
45
2
3.90
12
3.86
55
7
3.92
16
3.91
77
6
4.10
20
4.00
32
2
4.03
375
8
2.1%
4.07
52
1
4.04
588
34
5.8%
4.08
66
2
4.06
684
46
6.7%
4.11
196
15
3.92
113
7
4.23
31
4
4.16
24
0
4.11
68
7
3.83
30
7
4.10
99
8
4.00
68
4
3.83
7
1
4.24
25
1
4.01
68
5
3.59
8
2
4.88
1
5
3.80
8
2
4.23
33
3
4.18
167
10
3.99
94
8
3.79
13
1
3.99
65
6
4.40
65
2
4.01
108
12
4.18
21
1
6
3.75
1
4.05
398
35
8.8%
4.11
574
45
7.8%
4.35
18
9
3.78
1
L
4.19
4
4.15
2
4.78
1
4.51
11
4.24
4
4.20
4
3.75
1
4.18
7
4.20
4
Grand Total
4.14
131
6
4.13
627
45
4.05
531
29
4.09
272
17
3.94
206
13
3.97
175
15
4.32
105
3
3.98
372
32
4.00
78
3
4.06
151
5
4.06
2648
168
6.3%
9
5. CHANGES OVER TIME Figures 4 and 5 display the faculty‐level trends in unit evaluations between 2011 and 2014 with regard to the percentage of units classified as ‘outstanding’ (Figure 4) or as ‘needing critical attention’ (Figure 5). The percentage of unit offerings classified as ‘outstanding’ (all faculties combined) decreased from 13.4% in 2013 to 13.1% in 2014. Increases in the percentage of unit offerings classified as ‘outstanding’ from 2013 to 2014 are seen in three faculties. In decreasing magnitude they are Pharmacy & Pharmaceutical Sciences (4.3%), Law (2.9%) and Business & Economics (1.1%). The largest decreases from 2013 to 2014 were seen in the Engineering and Art, Design & Architecture (1.7% and 1.6%, respectively). The percentage of unit offerings classified as ‘needing critical attention’ (all faculties combined) continues to decrease (8.2% in 2011; 7.7% in 2012; 6.9% in 2013; 6.3% in 2014). At the faculty level, Science showed the largest decline in the percentage of units in the ‘needing critical attention’ range from 2013 to 2014 (‐2.8%) closely followed by the Pharmacy & Pharmaceutical Sciences faculty (‐2.7%). Other, signs of improvement from 2013 to 2014 are observed across all faculties with three exceptions: Business & Economics had the largest percentage increase in the number of unit offerings classified as ‘needing critical attention’ from 2013 to 2014 (0.7%) followed by Arts (0.5%) and Information Technology (0.3%). 10
Figure 4: Percentage of evaluated unit offerings classified as 'outstanding' by faculty, 2011‐20145, 6 30%
% of unit offerings 'outstanding'
25%
20%
15%
10%
5%
0%
Pharmacy &
Medicine, Pharmaceutical Nursing &
Sciences
Health Sciences
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
2011‐FullYear
12.1%
19.2%
8.6%
12.4%
6.6%
6.8%
18.3%
8.6%
2012‐FullYear
13.0%
18.2%
9.1%
11.8%
4.4%
9.3%
22.8%
9.9%
2013‐FullYear
18.4%
17.3%
11.0%
14.9%
7.5%
10.0%
24.7%
2014‐FullYear
16.8%
16.9%
12.1%
14.3%
5.8%
9.1%
27.6%
Science
Grand Total
8.2%
5.6%
12.4%
9.8%
5.5%
12.3%
11.2%
7.2%
8.4%
13.4%
10.2%
11.5%
7.9%
13.1%
5
Adjustment 1: 2011 data has been adjusted to include Summer Semester data. Adjustment 2: Non‐standard unit offerings originated from ‘Malaysia National Subjects’ and ‘Enhancement Studies Program’ were excluded. 6
2012, 2013 and 2014 figures include full year data, i.e. Semester 1, Semester 2 and Summer Semester data. 11
Figure 5: Proportion of evaluated unit offerings classified as 'needing critical attention' by faculty, 2011‐20145, 6 25%
% of unit offerings 'needing critical attention'
20%
15%
10%
5%
0%
Medicine, Pharmacy &
Nursing &
Pharmaceutical Health Sciences
Sciences
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
2011‐FullYear
6.2%
8.6%
5.2%
10.5%
8.8%
8.9%
4.1%
11.4%
11.2%
6.9%
8.2%
2012‐FullYear
8.3%
7.9%
6.0%
11.3%
5.7%
7.4%
4.7%
10.7%
8.4%
5.2%
7.7%
2013‐FullYear
6.3%
6.7%
4.8%
7.8%
7.3%
8.3%
3.5%
11.1%
6.5%
6.1%
6.9%
2014‐FullYear
4.6%
7.2%
5.5%
6.3%
6.3%
8.6%
2.9%
8.6%
3.8%
3.3%
6.3%
12
Science
Grand Total