STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT

advertisement
STUDENT EVALUATION OF TEACHING AND UNITS (SETU) SUMMARY REPORT
OF UNIT EVALUATION RESULTS
SEMESTER 2, 2013
19 December 2013
INTRODUCTION
The Student Evaluation of Teaching and Units (SETU) is an integrated version of two previous surveys; the unit
evaluation and MonQueST. SETU includes the original unit evaluation questions and four new teaching
questions.1
This report summarises the semester 2 2013 responses to SETU’s ‘overall satisfaction’ item. The overall item
required students to rate the statement; ‘Overall I was satisfied with the quality of this unit’, with either
‘Strongly Agree’ (5), ‘Agree’ (4), ‘Neutral’ (3), ‘Disagree’ (2)’ or ‘Strongly Disagree’ (1).
From these data, a ‘median’2 score was calculated for each unit offering3. Unit offerings were then classified into
one of the four groups using the following median score cut-offs:
 Outstanding: ≥4.70
 Meeting aspirations: 3.60 – 4.69
 Needing improvement: 3.01 – 3.59
 Needing critical attention: ≤3.00
FINDINGS
Overall satisfaction data was collected for 2,654 degree level4 unit offerings in Semester 2, 2013. Note that this
figure only includes offerings with overall satisfaction responses.
1. FACULTY VARIATION
Table 1 and Figure 1 both show that the vast majority of unit offerings were categorised as either ‘meeting
aspirations’ (72.0%) or ‘outstanding’ (13.4%) and the highest performing faculty was Law (23.9% of units
‘outstanding’).
Notwithstanding high overall satisfaction levels, 6.9% of unit offerings were classified as ‘needing critical
attention’. Among all faculties, Information Technology (9.7%) and Medicine Nursing & Health Sciences (9.3%)
had a relatively high percentage of units falling into this category.
Table 2 and Figure 2 show the results for unit offerings with 5 or more responses. Results for unit offerings with
less than 5 responses are in Table 3 and Figure 3.
1
Further detail on the SETU can be found at: http://www.opq.monash.edu.au/us/surveys/setu/index.html
Further detail on how the median is calculated can be found at:
http://www.opq.monash.edu.au/us/surveys/setu/setu_median_calculation.pdf
3
A unit offering is defined here as a unit at a specific location, in a specific mode (e.g. on campus/off campus). It does not
take into account instances where the unit offering may run in different calendar types within the same survey period or
different unit classes within the same mode ie. DAY, EVENING (both ON campus mode). Also on-shore non-campus
locations are grouped into ‘Other Australian Locations’ and off-shore non-campus locations are grouped into ‘Other
Overseas locations’.
4
Evaluations were also undertaken for Monash College diploma, Malaysia National Subject units and Sth Africa Foundation
Program units but these are not included in this report.
2
1
Office of Planning and Quality – University Planning and Statistics
Table 1: No. of unit offerings falling into each "traffic light" category by faculty, Semester 2 2013
Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical
Attention
(≤3.0)
Needing
Improvement
(3.01-3.59)
Meeting
Aspirations (3.64.69)
Outstanding
(≥4.7)
Total
7
43
27
19
11
17
5
35
6
13
183
8
42
42
13
24
14
2
38
16
6
205
70
497
408
158
147
122
82
259
48
119
1,910
18
120
60
29
15
22
28
43
5
16
356
103
702
537
219
197
175
117
375
75
154
2,654
Table 2: No. of unit offerings with 5 or more responses falling into each "traffic Light" Category by Faculty,
Semester 2 2013
Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical
Attention
(≤3.0)
Needing
Improvement
(3.01-3.59)
Meeting
Aspirations (3.64.69)
Outstanding
(≥4.7)
Total
6
12
10
13
9
8
2
22
3
10
95
7
28
32
11
22
12
2
24
13
6
157
64
409
365
141
140
101
68
213
41
111
1,653
10
60
35
23
8
11
15
20
3
10
195
87
509
442
188
179
132
87
279
60
137
2,100
Table 3: No. of unit offerings with less than 5 responses falling into each "traffic Light" Category by faculty,
Semester 2 2013
Median Response Band for Overall Satisfaction Item
Owning Faculty
Art Des & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Grand Total
Needing Critical
Attention
(≤3.0)
Needing
Improvement
(3.01-3.59)
Meeting
Aspirations (3.64.69)
Outstanding
(≥4.7)
Total
1
31
17
6
2
9
3
13
3
3
88
1
14
10
2
2
2
0
14
3
0
48
6
88
43
17
7
21
14
46
7
8
257
8
60
25
6
7
11
13
23
2
6
161
16
193
95
31
18
43
30
96
15
17
554
2
Figure 1: Proportion of unit offerings falling into each ‘traffic light category’ by faculty, Semester 2 2013
3
Figure 2: Proportion of unit offerings with 5 or more responses falling into each ‘traffic light category’ by faculty, Semester 2 2013
4
Figure 3: Proportion of unit offerings with less than 5 responses falling into each ‘traffic light category’ by faculty, Semester 2 2013
5
2. CAMPUS/FACULTY VARIATION
Table 4 displays average median satisfaction (overall) scores across unit offerings, with data separated by
campus and faculty. There were no campus/faculty groups which had average median scores in the ‘needing
critical attention’ range. The only campus/faculty group reached ‘Outstanding’ criteria for the average median
score was Arts at ‘Other Australian Locations’ (5.00), although it had only one unit with one response count. It
also noted that the average median score for Information Technology at Malaysia (3.58) fell into ‘needing
improvement’ range, with 5 of the 24 units reached ‘needing critical attention’ range.
Examining Campus alone (far right column) indicates a relatively high percentage of units with the critical
attention flag at Parkville (10.2%) and Gippsland (15.0%). For all other campuses, less than 10% of units fell into
the critical range. Examining Faculty alone (third bottom row of table 4), shows the average median for all
faculties is in the ‘meeting aspirations’ range.
3. UNIT MODE VARIATION
Table 5 displays average median satisfaction broken down by faculty and mode (on-campus, off-campus, or
on/off campus). The majority of faculty/mode groups fell into the ‘meeting aspirations’ category. The ‘off
campus’ for Information Technology group (3.56) was the exception, falling into the ‘needing improvement’
category.
Collapsing across faculty groups (bottom 4 rows of table 5) shows that average median scores from all three
modes (on, off, on/off-campus) are in the ‘meeting aspirations’ range. However, ‘off campus’ had a relatively
higher percentage of units falling into ‘needing critical attention’ range (15.6%) than ‘on campus’ (5.4%) and
‘on/off campus’ (8.3%).
4. UNIT LEVEL VARIATION
Table 6 shows average median satisfaction split by faculty and unit level. The average median satisfaction with
level 6 units offered by the faculty of Business & Economics was well into the ‘outstanding’ range (5.00), with
eight responses across the two units forming the result. In addition, level 4 in Science may need attention for
the falling of average median score (3.55) into ‘needing improvement’ category.
Collapsing across faculty (bottom 4 rows of table 6), it shows the average median score from each unit level is in
the ‘Meeting Aspirations’ range, and all levels obtained a percentage of units in ‘needing critical attention’ at or
below 10%.
6
Table 4: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and location, Semester 2 2013
Location
Berwick
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Caulfield
Av. Median
No. of unit offerings
No. needing critical attention
Clayton
Av. Median
No. of unit offerings
No. needing critical attention
Gippsland Av. Median
No. of unit offerings
No. needing critical attention
Parkville
Av. Median
No. of unit offerings
No. needing critical attention
Peninsula Av. Median
No. of unit offerings
No. needing critical attention
South Africa Av. Median
No. of unit offerings
No. needing critical attention
Malaysia
Av. Median
No. of unit offerings
No. needing critical attention
Prato
Av. Median
No. of unit offerings
No. needing critical attention
Os China
Av. Median
South East No. of unit offerings
University No. needing critical attention
Other
Av. Median
Australian No. of unit offerings
Locations No. needing critical attention
Other
Av. Median
Offshore
No. of unit offerings
Locations No. needing critical attention
Overall Av. Median
Total no. of unit offerings
Total no. needing critical attention
Art,
Des &
Arch
Arts
4.19
25
1
4.09 4.18
97 150
7
10
4.29
327
6
3.86
96
15
4.33
1
4.61
6
3.98
62
7
3.84
31
4
4.46
9
Bus
Eco Educ Eng
3.99 4.02
37
14
3
2
4.10
203
6
4.05 4.01 4.00
95 106 124
2
11
6
3.96 4.00 3.74
69
32
16
9
3
3
4.06
28
4
4.17
32
1
4.00
73
2
Info
Tech
4.18
76
3
3.93
35
3
3.83
22
6
4.08
47
2
4.03
16
4.13 3.91
3
57
2
Pharm
&
MNH Pharm
Law
S
Sci
3.84
17
1
3.94
53
7
4.11 4.03
42 117
4
5
3.99
26
3
3.90
59
6
3.96
65
8
4.13
12
3.58 4.64
24
8
5
3.97
22
3.70
16
Sci
4.06
93
7
3.96
32
5
3.92
28
1
4.62
2
5.00
1
4.12
103
7
4.16
702
43
4.05
537
27
4.23
17
1
4.04 3.95
219 197
19
11
4.31
67
1
3.72
63
11
4.00 4.26
175 117
17
5
3.94
375
35
4.50
1
3.86
75
6
4.02
154
13
%
needing
critical
Total attention
4.02
93
7
7.5%
4.11
579
33
5.7%
4.12
939
44
4.7%
3.91
293
44
15.0%
3.90
59
6
10.2%
4.02
141
14
9.9%
4.05
122
8
6.6%
3.92
262
14
5.3%
4.52
15
0.0%
4.62
2
0.0%
4.04
132
12
9.1%
4.23
17
1
5.9%
4.06
2654
183
6.9%
7
Table 5: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and Mode, Semester 2 2013
Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Off Campus On Campus
4.12
103
7
3.95
4.19
93
598
14
29
3.73
4.09
54
483
10
17
3.95
4.09
59
149
6
11
3.87
3.96
17
180
1
10
3.56
4.05
18
157
7
10
4.26
117
5
3.88
3.95
105
256
15
19
3.99
3.81
22
53
3
3
3.81
4.03
10
144
3
10
3.88
4.09
378
2240
59
15.6%
121
5.4%
On/Off
Campus
4.13
36
Grand
Total
4.12
103
7
4.16
702
43
4.05
537
27
4.04
219
19
3.95
197
11
4.00
175
17
4.26
117
5
3.94
375
35
3.86
75
6
4.02
154
13
4.06
2654
3
8.3%
183
6.9%
4.24
11
3.92
11
2
4.22
14
1
8
Table 6: Average ‘Overall Satisfaction’ median score, and number and % of unit offerings ‘needing critical
attention’ by faculty and unit level, Semester 2 2013
Owning Faculty
Art, Design & Architecture
Arts
Business & Economics
Education
Engineering
Information Technology
Law
Med Nursing & Health Sci
Pharmacy & Pharm Science
Science
Overall Av. Median
Total No. of unit offerings
No. needing critical attention
% needing critical attention
Measure
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
Av. Median
No. of unit offerings
No. needing critical attention
1
4.08
26
2
4.10
131
3
3.97
75
4
4.08
17
3.85
23
2
3.98
24
2
3.82
2
4.03
46
3.79
12
4.10
34
1
4.03
390
14
3.6%
2
4.03
29
4
4.16
223
14
3.94
129
8
4.05
18
1
3.85
46
3
3.92
40
3
3.99
3
3.73
58
9
3.80
16
1
3.94
53
6
4.00
615
49
8.0%
3
4.27
25
1
4.18
238
15
4.08
160
5
4.15
17
1
3.97
52
4
4.01
44
4
4.11
10
1
3.90
70
7
3.86
18
1
4.07
62
4
4.08
696
43
6.2%
4
5
4.14 4.21
18
3
0
4.23 4.11
73
35
6
5
4.40 4.13
25
77
2
4
4.06 3.98
108
59
9
8
4.05 3.97
62
14
2
3.82 4.04
9
41
2
4
4.22 4.32
30
72
2
2
4.11 3.87
62
111
3
16
3.71 3.81
7
7
1
2
3.55
5
2
4.12 4.05
399
419
29
41
7.3% 9.8%
6
3.50
1
0
3.75
2
9
4.14
1
5.00
2
4.07
69
4
4.31
2
4.11
15
2
4.35
15
0
4.26
5
L
4.11
8
4.08
15
1
4.31
22
4.09
105
7
6.7%
4.11
8
Grand
Total
4.12
103
7
4.16
702
43
4.05
537
27
4.04
219
19
3.95
197
11
4.00
175
17
4.26
117
5
3.94
375
35
3.86
75
6
4.02
154
13
4.06
2654
183
6.9%
9
5.
CHANGES OVER TIME
Figures 4 and 5 display the faculty-level trends in unit evaluations between 2010 and 2013, with regard to the
percentage of units classified as ‘outstanding’ (Figure 4) or as ‘needing critical attention’ (Figure 5).
It is worth noting that the average median satisfaction scores in Figure 4 and 5 have been adjusted with regard
to the following changes:
 Adding of Summer Semester data in 2010 and 2011 to reflect the true full year results
 Removing non-standard unit offerings originated from Malaysia National Subjects and Enhancement
Studies Program.
The percentage of unit offerings classified as ‘outstanding’ (all faculties combined) has increased gradually since
2010 (Grand Total column: 11.4% in 2010; 12.4% in 2011; 12.3% in 2012; 13.4% in 2013). At the faculty level the
Art Deigns & Architecture shows a sharp improvement (18.4% in 2013 following 13% in 2012). Decreases are
noted in 2013 results in Arts (19.3% in 2010 dropping to 17.3% in 2013) and Pharmacy & Pharmaceutical
Sciences (9.8% in 2012 decreasing to 7.2% in 2013). In addition, improvement is also seen in all other faculties in
2013.
The percentage of unit offerings classified as ‘needing critical attention’ (all faculties combined) has also
gradually decreased (8.3% in 2010, 8.2% in 2011; 7.8% in 2012, 6.9% in 2013). At faculty level, Arts has shown a
graduate decline in the percentage of units in the ‘needing critical attention’ range since 2010 (9.4% in 2010 to
6.6% in 2013). In addition, signs of improvement are observed at Art Design & Architecture, Business &
Economics, Education, Law and Pharmacy & Pharmaceutical Sciences, where the percentage of units in the
‘needing critical attention’ range has decreased in 2013. On the other hand, the percentage of units in critical
range has increased in four faculties: Engineering, Information Technology and Medicine, Nursing and Health
Sciences. The percentage of units in Medicine, Nursing and Health Sciences faculties has exceeded 10% every
year since 2010.
10
Figure 4: Percentage of evaluated unit offerings classified as 'outstanding' by faculty, 2010-20135, 6
5
Adjustment 1: 2010 to 2011 data has been adjusted to include Summer Semester data.
Adjustment 2: Non-standard unit offerings originated from ‘Malaysia National Subjects’ and ‘Enhancement Studies Program’ were excluded.
6
2012 and 2013 figures include full year data, i.e. Semester 1, Semester 2 and Summer Semester data.
11
Figure 5: Proportion of evaluated unit offerings classified as 'needing critical attention' by faculty, 2010-20135, 6
12
Download