iLearn Evaluation Report
ILEARN STAFF EXPERIENCE SURVEY 2014
Produced by Jayde Cahir (Chief Investigator), Helen Carter (Co-Investigator), Cathy Rytmeister (Statistical analysis and advice),
Lucy Arthur (Faculty iLearn Support Coordination) and Rebecca Ritchie (Central iLearn Support Coordination).
May 2015
1
TABLE OF CONTENTS
Table of Contents ...................................................................................................................................................................... 2
List of Figures ............................................................................................................................................................................ 3
List of Tables ............................................................................................................................................................................. 3
iLearn Staff Experience Survey ................................................................................................................................................ 4
Results and Discussion ............................................................................................................................................................. 4
Demographic Details................................................................................................................................................................. 4
Technologies used to access iLearn: Staff Survey Results ....................................................................................................... 4
Staff Satisfaction with iLearn as a support for learning and teaching .................................................................................... 5
Staff Skills and Confidence Levels ............................................................................................................................................ 7
Staff Confidence Levels ............................................................................................................................................................. 7
Changing Practice ................................................................................................................................................................... 10
Staff Experience with iLearn .................................................................................................................................................. 12
Statistics on the use of iLearn Functions: data from the iLearn system ............................................................................... 12
Scaled responses on the perceived value of iLearn Functions: data from the surveys ........................................................ 14
iLearn Functions for organising students learning ............................................................................................................... 14
iLearn Functions for engaging students with learning activities .......................................................................................... 18
Staff Satisfaction with Services and Support ......................................................................................................................... 25
Satisfaction levels with support services for iLearn and iTeach ........................................................................................... 25
Staff Satisfaction with iTeach ................................................................................................................................................. 25
Staff Satisfaction with iTeach Support ................................................................................................................................... 25
Staff Satisfaction with UNIT Guides ...................................................................................................................................... 26
Staff Satisfaction with iTeach Training and Support............................................................................................................. 26
Staff Satisfaction with OneHelp for iLearn support .............................................................................................................. 26
Support services for iLearn provided by the Faculties and the LTC ..................................................................................... 27
Staff Satisfaction with Online Resources ............................................................................................................................... 27
Staff Satisfaction with the iLearn Drop-in Clinics ................................................................................................................. 29
Staff Satisfaction with iLearn training workshops ................................................................................................................ 29
Staff Satisfaction with Faculty-based iLearn Support ...........................................................................................................30
iLearn Status Page................................................................................................................................................................... 31
Appendices .............................................................................................................................................................................. 32
Appendix 1: iLearn Staff Experience Survey Questions ........................................................................................................ 32
Appendix 2: Survey Advertisements ...................................................................................................................................... 37
Appendix 3: Improvements to the iLearn platform ..............................................................................................................38
Appendix 4: Statistical Analysis of Technologies Staff used to access iLearn ...................................................................... 39
Frequency of using iLearn with… ...................................................................................................................................... 39
2.1 Using iLearn with…...................................................................................................................................................... 40
Using iLearn with… ............................................................................................................................................................ 41
Appendix 5: Statistical Analysis of Staff Skills, Confidence Levels and Satisfaction ........................................................... 42
Overall teaching experience with iLearn ........................................................................................................................... 42
Overall teaching experience with iLearn ........................................................................................................................... 43
Overall teaching experience with iLearn ...........................................................................................................................44
Appendix 6: Statistical Analysis of Staff Experience with iLearn .........................................................................................46
Organising students' learning. ...........................................................................................................................................46
Engaging students in learning activities ............................................................................................................................46
The iLearn platform was effective in meeting my needs… ............................................................................................... 47
2
with respect to organising students' learning.................................................................................................................... 47
for engaging my students in learning activities................................................................................................................. 47
The iLearn platform was effective in meeting my needs… ...............................................................................................48
with respect to organising students' learning....................................................................................................................48
for engaging my students in learning activities.................................................................................................................48
Appendix 7: Statistical Analysis of Staff Satisfaction with Services and Support ................................................................49
Satisfaction with support and services ..............................................................................................................................49
Satisfaction with support and services ..............................................................................................................................50
Satisfaction with support and services .............................................................................................................................. 51
LIST OF FIGURES
Figure 1: Satisfaction with iLearn as an online support for unit content ..................................................................... 5
Figure 2: Satisfaction with iLearn as a support for organising my teaching ................................................................ 6
Figure 3: Satisfaction with iLearn as a support for interaction and communication with students ........................... 6
Figure 4: Satisfaction with the way iLearn supports student engagement .................................................................. 7
Figure 5: Development of skills to effectively use iLearn..............................................................................................8
Figure 6: Staff confidence when using iLearn ...............................................................................................................8
Figure 7: Staff confidence when using Echo360 ........................................................................................................... 9
Figure 8: Staff confidence when using iTeach ............................................................................................................... 9
Figure 9: Staff confidence when using iTeach to create Unit Guides ......................................................................... 10
Figure 10: Staff confidence when using iTeach to map student learning outcomes .................................................. 10
Figure 11: Change in teaching practice ......................................................................................................................... 11
Figure 12: Change in future teaching practice .............................................................................................................. 11
Figure 13: Staff Experience of iLearn / Moodle .......................................................................................................... 12
Figure 14: Undergraduate and Postgraduate Units used of iLearn Functions .......................................................... 13
Figure 15: iLearn Function Announcements ............................................................................................................... 15
Figure 16: iLearn Function Online Assignment Submission ...................................................................................... 15
Figure 17: iLearn Function Turnitin ............................................................................................................................ 16
Figure 18: iLearn Function GradeMark ....................................................................................................................... 17
Figure 19: iLearn Function Grades .............................................................................................................................. 17
Figure 20: iLearn Function Calendar .......................................................................................................................... 18
Figure 21: iLearn Function Discussion Forums .......................................................................................................... 19
Figure 22: iLearn Function Quizzes ............................................................................................................................ 20
Figure 23: iLearn Function Dialogue Module ............................................................................................................. 21
Figure 24: iLearn Function Database .......................................................................................................................... 21
Figure 25: iLearn Function Workshop Tool ................................................................................................................ 22
Figure 26: iLearn Function Blog .................................................................................................................................. 22
Figure 27: iLearn Function Wiki .................................................................................................................................. 23
Figure 28: iLearn Function Chat.................................................................................................................................. 24
Figure 29: iLearn Function Twitter Feeds ................................................................................................................... 24
Figure 30: Staff satisfaction with the automatic enrolment of students facilitated by iTeach .................................. 25
Figure 31: Staff satisfaction with OneHelp for resolution of issues relating to iTeach .............................................. 26
Figure 32: Staff satisfaction with OneHelp for resolution of issues relating to iLearn functions ............................. 27
Figure 33: Staff satisfaction with the ease of locating online "self-help" resources ..................................................28
Figure 34: Staff satisfaction with the effectiveness of the online "self-help" resources ............................................28
Figure 35: Staff Satisfaction with the iLearn Drop-in Clinics ..................................................................................... 29
Figure 36: Staff Satisfaction with the iLearn Training Workshops ............................................................................30
Figure 37: Staff Satisfaction with Faculty iLearn Support ..........................................................................................30
LIST OF TABLES
Table 1: The iLearn Staff Experience Survey Demographic Information for 2012 - 2014 ......................................................... 4
Table 2: Technologies used to access iLearn according to the 2014 staff survey ........................................................................ 5
Table 3: Staff Satisfaction with iLearn as a support for learning and teaching .......................................................................... 5
Table 4: Staff Confidence levels in using iLearn as a support for learning and teaching ........................................................... 7
Table 5: Staff members changing practice by using iLearn as a support for learning and teaching ........................................ 10
Table 6: The percentage of units that used the various iLearn functions ................................................................................. 13
Table 7: iLearn functions that facilitate organising students learning ...................................................................................... 14
Table 8: iLearn functions that facilitate engagement with learning activities .......................................................................... 19
Table 9: Staff Satisfaction with support services for iLearn and iTeach ................................................................................... 25
Table 10: Staff Satisfaction with support services provided by the Faculties and the LTC ...................................................... 27
3
ILEARN STAFF EXPERIENCE SURVEY
The 2014 iLearn Staff and Student Experience Surveys are part of a wider Learning and Teaching Centre (LTC)
quality enhancement strategy for the University’s Learning Management System (LMS) and learning technologies
more generally. In 2014, iLearn had moved into the second year of its operational phase thus the focus of the 2014
survey was to evaluate staff satisfaction with:

the functionality of iLearn from a teaching practice and technical perspective;

the University’s services and support for iLearn; and

to document trends in use to inform future needs in terms of functionality and support.
This focus aligns with the findings and recommendations of the iLearn Implementation 2012: Evaluation Report
and the iLearn Evaluation Report 2013 in order to capture the needs and expectations of staff and to continue the
ongoing quality enhancement cycle.
RESULTS AND DISCUSSION
This evaluation study examined the second year of the operationalisation phrase of the iLearn platform. Data for
this evaluation study was compiled through the iLearn Staff Experience Survey (Appendix 1). Invitations to
complete the iLearn Staff Experience Survey were sent over a three-week period in Session 2, 2014, between the 8th
and 29th October. The purpose of this survey was to:

monitor engagement with iLearn;

evaluate satisfaction with services and support; and

map changes in learning and teaching across the University.
The survey was sent only to members of staff using iLearn in Session 2, 2014. This approach to selecting the sample
aligned with the creation of iLearn Staff Experience Survey in 2012 and 2013. As in previous years, the survey was
delivered through University evaluation system, Teaching Evaluation for Development Service (TEDS), using
EvaSys.
DEMOGRAPHIC DETAILS
In Session 2, 2014 invitations were sent to 1,713 iLearn users and a total of 210 responses were received, which
represented 12.3% of the iLearn user population at that time. The response rate remains consistent with last year
and provides adequate insight into the environment in which the technology is used.
iLearn Staff Experience
Survey
Faculty of Arts
Faculty of Science
Faculty of Human Science
Faculty
of
Business
&
Economics
Session 2, 2012
127 respondents
28.3%
26.0%
24.4%
21.3%
Session 2, 2013
208 respondents
27.2%
20.9%
31.6%
20.4%
Session 2, 2014
210 respondents
22.9%
29.5%
21.0%
26.7%
19.7%
13.6%
12.0%
First Experience of iLearn
23.2%
39.0%
41.7%
Sessional Staff
72.8%
61.0%
58.3%
Fixed Term or Continuing
Table 1: The iLearn Staff Experience Survey Demographic Information for 2012 - 2014
TECHNOLOGIES USED TO ACCESS ILEARN: STAFF SURVEY RESULTS
In order to ascertain the technologies used to access iLearn, the survey respondents were asked to: “Please indicate
the frequency with which you use the following devices to access iLearn”.
Technologies used to access iLearn 2014
Desktop or laptop computer at other locations (n =
210)
University provided computer on campus (n = 210)
Smartphone using another network provider (n = 199)
My own laptop using OneNet (n = 197)
Tablet at other locations (n = 200)
Smartphone at MQ using OneNet (n = 203)
Tablet at MQ using OneNet (n = 202)
Other device/network combination (n = 166)
At
least
once a
day
51.9%
A few
times a
week
A few
times a
semester
Never
or
rarely
35.7%
A few
times
a
month
5.2%
2.9%
4.3%
42.9%
14.1%
13.2%
9.5%
7.9%
3.5%
1.2%
32.9%
9.0%
18.3%
9.0%
10.8%
12.4%
1.2%
5.2%
6.5%
12.2%
11.0%
9.9%
11.9%
0.6%
6.2%
14.6%
12.2%
9.0%
12.8%
11.4%
0.6%
12.9%
55.8%
44.2%
61.5%
58.6%
60.9%
96.4%
4
Table 2: Technologies used to access iLearn according to the 2014 staff survey
The table above relates to the use of various technologies to access iLearn, as it is important to monitor if the system
is meeting the needs of new technological devices and if the network is able to support the system and devices in all
circumstances. The staff responses to the last three surveys have shown a consistent approach to which technologies
are relied upon to access iLearn with ‘Desktop or laptop computer at other locations’ and ‘University provided
computer on campus’ being the most popular choices. It is possible that there might be some changes in 2015 with
the use of the iLearn mobile enabled interface and the further proliferation of technologies like Smartphones and
Tablets, however, at the moment, the use of these technologies ranked highly in the ‘never or rarely’ category.
Further statistical analysis revealed that staff members in the Faculty of Arts indicate frequent use of 'Desktop computers
at other locations' (p<0.01) in greater proportion than surveyed staff members in other faculties. Staff members that are
employed in Full-time fixed or permanent positions indicate frequent use of ‘University provided computer on
campus’ (p < 0.05) in greater proportion than the casual or sessional staff members surveyed. Staff members that are
employed as casual or sessional staff indicate frequent use of 'Own laptop at MQ using OneNet' (p<0.01) to access
iLearn in greater proportion than the surveyed staff members in Full-time fixed or permanent positions. These
findings seem consistent with employment conditions of staff members (see Appendix 4).
STAFF SATISFACTION WITH ILEARN AS A SUPPORT FOR LEARNING AND TEACHING
The table below provides a summary of the results from the 2014 survey.
2014 iLearn Staff Experience Survey Results
Agreement
Neutral /
Mixed
Feelings
Disagreement
72.4%
15.8%
11.8%
69.5%
17.4%
13.1%
62.5%
18.8%
18.8%
197
60.3%
24.2%
15.5%
194
I am satisfied with iLearn as an online support for my unit
content.
I am satisfied with iLearn as a support for organising my
teaching.
I am satisfied with the way iLearn supports interaction and
communication with my students.
I am satisfied with the way iLearn supports my students'.
engagement with learning activities.
N
196
190
Table 3: Staff Satisfaction with iLearn as a support for learning and teaching
The figures below offer an indication the surveyed staff members’ responses since 2012. The number of the surveyed
staff members’ responses vary thus the number of responses (n=) received are indicated in each.
I am satisfied with iLearn as an online
support for my unit content
2014, S2 (n = 196)
2013, S2 (n = 194)
72.4%
68.0%
70.4%
Agreement
Neutral / Mixed Feelings
Disagreement
2012, S2 (n = 125)
15.8%
21.1%
16.0%
11.8%
10.8%
13.6%
Figure 1: Satisfaction with iLearn as an online support for unit content
According to the 2014 staff survey responses 72.4% are in agreement that they are satisfied with iLearn as an online
support for their unit content. This has increased since Session 2, 2013 (68%) and Session 2, 2012 (70.4%). The
neutral / mixed feelings and the level of disagreement with the statement have also decreased since Session 2, 2012.
5
I am satisfied with iLearn as a support for
organising my teaching
2014, S2 (n = 190)
2013, S2 (n = 196)
Agreement
Neutral / Mixed Feelings
Disagreement
2012, S2 (n = 122)
60.7%
69.5%
68.8%
17.4%
22.4%
20.5%
13.1%
16.8%
10.6%
Figure 2: Satisfaction with iLearn as a support for organising my teaching
Staff satisfaction with iLearn as a support for organising teaching, according to the survey responses in 2014, is
69.5%, this is higher than previous years. The neutral / mixed feelings category and the percentage of staff members
that disagreed with this statement are also less than 2013 and 2012.
I am satisfied with the way iLearn supports
interaction and communication with my
students
2014, S2 (n = 197)
2013, S2 (n = 192)
2012, S2 (n = 123)
62.5%
59.9%
56.1%
Agreement
Neutral / Mixed Feelings
18.8%
19.8%
22.0%
Disagreement
18.8%
20.4%
21.9%
Figure 3: Satisfaction with iLearn as a support for interaction and communication with students
According to the 2014 staff responses 62.5% are in agreement that they are satisfied with the way iLearn supports
interaction and communication with students, which is higher than previous years. The neutral / mixed feelings
category and the percentage of staff members that disagreed with this statement are also less than 2013 and 2012.
Furthermore, staff members employed as casual or sessional staff indicated that they are satisfied with the way
iLearn supports their interaction and communication with students (p<0.05) to a greater extent than surveyed staff
members in Full-time fixed or permanent positions (see Appendix 5).
6
I am satisfied with the way iLearn supports
my students' engagement with learning
activities
2014, S2 (n = 194)
2013, S2 (n = 186)
2012, S2 (n = 123)
60.3%
64.5%
62.6%
Agreement
24.20%
19.90%
22.00%
Neutral / Mixed Feelings
Disagreement
15.5%
15.6%
15.5%
Figure 4: Satisfaction with the way iLearn supports student engagement
Staff satisfaction with the way iLearn supports student engagement with learning activities, according to the survey
responses in 2014, is 60.3%. The neutral / mixed feelings category has risen while the percentage of staff members
that disagreed with this statement is almost identical to previous years.
STAFF SKILLS AND CONFIDENCE LEVELS
Measuring staff skills and confidence levels is difficult when relying upon responses to scaled questions, however the
following results offer an indication of the changes that have taken place between the initial introduction and
implementation of iLearn to the second year of the operational phase.
STAFF CONFIDENCE LEVELS
The table below provides a summary of the results from the following six scaled questions in the 2014 survey:
2014 iLearn Staff Experience Survey Results
I have developed the necessary skills to effectively use
iLearn.
I feel confident when using iLearn in my teaching.
I feel confident when using Echo360 in my teaching.
I feel confident when using iTeach to manage
enrolments.
I feel confident when using iTeach to create my Unit
Guide.
I feel confident when using iTeach to map learning
outcomes, assessment tasks and graduate capabilities.
Agreement
Neutral /
Mixed
Feelings
Disagreement
N
70.6%
21.6%
7.9%
77.1%
62.6%
62.5%
19.4%
27.2%
21.5%
3.4%
10.1%
16%
206
158
144
61.3%
19.4%
19.3%
155
58.6%
19.7%
21.8%
152
204
Table 4: Staff Confidence levels in using iLearn as a support for learning and teaching
The figures below display the staff responses to the above statements in comparison to the survey results in 2012
and 2013.
7
I have developed the necessary skills to
effectively use iLearn
2014, S2 (n = 204)
2013, S2 (n = 201)
2012, S2 (n = 124)
70.6%
69.2%
67.7%
Agreement
21.6%
16.9%
21.0%
Neutral / Mixed Feelings
7.9%
Disagreement
13.9%
11.3%
Figure 5: Development of skills to effectively use iLearn
According to the staff members surveyed 70.6% agree that they have developed the necessary skills to effectively use
iLearn. This is almost identical to the 2013 results, however, there has been an increase in the ‘neutral / mixed
feelings’ category and a decrease in the percentage of staff members that disagreed with this statement.
I feel confident when using iLearn in my
teaching
201, S2 (n = 206)
2013, S2 (n = 202)
2012, S2 (n = 123)
77.1%
75.7%
78.1%
Agreement
19.4%
15.8%
14.6%
Neutral / Mixed Feelings
Disagreement
3.4%
8.4%
7.3%
Figure 6: Staff confidence when using iLearn
According to the staff members surveyed 77.1% agree that they feel confident when using iLearn in their teaching.
This is slightly higher than Session 2, 2013 (75.7%) and slightly less than Session 2, 2012 (78.1%) nevertheless there
was a decrease in the number of staff members surveyed that disagreed with this statement and consequently an
increase in the ‘neutral / mixed feelings’ category.
8
I feel confident when using Echo360 in my
teaching
2014, S2 (n = 158)
2013, S2 (n = 150)
2012, S2 (n = 90)
62.6%
64.6%
60.0%
Agreement
27.2%
23.3%
26.7%
Neutral / Mixed Feelings
Disagreement
10.1%
12.0%
13.4%
Figure 7: Staff confidence when using Echo360
A total of 62.6% of the staff members surveyed agree that they feel confident when using Echo360 in their teaching.
This result shows a slight increase since Session 2, 2012 (60%) and a slight decrease from Session 2, 2013 (64.6%).
I feel confident when using iTeach to
manage enrolments
2014, S2 (n = 144)
2013, S2 (n = 143)
2012, S2 (n = 106)
62.5%
Agreement
56.0%
47.2%
Neutral / Mixed Feelings
Disagreement
21.5%
22.4%
28.3%
16.0%
21.7%
24.5%
Figure 8: Staff confidence when using iTeach
A total of 62.5% of the staff members surveyed agree that they feel confident when using iTeach in their teaching.
This is the highest result across the three surveys and accordingly there was a reduction in ‘neutral / mixed feelings’
category and a decrease in the ‘disagree’ and ‘strongly disagree’ categories. There were no statistical differences
between Faculties or Staff Appointment, however, staff members with previous experience of iLearn are confident to
a greater extent when using iTeach to manage enrolments than staff members using iTeach for the first time
(p<0.01) (see Appendix 5).
9
I feel confident when using iTeach to
create my Unit Guide
2014, S2 (n = 155)
2013, S2 (n = 155)
2012
61.3%
Agreement
49.7%
19.4%
18.1%
Neutral / Mixed Feelings
19.3%
Disagreement
32.3%
Figure 9: Staff confidence when using iTeach to create Unit Guides
The percentage of staff members that agree they feel confident when using iTeach (UNITS) to create their Unit
guide has significantly increased since 2013. There has been a slight rise in the neutral/mixed feelings category but a
significant drop in the levels of disagreement.
I feel confident when using iTeach to map
learning outcomes, assessment tasks and
graduate capabilities
2014, S2 (n = 152)
2013, S2 (n = 156)
Agreement
Neutral / Mixed Feelings
Disagreement
2012
46.8%
58.6%
19.7%
17.9%
21.8%
35.2%
Figure 10: Staff confidence when using iTeach to map student learning outcomes
From the 152 staff members that responded to this scaled question 58.6% agree with this statement, while there has
been a slight rise in the neutral/mixed feelings category there has also been a significant drop in the levels of
disagreement.
CHANGING PRACTICE
Staff confidence levels contribute to maintaining an environment for change in practice. The table below provides a
summary of the results from the following two scaled questions:
2014 iLearn Staff Experience Survey Results
As a result of using iLearn I have reconsidered the way I teach my unit.
Agreement
51.9%
Neutral
/ Mixed
Feelings
17.7%
Disagreement
N
30.4%
181
I would like to utilise the tools in iLearn to enhance my teaching in the
76.4%
12.3%
11.3%
future.
Table 5: Staff members changing practice by using iLearn as a support for learning and teaching
10
195
The figures below display the staff responses to the above statements in comparison to the survey results in 2012
and 2013.
As a result of using iLearn I have
reconsidered the way I teach my unit
2014, S2 (n = 181)
2013, S2 (n = 156)
2012, S2 (n = 114)
51.9%
49.4%
47.4%
Agreement
17.7%
18.8%
22.8%
Neutral / Mixed Feelings
30.4%
31.8%
29.8%
Disagreement
Figure 11: Change in teaching practice
The level of agreement to this survey question 51.9% continues to marginally rise and as a consequence the level of
neutral / mixed feelings slowly decreasing over the past three years. Significantly almost a third of those surveyed
continue to be in disagreement. Conversely, staff members employed as casual or sessional staff indicated that as a
result of using iLearn they have reconsidered the way they teach their unit (p<0.05) to a greater extent than
surveyed staff members in Full-time fixed or permanent positions. Whereas staff members, using iLearn for the first
time are less likely to agree with this statement (p<0.05) (see Appendix 5).
I would like to utilise the tools in iLearn to
enhance my teaching in the future
2014, S2 (n = 195)
2013, S2 (n = 193)
2012, S2 (n = 124)
76.4%
71.0%
Agreement
87.9%
Neutral / Mixed Feelings
Disagreement
12.3%
17.1%
5.6%
11.3%
11.9%
6.4%
Figure 12: Change in future teaching practice
Of the 195 staff members who responded to this survey question, 76.4% agree with this statement, which is higher than
Session 2, 2013 but still lower than Session 2, 2012. The neutral / mixed feelings and the levels of disagreement have also
decreased.
11
While the system data shows a slight increase in the use of alternative iLearn functions, it may be that there needs to
be a stronger link between alternative iLearn functions and how it can benefit or enhance online pedagogy. This
category also includes the percentage of staff members surveyed that had no previous experience of iLearn before
Session 2, 2014.
I had no experience of iLearn/Moodle
until this year
2014, S2 (n=208)
2013, S2 (n = 208)
2012, S2 (n = 127)
12%
Surveyed Staff Members
Surveyed Staff Members
13.60%
Surveyed Staff Members
19.70%
Figure 13: Staff Experience of iLearn / Moodle
In Session 1, 2012 a total of 74.4% of staff members surveyed had no previous experience of Moodle, however, in
Session 2, 2014 only 12% of staff surveyed had no previous experience of iLearn / Moodle. This is a slight decrease
from Session 2, 2012-2013 and a significant decrease from the iLearn implementation in Session 1, 2012. This result
indicates that there needs to be a focus on the next phase of change, specifically, how the available technology can
benefit or enhance online pedagogy.
STAFF EXPERIENCE WITH ILEARN
This section focuses on staff engagement with iLearn by incorporating four forms of data sourced from: the iLearn
system; the scaled responses; open ended responses; and statistical analysis.
STATISTICS ON THE USE OF ILEARN FUNCTIONS: DATA FROM THE ILEARN SYSTEM
The percentages listed in the table below were generated through the iLearn system. The purpose of generating this
data is to compare the results between the use of the iLearn functions with the surveyed staff members responses to
the ‘usefulness’ of each iLearn function. This comparison will not be possible for some of the iLearn functions, for
example, there is no data available on Twitter Feeds, RSS Feeds, Announcements, Videos, Links to eReserve,
Gradebook and Calendar. This is because they are considered to be ‘Moodle block instances’, thus it is problematic
to extract this information from the iLearn platform. There are also multiple assignments used in each unit and the
introduction of two classifications for assignments in 2013, however, the system showed in Session 2, 2014 there
were: 317 units with one or more moodle assignments and 641units with one or more Turnitin assignments. This
equates to 95.6% of iLearn units had either one or more moodle assignments or one or more Turnitin assignments
in Session 2, 2014.
S2, 2012
S2, 2013
S2, 2014
Number of iLearn Units
1,160
1,034
1,002
iLearn Functions –
used…
Discussion Forum
99.0%
99.1%
99.9%
--------
--------
95.6%
Labels
94.1%
90.3%
94.0%
Links to readings & external sites
83.5%
89.2%
92.5%
The % of units that
Online Assignment Submission
12
Echo360
61.3%
67.9%
75.0%
Turnitin (includes GradeMark)
31.0%
47.8%
61.6%
Dialogue Module
42.5%
36.9%
38.2%
Quiz
18.8%
20.6%
25.5%
Grouping/Groups
4.9%
6.2%
7.0%
Blog
2.4%
4.4%
5.5%
Chat
4.4%
6.7%
5.0%
Wiki
2.4%
4.3%
3.5%
Book Module
3.2%
3.7%
3.5%
Database
2.0%
2.4%
2.3%
Workshop Tool
1.2%
1.5%
1.4%
Lesson
0.6%
0.9%
1.0%
Table 6: The percentage of units that used the various iLearn functions
During the implementation of iLearn all units were required to have an online presence, however, some
postgraduate courses do not have a corresponding iLearn unit thus only postgraduate coursework units were
included in the survey’s research sample. The total numbers of units that these percentages are based, do not
include the units in the system that were labelled as: ‘test’; ‘iLearn Admin’ or ‘Do not use’. It is also important to
note that some internal and external units might be listed as separate courses however they only have one iLearn
instance, hence the number of iLearn units listed above appear to be less than the courses offered by the University.
The highest increase in use, according to the table above, was Turnitin, which also includes GradeMark. This could
be attributed to this function having a well-defined purpose that relates to all units with a written assessment task. It
could also be attributed to the fact that GradeMark was mandated in some Faculties, as this would have had an
impact on the uptake. There was consistency in the use of: Discussion Forums; links to external sites; Echo360;
Dialogue Module; Blogs; Book Module and Databases. There was a slight decrease in the use of Chat; Wiki and
Workshop Tool. The 2014 iLearn Staff Experience Survey was the first year that we have included Grouping/Groups
and Labels, however, Grouping and Groups has shown a slight increase in use over the past three years and the use
of Labels has slightly increased since 2013.
Learn functions in UG and PG Units
120.00%
100.00%
80.00%
60.00%
40.00%
20.00%
0.00%
653 UG Units, S2 2014
349 PG Units, S2 2014
Figure 14: Undergraduate and Postgraduate Units used of iLearn Functions
13
The above figure shows the percentage of undergraduate and postgraduate units that used the various iLearn
functions. The percentage of use across all undergraduate and postgraduate units are very similar for all the iLearn
functions except there is greater use of the Dialogue Module in postgraduate units and great use of Quizzes in
undergraduate units.
SCALED RESPONSES ON THE PERCEIVED VALUE OF ILEARN FUNCTIONS: DATA FROM THE
SURVEYS
All the iLearn functions are used for different purposes thus the staff experiences of learning and teaching using the
range of iLearn functions, were organised in two categories:

iLearn Functions for organising teaching

iLearn Functions for engaging with learning activities
At the beginning of each category there is a short summary interpreting the results, which is then summarised in a
table. Further information is provided on each iLearn function that situates the results within any system data, open
ended comments by staff members and statistical analysis by Faculty and Staff Appointment (see Appendix 6).
The feedback in the open ended responses to: “Please tell us about any ongoing technical issues you have
experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” and “What
would enable you to use iLearn more effectively in your teaching?” These questions had a total of 151 and
171 comments respectively will be explored in the summaries below.
ILEARN FUNCTIONS FOR ORGANISING STUDENTS LEARNING
According to the staff members surveyed in 2014 Announcements were the most effective in meeting staff members
needs with regards to organising students’ learning (88.5%). This statement also had the highest response rate
(n=191) from the 210 survey responses received and has continued to be ranked highly by staff members since the
implementation of iLearn. Online Assignment Submission had the second highest level of agreement (85.2%) in this
category followed by Turnitin (82.1%), Labels (72.2%), GradeMark (71.2%), Grades (70.1%), Activity logs (56.8%),
Reports (49.5%) and Calendar (44.3%).
2014 iLearn Staff
Experience Survey Results
Agreement
Neutral /
Mixed
Feelings
6.3%
12.1%
Disagreem
ent
Announcements
88.5%
5.3%
Online Assignment
85.2%
2.7%
Submission
Turnitin
82.1%
14.3%
3.6%
Labels
72.2%
20.4%
7.2%
Grademark
71.2%
14.7%
14.1%
Grades
70.1%
17.5%
12.4%
Activity Logs
56.8%
28%
15.2%
Reports
49.5%
34.2%
16.2%
Calendar
44.3%
32.2%
23.5%
Table 7: iLearn functions that facilitate organising students learning
N
191
182
168
137
163
177
125
111
115
The figures below display the staff responses to the above iLearn Functions in comparison to the survey results in
2012 and 2013.
ANNOUNCEMENTS
The results showed that 88.5% of staff members surveyed agree that Announcements were effective in meeting their
needs with regard to organising students' learning (n=191). This has been a consistent finding since iLearn’s
implementation in 2012.
14
Announcements
2014, S2 (n = 191)
2013, S2 (n = 193)
2012, S2 (n = 124)
88.5%
86.0%
88.7%
Agreement
6.3%
8.3%
5.6%
Neutral / Mixed Feelings
5.3%
5.7%
5.6%
Disagreement
Figure 15: iLearn Function Announcements
There is no system data for Announcements as they are considered to be ‘Moodle block instances’. In response to
“Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn,
Echo360, iTeach or the network)” There were two comments (n = 151) that suggested Announcements have
vanished or that students consider these announcements to be spam from the University. There were five comments
that related to Announcements in response to “What would enable you to use iLearn more effectively in your
teaching?” (n = 171) that suggested improvements are required in how and where the message is sent as well as how
it is categorised.
ONLINE ASSIGNMENT SUBMISSION
Online Assignment Submission was also ranked highly by staff members who have completed the 2014 survey. A
total of 85.2% agree Online Assignment Submission was effective in meeting their needs with regard to organising
students' learning (n=182).
Online Assignment Submission
2014, S2 (n = 182)
2012, S2 (n = 96)
2012, S1 (n = 153)
85.2%
Agreement
72.9%
68.0%
12.1%
15.6%
12.4%
Neutral / Mixed Feelings
2.7%
Disagreement
11.5%
19.7%
Figure 16: iLearn Function Online Assignment Submission
The system data shows that 95.6% of iLearn units had either one or more moodle assignments or one or more
Turnitin assignments in Session 2, 2014. In response to “Please tell us about any ongoing technical issues you have
experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There were five comments (n
= 151) that revealed staff members have experienced issues with: the slowness of the system, delays in the email
notifications and students uploading their assignments or reading feedback. There were three comments that
related to Online Assignment Submission in response to “What would enable you to use iLearn more effectively in
15
your teaching?” (n = 171) there were calls for improvements to track late submissions and process of assigning
marks.
TURNITIN
Turnitin was ranked highly by staff members who have completed the 2014 survey; however, the open-ended
responses revealed the difficulties experienced when using the system. Nevertheless, 82.1% agree Turnitin was
effective in meeting their needs with regard to organising students' learning (n=168).
Turn-it-in
2014, S2 (n = 168)
2013, S2 (n = 158)
82.1%
74.7%
Agreement
Neutral / Mixed Feelings
Disagreement
2012
14.3%
16.5%
3.6%
8.8%
Figure 17: iLearn Function Turnitin
According to the system data the highest increase in use over the last two years was Turnitin, which also includes
GradeMark. This could be attributed to this function having a well-defined purpose that relates to all units with a
written assessment task. The iLearn system data indicates that the use of Turnitin has had an increase from 47.8%
of units in Session 2, 2013 to 61.6% of units in Session 2, 2014. In response to “Please tell us about any ongoing
technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”
there were 18 comments (n = 151) that revealed staff members have experience issues with: the site being
unavailable, submission of marks, system slowness, non-compatibility with certain mobile devices, students having
difficulties uploading their assignments and the need to access to a reliable manual. There were five comments that
related to Turnitin in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171)
one staff member said: “Marking on Turnitin is faster but the depth of the marking and feedback is not as good”.
The other four comments referred to a faster and simpler system that is stable. Staff members in the Faculty of Arts
and the Faculty of Business and Economics found Turnitin effective in meeting their needs with regards to organise
students learning (p<0.05) to a greater extent than staff members surveyed in other faculties (see Appendix 6).
LABELS
The system data shows that 94.0% of iLearn units in Session 2, 2014 used Labels. The survey results revealed that
72.2% of staff members agree Labels were effective in meeting their needs with regard to organising students' learning
(n=137). There is no survey data for this iLearn Function prior to 2014 thus no comparison graph can be shown.
However, the iLearn system data indicates that the use of Lables has had an increase from 90.3% of units in Session
2, 2013 to 94.0% of units in Session 2, 2014. In response to “Please tell us about any ongoing technical issues you
have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there one comment
(n=151) that related to labels. This staff member said: “The default iLearn style this year was overly complex and
space wasting on the screen. I opted for an older style and simplified it further by removing icons from the labels in
each week. It was tedious to have to recreate the labels for lectures, reading etc. it is a pity to waste screen space
particularly with iPhone and iPad users whose screen is limited”. There was also one comment that related to Labels
in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171). This staff
member said: “More flexible design. The main collapsible "week" sections are fine, but I also need collapsible
subsections within the weeks. Labels is an interim measure but not collapsible. Also an icon library with easily
accessible icons including but certainly not limited to the standard iLearn ones”.
GRADEMARK
GradeMark was also ranked highly by staff members who have completed the 2014 survey; however, the openended responses revealed some difficulties when using the system. Despite this a total of 71.2% agree GradeMark was
effective in meeting their needs with regard to organising students' learning (n=163).
16
GradeMark
2014, S2 (n = 163)
2013, S2 (n = 144)
2012
71.2%
Agreement
59.7%
14.7%
Neutral / Mixed Feelings
22.2%
14.1%
18.0%
Disagreement
Figure 18: iLearn Function GradeMark
According to the system data the highest increase in use over the last three years was Turnitin, which also includes
GradeMark. Any ongoing issues related to the technology are also outlined in Turnitin. In response to “Please tell us
about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or
the network)” there were seven comments (n=151) that related to GradeMark being an unreliable system, the need
for more flexibility with marking, being too time consuming to set-up and not saving grades. There were three
comments that related to GradeMark in response to “What would enable you to use iLearn more effectively in your
teaching?” (n = 171) the suggested improvements were capabilities for: allocating half marks, improved marking
rubric and one staff member said “Grademark is great on Turnitin iPad app except that it is not possible to link
comment to rubric in app version”. Staff members employed as casual or sessional staff found GradeMark to be
effective in meeting their needs with regards to organising students learning (p<0.05) to a greater extent than
surveyed staff members in Full-time fixed or permanent positions (see Appendix 6).
GRADES
The results showed that 70.1% of staff members surveyed agree that Grades (Gradebook) was effective in meeting their
needs with regard to organising students' learning (n=177), which is a significant rise from last year.
Grades
2014, S2 (n = 177)
2013, S2 (n = 140)
2012, S2 ( n = 80)
70.1%
60.0%
66.3%
Agreement
Neutral / Mixed Feelings
Disagreement
17.5%
22.9%
20.0%
12.4%
17.1%
13.8%
Figure 19: iLearn Function Grades
There is no system data for Grades as it is considered to be a ‘Moodle block instance’. In response to “Please tell us
about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or
the network)” There were eight comments (n = 151) that indicated issues with the design, slowness, difficulty
uploading grades and problems with hiding grades. There were nine comments that related to Grades in response to
17
“What would enable you to use iLearn more effectively in your teaching?” (n = 171) the suggested improvements
were: improved functionality including a red pen, calculation of grades and upload capability.
ACTIVITY LOGS
The results showed that 56.8% of staff members surveyed agree that Activity logs were effective in meeting their needs
with regard to organising students' learning (n=125). There is no previous survey data for this iLearn Function thus no
comparison graph can be shown. In response to “Please tell us about any ongoing technical issues you have
experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there one comment (n=151)
that related to Activity Logs commenting on the long loading time. There were four comments that related to
Activity Logs in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171),
which referred to a more streamlined process for monitoring students activity. One staff member said: “Activity logs
are difficult to use, time consuming (having to check each individual students activity log is a huge time burden on
teaching staff) and arduous as a mechanism for monitoring student activity”. Staff members in the Faculty of
Business and Economics and Faculty of Human Sciences found Activity Logs effective in meeting their needs with
regards to organise students learning (p<0.01) to a greater extent than staff members surveyed in other faculties
(see Appendix 6).
REPORTS
The results showed that 49.5% staff members surveyed agree that Reports were effective in meeting their needs with
regard to organising students' learning (n=111). There is no previous survey data for this iLearn Function thus no
comparison graph can be shown. This function requires further publicity because there are calls for reporting
features by some staff members but it was unclear whether these comment were referring to Activity Logs and
calling them Reports or if the comments were regarding the Reports that iLearn can generate.
CALENDAR
A total of 43.3% of staff members surveyed agree that the Calendar was effective in meeting their needs with regard to
organising students' learning (n=115), which is a significant increase from last year.
Calendar
2014, S2 (n = 115)
2013, S2 (n = 117)
2012, S2 (n = 60)
44.3%
Agreement
36.7%
43.3%
32.2%
35.9%
35.0%
Neutral / Mixed Feelings
Disagreement
23.5%
27.3%
21.7%
Figure 20: iLearn Function Calendar
There is no system data for Calendar as it is considered to be a ‘Moodle block instance’. There were no comments (n
= 151) regarding the calendar in response to “Please tell us about any ongoing technical issues you have experienced
during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There was one comment in response to
“What would enable you to use iLearn more effectively in your teaching?” (n = 171) that staff member said: “The
calendar isn't particularly useful. Last time I checked I wasn't flexible enough to be an effective reminder about
deadlines, upcoming events, etc”. Conversely, staff members in Faculty of Business and Economics found the
calendar effective in meeting their needs with regards to organise students learning (p<0.05) to a greater extent
than staff members surveyed in other faculties (see Appendix 6).
ILEARN FUNCTIONS FOR ENGAGING STUDENTS WITH LEARNING ACTIVITIES
According to the staff members surveyed in 2014 Discussion Forms were the most effective in meeting staff
members needs with regards to engaging students’ with learning activities (72.6%). This statement also had the
highest response rate (n=183) from the 210 survey responses received and has continued to be ranked highly by
18
staff members since the implementation of iLearn. Quizzes had the second highest level of agreement (70.3%) in
this category followed by Groupings and Groups (62.1%), Dialogue Module (60.4%), Database (42.8%), Workshop
Tool (38.2%), Blog (34.2%), Wiki (30.5%), Chat (27.3%) and Twitter Feeds (26%). The three iLearn functions with
the highest Neutral / Mixed Feelings rankings were: Workshop Tool (38.2%) Twitter Feeds (34.8%) and Blogs
(34.2%).
2013 iLearn Staff
Experience Survey Results
Agreement
Neutral / Disagreement
N
Mixed
Feelings
Discussion Forums
72.6%
13.7%
13.7%
183
Quizzes
70.3%
16.1%
13.6%
118
Groupings and Groups
62.1%
19.4%
18.5%
103
Dialogue Module
60.4%
19.4%
20.2%
139
Database
42.8%
28.6%
28.5%
56
Workshop Tool
38.2%
38.2%
23.7%
55
Blog
34.2%
34.2%
31.5%
73
Wiki
30.5%
30.5%
38.9%
59
Chat
27.3%
33.3%
39.4%
66
Twitter Feeds
26%
34.8%
39.1%
46
Table 8: iLearn functions that facilitate engagement with learning activities
DISCUSSION FORUMS
There was a slight increase from last year in the agreement levels as 72.6% of staff members surveyed agree that
Discussion Forums were effective in meeting their needs for engaging their students in learning activities (n=183).
Discussion Forums
2014, S2 (n = 183)
2013, S2 (n = 180)
2012, S2 (n = 117)
72.6%
68.3%
71.8%
Agreement
Neutral / Mixed Feelings
Disagreement
13.7%
20.0%
16.2%
13.7%
11.6%
12.0%
Figure 21: iLearn Function Discussion Forums
The iLearn system data indicates that the use of Discussion Forums has remained for the last three years (2012 –
99%; 2013 – 99.1% and 2014 – 99.9%).There was one comment (n = 151) regarding Discussion Forums in response
to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn,
Echo360, iTeach or the network)”. This staff member said: “Students report receiving so much spam from iLearn
discussion boards that they miss important announcements”. In response to “What would enable you to use iLearn
more effectively in your teaching?” (n = 171) there were nine comments that referred to providing: an anonymous
option, like and rate capabilities, group specific discussion forums. There were also comments requesting an end to
the email notices and better organisation of the discussion threads.
QUIZZES
There was an increase from previous years as 70.3% of staff members agree Quizzes were effective in meeting their
needs for engaging their students in learning activities (n=118).
19
Quizzes
2014, S2 (n = 118)
2013, S2 (n = 95)
2012, S2 (n = 49)
70.3%
Agreement
Neutral / Mixed Feelings
60.0%
61.2%
16.1%
17.9%
18.4%
13.6%
Disagreement
22.1%
20.4%
Figure 22: iLearn Function Quizzes
The iLearn system data indicates that the use of Quizzes has increased from 20.6% of iLearn units in Session 2, 2013
to 25.5% of iLearn units in Session 2, 2014. There were four references (n = 151) to Quizzes in response to “Please
tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360,
iTeach or the network)” that revealed staff members have experience issues with the new theme not showing the
answers to quiz questions. There was also a request to streamline the method of implementing an online quiz. In
response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were seven
comments that referred to flexibility in the type of quiz questions, for example, an open-ended response text box.
There were also comments regarding the limitations of quizzes, experiencing problems with the inclusion of generic
questions in quizzes, issues with using embedded quizzes as well as access to software for creating quizzes in Word.
There were no statistical differences between Faculties or Staff Appointment, however, staff members using iLearn
for the first time were less likely to agree that Quizzes were effective in meeting their needs for engaging their students in
learning activities (p<0.05) than staff members with previous experience (see Appendix 6).
GROUPING AND GROUPS
A total of 62.1% of staff members who have completed the 2014 survey, consider that grouping and groups were
effective in meeting their needs for engaging their students in learning activities (n=103). There is no previous survey data
for this iLearn Function. However, the iLearn system data indicates that the use of Grouping and Groups has had a
slight increase from 6.2% of units in Session 2, 2013 to 7.0% of units in Session 2, 2014. There were three references
(n = 151) to Grouping and Groups in response to “Please tell us about any ongoing technical issues you have
experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” that revealed staff members
have experience issues with: resolving groupings and problems with enrolling students when they change tutorials
or units. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there
were six comments that referred to providing different group scenarios each week, easier grouping by class, the need
for auto organisation of students into groups and the procedure to create groups being too cumbersome. There were
also requests for more workshops including more time allocated in workloads to attend these workshops as well as
the need for the system to be able to produce a list of students that do not sign up for a group. Staff members in the
Faculty Business and Economics, Faculty of Science, Faculty of Human Sciences found Groupings and Groups
effective in meeting their needs with regards to engaging their students in learning activities (p<0.01) to a greater
extent than staff members surveyed in the Faculty of Arts (see Appendix 6).
DIALOGUE MODULE
There was an increase from the 2013 survey results as 60.4% of staff members that the Dialogue Module (sending
and receiving messages) was effective in meeting their needs for engaging their students in learning activities
(n=139).
20
Dialogue Module
2014, S2 (n = 139)
2013, S2 (n = 149)
2012, S2 (n = 95)
60.4%
Agreement
51.7%
60.0%
19.4%
Neutral / Mixed Feelings
26.2%
12.6%
20.2%
22.2%
27.3%
Disagreement
Figure 23: iLearn Function Dialogue Module
The iLearn system data indicates that the use of Dialogue Module has slightly increased from 36.9% of units in
Session 2, 2013 to 38.2% of units in Session 2, 2014.There were two references (n = 151) to the Dialogue Module in
response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with
iLearn, Echo360, iTeach or the network)” that requested one click access to dialogues and providing Carbon Copy
(CC) and Blind Carbon Copy (BCC) options for messages. In response to “What would enable you to use iLearn
more effectively in your teaching?” (n = 171) there were ten comments that focused on having a dialogue module
that operates like email and has a group email function.
DATABASE
There was an increase in levels of agreement from previous surveys as a total of 42.8% agree Databases were
effective in meeting their needs for engaging their students in learning activities (n=56).
Database
2014, S2 (n = 56)
2013, S2 (n = 53)
2012, S2 (n = 26)
42.8%
Agreement
35.9%
34.6%
28.6%
28.3%
Neutral / Mixed Feelings
23.1%
28.5%
Disagreement
35.9%
42.3%
Figure 24: iLearn Function Database
The iLearn system data indicates that the use of the Database has marginally decreased from 2.4% of iLearn units in
Session 2, 2013 to 2.3% of iLearn units in Session 2, 2014. There were no references to the Database in response to
“Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn,
Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively
in your teaching?” (n = 171) there was one comment that called for further enhancements. This staff member said:
“We use the database tool. However, we'd like to see a voting component added. So students can like a post and
easily make comments similar to Facebook. Right now we have a work around using the choice activity to vote
however, it's cumbersome”.
21
WORKSHOP TOOL
There has been a significant decrease in staff levels of agreement as, 38.2% of staff members agree that the
Workshop Tool were effective in meeting their needs for engaging their students in learning activities (n=55).
Workshop Tool
2014, S2 (n = 55)
2013, S2 (n = 54)
2012, S2 (n = 21)
38.2%
Agreement
44.4%
47.6%
38.2%
Neutral / Mixed Feelings
24.1%
19.0%
23.7%
Disagreement
31.5%
33.3%
Figure 25: iLearn Function Workshop Tool
The iLearn system data indicates that the use of the Workshop Tool has marginally decreased from 1.5% of iLearn
units in Session 2, 2013 to 1.4% of iLearn units in Session 2, 2014. There were no references (n = 151) to Workshop
Tool in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for
example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use
iLearn more effectively in your teaching?” (n = 171) there were two comments that called for greater flexibility such
as submission of late files and the workshop tool for voluntary participation. Staff members employed as casual or
sessional staff found Workshops to be effective in meeting their needs with regards to engaging students in learning
activities (p<0.05) to a greater extent than surveyed staff members in Full-time fixed or permanent positions (see
Appendix 6).
BLOG
A total of 34.2% of staff members consider the Blogs to be effective in meeting their needs for engaging their
students in learning activities.
Blog
2014, S2 (n = 73)
2013, S2 (n = 63)
2012, S2 (n = 31)
34.2%
36.5%
32.3%
Agreement
34.2%
38.1%
32.3%
Neutral / Mixed Feelings
31.5%
Disagreement
25.4%
35.5%
Figure 26: iLearn Function Blog
The iLearn system data indicates that the use of the Blog has marginally increased from 4.4% of iLearn units in
Session 2, 2013 to 5.5% of iLearn units in Session 2, 2014. There were no references to blogs in response to “Please
tell us about any ongoing technical issues you have experienced during 2013 (for example, with iLearn, Echo360,
22
iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your
teaching?” (n = 171) there were three comments, two staff members who would like to use it and one staff member
who used it in 14 tutorials and commented on how clunky it was. There were no statistical differences between
Faculties or Staff Appointment, however, staff members using iLearn for the first time were less likely to agree that
the Blog was effective in meeting their needs for engaging their students in learning activities (p<0.05) than staff members
with previous experience (see Appendix 6).
WIKI
A total of 30.5% agree that the Wiki was effective in meeting their needs for engaging their students in learning
activities (n=59).
Wiki
2014, S2 (n = 59)
2013, S2 (n = 55)
2012, S2 (n = 28)
30.5%
Argeement
25.5%
35.7%
30.5%
32.7%
28.6%
Neutral / Mixed Feelings
38.9%
41.8%
35.7%
Disagreement
Figure 27: iLearn Function Wiki
The iLearn system data indicates that the use of the Wiki has marginally decreased from 4.3% of iLearn units in
Session 2, 2013 to 3.5% of iLearn units in Session 2, 2014. There was one comment referring to the Wiki in response
to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn,
Echo360, iTeach or the network)”. This staff member said: “Group management in Wikis is badly designed and does
not reflect user needs”. In response to “What would enable you to use iLearn more effectively in your teaching?” (n =
171) there were four comments that focused on improvements to the Wiki for ease of use and one comment that
stated how it had been dumped for Google Docs to allow for synchronous editing. Staff members in the Faculty of
Science found Wikis effective in meeting their needs with regards to engaging their students in learning activities
(p<0.01) to a greater extent than staff members surveyed in other faculties (see Appendix 6).
CHAT
A total of 27.3% agree Chat was effective in meeting their needs for engaging their students in learning activities
(n=66).
23
Chat
2014, S2 (n = 66)
2013, S2 (n = 61)
2012, S2 (n = 29)
27.3%
27.9%
24.1%
Agreement
33.3%
31.1%
34.5%
Neutral / Mixed Feelings
39.4%
41.0%
41.4%
Disagreement
Figure 28: iLearn Function Chat
The iLearn system data indicates that the use of the Chat has marginally decreased from 6.7% of iLearn units in
Session 2, 2013 to 5.0% of iLearn units in Session 2, 2014. There were no references to Chat in response to “Please
tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360,
iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your
teaching?” (n = 171) there was one comment “We've used the chat a couple of times and it's difficult to see the
connections between parties. Again can someone look outside the LMS for a Chat tool that makes having an online
Chat with more than a dozen people easy to follow”.
TWITTER FEEDS
A total of 26% agree Twitter feeds were effective in meeting their needs for engaging their students in learning
activities (n=46), which is an increase from previous years.
Twitter Feeds
2014, S2 (n = 46)
Agreement
Neutral / Mixed Feelings
Disagreement
2013, S2 (n = 41)
2012, S2 (n = 14)
26.0%
19.5%
21.4%
34.8%
39.0%
28.6%
39.1%
41.5%
50.0%
Figure 29: iLearn Function Twitter Feeds
There is no system data for Twitter Feeds as they are considered to be ‘Moodle block instances’. There were no
references to Twitter Feeds in response to “Please tell us about any ongoing technical issues you have experienced
during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would
enable you to use iLearn more effectively in your teaching?” (n = 171) One staff member said: “The twitter feed tool
works well, but it would be great if it could filter re-tweets. When a tweet is retweeted it currently appears multiple
times - and this is less than ideal”.
24
STAFF SATISFACTION WITH SERVICES AND SUPPORT
In order to ascertain levels of staff satisfaction with the services and support for the iLearn platform, the survey
focused on two categories:

Support services for iLearn and iTeach;

Support services for iLearn provided by the Faculties and the Learning and Teaching Centre (LTC)
including online resources.
SATISFACTION LEVELS WITH SUPPORT SERVICES FOR ILEARN AND ITEACH
The table below displays the surveyed staff members’ satisfaction with the support services for iLearn and iTeach.
The number of the surveyed staff members’ responses vary thus the number of responses (n=) received are
indicated in each.
2014 iLearn Staff Experience Survey Results
Satisfied
Neutral /
Mixed
Feelings
17%
21.1%
Dissatisfied
Automatic enrolment of students facilitated by iTeach.
77.5%
OneHelp for resolution of issues relating to iTeach
62.6%
functions.
Unit Guides (unitguides.mq.edu.au) for providing
53%
20.5%
essential information to students.
Training and support for using iTeach to create Unit
60%
24.6%
Guides.
OneHelp for resolution of issues relating to iLearn
60.1%
22.1%
functions.
Table 9: Staff Satisfaction with support services for iLearn and iTeach
5.5%
16.3%
N
147
123
26.6%
166
15.4%
130
17.8%
163
STAFF SATISFACTION WITH ITEACH
A total of 77.5% of staff members surveyed are in agreement that they are satisfied with Automatic enrolment of
students facilitated by iTeach (n=147).
Automatic enrolment of students
facilitated by iTeach
2014, S2 (n = 147)
2013, S2 (n = 148)
77.5%
73.7%
68.4%
Satisfied
Neutral / Mixed Feelings
Dissatisfied
2012, S2 (n = 111)
17.0%
15.5%
18.0%
5.5%
10.8%
13.5%
Figure 30: Staff satisfaction with the automatic enrolment of students facilitated by iTeach
Since 2012, there has been an increase in the staff responses to the automatic enrolment of students facilitated by
iTeach satisfied categories thus a significant drop in the ‘Netural / Mixed Feelings’ and a decrease in the dissatisfied
categories. This could signify how the system has improved since its introduction. It could also be attributed to the
time it takes to embed a significant cultural change.
STAFF SATISFACTION WITH ITEACH SUPPORT
A total of 62.6% of staff members surveyed are in agreement that they are satisfied with OneHelp for resolution of
issues relating to iTeach (n=123).
25
OneHelp for resolution of issues relating
to iTeach
2014, S2 (n = 123)
2013, S2 (n = 140)
62.6%
64.2%
Satisfied
Neutral / Mixed Feelings
Dissatisfied
2012
21.1%
16.4%
16.3%
19.3%
Figure 31: Staff satisfaction with OneHelp for resolution of issues relating to iTeach
In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example,
with iLearn, Echo360, iTeach or the network)” there were three comments that related to iTeach (n = 151). These
comments focused on: the system being cumbersome, problems giving access to guest lecturers or industry
specialists and the systems structural problems. There were two comments in response to “Do you have any
suggestions for improving the level and /or effectiveness of iLearn support and services?” (n = 82) one staff member
said: “I find compiling unit guides on iTeach cumbersome and time consuming. I hate having to enter data in
separate fields then combine it as I have 6 separate iLearn units for what is essentially the same unit guide. I would
prefer to just upload a word document”. The other comment referred to the limited time for training in systems such
as iTeach, when employed as a session staff member a few days before the session begins.
STAFF SATISFACTION WITH UNIT GUIDES
A total of 53% of staff members surveyed are in agreement that they are satisfied with the Unit Guides
(unitguides.mq.edu.au) for providing essential information to students (n=166). This was a new question thus there
is no comparison data. In response to “Please tell us about any ongoing technical issues you have experienced
during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there were six comments that related to
iTeach (n = 151). These comments focused on: the unnecessary detail, how it is a time consuming process, how the
language is condescending, and there are rendering problems when published and printed. There were five
comments in response to “Do you have any suggestions for improving the level and /or effectiveness of iLearn
support and services?” (n = 82) that focused on: the difficulty using unit guides, navigation problems from iLearn to
the unit guides, the document is too long and writing a unit and study guide for students is double the work. There
were also nine comments in response to “Do you have any other comments you would like to add?” (n=87). These
comments referred to: doubling up on information, the ugly interface, having an edit capacity after publication and
connecting the assessments in the unit guide with grades. There were no other references to unit guides in the openended responses. Staff members employed as casual or sessional staff are satisfied that UNITS/Unit guides provide
essential information to students (p<0.01) to a greater extent than surveyed staff members in Full-time fixed or
permanent positions (see Appendix 7).
STAFF SATISFACTION WITH ITEACH TRAINING AND SUPPORT
A total of 60% of staff members surveyed are in agreement that they are satisfied with Training and support for
using iTeach to create Unit Guides (n=130). This was a new survey question to mirror the change in the system thus
it is not possible to generate a comparison graph. However, in the 2013 survey staff members were asked to indicate
their overall satisfaction with the training and support provided for UNITS (now iTeach) and 43.9% of the staff
members surveyed indicated that they were satisfied thus there has been a significant increase in the last year.
STAFF SATISFACTION WITH ONEHELP FOR ILEARN SUPPORT
A total of 60.1% of staff members surveyed are in agreement that they are satisfied with OneHelp for resolution of
issues relating to iLearn functions (n=163).
26
OneHelp for resolution of issues
relating to iLearn functions
2014, S2 (n = 163)
2013, S2 (n = 161)
2012, S2 (n = 84)
60.1%
66.5%
65.4%
Satisfied
Neutral / Mixed Feelings
Dissatisfied
22.1%
14.3%
19.0%
17.8%
19.3%
15.5%
Figure 32: Staff satisfaction with OneHelp for resolution of issues relating to iLearn functions
There were two comments in response to “Please tell us about any ongoing technical issues you have experienced
during 2014 (for example, with iLearn, Echo360, iTeach or the network)” (n = 151) that related to OneHelp. These
comments related to issues with Turnitin and students that were given the wrong advice through OneHelp. There
were also seven comments in response to “Do you have any suggestions for improving the level and/or effectiveness
of iLearn support and services?” (n = 82) that related to OneHelp. While staff members in general seem satisfied
with OneHelp, there were comments that indicated they are would like the option of phone support. There were
calls for an ‘urgent’ function within OneHelp and better student support.
SUPPORT SERVICES FOR ILEARN PROVIDED BY THE FACULTIES AND THE LTC
The table below displays the surveyed staff members’ satisfaction with the support services provided by the Faculties
and the LTC. The number of the surveyed staff members’ responses vary thus the number of responses (n=) received
are indicated in each.
2014 iLearn Staff Experience Survey Results
Satisfied
Neutral /
Mixed
Feelings
23.7%
29.2%
22%
17.7%
Dissatisfied
N
The ease of locating the iLearn online "self-help" resources.
52%
24.3%
The effectiveness of the iLearn online "self-help" resources.
45.6%
25.2%
The effectiveness of iLearn training workshops.
68.8%
9.3%
The support for using iLearn provided by the iLearn drop-in
75%
7.2%
clinic.
The level of Faculty-based support for using iLearn.
63%
19.7%
17.2%
Table 10: Staff Satisfaction with support services provided by the Faculties and the LTC
177
171
141
124
157
STAFF SATISFACTION WITH ONLINE RESOURCES
A total of 52% of staff members surveyed are in agreement that they are satisfied with ease of locating online "selfhelp" resources to assist me in using iLearn (n=177).
27
Ease in locating online "self-help" resources
2014, S2 (n = 177)
2013, S2 (n = 166)
2012
52.0%
Satisfied
66.5%
23.7%
Neutral / Mixed Feelings
14.3%
24.3%
19.3%
Dissatisfied
Figure 33: Staff satisfaction with the ease of locating online "self-help" resources
A total of 45.6% of staff members surveyed are in agreement that they are satisfied with the effectiveness of the
online "self-help" resources in assisting my use of iLearn (n=171). There were two comments made in response to
“Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n=82).
One staff member said they were satisfied and the other requested ease of access to Quick Guides.
The effectiveness of the online "self-help"
resources
2014, S2 (n = 171)
2013, S2 (n = 163)
2012
45.6%
Satisfied
41.2%
29.2%
Neutral / Mixed Feelings
Dissatisfied
34.4%
25.2%
24.6%
Figure 34: Staff satisfaction with the effectiveness of the online "self-help" resources
The scaled responses indicate that has been a drop in the satisfaction levels with regards to the online resources ease
of access; however there has been a slight increase in the satisfaction levels when staff members consider the
effectiveness of the online resources. There were five comments in response to “Do you have any suggestions for
improving the level and /or effectiveness of iLearn support and services?” (n = 82) two of these comments were very
positive and the other two commented on how phone support is still needed when the self-help resources are not
adequate. There was also one comment that suggested more resources for students were needed. There were also
two comments in response to “Do you have any comments you would like to add?” (n = 80) both suggested
improvements to tailoring the online resources.
28
STAFF SATISFACTION WITH THE ILEARN DROP-IN CLINICS
A total of 75% of staff members surveyed are in agreement that they are satisfied with the support for using iLearn
provided by the iLearn drop-in clinic (n=124).
Staff Satisfaction with the iLearn Drop-in
Clinics
2014, S2 (n = 124)
2013, S2 (n = 134)
2012
75.0%
79.9%
Satisfied
17.7%
16.4%
Neutral / Mixed Feelings
Dissatisfied
7.2%
3.7%
Figure 35: Staff Satisfaction with the iLearn Drop-in Clinics
In 2014, there were 159 visits and 137 visits from staff members to the iLearn Drop-in-Clinics in Session 1 and 2
respectively. In general, there has been very positive feedback. In response to “Do you have any suggestions for
improving the level and/or effectiveness of iLearn support and services?” (n = 82) there were nine comments that
focused on: the clinics need to begin earlier in the session, run all session and run at different times. One staff
member also said that they were unaware of the drop-in clinics. There were also five comments in response to “Do
you have any other comments you would like to add?” (n=87). One staff member said: “I can’t stress enough how
valuable the drop in centre is for me pre-semester start up. The staff members are endlessly patient and helpful.
Having the drop in centre reduces my stress re needing to get things ready to commence each semester. This
semester I am convening 8 units so the work to set up each iLearn site is daunting but made much easier by the
wonderful staff at the drop in centres. Please keep it going, same informal format and congratulate all involves with
this practical and invaluable initiative. Thanks”.
STAFF SATISFACTION WITH ILEARN TRAINING WORKSHOPS
A total of 68.8% of staff members surveyed are in agreement that they are satisfied with the effectiveness of iLearn
training workshop (n=141).
29
Staff Satisfaction with the iLearn Training
Workshops
2014, S2 (n = 141)
2013, S2 (n = 153)
2012
68.8%
68.6%
Satisfied
22.0%
22.2%
Neutral / Mixed Feelings
9.3%
9.2%
Dissatisfied
Figure 36: Staff Satisfaction with the iLearn Training Workshops
In response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and
services?” (n = 82) there were nine comments that related to iLearn Training Workshops. These comments referred
to the high quality of the workshops, problems relating to casual staff appointments, specifically, not having the
time or not having access to remuneration for time spent in training. There were also suggestions of linking the
advertisement of training sessions to the academics first use of iLearn as well as offering training outside of peak
times. In response to “Do you have any other comments you would like to add? (n=87)” there were four comments
again reiterating not having the time or not having access to remuneration for time spent in training. In response to
“What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were six comments that
simply listed ‘more training’ but no further details were provided.
STAFF SATISFACTION WITH FACULTY-BASED ILEARN SUPPORT
A total of 63% of staff members surveyed are in agreement that they are satisfied with the level of Faculty-based
support for using iLearn (n=157).
Staff Satisfaction with Faculty iLearn
Support
2014, S2 (n = 157)
2013, S2 (n = 158)
2012, S2 (n = 115)
63.0%
60.7%
Satisfied
87.8%
19.7%
25.3%
Neutral / Mixed Feelings
7.0%
17.2%
13.9%
Dissatisfied
5.2%
Figure 37: Staff Satisfaction with Faculty iLearn Support
In response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and
services?” (n = 82) there were eleven comments regarding Faculty Support. While there were several compliments it
was obvious that some staff members would like more ‘hands-on’ support and that they are very grateful when they
30
do receive it. In response to “Do you have any other comments you would like to add?” (n=87) there were twelve
comments that referred iLearn support and services. Again all but two of these comments were compliments about
the excellent support staff members have received, for example, one staff member said: “I think the iLearn support
team, at least the majority of individuals I have personally worked with, are fantastic. They are enthusiastic,
professional, personable, and really want to help. They are a pleasure to work with”.
ILEARN STATUS PAGE
A total of 13.3% of staff members surveyed have accessed the iLearn Status Page this year, 70.9% had not heard of it
and 15.8% of staff were unsure (n=196). There is now a direct link to the status page from the iLearn page
https://ilearn.mq.edu.au/login/index.php, thus these numbers should increase.
31
APPENDICES
APPENDIX 1: ILEARN STAFF EXPERIENCE SURVEY QUESTIONS
The surveys consisted of a combination of multiple-choice, scaled and open-ended questions that focused on:

Demographics relating to the learning and teaching context.

Technologies used to access iLearn.

Experiences of learning and teaching using the iLearn functions, which focused on:
1.
Organising teaching; or
2.
Engaging with learning activities.

Most valuable features and suggested improvements.

Overall satisfaction with iLearn, iTeach and associated support services.
In 2014, additional questions were included in the iLearn Staff Experience Survey regarding the Online Unit Guide
Facility.
It is important to note that the iLearn Staff Experience Survey used a five point Likert scale (strongly agree, agree,
neutral/mixed feelings, disagree and strongly disagree), which has been reduced to Agreement (strongly agree +
agree), Neutral/mixed feelings and Disagreement (strongly disagree + disagree) in all the tables and graphs in this
report. This same approach is used for the satisfaction questions, again the five point Likert scale (very satisfied,
satisfied, neutral/mixed feelings, dissatisfied and very dissatisfied) has been reduced to Satisfied (very satisfied +
satisfied), Neutral/mixed feelings and Dissatisfied (very dissatisfied + dissatisfied) this decision was made for
readability purposes. In each of the tables and graphs the number of staff responses vary thus the number of
responses (n=) received are indicated in each, in order to provide further clarity to the findings. Where possible, the
iLearn Staff Experience Survey results are presented with the corresponding results from the 2012 and 2013
surveys. This was not always possible as each year the survey questions have slightly changed to correspond with the
modifications within the system and the transition from the iLearn implementation to the operational phase.
32
33
34
35
36
APPENDIX 2: SURVEY ADVERTISEMENTS
Various communication strategies were used to raise awareness of the iLearn surveys; for example, survey
advertisements on screens across campus including the central courtyard, Campus Hub, Macquarie University Sport
& Aquatic Centre, E4B, W6B and the Library. A link to the iLearn improvements document and the advertisements
with further information regarding the surveys were published in LTC Faculty Reports, the LTC Blog Teche and the
iLearn homepage (once the user had logged into the system). Individual emails were sent to all Departmental
Administrators asking that they notify teaching staff about the surveys.
ADVERTISEMENTS FOR THE ILEARN HOMEPAGE
ADVERTISEMENT FOR CAMPUS SCREENS, POSTERS AND ATTACHMENTS TO EMAIL NOTIFICATIONS
37
APPENDIX 3: IMPROVEMENTS TO THE ILEARN PLATFORM
Since the iLearn implementation in 2012 the iLearn team has made continual improvements to the system. The
improvements made in 2014 include:

the new iLearn theme;

the introduction of a mobile friendly iLearn site;

the iTeach-UNITS integration;

the launch of iShare;

the introduction of iLearn status page (http://status.ilearn.mq.edu.au/);

the inclusion of a new question type in the IPA transcription;

updates to Unit readings block and Category block;

updates to Echo360 to enhance stability and maintain security;

the streaming and screen capture capabilities for Echo360 were extended to an additional 20 venues across
campus, and the new iTeach integration improved convenor access methods, resulting in fewer requests to
grant academics access to their units within Echo360.
38
APPENDIX 4: STATISTICAL ANALYSIS OF TECHNOLOGIES STAFF USED TO ACCESS ILEARN
STATISTICAL ANALYSIS BY FACULTY
Note: Faculty N = Maximum N: actual Faculty total for each row varies slightly due to missing values or NA
responses. Furthermore, two respondents did not nominate their Faculty.
Differences by Faculty significant at *α = 0.05, ** α = 0.01
Frequency of using
iLearn with…
Arts
N = 48
FBE
N = 56
FHS
N = 44
Science
N = 62
ALL
N = 210
Uni-provided computer at MQ
col %
col %
col %
col %
col %
Never or rarely
6
12
18
14
13
Moderate frequency
10
20
9
6
11
High frequency
83
68
73
79
76
Desktop or laptop computer at other location/s**
Never or rarely
2
2
2
10
4
Moderate frequency
0
4
7
19
8
High frequency
98
95
91
71
88
Never or rarely
44
53
25
51
44
Moderate frequency
29
22
34
16
24
High frequency
27
25
41
33
32
Never or rarely
59
57
52
65
59
Moderate frequency
26
26
23
17
23
High frequency
15
17
25
18
19
Own laptop at MQ using OneNet
Smartphone at MQ using OneNet
Smartphone using another network provider
Never or rarely
52
52
60
58
56
Moderate frequency
30
17
16
22
21
High frequency
18
31
23
20
23
Never or rarely
60
71
42
67
61
Moderate frequency
30
17
33
17
23
High frequency
11
12
26
17
16
Never or rarely
68
59
49
68
62
Moderate frequency
23
18
23
17
20
High frequency
9
22
28
15
18
Never or rarely
100
98
90
96
96
Moderate frequency
0
0
3
2
1
High frequency
0
2
6
2
2
Tablet at MQ using OneNet
Tablet using another network provider
Other device/network combination
39
STATISTICAL ANALYSIS BY STAFF APPOINTMENT
Comparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Staff Appointment significant at *α = 0.05, ** α = 0.01
2.1 Using
iLearn
with…
Uni-provided
computer at
MQ **
Employment Status
CFT
N = 116
CAS
N = 83
Employment Status
ALL
N = 210
Desktop or
laptop
computer at
other location/s
CFT
N = 116
CAS
N = 83
ALL
N = 210
col %
col %
col %
5
4
7
8
88
88
54
59
26
23
20
19
68
61
18
23
15
16
94
96
1
1
4
2
col %
col %
col %
Never or rarely
4
27
13
Never or rarely
3
Moderate
frequency
5
21
11
Moderate
frequency
9
High frequency
91
53
76
High frequency
89
Own laptop at MQ using OneNet *
Never or rarely
45
Moderate
frequency
29
High frequency
26
Smartphone at MQ using OneNet
39
44
Never or rarely
62
19
24
Moderate
frequency
20
43
32
High frequency
19
Smartphone using another network provider
Never or rarely
55
Moderate
frequency
21
High frequency
24
Tablet at MQ using OneNet
54
56
Never or rarely
57
22
21
Moderate
frequency
25
24
23
High frequency
18
Tablet using another network provider
Never or rarely
60
A few times a
week
19
At least once a
day
21
Other device/network combination
63
62
Never or rarely
98
21
20
A few times a
week
1
16
18
At least once a
day
1
40
STATISTICAL ANALYSIS BY EXPERIENCE
Comparison between views of staff for whom this session was the first experience of iLearn at Macquarie (First
time) and those who had previously used iLearn at Macquarie (Prev Exp).
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
Experience using iLearn
Using iLearn
with…
Uni-provided
computer at
MQ
First
time N
= 25
Prev Exp
N = 183
Experience using iLearn
ALL
N = 208
Desktop or
laptop
computer at
other
location/s
First
time N
= 25
Prev Exp
N = 183
ALL
N = 208
col %
col %
col %
3
4
8
8
89
88
col %
col %
col %
Never or rarely
32
10
13
Never or rarely
12
Moderate
frequency
16
11
11
Moderate
frequency
12
High frequency
52
79
76
High frequency
76
Own laptop at MQ using OneNet
Never or rarely
52
Moderate
frequency
12
High frequency
36
Smartphone at MQ using OneNet
43
44
Never or rarely
64
26
24
Moderate
frequency
16
31
32
High frequency
20
Smartphone using another network provider
Never or rarely
56
Moderate
frequency
20
High frequency
24
Never or rarely
68
A few times a
week
28
At least once a
day
4
59
24
23
19
19
60
61
23
23
17
16
Tablet at MQ using OneNet
55
56
Never or rarely
68
22
21
Moderate
frequency
24
23
23
High frequency
8
Tablet using another network provider
57
Other device/network combination
61
62
Never or rarely
100
19
20
A few times a
week
0
21
18
At least once a
day
0
96
96
1
1
3
2
41
APPENDIX 5: STATISTICAL ANALYSIS OF STAFF SKILLS, CONFIDENCE LEVELS AND
SATISFACTION
STATISTICAL ANALYSIS BY FACULTY
Differences by Faculty significant at *α = 0.05, ** α = 0.01
Overall teaching
experience with
iLearn
Arts
N = 48
FBE
N = 56
FHS
N = 44
Science
N = 62
ALL
N = 210
Mean
SDev
Mean
SDev
Mean
SDev
Mean
SDev
Mean
SDev
I have developed the
necessary skills to
effectively use iLearn
3.9
0.9
3.9
0.9
3.9
1.0
3.8
0.9
3.9
0.9
I feel confident when
using iLearn in my
teaching.
4.2
0.8
4.1
0.7
4.1
0.9
3.9
0.9
4.1
0.8
I feel confident when
using Echo360 in my
teaching.
3.7
1.1
3.7
1.2
3.9
1.1
3.6
0.9
3.7
1.1
I feel confident when
using iTeach to
manage enrolments.
3.6
1.1
3.5
1.2
4.0
1.2
3.8
1.2
3.7
1.2
As a result of using
iLearn I have
reconsidered the way I
teach my unit
3.5
1.4
3.5
1.1
3.2
1.3
3.0
1.3
3.3
1.3
I would like to utilise
the tools in iLearn to
enhance my teaching
in the future.
4.0
1.1
4.3
0.9
4.0
1.3
4.0
1.1
4.1
1.1
I am satisfied with
iLearn as an online
support for my unit
content.
3.8
1.2
3.9
1.0
4.0
1.0
3.8
1.0
3.8
1.0
I am satisfied with
iLearn as a support for
organising my
teaching.
3.6
1.2
3.9
0.9
3.8
1.0
3.6
1.0
3.7
1.0
I am satisfied with the
way iLearn supports
interaction and
communication with
my students.
3.3
1.3
3.7
0.9
3.5
1.0
3.7
1.1
3.6
1.1
I am satisfied with the
way iLearn supports
3.2
1.2
3.7
1.0
3.5
0.9
3.7
0.9
3.5
1.0
42
my students'
engagement with
learning activities.
I feel confident when
using iTeach to create
my Unit Guide
I feel confident when using
iTeach to map learning
outcomes, assessment
tasks and Graduate
Capabilities
3.6
1.1
3.7
1.2
3.6
1.1
3.6
1.2
3.6
1.1
3.3
1.3
3.7
1.2
3.4
1.3
3.6
1.2
3.5
1.2
STATISTICAL ANALYSIS BY STAFF APPOINTMENT
Comparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).
Differences by Faculty significant at *α = 0.05, ** α = 0.01
Overall teaching experience
with iLearn
CFT
N = 116
CAS
N = 83
ALL
N = 210
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
I have developed the necessary
skills to effectively use iLearn
114
3.9
0.9
79
3.9
1.0
20
4
3.9
0.9
I feel confident when using
iLearn in my teaching.
115
4.0
0.8
80
4.1
0.9
20
6
4.1
0.8
I feel confident when using
Echo360 in my teaching.
105
3.7
1.0
46
3.7
1.2
15
8
3.7
1.1
I feel confident when using
iTeach to manage enrolments.
106
3.8
1.2
31
3.6
1.2
14
4
3.7
1.2
As a result of using iLearn I have
reconsidered the way I teach
my unit *
113
3.1
1.3
59
3.5
1.2
18
1
3.3
1.3
I would like to utilise the tools
in iLearn to enhance my
teaching in the future.
114
3.9
1.2
71
4.3
1.0
19
5
4.1
1.1
I am satisfied with iLearn as an
online support for my unit
content.
113
3.8
1.1
72
3.9
1.0
19
6
3.8
1.0
I am satisfied with iLearn as a
support for organising my
teaching.
110
3.6
1.1
69
3.9
0.9
19
0
3.7
1.0
I am satisfied with the way
iLearn supports interaction and
communication with my
students.*
113
3.4
1.2
74
3.8
0.9
19
7
3.6
1.1
I am satisfied with the way
111
3.4
1.1
73
3.8
0.9
19
3.5
1.0
43
iLearn supports my students'
engagement with learning
activities. **
4
I feel confident when using
iTeach to create my Unit Guide.
110
3.7
1.2
39
3.6
1.1
15
5
3.6
1.1
I feel confident when using
iTeach to map learning
outcomes, assessment tasks
and Graduate Capabilities
108
3.6
1.3
38
3.5
1.1
15
2
3.5
1.2
STATISTICAL ANALYSIS BY EXPERIENCE
Comparison between views of staff for whom this session was the first experience of iLearn at Macquarie (First
time) and those who had previously used iLearn at Macquarie (Prev Exp).
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
Overall teaching experience
with iLearn
First time
N = 25
Prev Exp
N = 183
ALL
N = 210
iLearn tool
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
I have developed the necessary
skills to effectively use iLearn
24
3.5
0.9
178
3.9
0.9
20
4
3.9
0.9
I feel confident when using
iLearn in my teaching.
25
3.8
0.9
179
4.1
0.8
20
6
4.1
0.8
I feel confident when using
Echo360 in my teaching.
12
3.5
1.2
144
3.8
1.1
15
8
3.7
1.1
I feel confident when using
iTeach to manage enrolments.
**
7
2.3
1.1
136
3.8
1.1
14
4
3.7
1.2
As a result of using iLearn I have
reconsidered the way I teach
my unit *
20
2.7
1.2
159
3.3
1.3
18
1
3.3
1.3
I would like to utilise the tools
in iLearn to enhance my
teaching in the future.
23
3.7
1.4
170
4.1
1.1
19
5
4.1
1.1
I am satisfied with iLearn as an
online support for my unit
content.
22
3.5
1.1
172
3.9
1.0
19
6
3.8
1.0
I am satisfied with iLearn as a
support for organising my
teaching.
21
3.7
1.0
167
3.7
1.0
19
0
3.7
1.0
I am satisfied with the way
23
3.3
1.0
172
3.6
1.1
19
3.6
1.1
44
iLearn supports interaction and
communication with my
students.
7
I am satisfied with the way
iLearn supports my students'
engagement with learning
activities.
24
3.2
1.1
168
3.6
1.0
19
4
3.5
1.0
I feel confident when using
iTeach to create my Unit Guide
12
3.5
1.2
142
3.7
1.1
15
5
3.6
1.1
I feel confident when using
iTeach to map learning
outcomes, assessment tasks
and Graduate Capabilities
11
3.3
0.8
140
3.6
1.3
15
2
3.5
1.2
45
APPENDIX 6: STATISTICAL ANALYSIS OF STAFF EXPERIENCE WITH ILEARN
STATISTICAL ANALYSIS BY FACULTY
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
Organising
students'
learning.
Arts
FBE
FHS
Science
ALL
iLearn tool
N
Mean
SDe
v
N
Mea
n
SDe
v
N
Mean
SDe
v
N
Mean
SDe
v
N
Mean
SDe
v
Announcements
47
4.2
1.1
50
4.4
0.7
40
4.3
1.0
54
4.3
0.9
191
4.3
0.9
Calendar *
31
3.0
1.2
27
3.8
1.1
22
2.9
1.1
35
3.3
1.1
115
3.3
1.2
Labels
37
3.8
1.0
32
4.0
1.0
32
4.0
1.1
36
4.0
1.0
137
3.9
1.0
Online
Assignment
Submission
48
4.4
0.8
44
4.5
0.6
38
4.2
1.0
52
4.2
1.0
182
4.3
0.8
Turnitin *
46
4.5
0.7
43
4.4
0.7
39
4.0
1.0
40
4.0
1.0
168
4.2
0.9
Grademark
44
4.0
1.1
45
4.0
1.1
35
3.6
1.3
39
3.7
1.2
163
3.9
1.2
Grades
42
3.9
1.0
51
4.1
1.1
39
3.7
1.2
45
3.7
1.2
177
3.9
1.1
Activity logs **
33
3.3
1.2
33
3.8
1.1
26
4.1
0.9
33
3.2
0.9
125
3.6
1.1
Reports
28
3.1
1.2
33
3.7
1.1
22
3.9
0.9
28
3.2
0.9
111
3.5
1.1
Engaging
students in
learning
activities
Arts
FBE
FHS
Science
ALL
iLearn tool
N
Mean
SDe
v
N
Mean
Sde
v
N
Mean
Sde
v
N
Mean
Sde
v
N
Mean
Sd
ev
Dialogue
35
3.3
1.2
37
3.5
1.2
33
3.6
1.3
34
3.8
1.2
139
3.5
1.2
Discussion
Forums
46
3.7
1.2
46
3.9
1.1
41
4.0
1.1
50
3.9
1.1
183
3.9
1.1
Blog
22
2.6
1.2
19
3.1
1.3
15
3.1
1.2
17
3.1
1.4
73
3.0
1.3
Chat
18
2.3
1.2
16
3.1
1.3
3.0
1.3
17
2.7
1.4
66
2.7
1.3
Quizzes
32
3.5
1.2
28
4.1
1.1
24
3.8
1.1
34
3.9
1.1
118
3.8
1.2
Wiki *
15
2.1
1.0
19
2.7
1.2
11
2.9
1.3
14
3.4
1.1
59
2.8
1.2
Database
16
2.4
1.2
16
3.4
1.2
11
3.0
1.2
13
3.4
1.3
56
3.0
1.3
Workshops
18
2.7
1.1
17
3.1
1.2
8
3.3
0.9
12
3.4
1.2
55
3.1
1.2
Twitter feeds
16
2.1
1.1
13
2.8
1.1
8
2.5
1.2
9
3.3
1.5
46
2.6
1.3
Groupings
and Groups
**
29
2.9
1.4
30
3.8
1.2
20
3.7
1.2
24
3.9
1.1
103
3.5
1.3
15
46
STATISTICAL ANALYSIS BY STAFF APPOINTMENT
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
The iLearn platform was effective in meeting my needs…
with respect to organising
students' learning
CFT
N = 116
CAS
N = 83
ALL
N = 210
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Announcements
114
4.3
1.0
67
4.3
0.9
191
4.3
0.9
Calendar
64
3.2
1.1
44
3.5
1.2
115
3.3
1.2
Labels
90
3.9
1.1
40
4.0
0.9
137
3.9
1.0
Online Assignment Submission
107
4.3
0.9
68
4.4
0.8
182
4.3
0.8
Turnitin
99
4.2
0.8
64
4.2
1.0
168
4.2
0.9
Grademark *
94
3.7
1.3
63
4.1
1.0
163
3.9
1.2
Grades
100
3.8
1.2
70
4.0
1.0
177
3.9
1.1
Activity logs
75
3.4
1.1
42
3.8
1.1
125
3.6
1.1
Reports
70
3.4
1.1
35
3.7
1.0
111
3.5
1.1
iLearn tool
for engaging my students in
learning activities.
CFT
N = 116
CAS
N = 83
ALL
N = 210
iLearn tool
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Dialogue
80
3.5
1.3
51
3.6
1.3
139
3.5
1.2
Discussion Forums
104
3.8
1.2
70
3.9
1.0
183
3.9
1.1
Blog
43
2.8
1.2
25
3.2
1.3
73
3.0
1.3
Chat
36
2.5
1.3
25
3.0
1.3
66
2.7
1.3
Quizzes
75
3.8
1.2
36
3.9
1.1
118
3.8
1.2
Wiki
35
2.6
1.2
21
2.9
1.3
59
2.8
1.2
Database
30
2.7
1.1
23
3.4
1.4
56
3.0
1.3
Workshops *
28
2.7
1.1
23
3.4
1.2
55
3.1
1.2
Twitter feeds
26
2.4
1.2
18
2.8
1.3
46
2.6
1.3
Groupings and Groups
65
3.4
1.4
32
3.8
1.1
103
3.5
1.3
47
STATISTICAL ANALYSIS BY EXPERIENCE
Comparison between views of staff for whom this session was the first experience of iLearn at Macquarie (First
time) and those who had previously used iLearn at Macquarie (Prev Exp).
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
The iLearn platform was effective in meeting my needs…
with respect to organising
students' learning
First time
N = 25
Prev Exp
N = 183
ALL
N = 210
iLearn tool
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Announcements
21
4.1
1.0
168
4.3
0.9
191
4.3
0.9
Calendar
14
3.3
1.0
100
3.3
1.2
115
3.3
1.2
Labels
11
3.5
1.1
125
4.0
1.0
137
3.9
1.0
Online Assignment Submission
21
4.2
0.9
159
4.3
0.8
182
4.3
0.8
Turnitin
21
4.1
1.0
146
4.2
0.9
168
4.2
0.9
Grademark
20
3.9
1.2
142
3.9
1.2
163
3.9
1.2
Grades
17
3.9
1.2
158
3.9
1.1
177
3.9
1.1
Activity logs
14
3.5
1.0
109
3.6
1.1
125
3.6
1.1
Reports
8
3.3
1.2
101
3.5
1.1
111
3.5
1.1
for engaging my students in
learning activities.
First time
N = 25
Prev Exp
N = 183
ALL
N = 210
iLearn tool
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Dialogue
14
3.3
1.6
125
3.6
1.2
139
3.5
1.2
Discussion Forums
17
3.4
1.3
164
3.9
1.1
183
3.9
1.1
Blog *
6
1.8
1.0
67
3.1
1.3
73
3.0
1.3
Chat
7
2.3
1.5
59
2.8
1.3
66
2.7
1.3
Quizzes *
11
3.1
1.4
106
3.9
1.1
118
3.8
1.2
Wiki
4
2.3
1.9
54
2.8
1.2
59
2.8
1.2
Database
7
3.0
1.7
49
3.0
1.2
56
3.0
1.3
Workshops
7
3.0
1.6
48
3.1
1.1
55
3.1
1.2
Twitter feeds
5
2.0
1.7
41
2.7
1.2
46
2.6
1.3
Groupings and Groups
10
3.3
1.5
93
3.6
1.3
103
3.5
1.3
48
APPENDIX 7: STATISTICAL ANALYSIS OF STAFF SATISFACTION WITH SERVICES AND
SUPPORT
STATISTICAL ANALYSIS BY FACULTY
Differences by Faculty significant at *α = 0.05, ** α = 0.01
Satisfaction with
support and
services
Arts
N = 48
FBE
N = 56
FHS
N = 44
Science
N = 62
ALL
N = 210
Mean
SDe
v
Mean
SDev
Mean
SDev
Mean
SDev
Mean
SDev
Automatic enrolment of
students facilitated by
iTeach.
4.0
1.0
4.1
1.0
4.4
1.0
4.3
0.7
4.2
0.9
OneHelp for resolution
of issues relating to
iTeach functions.
3.5
1.2
3.6
1.1
3.9
1.1
3.9
0.9
3.7
1.1
OneHelp for resolution
of issues relating to
iLearn functions.
3.4
1.2
3.7
1.2
3.8
1.2
3.9
1.0
3.7
1.2
UNITS/Unit Guides for
providing essential
information to students
3.1
1.4
3.7
1.3
3.1
1.3
3.4
1.2
3.3
1.3
Training and support for
using iTeach to create
Unit Guides
3.6
1.0
3.5
1.2
3.7
1.1
3.6
1.1
3.6
1.1
Ease of locating online
“self-help” resources to
assist me in using
iLearn.
3.3
1.0
3.4
1.0
3.4
1.2
3.2
1.1
3.3
1.1
The effectiveness of the
online “self-help”
resources in assisting
my use of iLearn
3.1
1.1
3.3
1.0
3.3
1.2
3.3
1.1
3.2
1.1
The effectiveness of
iLearn training
workshops
3.7
1.0
3.8
1.1
4.1
0.9
3.7
1.2
3.8
1.0
The support for using
iLearn provided by the
iLearn drop-in clinic
4.1
1.0
3.9
1.0
4.2
1.1
4.2
1.1
4.1
1.0
The level of Facultybased support for using
iLearn
3.9
1.1
3.9
1.2
3.8
1.4
3.3
1.1
3.7
1.2
49
STATISTICAL ANALYSIS BY STAFF APPOINTMENT
Comparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).
Differences by Faculty significant at *α = 0.05, ** α = 0.01
Satisfaction with support
and services
CFT
N = 116
CAS
N = 83
ALL
N = 210
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Automatic enrolment of
students facilitated by iTeach.
105
4.1
1.0
34
4.3
0.8
147
4.2
0.9
OneHelp for resolution of issues
relating to iTeach functions.
89
3.7
1.1
27
3.8
1.1
123
3.7
1.1
OneHelp for resolution of issues
relating to iLearn functions.
104
3.6
1.2
51
3.8
1.1
163
3.7
1.2
UNITS/Unit Guides for
providing essential information
to students. **
109
3.2
1.4
48
3.8
1.0
166
3.3
1.3
Training and support for using
iTeach to create Unit Guides
87
3.5
1.0
37
3.7
1.1
130
3.6
1.1
Ease of locating online “selfhelp” resources to assist me in
using iLearn.
106
3.3
1.1
61
3.4
1.1
177
3.3
1.1
The effectiveness of the online
“self-help” resources in assisting
my use of iLearn
104
3.1
1.1
57
3.4
1.1
171
3.2
1.1
The effectiveness of iLearn
training workshops
88
3.7
1.1
46
4.0
0.9
141
3.8
1.0
The support for using iLearn
provided by the iLearn drop-in
clinic
84
4.0
1.1
33
4.2
0.7
124
4.1
1.0
The level of Faculty-based
support for using iLearn
96
3.6
1.2
51
3.9
1.0
157
3.7
1.2
50
STATISTICAL ANALYSIS BY EXPERIENCE
Comparison between views of staff for whom this session was the first experience of iLearn at Macquarie (First
time) and those who had previously used iLearn at Macquarie (Prev Exp).
Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses.
In the following table, percentages may not sum to 100 due to rounding error.
Differences by Experience significant at *α = 0.05, ** α = 0.01
Satisfaction with support
and services
First time
N = 25
Prev Exp
N = 183
ALL
N = 208
N
Mean
SDev
N
Mean
SDev
N
Mean
SDev
Automatic enrolment of
students facilitated by iTeach.
9
3.8
0.7
136
4.2
1.0
147
4.2
0.9
OneHelp for resolution of issues
relating to iTeach functions.
10
3.8
1.0
113
3.7
1.1
123
3.7
1.1
OneHelp for resolution of issues
relating to iLearn functions.
16
3.6
1.2
145
3.7
1.2
163
3.7
1.2
UNITS/Unit Guides for
providing essential information
to students
13
3.7
1.3
151
3.3
1.3
166
3.3
1.3
Training and support for using
iTeach to create Unit Guides
9
3.4
1.1
121
3.6
1.1
130
3.6
1.1
Ease of locating online “selfhelp” resources to assist me in
using iLearn.
18
3.2
1.1
157
3.3
1.1
177
3.3
1.1
The effectiveness of the online
“self-help” resources in assisting
my use of iLearn
18
3.1
1.1
151
3.2
1.1
171
3.2
1.1
The effectiveness of iLearn
training workshops
15
3.9
1.2
126
3.8
1.0
141
3.8
1.0
The support for using iLearn
provided by the iLearn drop-in
clinic
7
3.4
1.1
117
4.1
1.0
124
4.1
1.0
The level of Faculty-based
support for using iLearn
15
3.5
1.3
142
3.7
1.2
157
3.7
1.2
51