Open-source vs proprietary VLE: An exploratory study
of staff perceptions
Harsh Suri and Margot Schuhmacher
Monash University
This paper explores staff perceptions of two open-source Virtual learning environments
(VLEs), Sakai 2.4.1 and Moodle 1.8, compared to Blackboard Vista 4. In general, staff felt
that Sakai was simpler to use, but lacked some of the richer functionality available in
Blackboard. A general dissatisfaction with both Sakai and Blackboard led to trialling of
Moodle which was perceived more favourably than Blackboard.
Keywords: staff perceptions of open-source and proprietary virtual learning environments
Contextual background
It has been argued that open-source software can help in saving the ever-increasing licensing fee of any
commercial provider such as Blackboard (Wheeler, 2004). Additionally, it opens up opportunities to
further develop the VLE to meet specific needs of an institution or a group of institutions. However,
sometimes they can become compromising with inadequate documentation, less functional features and
numerous bugs, with commercial systems seen as a “safer” option (Young, 2004, p. B12). Further, using
more than one VLE can have workload implications for staff, students and central support.
Blackboard Vista 4i is the centrally supported proprietary Virtual learning environment (VLE) used at
Monash University. With worldwide exploration of the suitability of open-source VLEs such as Moodle,
Sakai, Atutor, Bodington, Claroline, Course Manager and ILIAS (Özdamlı, 2007; Uzunboylu, Ozdamli,
& Ozcinar, 2006), small scale pilots of Sakai 2.4.1 and Moodle 1.8 were trialled in the Faculty of
Information Technology (FIT) of Monash University. This paper reports on perceptions of staff who were
involved in these trials.
Key themes emerging from the interviews about Sakai and Blackboard
In May 2008, six staff from the Faculty of Information Technology (FIT) were interviewed who had used
Sakai 2.4.1 and Blackboard. Two of them had used Sakai for an entire semester while the other four had
used Sakai in lieu of Blackboard when the central system was unavailable for five days (Monash
University upgraded from WebCT Vista 3 to Blackboard Vista 4 in November 2007). In general, staff felt
that Sakai was simpler to use, but lacked some of the richer functionality available in Blackboard.
Nonetheless, neither system was seen as fulfilling staff needs at a satisfactory level.
Interface and ease of use: Blackboard was seen as being “overly complicated for seemingly simple
tasks”. It was described as having a “highly compartmentalised” approach and a “terrible interface”. On
the other hand, Sakai was seen as being “full and clean”; “simpler”; and “typical of the interface that
students would engage with”ii
Accessibility and interoperability: Sakai was regarded as being much more accessible as it was “less
particular as to which browser it runs in” and “more portable, lightweight and responsive”. The restriction
to use a supported version of Java when using Blackboard was regarded as a limitation. Interoperability of
data between systems and “multi platform accessibility” were considered important in any VLE.
Communication tools: Discussion forum of Blackboard was perceived better with an intuitive layout
whereas Sakai was seen as taking a “sequential path” which forced the opening of threads. The chat
feature of Sakai was regarded as being better as it allowed seamless integration of synchronous as well as
asynchronous chats. Although many staff did not use Web 2.0 tools in their current teaching, they felt that
these tools were desirable in a VLE. Whiteboard functionality (available in Blackboard) and Wikis
(available in Sakai) were regarded as highly desirable in any VLE. One interviewee who used Wikis with
his students felt that this was a serious limitation in Blackboard. The Blogs tool of Sakai was also
regarded as being more powerful.
Proceedings ascilite Melbourne 2008: Poster: Suri & Schuhmacher
1002
Distributing resources: Blackboard was preferred for allowing a complex network of folders, documents
and activities to be layered in presentation within a module system (unavailable in Sakai). Sakai was
preferred for allowing RSS feeds.
Assessment and feedback tools: Grade Book, quizzes, assignment and announcement tools were generally
regarded as essential for assessment and feedback. The Grade Book in Blackboard was preferred even
though it was “overly complicated”, with low interoperability, and required “significant effort to use”.
Sakai did not offer any option for group assignment submission. Neither system allowed the recording of
assignment extensions.
Using more than one VLE: Three interviewees noted that using more than one system could be confusing
for students, waste staff time, increase training time and “compound the problem of ignorance”. However,
the other three interviewees felt that it would be good, or unethical not to, provide an alternative method
of accessible learning.
Open Source versus Commercial: There was a general preference for a non-commercial VLE provided it
was “comparable” in functionality and “not detrimental to productivity”. Advantages of an open-source
VLE that were noted included: cost factors, ability to add desirable features and a philosophical
preference for open-source software.
Summary: The overall impression from the feedback of all 6 interviewees suggested that while they liked
the simple interface of Sakai and its Web 2.0 tools, it did not have the functionality to compete at the
same level as Blackboard. Some thought that with improved functionality, modality and architecture,
Sakai could become highly competitive. Even though there was a general dissatisfaction with many
aspects of Blackboard, it was seen as the preferred system. The comparison between the two systems can
be summed up in the following quote: There was a “trade off between the simplicity of Sakai against a
more powerful Blackboard”. Also, a general preference was clearly indicated for an open-source VLE
which was sufficiently advanced to be customised for meeting the specific needs of staff without
compromising on productivity.
Some themes emerging from the survey about Moodle and Blackboard
A general dissatisfaction with both Sakai and Blackboard and an overall support for an open-source VLE
led to trialling of Moodle in 17 FIT units in semester 1, 2008. All the 35 staff involved in these units were
invited to respond to a web-based survey. This section reports on some of the key themes emerging from
the survey responses of 25 staff who had used Moodle.
The first set of questions asked staff to rate the importance of a list of features within a VLE for their
teaching. When staff were asked as to how important was it for a VLE to be open-source, 8% (2) regarded
it as being essential, 24% (6) regarded as being highly desirable, 40% (10) felt it was desirable, 24% (6)
said that it didn’t matter and 4% (1) ticked on the don’t know option. This is an interesting finding as
these staff from FIT are well aware of the pros and cons of open-source versus proprietary software.
As illustrated in Figure 1, the following features were regarded as essential or highly desirable for
teaching by most of the staff: compatibility with most web browsers (76%); portability across platforms
such as Linux, Windows, Mac, etc. (76%); being able to link directly to a specific unit page rather than
having to go through multiple screens to get to a specific page (72%); a functional back button (72%);
and independence from any Java Applet (56%). Although the other features included in the list were not
regarded as essential or highly desirable by a majority of respondents, they were still regarded as
essential, highly desirable or desirable by a majority: sophisticated Gradebook (84%); integration with a
single sign-on, i.e., my.monash portal (72%); tracking data (68%); and Wikis (60%).
Proceedings ascilite Melbourne 2008: Poster: Suri & Schuhmacher
1003
Compatibility with most webbrowsers
56%
20%
52%
Portability across platforms
Link directly to a specific unit
page
24%
48%
Independence from any Java
Applet
32%
Integration with my.monash portal
32%
12%
16%
Tracking data
12%
Sophisticated Gradebook
20%
24%
32%
Essential
12%
20%
40%
25%
50%
Highly desirable
4%
28%
32%
0%
4% 4%
12%
32%
28%
20%
8% 4%
32%
8%
Desirable
8%
8%
16%
36%
24%
20%
Wikis
4%
16%
24%
36%
A functional "back" button
12%
12%
12%
75%
Doesn't matter
4%
100%
Don't know
Figure 1: Rating the features of a VLE on the importance for your teaching
Ease of accessing the system
(n=22)
59%
36%
55%
Overall ease of use (n=22)
Overall reliability of the system
(n=21)
36%
48%
Ease of uploading the resource
files (19)
42%
User-friendly interface (n=22)
41%
24%
32%
Ease of linking the files
appropriately (n=18)
20%
19%
5% 5%
16%
5% 5%
9%
47%
33%
0%
5% 5%
45%
35%
Context-sensitive help (n=17)
5%
33%
40%
12%
22%
60%
6%
80%
Moodle is a lot better
Moodle is a little better
Both are equally good
Both are equally bad
Blackboard is a little better
Blackboard is a lot better
5%
6%
6%
100%
Figure 2: Presenting learning materials and overall ease of use
In questions which compared different features of Moodle and Blackboard, “don’t know” responses were
excluded when computing the percentages as it was assumed that these respondents may not have had
sufficient experience of using those features. As illustrated in Figure 2, most found Moodle easier to use
when compared with Blackboard: ease of accessing the system (95%); overall ease of use (91%); userfriendly interface (86%); context-sensitive help (82%); and overall reliability of the system (72%).
Similarly, staff also preferred Moodle for presenting learning materials: ease of uploading the resource
files (74%); and ease of linking the files appropriately (66%).
As illustrated in Figure 3, only a small number of respondents could compare the tools for assessing and
providing feedback to students. For managing students’ marks, 40% respondents preferred Moodle while
another 40% preferred Blackboard. On all other aspects, Moodle was preferred by a majority: creating a
group project (100%); designing a quiz (88%); giving feedback to students (85%); creating an assignment
(73%); opportunities for students to give peer feedback (63%); marking a group project (60%) and
marking an assignment (59%).
Proceedings ascilite Melbourne 2008: Poster: Suri & Schuhmacher
1004
31%
Giving feedback to students (n=13)
54%
25%
Designing a quiz (n=8)
Opportunities for students to give
peer feedback (n=8)
63%
25%
Marking an assignment (n=12)
17%
Creating a group project (n=6)
17%
45%
31%
18%
9%
17%
15%
9%
17%
8%
8%
31%
60%
Marking a group project (n=6)
0%
13%
83%
8%
Managing students' marks (n=13)
25%
42%
8%
13%
38%
18%
Creating an assignment (n=11)
8%
20%
8%
20%
40%
60%
20%
80%
100%
Moodle is a lot better
Moodle is a little better
Both are Equally Good
Both are Equally Bad
Blackboard is a Little Better
Blackboard is a Lot Better
Figure 3: Assessment and feedback
Blog (n=6)
50%
Sending email notifications
(n=16)
50%
Discussion forum (n=20)
13%
45%
Email (n=16)
38%
Chat (n=11)
36%
0%
33%
17%
19%
25%
19%
6%
6%
5% 5%
20%
31%
45%
6%
6%
6%
9%
9%
20%
Moodle is a lot better
40%
60%
Moodle is a little better
80%
Both are equally good
100%
Both are equally bad
Blackboard is a Little Better
Blackboard is a Lot Better
Figure 4: Communicating with students
Similarly, even though many respondents could not compare specific communication tools within the two
systems, most of those who indicated their preference did so in favour of Moodle: Blog (83%); chat
(81%); discussion forum (70%); sending email notifications (63%); and email (57%). In addition to the
data reported here, several open-ended questions were also included in the survey which provided a
clearer understanding of staff preferences.
Moving forward
As the majority of staff preferred an open-source VLE to a proprietary VLE, the University is seriously
exploring the option of replacing the current proprietary VLE with an open-source VLE. Further
evaluation of Sakai was discontinued as there was little interest among staff to participate in further trials
of Sakai. In addition to the data about staff perceptions that is reported here, surveys were also conducted
to understand how students compared Moodle and Blackboard. As data from staff and student surveys
clearly indicated a preference for Moodle, more trials of Moodle are being conducted in Semester 2 to
examine if this preference is evident even when Moodle is implemented on a wider scale within the
faculty.
Proceedings ascilite Melbourne 2008: Poster: Suri & Schuhmacher
1005
References
Özdamlı, F. (2007, May 3-5). An evaluation of open source learning management systems according to
administration tools and curriculum design. Paper presented at the Seventh International Educational
Technology (IETC) Conference, Nicosia, Turkish Republic of Northern Cyprus,.
Uzunboylu, H., Ozdamli, F., & Ozcinar, Z. (2006). An evaluation of open source learning management
systems according to learners tools. Current Developments in Technology-Assisted Education, 1, 8-11.
Wheeler, B. (2004). The Open Source Parade. EDUCAUSE Review, 39(5), 68-69.
Young, J. R. (2004, September 24). Sakai project offers an alternative to commercial course-management
programs. The Chronicle of Higher Education, p. B12. Retrieved 2008, February 7, from
http://chronicle.com.ezproxy.lib.monash.edu.au/weekly/v51/i05/05b01201.htm
Author: Dr Harsh Suri. Email: Harsh.Suri@calt.monash.edu.au
Margo Schumacher. Email: Margot.Schuhmacher@infotech.monash.edu.au
Please cite as: Suri, H. & Schuhmacher, M. (2008). Open-source vs proprietary VLE: An exploratory
study of staff perceptions. In Hello! Where are you in the landscape of education technology?
Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/suriposter.pdf
Copyright 2008 Harsh Suri and Margot Schuhmacher
The authors assign to ascilite and educational non-profit institutions a non-exclusive licence to use this
document for personal use and in courses of instruction provided that the article is used in full and this
copyright statement is reproduced. The authors also grant a non-exclusive licence to ascilite to publish this
document on the ascilite web site and in other formats for Proceedings ascilite Melbourne 2008. Any
other use is prohibited without the express permission of the authors.
i The authors recognise that Blackboard Learning System (CE & Vista) and Blackboard Academic Suite
are distinct products offered by Blackboard Inc. Perceptions of staff in this study are based on their
experiences with WebCT Vista 3 and/or Blackboard Vista 4. In this paper, Blackboard is used as an
umbrella term for WebCT Vista 3 or Blackboard Vista 4.
ii Throughout this paper, double quotation marks have been used to highlight direct quotes from the
interviewees comments.
Proceedings ascilite Melbourne 2008: Poster: Suri & Schuhmacher
1006