Standard Based Requirements
for LMS Profile Assessment
Gerry Paul C. Genove
Saint Louis University
gepaul_genove@yahoo.com
Cecilia A. Mercado
Saint Louis University
cicsdean@slu.edu.ph
Abstract- One of the major setbacks raised
by the continually changing developments
in technology solutions is the capacity of
the organizations to keep up with these
changes. An upgrade in features and the
introduction of additional modules of
technology solutions do not necessitate an
immediate upgrade of an application. The
proliferation of Learning Management
System (LMS) solutions both proprietary
and open source raises a growing concern
on organizations implementing LMS
particularly on whether existing LMS
solutions are operating within acceptable
service standards both in the perspective
of the users and the industry perspectives.
This study aimed to identify the
standards-based requirements within
which any existing LMS may be assessed
using gap analysis. Requirements include
1) features, 2) technical and 3) support
structure. Result of such can equip
educational
and
technology
administrators
informed decisions on
upgrade or purchase or development of
new LMS solutions. As an application,
the measures developed was used in the
assessment of MySLU LMS. The study
established that the features and technical
profile
satisfied
the
standard
requirements that were set. Overall the
LMS did not satisfy the standard-based
requirements for support structure.
Majority of the components of both
support structure categories identified
gaps in the current support structure set-
up of the My Classes LMS. The results of
gap analysis will not be part of this paper.
Categories and Subject Descriptors- ICT in
Education, eLearning
Keywords- Learning Management System,
LMS features , technical features, support
structure, LMS, Open BBR, profiling
1. INTRODUCTION
Education has become a commodity in
which people seek to invest for their own
personal gain, to ensure equality of
opportunity and as a route to a better life[1].
The introduction of e-learning poised the
Philippine government to effect changes in
the landscape of education in the Philippines.
It created a number of initiatives in
partnership with external agencies since the
year 2000. One form of e-learning tool
that is widespread today is the use of a
Learning Management System.
This is
sometimes
referred
to
as
Course
Management Systems(CMS) or Virtual
Learning Environments(VLE). A Learning
Management System(LMS) is a software
application or Web-based technology used to
plan, implement, and assess a specific
learning process[2]. Vovides et. al [3] noted
that LMS are utilized in education in
different ways and are evolving. It can be
used as a supplement to the traditional
classroom curriculum, i.e., as an electronic
repository of course materials. Instructors
who teach in-class courses may also choose
to use a ‘blended’ approach by utilizing the
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.1
Gerry Paul C. Genove and Cecilia A. Mercado
CMS as a tool to deliver additional or
supplemental course materials to students.
Traditional teaching environments tend to be
teacher-centered while a blended approach
allows instructors to mix things up and to
offer students a more intellectually engaging
learning experience by combining in-class
time with on-line components through the
use of synchronous and asynchronous tools.
Finally, a CMS can be used in distance
education for the delivery of fully online
courses.
The implementation of CMS in
universities followed on the revolution of
educational technology that promised better
quality, learner-centered education and
stipulates that it would deliver more
independent and active students[4]. There
are evidences that LMS is a preferred elearning tool that is continuously gaining
popularity and is being embraced as a major
platform in the educational technology
revolution.
In answer to growing demands of
integrating ICT in education, SLU was able
to implement an e-learning web portal in the
year 2004 which was coined as the MySLU
web portal. The MySLU web portal is a
personalized, consistent, single sign-on web
interface to different services provided by
the SLU. It enables users (students, alumni,
faculty, and other employees) to become
more productive and efficient in using
information
and
resources
intended
specifically for them. It allows different
departments, units, or entities to give
personalized information regarding grades,
classes, library services, research materials,
events and others. Figure
1 shows a
snapshot of the MySLU LMS.
One major component of the MySLU
web portal includes an LMS. Despite the
introduction of such tools, a major problem
besetting the university is the under
utilization of the LMS. According to Evans
[5] “There are many reasons for determining
why teaching staff is not integrating
technology into their classroom lessons”.
This study aimed to develop a measure of
assessing the existing LMS based on
requirements of existing standards at the
same time, address the under-utilization
problems that the current LMS is facing.
Figure 1. Snapshot of MySLU LMS
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.2
The study examines the implementation
issues of the Learning Management System
as well as the environment where this system
operates which includes its stakeholders and
other factors that affects them. The study
also examines the personnel and procedures
in the administration, maintenance and
support given to the LMS. An analysis of
factors directly or indirectly affecting the My
Classes LMS is done in order to pinpoint
requirements that are needed to address
current
problems
primarily
the
underutilization of the LMS. All these are
necessary in the profiling activity.
2. METHODOLOGY
Evaluating software is a significant task
for corporate IT managers, but probable
users of open source software lack an easy,
effective, and trustworthy process for
decision making. There is no widely used
model for assessment. This complicates open
source adoption, as companies assessing
open source software can rarely learn from
each other’s experiences [6]. Several
methodologies exist in evaluating open
source software. These often include a set of
criteria for the evaluation. Some examples
of these evaluation methodologies are the
Open Source Maturity Model by CapGemini
(OSMM CapGemini), Open Source Maturity
Model developed by Navicasoft’s Bernard
Golden (OSMM Navica) and Qualification
and
Selection
of
Open
Source
Software(QSOS) developed by Atos Origin.
Another methodology for an Open Standard
in evaluating OSS is the Open Business
Readiness Rating(Open BRR) developed by
Carnegie Mellon West, SpikeSource,
O'Reilly, and Intel. Its goal is to enable the
entire community (enterprise adopters and
developers) to rate software in an open and
standardized way.
After
careful
and
methodical
considerations, the Open BRR Model was
used to identify the features profile and the
LMS standard profile. Below are the notable
characteristics that led to the choice of
assessment.
a) Open BRR is open and customizable, so
that it can be applied to any business
situation.[6] ;
b) Scoring procedure that assigns a discreet
score between 1 and 5 better
differentiates how a criterion is assessed
[7]. “This is verbally translatable to
Unacceptable, Poor, Acceptable, Very
Good, and Excellent” [6];
c) The practicality of using a 5 point scale
rating will not be applicable for every
criterion. According to Spikesource et.
al [6] “For qualitative metrics, we realize
that not all metrics can be measured in a
range, or if they can, their range may not
fit nicely into a scale of 5” ;
d)
Wasserman as cited by Letillier [8]
argues that Open BRR makes reference
to
standard
evaluation
processes
particularly ISO/IEC 9126 and ISO/IEC
25000 ;
e) Scoring procedure/model is strict and
clear with fewer ambiguities; Wording of
metrics is accurate and extensive [7 ,9];
and
f) Evaluation procedure is simple and
designed to create a specific assessment
of a software [7].
The Open BRR identifies suggested
criteria that can be modified and fine tuned
depending on the software being evaluated.
The identified criteria suggested by the Open
BRR model and its methodologies were used
to create the features profile of the My
Classes LMS.
In the profiling method which includes
review of product documentation, the
Dokeos LMS website and web searches for
related documents were used to complete the
needed information. MySLU LMS is
customized from Dokeos platform. These
information were summarized using a
features mapping to facilitate the easy
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.3
Gerry Paul C. Genove and Cecilia A. Mercado
justification of how each category was
evaluated and scored using the features
assessment rubric. The technical profile
covered two main components : the
hardware and software profile used to
implement the LMS and the SLU Network’s
Intranet and Internet profile. The rubric for
assessing ICT infrastructure [10] was used to
identify the
network and
internet
connection profile of SLU and identify
acceptable standards which was used in the
assessment in this area. Mohktar et. al [10]
presented in his study an evaluation tool that
will evaluate the readiness of an institution
for e-learning. Rubrics are set of categories
that define and describe important
components of the areas being assessed.
Each category contains a gradation of levels
of implementation with a score assigned to
each level. The rubric adopts a 3-point scale
to differentiate the levels of ICT
implementation. The three categories of ICT
implementation are descriptively labelled as
low, moderate and high to represent the
lower, middle and upper tier of the rubric
scale.
In measuring the support structure,
assessment rubric was derived from User
Support Best practices identified by Towns
et. al [11] and variables found on the ICT
Maturity
Tool
specifically
the ICT
Organizational
Support
Assessment
identified by Working Groups Experts[12].
The ICT Maturity tool was developed by
Carnegie
Mellon
and
International
Development Research Center (IDRC) to
assess the readiness of an institution in
integrating ICT in HEIs.
The assessment is based on knowledge
base of global trends in ICT applications in
different universities and studies of academic
experiences and best practices. A component
of this tool addresses the ICT Organizational
Support which is aimed in measuring key
indicators addressing the user support
structure. The ICT Organizational Support
determines the success or failure of ICT
applications in higher education. Since the
study was limited to the support structure
concerns of the LMS system, other
components found on the tool were not
included. Towns et. al [11] identified a
support model that enumerates key
components that should be present in a
support system. These components were
used to identify the profile of the LMS
system particularly addressing the support
model components.
The support model
components identify functional areas or
variables that are critical in the
implementation of a support system
structure. Best practices and experiences by
LMS experts in the industry identified by the
eLearning Guild were reviewed to formulate
metrics for each component found on the
rubric.
The rubric measured the existing support
structure of the LMS system and presented
the efficiency of the current support structure
that was in place. This rubric was used in
identifying the support structure standards
and was also used in the assessment of the
support structure. Lastly the extent of
utilization of the My Classes LMS was
measured using server statistics coming
from the LMS was used to capture how the
system was utilized on the previous
semesters and how it is was utilized in the
current semester.
Information like the distribution of
faculty and student users was presented.
Information on the tools and the time the
LMS is utilized was also presented.
Interviews were conducted with school
administrators that defined the ideal
utilization profile of the LMS.
The study
was guided by the framework as presented in
Figure 2.
3. RESULTS AND DISCUSSION
This section presents the assessment of
current profile of MyClassess LMS using
standards identified. The current profile
includes the features profile, the technical
profile, support structure and the utilization
profile.
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.4
Standard Based Requirements for LMS Profile Assessment
3.1.
MY
PROFILE .
CLASSES
FEATURES
The My Classes LMS was evaluated
based on the evaluation steps given by the
OpenBRR Model. An evaluation was done
for each criterion based on information
collected from the Dokeos product website
and relevant documents and websites.
Table 1 presents the My Classes LMS
Features
Mapping
which
identifies
information sources like related documents
and websites, delineated factors as well as
factors identified to be present on the
My Classes LMS.
Further processing the results, a detailed
assessment using weighted scoring was
done. Each criterion is given a score based
on the metrics specified on the evaluation
rubric.
The weighted rating can be
computed by multiplying the score to the
weight assigned for each criterion.
The
overall rating can be computed by getting the
summation of all weighted scores computed
for each criterion.
From the results, An overall score of
4.506 was given to the My Classes LMS
which has a qualitative interpretation of
excellent. In six out of the seven criteria
specified on the rubric, the My Classes was
given a rating of excellent rating in which
scores ranging from 4.2 to 4.75 was given.
In one of these seven criteria, it was given
very good rating. Table 2 presents the My
Classes Features profile.
3.2. MY CLASSES
PROFILE .
TECHNICAL
This section presents the profile of the
technical infrastructure used to implement
the My Classes LMS. This is divided into
two main components, the hardware and
software components used to implement the
LMS and the SLU Network’s Intranet and
Internet components.
3.2.1 Hardware and Software Profile
Among the things profiled in this section
are, the hardware specifications of the server
running the LMS, the server technologies
that are utilized by the LMS software and the
operating system installed on the server.
The My Classes LMS and its supporting
server applications, MySQL Database server,
Apache web server and the PHP scripting
technology are housed on a single machine
or server. Since the upgrade of the My
Classes to version 2.0, this server is being
utilized for the LMS service of SLU. The
hardware model of the server is the IBM
X3500 Tower Server.
Most of its
components follow the manufacturer’s
default configuration except for the memory
module and the disk devices. The memory
module was upgraded from 1 GB to 4 GB of
fully buffered DIMM running at as speed of
667 MHz. The disk component was also
upgraded to facilitate RAID 1 which
supports a disk mirroring capability
implemented using 2 Serial Attached
SCSI(SAS) devices. The disk capacity for
each disk is 73GB SAS which allows hot
swapping of similar disk devices in case one
of these two devices fails. An extra disk was
added having a capacity of 150GB SAS
whose main purpose is to facilitate the
storage of backup for the system and data
files.
The server is running the operating
system CENTOS(Community ENTerprise
OS) version 5.1. Server technologies or
server application programs to satisfy the My
Classes LMS pre-requisites are pre-installed
components
of
CENTOS.
These
components were further upgraded to
address issues involving security and fine
tuned according to the configuration needs of
the My Classes LMS and other hosted
systems. Table 3 the key components of the
hardware, the OS and server technologies
utilized by the My Classes LMS.
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.5
Gerry Paul C. Genove and Cecilia A. Mercado
TABLE 3.1 :
My Classes LMS Features Mapping
Feature
Information
Sources
Delineated factors
MySLU Classes Assessed Value
Identifies the following features authoring,
interactivity/games/simulation, quizzing/testing, social
functionality(communication and collaboration tools),
delivery, tracking and mobile enabled.
Identifies 9 key features (syllabus, online readings and links
to other text-based courses, keeping track of grades on
assignments and tests, online discussion board, turning
assignments online, access to sample exams and quizzes
for learning purposes, taking exams and quizzes online for
grading purposes, getting assignments back from
instructors from instructors with comments and grades and
sharing materials
Identifies 3 main groups of features. They are the following:
Course based which includes basic course information, file
sharing, communication, groups and test and quizzes;
Access control; Administration and Language Support.
list of features includes course management, content
management, communications, manage assessments,
manage students, groups within courses and gradebook
All key features are present except mobile enabled
FUNCTIONALITY
General
Features of LMS
Center for learning &
Performance technology
for Instructional Course
tools[13]
ECAR Study of
Undergraduate Students
and Information
Technology [14]
Edutools and Common
Wealth of Learning as
cited by Van den
Burg[15]
UC[16]
Smith et. al [17]
McHenry[18]
Special Features
Bersin et. al[19]
Edutools[15]
Manageability of
Functions
24eSolutions [20]
mobile computing
annotation, natural language integration, live multimedia
interaction, quality control, spontaneous group formation,
credits-royalties-modularity, and other structured
interactions--for learning network users
Analytics and Reporting; Integrated performance and talent
management; Web Conferencing, Virtual Classrooms and
Telepresence; Search; and Informal Learning
Management.
online journal, student homepage, accessibility compliance,
content sharing, video services, collaborative site(wiki),
support LaTex, support Tex
ability to update and maintain web applications without
distributing and installing software on potentially thousands
of client computers is a key reason for their popularity as is
the inherent support for cross-platform compatibility
All key features are present
All key features are present
All key features are present except gradebook
which is disabled due to the presence of an inhouse developed class record system called Jcard
or My Grades System.
Not available
Only live multimedia interaction is present with the
option of this feature being activated
Only Web Conferencing & Virtual Classrooms are
present and may be activated as cited by
Kineo(2010)
supports accessibility compliance via SCORM,
support LaTex, support Tex, content sharing and
video services
Web interface is present and is the main UI used to
manage various functions
USABILITY
Bplans[21]
End-user UI
experience
Time for setup
pre- requisites
For installing
open source
software
Kineo[22]
The User Interface(UI) of a website is ultimately how it lets
users know what it has to offer them. If it lacks an easy
navigation scheme users get lost, and never find the
information on a site.
Dokeos UI
user experience
Pre-requisite setup time
Time for vanilla
Dokeos [23]
installation/
configuration
DOCUMENTATION
Dokeos [23]
Dokeos portable
Documentation available and format of documents
Existence of
various kinds of
documentation
Navigation structure or UI is easy and intuitive
The Dokeos LMS is easy to get started and it is
very straightforward to create courses and to
choose what resources you want to make available
Pre-requisites are pre-installed on the CENTOS
Enterprise Server that was installed. Binaries for
windows, Mac and Linux are available and set-up
time is 10 to 30 minutes. The variance in time will
depend on customizations made for the prerequisite
software that is installed. For default setup settings,
Dokeos portable is a intended for vanilla
installation. , Set-up time is less than 15 minutes
Useful documents which include flash tutorials that
contain a step by step instruction that is visually
appealing are present. It also contains PDF
manuals which is an in-depth documentation which
includes the installation and teacher manuals. The
documentation is available in multiple formats like
html, pdf and swf formats. Other useful documents
are also available like the e-learning management
guide, white papers about e-learning, the SCORM
standard, Dokeos LMS as compared to other
Learning or Course Management Systems, tips and
tricks to further enhance the Dokeos LMS.
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.6
Standard Based Requirements for LMS Profile Assessment
TABLE 1 continued.
Feature
Information
Sources
Delineated
factors
My Classes Assessed Value
DOCUMENTATION
Dokeos [23]
Documentation
available and format
of documents
Dokeos [23]
Community
contribution
Dokeos (23]
Kineo [22]
Tally of posted
messages from June
1 - Nov 31, 2009
Community of users
SourceForge
[24]
Dokeos
Subversion(SVN)
Dokeos [23]
Code contributors
Ohloh[25]
Development activity
Dokeos [23]
Tally of posted
messages
From June 1, 2009 to November 31, 2009, a total of 1,320 messages posted. The
average number of messages posted per month during that period is 220.
Dokeos [23]
Support services
Dokeos [23]
Paid support
services
Ohloh[25]
Size of development
team
The site contains active support for installation, development, configuration
customizations and bugs to name a few. This is where the lead developers of the
software interact with the community of users and where solutions to these problems are
also given.
A prepaid technical support assistance is also an option given by the Dokeos company.
This contract involves a 40 hour technical assistance which guarantees for a quick
intervention on almost any topic related to the Dokeos software.
This is one of the largest open-source teams in the world, and is in the top 2% of all
project teams on Ohloh.
Secunia as
cited by
Naraine [26]
Software security
vulnerabilities
Secunia(2009) reports that that for the year 2009, 3 security issues were reported. Of
these 3 security vulnerabilities, 1 was reported during the period starting from June to
November 2009. This security issue was classified as highly critical.
Secunia [26]
Number of
unpatched security
issues
Wiki websites for
Dokeos security
From June 2009 to November 2009, the number of security vulnerabilities still open or
unpatched for the Dokeos LMS is zero(0).
Existence of various kinds
of documentation
User contribution
framework
Average volume of general
mailing list in the last 6
months
Number of unique code
contributors in the last 6
months
SUPPORT
Average volume of general
mailing list in the last 6
months
Quality of professional
support
SECURITY
Number of security
vulnerabilities in the last 6
months that are moderately
to extremely critical
Number of security
vulnerabilities still open
(unpatched)
Is there a dedicated
information (web page,
wiki, etc) for security?
ADOPTION
How many book titles does
Amazon.com give for Power
Search query: “subject:
computer and title:
component name”?
Reference deployment
Dokeos [23]
Useful documents which include flash tutorials that contain a step by step instruction that
is visually appealing are present. It also contains PDF manuals which is an in-depth
documentation which includes the installation and teacher manuals. The documentation is
available in multiple formats like html, pdf and swf formats. Other useful documents are
also available like the e-learning management guide, white papers about e-learning, the
SCORM standard, Dokeos LMS as compared to other Learning or Course Management
Systems, tips and tricks to further enhance the Dokeos LMS.
Community contribution is supported through the forums component of the Dokeos
website, the Dokeos mailing list and the Dokeos SVN website. Several authors forming
the Dokeos community contributes to these set of support documents. The main support
documents are selected and filtered to provide the best set of documentation for the
community of users. These documents are posted and are available on the Dokeos
company website.
A total of 1,320 messages posted. The average number of messages posted per month
during that period is 220.
The Dokeos forums are fairly well populated. There are over 4,000 registered members.
Moodle tops the list of the number of community users with half million users. Compared
to the other non-Moodle open source LMS communities, the Dokeos community has a
smaller number of registered users but a much larger number of forum topics and posts,
which means that the community is more active and supportive.
In the last six months particularly from June to December 2009, Dokeos Subversion(SVN)
21150 to 23338 are listed which represents code modifications and improvements. The
total number of code modifications during this period totals to 2188. The average code
contribution per month is 364.67.
a total of 65 members who act as code contributors or testers for the Dokeos LMS. This
team is lead by Thomas De Praetere, the Dokeos project manager.
Increasing year-over-year development activity. Over the last twelve months, Dokeos has
seen a substantial increase in activity. This is probably good sign that interest in this
project is rising, and that the open source community has embraced this project.
Ohloh makes this determination by comparing total number of commits made by all
developers during the most recent twelve months with the same figure for the twelve
months before that.
There are 3 wiki websites for Dokeos which is dedicated for security. This sites are
updated on a regular basis, reporting on issues concerning security. They are the
following: http://www.dokeos.com/wiki/index.php/Security,
http://securitytracker.com/archives/target/6829.html http://secunia.com/product/4508/?task=advisories
Amazon[27]
Dokeos books
A single book matching the suggested criteria. This book is entitled, Développer des
cours en ligne avec Dokeos 1.8.x.
Dokeos
website
Worldwide map of
Dokeos
deployment
There are a total of 8108 portals, 157,442 courses and 2,090,526 users. The main
bulk of users supporting the Dokeos system can be found in Europe, Brasil, Chile
and the United States of America
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.7
Gerry Paul C. Genove and Cecilia A. Mercado
TABLE 2.
My Classes Features Profile
CATEGORIES/METRICS
Functionality
%
25%
ACTUAL VALUES
-
SCORE
INTERPRETATION
4.4
Excellent
General Features of LMS
70
More than 90% percent of standard features
are present
5
Excellent
Special Features
10
3 Special features namely video
conferencing, virtual meeting and
compliance to standards
3
Acceptable
20
All interfaces are web-based, main interface
is GUI with a standard point and click
interface, web browser is the only
requirement (thin client)
5
Excellent
4.75
Excellent
Manageability of Functions
Usability
20%
-
End-user UI experience
50
Simple & Intuitive, information is well
organized, no manual required
5
Excellent
Time for setup pre- requisites
For installing open source
software
25
10 – 30 minutes
4
Very Good
Time for vanilla installation/
configuration
25
< 10 minutes
5
Excellent
-
5
Excellent
5
Excellent
5
Excellent
4.2
Excellent
3
Acceptable
5
Excellent
Documentation
15%
Existence of various kinds of
documentation
70
Install/deploy, user, admin, optimization,
upgrading, development documentation is
available in multiple formats (pdf, single html,
multi-file html).
User contribution framework
30
People are allowed to contribute, and
contributions are edited / filtered by experts
Community
12%
Average volume of general mailing list in the last 6 months
40
Number of unique code
contributors in the last 6 months
60
Support
10%
220 messages per month
2188
-
4
Very
Good
Average volume of general
mailing list in the last 6 months
50
220 messages per month
3
Acceptable
Quality of professional support
50
Installation + troubleshooting + integration /
customization support
5
Excellent
Security
Number of security
vulnerabilities in the last 6
months that are moderately to
extremely critical
Legend: 1.0 to 1.79
1.8 to 2.59
2.6 to 3.39
3.4 to 4.19
4.2 to 5.0
10%
-
4.5
Excellent
50
1
4
Very Good
- Unacceptable
- Poor
- Acceptable
- Very Good
- Excellent
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.8
3.2.2 Network and Internet Connection
Profile
Using a rubric formulated to measure the
Internet and Network indicators, the network
and internet component supporting the LMS
were evaluated. The findings on this section
were collated using interviews from the SLU
Net Office Director and the Network &
System and relevant documents they have
provided. The Speed test utility was also
used to further support the findings on the
interviews and documents collected.
when accessed outside the SLU network
would need the uplink traffic of the
university’s internet connection to download
information from the My Classes server.
Since the LMS server is installed within the
SLU network, LMS users would need the
local area network facilities when they are on
the SLU campus plus the downlink traffic of
the university’s internet connection when
accessing links specified on a course found
on the LMS.
B. Network and Internet Indicators
A. Brief Overview of the SLU Network
B.1. Network Specification.
The SLU Network maintains approximately
1400 Personal Computer(PC) clients,
majority of these machines are found on the
13 computer laboratories spread in different
buildings in the SLU main campus. Two
hundred sixteen (216) computers are
stationed on the Internet Library on the
Charles Vath Building.
The remainder
represents the computer units delegated to
different offices and other laboratories in the
university. All of these PCs have the
capability of using the services available on
the network including the internet service.
Office PCs can access the Internet services
by going thru a Network Address
Translation(NAT) server or commonly
known as a proxy server. The same facility
is also used by students to connect to the
internet through the use of their prepaid
internet accounts.
The SLU Network backbone runs a speed of
100 mbps and employs a combination of
fiber optic technology and fast ethernet
technology to link the different buildings
together. The backbone has the capability of
attaining speeds of 1 Gbps or more when
upgrades are done on the switching
equipments for the major nodes[28]
.
Computers found on each building run the
IEEE 802.3u protocol, commonly known as
fast Ethernet. Fast Ethernet supports a
maximum transfer speed of 100 mbps.
Figure 3.1 shows the network backbone
diagram of the SLU network. Using the
network and internet infrastructure rubric, a
score of 2 which has a qualitative
interpretation as moderate level of
implementation was given to this indicator.
The My Classes LMS utilizes the SLU
Network resources as well as the internet
connection resources of SLU. The SLU
Network resource is vital for connecting the
different servers to make the service
available to its users. It is also important in
providing the availability of the LMS service
within the university.
The internet connection of SLU is very
important in providing services to people
wanting to access the LMS outside the SLU
network. It has been observed that the My
Classes LMS is accessed by many users even
after class hours which are evident on the
LMS server access logs. The LMS service
B.2 Wireless Coverage
Wireless LAN services are available on the
SLU network. These services are available
to users inside the university campus. The
service is regulated to ensure that only
bonafide and legitimate users of SLU avail
of the service.
Wireless services are
available both on the main campus and the
Gonzaga campus. Figure 3.2 shows the SLU
Network wireless access points diagram.
There are exactly 20 hot spots positioned on
key areas on the SLU main campus and the
Gonzaga campus.
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.9
Gerry Paul C. Genove and Cecilia A. Mercado
TABLE 3.:
My Classes Hardware and Software Profile
Component
Current Profile
/Property
Hardware (IBM X3500 Tower Server)
Processor
Memory
Disk
Ethernet
’/Network
Product
Documentation
Other HW
components:
Optical
Devices
Monitor
Keyboard
Mouse
Software
Operating
System
SW Technology :
Apache
SW Technology :
PHP
SW:
Administrative
Tools (SSH,
MySQL
Administrator &
Query Browser,
SFTP)
1 Quad-Core Intel Xeon Processor X5450
(3.0GHz 12MB L2 1333MHz 120W)
4GB Fully Buffered DIMM 667MHz
2 units 73GB 10K 2.5" Hot-Swap SAS HDD
configured to support RAID 1 (disk mirroring)
- 1 unit 146GB 10K 2.5" Hot-Swap SAS HDD
2 built-in 1 GB Ethernet (Integrated dual
Gigabit Ethernet)
The product documentation is available online
and available using the Product’s DVD/CD
application and documentation add-ons
IBM 16X RAM-Read DVD-ROM IDE Drive
IBM C117 17 CRT Monitor
IBM Preferred Pro Keyboard USB - US
English 103P
IBM 2 Button Optical Wheel Mouse ( Black
USB)
CENTOS 5.1 (Community ENTerprise OS)
Version 2.2.3
Version 5.1.6
All of these technologies are installed on the
server and other access stations for system
administration
The main campus utilizes 18 wireless access
points while the Gonzaga campus utilizes 2
wireless access points.
These hotspots
supports the IEEE 802.11a, IEEE 802.11b
and IEEE 802.11g protocols which has a
maximum indoor propagation distance of 45
meters
and
a
maximum
outdoor
propagation distance of 90 meters and
a
peak
speed
of
54 mbps
(Compnetworking, 2010). It is estimated
that about 50% of the campus is covered by
the wireless LAN services provided by the
SLU Network. There are plans that these
wireless LAN devices be upgraded to a
better wireless technology particularly IEEE
803.11n which offers greater bandwidth of
over 100 mbps that supports a maximum
indoor propagation distance of 70 meters and
a maximum outdoor propagation distance of
250 meters. [28] Using the information
collected from the interview and the network
and internet infrastructure rubric, SLU’s
wireless infrastructure was given a score of 2
which has a qualitative interpretation of
moderate level of implementation.
B.3 Internet Bandwidth
The university is connected to Bayan
Telecommunication Inc.(BayanTel) using
FastE(Fast Ethernet). FastE can be used as
access method to the internet and is often
less expensive (10gea, 2010).
The
connection is capable of transmitting up to
14 mbps in synchronous mode for each
direction. Synchronous data transfer can
bring about a maximum bandwidth
throughput of 28 mbps, which can be
attributed to 14 mbps downlink and 14 mbps
uplink.
For the last five years, the
university’s
internet
connection
was
improved to provide a better internet service.
In 2005, the university’s Internet bandwidth
utilized the service called Frame Relay from
BayanTel having a bandwidth of 6 mbps.
For the year 2006, this was upgraded to 8
mbps. In 2007, the internet connection
service was upgraded to a broadband service
called FastE in order to accommodate greater
bandwidth and a more cost effective
connection. During this period, the internet
bandwidth was upgraded to 10 mbps. This
was further upgraded to 12 mbps in 2008 and
14 mbps in 2009 [29] Using the network and
internet infrastructure rubric, the SLU’s
internet bandwidth was given a score of 3
which has a qualitative interpretation of high
level of implementation.
B.4. Network Performance
SLU network services and applications
utilize the SLU local area network for its
delivery. There are no appreciable delays
that are encountered when using these
services from the LAN [28]. Some examples
of these systems are the online enrolment
system and the faculty and students’
information systems to name a few. Using
the network and internet infrastructure
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.10
Standard Based Requirements for LMS Profile Assessment
rubric, the SLU network performance was
given a score of 3 which has a qualitative
interpretation
of
high
level
of
implementation.
B.5 . Internet Performance
Figure 2 shows a recent snapshot using the
Multi Router Traffic Grapher(MRTG) server
of the network traffic status of the internet
traffic leading to the SLU network. The
graph shows a saturation of the downlink
data traffic starting from 8am to 10am and
from 1PM to 4 PM. A saturation of traffic
indicates that the bandwidth allocated is
being utilized to the maximum. The graph
also shows that the uplink traffic doesn’t
experience saturation in any point of the day.
The rating on the internet performance can
be evaluated using several approaches. The
downlink speed was evaluated and was given
a qualitative rating of “generally works well
but slow at busy times” especially for
applications which involves the transfer of
large data like video streaming. It was also
noted that on applications involving minimal
amount of data download, the internet
connection was evaluated to have a
qualitative rating of “always smooth without
appreciable delay”. The uplink speed of
SLU’s internet connection was evaluated and
was given a qualitative rating of “always
smooth without appreciable delay”. Overall
the internet connection was given a
qualitative rating of “always smooth without
appreciable delay”. [29]
Figure 2. Snapshot of the MRTG System
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.11
Gerry Paul C. Genove and Cecilia A. Mercado
The online speed testing facility provided by
Speedtest.net provides services to benchmark
an active internet connection using real-time
point to point testing. This tool is used by
various ISPs to measure the real time speed
or bandwidth from the Internet Service
Provider’s end to the client’s network. This
test was performed to measure the
performance of the SLU Internet connection
which is based on the actual bandwidth
received by a PC. This was done in order to
exactly know how much bandwidth is
available from the 14 mbps total that is
allotted or available for a PC in the SLU
network. The data gathered using this tool
was used for making recommendations and
also measure the real-time bandwidth that
propagates over PCs and servers on the SLU
network. As of February 15, 2010, Speedtest
ranks Philippines as 76th worldwide for the
download speed having an average of 2.58
mbps and 66th for the upload speed which
averages 0.68 mbps. Figure 3.4 shows a
snapshot of Speedtest.net’s top countries
ranked according to speed.
Figure 3 : Speedtest.net’s Top Countries Internet Speed Ranking
Source : http://www.speedtest.net (February 15, 2010)
The internet connection was sampled for
four times during school days particularly at
7:30am, 12pm, 4 pm and 7 pm. The initial
test done involves the ISP provider of SLU
which is Bayantel. From November 26 to
January 30, 2010, the average download
speed recorded was 8.31 mbps and the
average upload speed recorded was 8.37
mbps. Four other servers were used for the
testing of the bandwidth particularly located
in Los Angeles, New York, Paris and Sydney
Australia. These servers were part of the test
to generate a result which includes a set of
site variations where the ping value and the
distance covered would be larger and farther
respectively. From November 26 to January
30, 2010, the average download speed
recorded for all testing sites was 3.78 mbps
and the average upload speed recorded was
2.224 mbps.
Figure 4 shows the Speedtest results of the
SLU proxy server covering the period of
January 11, 2010 to February 15, 2010. This
information involves 341 total tests taken
wherein all tests that were conducted that
registers the IP address of SLU’s proxy
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.12
Standard Based Requirements for LMS Profile Assessment
server is used for the computation of the
average download and upload speeds. The
figure shows that the average download
speed covering this period is 2.57 mbps and
the average upload speed is 3.52 mbps. The
figure further gives the details of bandwidth
available for a PC during peak hours or a
saturation of the downlink of the internet
connection of SLU. In contrast, the uplink
traffic offers a better bandwidth. The figure
also shows benchmark information of SLU’s
internet connection in relation to other ISP
locations. This information is very useful in
benchmarking the internet connection of
SLU in relation to other institution in the
Baguio or Benguet locality, the country and
the international setting.
Factoring in the complexities of the internet
requirement of My Classes LMS, the uplink
traffic is considerably more important than
the downlink traffic because of the fact that
this connection supports users who are found
outside the SLU network. Using the data
gathered,, the internet connection of SLU
was described as “performance is always
smooth without appreciable delay”. My
Classes LMS internet connection profile was
given a score of 3 which has a qualitative
interpretation
of
high
level
of
implementation
In 3 out of the five network indicators, a high
level of implementation was observed
particularly for internet bandwidth, network
performance and internet performance. A
moderate level of implementation was
observed for the indicators, wireless
coverage and network specification.
TABLE 4.
SLU Network and Internet Profile
Network and
Internet
Indicators
Network
Specification
Internet
bandwidth
Actual Values
Qualitative
Interpretation
100 mbps fast Ethernet
Moderate
14 mbps (synchronous)
High
Wireless
around 50% of learning
coverage
area is covered
Moderate
Always smooth without
appreciable delay
Network
Propagates speeds of at
performance
least 10 mbps to
High
maximum of 100 Mbps
and 1 Gbps
Internet
Always smooth without
performance
appreciable delay
High
Table 4 summarizes the findings of the
network and internet infrastructure of SLU.
Figure 4. Speedtest.net’s Result for SLU’s Internet Connection
Source : http://www.speedtest.net (February 15, 2010)
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.13
Gerry Paul C. Genove and Cecilia A. Mercado
3.3. MY CLASSES
STRUCTURE PROFILE
SUPPORT
The support structure is profiled using two
catergories: support model components and
the support staff.
A.
Support Model Components
a. User Information and Tools
The My Classes LMS relies on the Dokeos
website for User Information and Tools such
as FAQs, discussion forums and online
documentation. During the first version of
the My Classes LMS, a customized version
of the Dokeos Teachers Manual and the
Dokeos Student manual was created and
posted online. During the upgrade of the My
Classes LMS, these manuals were not
updated nor were they posted online. The
lack of manpower support to build and
update these documents is one major reason
why there is no localized version of these
documents. The information tools that are
needed by the My Classes is highly
dependent of the resources provided by the
Dokeos website.
A teacher’s manual
corresponding to the version of Dokeos
implemented in SLU is posted on the Dokeos
company website. A list of useful tutorials
which is highly interactive is also part of the
documentation.
Using the metric on the Support Structure
rubric to score this variable, the My Classes
LMS was given a score of 2 which can be
interpreted as poor level of implementation.
The user information and tools are available.
However, they are incomplete and not
updated.
b. Service Level Agreements and Policies
A manual of operating procedures exists in
Saint Louis University to guide different
offices on how services should be rendered
to its clientele and how each office affect the
operation of another. Despite its presence,
there are no Service Level
Agreements that is present for any office
involving services being rendered.
Policies exist and guides users with their ICT
needs.
The ICT Research Lab has an
acceptable use, forum posting and web
content policies to guide its users in the
proper decorum in their use of various
systems maintained by the office. However,
a limited set of policies are present involving
the use of the LMS and no SLAs are present
for its users.
Using the metric on the Support Structure
rubric to score this variable, the My Classes
LMS was given a score of 1 which is
interpreted as unacceptable level of
implementation. A set of policies are present
but incomplete and no SLAs are present.
c. User Accounts and Allocation Procedures
Accounts in the My Classes system are
dependent on the MySLU portal system.
This is because the My Classes system is
implemented as a portlet on the portal server
called MySLU. This implies that in order to
avail of the services of the LMS, the user
goes through procedures of account
application in the MySLU portal.
Application of accounts is done online via
the registration page of the MySLU portal.
Student accounts are activated once these are
verified by the administrator of the portal.
The verification procedure involves a semiautomated system which is executed through
batch processing.
This verification makes sure that only
bonafide students, administrators, faculty
and alumni of SLU are given accounts.
Faculty and administration staff accounts are
activated after supporting papers such as
letter of request are forwarded to the ICTR
Laboratory head are approved.
These
accounts are given administrative roles on
the portal as well as some of its portlets
which include the My Classes LMS.
Problems concerning user accounts in the
portal as well as the LMS are forwarded to
the ICTR Laboratory email account. If
problems are unresolved, users are advised to
visit the ICTR Laboratory for a follow-up on
their support needs.
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.14
Standard Based Requirements for LMS Profile Assessment
Using the metric to score this variable, a
score of 5 which is interpreted as excellent
level of implementation was given to the My
Classes LMS. User accounts and allocation
procedures are supported online and
measures are in place in case problems
occur.
d. Education and Training
User support training is given by the ICTR
Laboratory for users of the LMS. Normally,
this is done only during semester breaks to
accommodate faculty users of different
departments. There are no regular scheduled
trainings for either faculty or students.
Trainings are normally conducted in a
request basis. Since the deployment of the
LMS in 2004, there were only five faculty
trainings conducted wherein most of the
participants were school administrators and
department heads.
These training were
either sponsored by the college or a
department requesting for LMS training
services. After a faculty member of a
college attends training, he is encouraged to
train his colleagues as well as his students.
No LMS training was conducted for student
users
Using the metric on the Support Structure
rubric to score this variable, a score of 2
which is interpreted as poor level of
implementation was given to the My Classes
LMS. Education and training are available
on-demand basis and attended by selected or
few users.
e. Help Desk Process
For all offices catering to the ICT needs of
the university, there is no dedicated help
desk to support user needs. The SLUNet
office provides key support services
concerning hardware and network services
but no dedicated help desk is available.
There are plans in the future to setup a
dedicated help desk [28].
The MISO
provides support for in-house developed
systems. However, no dedicated help desk is
present but similar functions exists which is
close to the functions of a help desk [30].
The ICTR Laboratory gives support to the
portal users as well as the LMS users.
However, there is no dedicated help desk to
support user needs.
Using the metric on the Support Structure
rubric to score this variable, a score of 2
which is interpreted as poor level of
implementation was given to the My Classes
LMS. User support in the form of a help
desk is limited only to the availability of
office staff and technical support response
time is greater than 24 hours.
f. Support Staff Information and Tools
The
ICTR
Laboratory
coordinator
administers major support services for the
LMS. In terms of tools to administer support
services, tools are almost complete. This is
attributed for the service being rendered
online and its tools to administer support
services are web-enabled.
In terms of administrative support for the
LMS system which is rendered by the
SLUNet Office and the Management
Information Systems Office (MIS Office),
the set of tools and information is sufficient.
This is attributed to the fact that there are
only minimal responsibilities delegated to
these two offices when it comes to the
administration of the system. Generally,
only
the
networking
configuration
parameters information as well as the
physical network connection is delegated to
the SLUNet Office. The MIS Office on the
other hand provides an online student
information database which is used by the
MySLU portal to activate accounts.
Using the metric on the Support Structure
rubric to score this variable, a score of 4
which is interpreted as very good level of
implementation was given to the My Classes
LMS. The set of support staff information
and tools which are digitally available are
almost complete.
g. Measuring Success through a Feedback
System
A feedback mechanism is available through
the use of the ICTR Lab e-mail account.
Since its introduction, there were no formal
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.15
Gerry Paul C. Genove and Cecilia A. Mercado
surveys except for the survey conducted in
this study to identify the views of the users.
The feedback mechanism using e-mail
suggests its inefficiency because of few
responses given by the users and only done
voluntarily.
The review of limited
information
becomes
a
pointless
responsibility for the ICTR Laboratory
coordinator because it does not represent the
majority of users.
Using the metric on the Support Structure
rubric to score this criterion, a score of 2
which is interpreted as poor level of
implementation was given to the My Classes
LMS. An online feedback is available
through the e-mail facility. Technology is
not used to review user assessment
information.
B. Support Staff Components
a. Support Responsibilities
Support for end-users is available for the My
Classes LMS users. The ICTR Laboratory
accommodates users having any problem
with the LMS. The structure available is
centralized.
User support is available
depending on the schedule of work duty of
the ICTR Lab Coordinator. This schedule
changes for every semester depending on the
schedule of the teaching load given to the
ICTR Lab coordinator. Urgent matters are
attended to immediately while less serious
matters are scheduled by appointment on
times the student or faculty can be
accommodated
by
the
ICTR
Lab
coordinator. This period usually takes a day
(minimum) to a couple of days (maximum).
No service level agreements are present in
terms of the end-user support.
Using the metric on the Support Structure
rubric to score this variable, the My Classes
LMS was given a score of 2 which is
interpreted as poor level of implementation.
End-user support is centrally available with
limited capacity.
b. Staff in the following technical ICT areas
Support staff responsibilities for the
administration and maintenance of the LMS
server is delegated to the ICTR Lab
Coordinator. Minor development issues with
regards to some bugs on the software are
also managed by the ICTR Lab coordinator.
There are no major code developments done
on the LMS system except for the video
conferencing and net meeting component
which involves more issues on system
administration. The SLUNet office whose
technical expertise includes hardware
maintenance and repair and network
management extends assistance in times
where there are configuration changes made
on the network. They are also responsible in
making sure that the network connection of
the LMS server is up 24/7. The role of the
MIS Office whose technical expertise
includes systems analysis and design, and
intranet and internet application development
are limited in terms of the LMS system.
There is no dedicated staff to act as help desk
for the LMS service.
Using the metric to score this variable, the
My Classes LMS was given a score of 4
which is evaluated as very good level of
implementation.
c. Staff in the following ICT functional areas
The following functional areas involving
system
administration
and
system
maintenance and control directly affecting
the LMS is delegated to the ICTR
Laboratory coordinator. Services rendered
by the SLUNet, MIS Office and ICTR Lab
are structured using a centralized system of
user support. Using the metric to score this
variable, the My Classes LMS was given a
score of 3 which is evaluated as acceptable.
Table 5 summarizes the findings for the My
Classes Support Structure Profile.
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.16
Standard Based Requirements for LMS Profile Assessment
TABLE 5.
My Classes Support Structure Profile
Support Structure
Component /
Variable
Actual Values
A.
Score
Qualitative
Interpretation
After an analysis of the LMS logs the
following were observed. Majority of users
belong to the School of Computing and
Information Sciences (SCIS). Most courses
created belong to SCIS totalling to 79% of
all courses . Some schools, particularly the
School of Teacher of Education and School
of Humanities do not have teachers or
courses that has been created under their
college
Support Model Components
User Information
and Tools
Available but
incomplete (less
than 50%) and not
updated
2
Poor
Service Level
Agreements &
Policies
None
1
Unacceptable
5
Excellent
2
Poor
User Accounts
and Allocation
Procedures
Education and
Training
Help Desk
Process
Supported online
and measures are in
place in case
problems occur
On-demand and
attended by selected
or few users
Limited to availability
of support staff
Technical support
response time
greater than 24
hours
2
Poor
Support Staff
Information and
Tools
More than 80%
available
4
Very Good
Measuring
Success through
a Feedback
System
An online feedback
is available.
Technology not used
to review student
assessment
information.
2
Poor
Support
responsibilities
Centrally available
with limited capacity
2
Poor
Staff in the
following
technical ICT
areas
Network
management only
plus administrative
system analysis and
design . Hardware
maintenance and
repair and Database
management
4
Very Good
Staff in the
following ICT
functional areas
Maintenance and
control plus system
administration
3
Acceptable
B. Students Survey
Support Staff
3.3. MY CLASSES
PROFILE
My Classes Server Statistics
UTILIZATION
The assessment of the My Classes utilization
was performed through a quantitative
analysis. A qualitative review of comments
given by faculty and students was also
performed. These information were used to
identify the factors that affect the
stakeholders’ utilization of the My Classes
LMS.
After an analysis of the Students Survey the
following were observed. Almost 7 out of 10
or 69.23% of students from other Colleges
have not taken a course that used the LMS.
Almost 1 out of 2 students or 45.43% who
do not use the LMS reports the reason that
they “do not know they were available” as
the main reason why they do not use the
LMS. 32.15% of non-LMS users reports the
reason that there teacher/s do not use the
LMS as the main reason why they do not use
the LMS. Most students are mainstream
adopters of technology. Students generally
agree that the use of IT tools and software
positively impacts them in their courses.
Students disagree with the statement “The
university’s IT services are always available
when I need them for my coursework”.
There is no significant difference on IT
perceptions of students from the SCIS and
students from other colleges of IT. Almost 3
out of 4 students or 72.38% of students who
do not use the LMS are willing to use the
LMS once some issues regarding the LMS
are resolved
C. Faculty Survey
After an analysis of the Faculty Survey the
following were observed. Ninety eight
percent or 98.2% of teachers from other
schools do not use the LMS, 71.33 percent
of them did not know that an LMS facility
was available and 37.06% of them indicated
that they have not received adequate training
for their use. Two (2) out of ten (10 )
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.17
Gerry Paul C. Genove and Cecilia A. Mercado
instructors in the SCIS do not use the LMS
but are willing to use it once some of their
concerns are addressed. Faculty users from
other colleges currently use the LMS less.
They attributed this due to the unavailability
of resources (computers) to access the LMS.
For faculty users of other colleges, the main
motivational reason why they use the LMS
was they learned about the LMS from
training
offered
by
on-campus
teaching/technology center.
For faculty
users from the SCIS, 75% was due to self
motivation. Fifty or 50% of faculty from
other schools do not know the advantages of
an LMS and 62.07% do not know the
disadvantages of the LMS. This leads to a
speculation that faculty from other colleges
do not know the benefits of using an LMS.
Eighty five and fifty two percent 85.52% of
faculty from other colleges are interested in
receiving LMS training. However, 14.48%
of them are not interested in receiving LMS
training.
D. Discussion of Utilization and the
Surveys’ open question
Almost 2 out of 100 faculty use of the LMS
based on the results of the survey conducted
for faculty belonging to other colleges.
Almost 31 out of 100 students from other
colleges use the LMS. In the SCIS , 8 out of
10 faculty use the LMS. Almost 95 out of
100 students in the CICS use the LMS. The
high utilization in the SCIS can be attributed
to courses that are inclined to use
technology. Moreover, the presence of
computer laboratories for SCIS courses
serves as resource for accessing the LMS.
Students depend on the availability of
courses created by teachers for them to enrol
as well as use the LMS facility which can
directly affect the LMS utilization of
students. It is also through the teacher’s
motivation that students are obliged to be
part of the LMS.
An open question was asked for student and
faculty respondents on their other insights of
the LMS. For students who were using the
LMS, a lot of positive comments were
attributed to how the LMS helped them in
their studies. One common comment which
was observed paid tribute to how easier for
them to download needed course materials.
They also praised how requirements can be
passed online. However, some students were
unhappy about the unavailable service or
down times during night times and
weekends. Since the technical support for
the LMS is limited to personnel and office
hours, this problem still persists. Some
comments also suggest that the LMS and its
support service be operational on a 24/7
basis.
For students who were not able to use the
LMS, most comments received was “we
really don’t know what an LMS is”. For
faculty respondents, a common comment
given was a complaint about information
dissemination about the LMS, lack of
orientation and training. Others comment on
their interest in attending orientation and
training. Others suggest that a manual for
the LMS be distributed. Other comments
addresses the lack of access station or
computers found on their faculty room and
indicated that it will be impossible for them
to use the LMS because of the needed
resources.
These comments validate the problem areas
that were identified in the gap assessment in
the support structure wherein most problems
identified by the end-users are direct effects
of components which are deficient. Overall,
these problems contribute to the low
utilization of the LMS for other colleges
apart from the CICS.
E. Acceptable Utilization Level
The LMS is surely one of the technology
initiative that the university has supported
over time, particularly so, as this has been
one of the supported project . The
development of the ICTR lab and the
creation of a position of an ICT research lab
coordinator to manage such system as well
as its continuing support despite the end of
project 3 is a proof of the university's support
for alternative form of learning.
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.18
Standard Based Requirements for LMS Profile Assessment
Given the current set-up, wherein the LMS is
used as a supplemental tool, the
quantification of acceptable level of LMS
utilization is difficult. In such case, while
there is a course that is created, deployed and
maintained in the LMS, the use of LMS is
acceptable. The need to emphasize on the
creation of policies, support for training,
infrastructure and resource requirements
should be set and supported before any
acceptable utilization level for the LMS can
be set.
ECAR sponsored a study of students and
information technology starting in the year
2004. These studies were done on the
succeeding years which include 2009 to
either follow up or survey new IT related
topics. These studies emphasized on IT used
by students, their IT skills and attitudes
towards IT. Part of this study asks students
about a CMS or an LMS. Figure 3.26 shows
the utilization of students on a yearly account
from 2006 to 2009. The figure shows that
there has been an increase of utilization of
students using a CMS starting to 2006 to
2009. The figure also indicates a high
utilization of students of the CMS which
ranges from 79.7% to 91.0%. According to
Salaway and Caruso (2007), the ECAR
findings about increased CMS activity are
corroborated by current data from both
EDUCAUSE and the Campus Computing
Survey. They also argue that these reports
point to course management systems’
accelerating
role
as
mission-critical
application for teaching and learning.
Figure 3.26 : Students Utilization of CMS on ECAR Surveys
The Educause 2008 Core Data Report as
cited by Smith et. al (2009), suggest that
there is plenty of growth in CMS use
because despite the near ubiquity of CMSs
on campuses, the vast majority of institutions
reported that faculty use CMSs selectively,
and fewer than 40% of institutions report that
the CMS is used for all or nearly or courses.
This shows that in spite a great number of
students who were reported to have used the
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.19
Gerry Paul C. Genove and Cecilia A. Mercado
CMS, the utilization of faculty of the LMS is
still on a moderate level.
The Educause studies define high utilization
levels in students and a moderate level of
utilization for faculty. These studies give a
benchmark on what level of utilization needs
to be reached by the My Classes LMS. This
clearly indicates that there is room for a lot
of growth in the utilization of the My Classes
LMS.
F. Factors Affecting the Utilization of
Stakeholders
The information from the logs as well as the
findings on the LMS survey for faculty and
students identify key factors resulting to the
low utilization of the LMS. The following
factors were identified:
a. lack of information dissemination about
the My Classes LMS ;
b. lack of or no training for LMS ;
c. lack of LMS support services;
d. LMS reliability issue ;
e. IT equipment problems to support the
use of the LMS.
4. CONCLUSION
The study established that the features and
technical profile of the My classes LMS
satisfied the standard requirements that were
set. This implies that in the implementation
of the My Classes LMS, the LMS software
and the technical infrastructure supporting
the LMS was proven to be effective.
On the other hand, majority of the support
structure used by My Classes LMS did not
satisfy the standard-based requirements for
support structure.
Majority of the
components of both support structure
categories identified gaps in the current
support structure set-up of the My Classes
LMS.
Almost all factors identified in the analysis
of LMS logs and surveys pointed to towards
the support structure problems. Secondary
problems brought about by these problems
were reflected on the comments given by the
respondents. One factor identified which
addresses the issue about IT equipment to
support students and faculty was identified
as another factor that affected the
stakeholders’ utilization of the LMS.
REFERENCES
[1] Davies, D. 1998, ‘The Virtual University: A
Learning University’, The Journal of Workplace
Learning, vol. 10, no. 4, pp. 175 – 213.
[2] Paulsen, M, 2002, ‘Online Education Systems in
Scandanavian and Australian Universities: A
Comparative Study’, The International Review of
Research in Open and Distance Learning Journal, vol.
3,
no.
2.,
viewed
2009
July,
<http://www.irrodl.org/index.php/irrodl/article/view/1
04/559. Retrieved July 10, 2009>
[3] Vovides, S., Sanchez-Alonzo, S., Mitropoulou,V.
and Nickmans, G., 2007. ‘The Use of e-Learning
Course Strategies and to Improve Self-regulated
Learning’, Education Research Review, vol. 2, pp.
64–74.
[4] Swinney, L. A. , 2004, ‘Why Faculty Use a
Course Management System (blackboard) to
Supplement
their
Teaching
of
Traditional
Undergraduate Courses’, Doctor of Philosophy
Dissertation, University of North Dakota
[5] Evans, K., 2005, ‘Front End Analysis Plan for the
Underutilization of Technology at Lincoln Middle
School’,
viewed
2009
September,
<http://www.kristenevans.info/su/reports/FEAplan.pd
f>
[6] SpikeSource, the Center for Open Source
Investigation at Carnegie Mellon West, and Intel
Corporation, 2005, ‘Business Readiness Rating for
Open source; A Proposed Open Standard to Facilitate
Assesment and Adoption of Open Source Software,
RFC1’
viewed
2009
July,
http://www.openbrr.org/docs/BRR_whitepaper_2005
RFC1.pdf
[7] Deprez, J.C. and Alexandre, S., 2008, ‘Comparing
Assessment Methodologies for Free/Open Source
Software: OpenBRR & QSOS’, viewed 2009 October,
<http://www.qualoss.org/dissemination/
DEPREZ_CompareFlOSS AssessMethodo-Camera02.pdf
[8] Letellier F., 2009, ‘FOSS-Bridge’, viewed 2009
October,<http://netnam.vn/fossbridge/uploads/Main/Foss-Bridge-Block27.pdf >
Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 18 No. SP1, December, 2010
71.20
Standard Based Requirements for LMS Profile Assessment
[9] De Silva, C., 2009, ‘Open source software
assessment methodologies’, viewed 2009 October,
<http://www.itpro.lk/?q=node/2814>
[10] Mokhtar, S., Alias R., Rahman A., 2007, ‘Rubric
for Assessing ICT Infrastructure in Malaysia Higher
Education’,
viewed
2009
September,
<http://eprints.utm.my/3115/1/Rubric_For_Assessing
_ICT_Infrastructure_in_Malaysia_Higher_Education_
2006.pdf
[11] Towns, J., Ferguson, J., Fredrick, D. and Myers,
G., 2001, ‘Grid User Support Best Practices’, viewed
2009
October,
<http://www.ggf1.nl/abstracts/GUS/GridUserServices
BestPractices-02221.pdf>
[12] Working Groups Experts 2000, ‘Guidelines for
Institutional Self-Assessment of ICT maturity in
African Universities’, viewed 2009 September,
<http://www.aau.org/english/documents/ICTGUID.pdf>
[13] Center for learning & Performance technology
for Instructional Course tools, 2009, ‘Learning Tools
Compendium’,
viewed
2009
December,
http://www.c4lpt.co.uk/Directory/Tools/instructional.
html
[14] Kvavik, R., Caruso, J., Morgan, G., 2004,
‘ECAR Study of Students and Information
Technology, 2005:
Convenience, Connection,
Control and Learning’, EDUCAUSE Center for
Applied Research, vol. 5
[15] Edutools 2005, ‘EduTolls Website’, viewed 2009
December,
http://www.edutools.info/glossary.jsp?pj=4
<http://www.b24esolutions.com/microsoftplatform/web-based applications/>
[21] Bplans 2009, ‘Business Plan Help & Small
Business Articles’,
viewed 2009 November,
http://articles.bplans.com/business-termglossary/business-terms-u
[22] Kineo, 2010, ‘Kineo.com on Dokeos LMS’,
viewed 2010 January <http://www.kineo.com/freetools/dokeos-lms-2.html>
[23] Dokeos, 2010, ‘Dokeos Website’, viewed 31
January 2010, http://www.dokeos.com
[24] Sourceforge, 2009, ‘Sourceforge.net on Dokeos
SVN’,
viewed
2009
December,
<
http://sourceforge.net/projects/dokeos/>
[25] Ohloh, 2010, ‘Dokeos’, viewed 2010 January,
http://www.ohloh.net/p/dokeos/
[26] Secunia, 2009, Secunia.com on Dokeos Security
Issues,
viewed
2009
December,
<
http://secunia.com/advisories/product/4508/>
[27] Amazon, 2010, Amazon.com on Dokeos Books,
viewed
February
2010,
http://www.amazon.com/s/ref=nb_sb_noss?url=searc
h-alias%
3Dstripbooks&fieldkeywords=dokeos&x=0&y=0
[28] Winston Chugsayan SLUNET Director, personal
communication, January 9, 2010)
[29] Willy Olivete, System Administrator, personal
communication, January 9, 2010).
[16] UC 2008, ‘University of Canterbury LMS
Review – Final Report and Recommendations’,
viewed
2009
August,
http://uctl.canterbury.ac.nz/files/staff/moodle/Final%2
0Report%20of%20the%20LMS%20Review%20Steer
ing%20Group%20-%20public%20version.pdf
[17] Smith, S., Salaway, G., Caruso, J., Katz, R.,
2009, ‘ECAR Study of Undergraduate Students and
Information Technology, 2009’, EDUCAUSE Center
for Applied Research, vol 6
[18] McHenry, B., 2010, ‘New Features for Learning
Management Systems’, viewed 2010 January,
http://www.sloanconsortium.org/publications/
magazine/v3n2/mchenry.asp
[19] Bersin, J., Mallon, D., Howard, C., O’Leonard,
K., 2009, ‘Learning Management Systems 2009
Executive Summary’, viewed 2010 January,
http://www.wsilms.com/Portals/0/WSILMS_Data_Fil
es/LMS_Market_Overview_Bersin.pdf
[20] 24eSolutions, 2009, ‘Web Applications’, viewed
2009
November,
The Seventh International Conference on eLearning for Knowledge-Based Society, 16-17 December 2010, Thailand
71.21