-- Although e-learning got off to a shak...> 4P> Chief Learning Officer Magazine
http://www.clomedia.com/content/anmviewer.asp?a=91&print
=
Assessing Learning Management Systems
John Hall
Although e-learning got off to a shaky start in the late 1990s, industry analysts are now
bullish about the enabling technologies, notably enterprise learning management
systems (LMSs), both in terms of growing adoption rates and quick return on
investment (ROI). Whereas early LMS adopters focused on discrete e-learning projects
that were limited in scope and required a lot of expensive customization, the trend today
toward consolidation and providing a single, common infrastructure to manage
learning andtraining initiatives across the organization.
js
A learning management system provides the platform for the enterprise's online
learning environment by enabling the management, delivery and tracking of blended
learning (i.e., online and traditional classroom) for employees, stakeholders and
customers. A robust LMS should integrate with other departments, such as human
resources, accounting and e-commerce, so administrative and supervisory tasks can be
streamlined and automated and the overall cost and impact of education can be
tracked and quantified.
Furthermore, an LMS should support a collaborative learning community, offering
multiple modes of learning—from self-paced coursework (Web-based seminars and
classes, downloadable, CD-ROM and video content) to scheduled classes (live
instruction in classroom settings or online) to group learning (online forums and
chats). In its ability to integrate, organize and standardize learning across broad
organizational requirements, the LMS model has been compared favorably to
enterprise resource planning (ERP) solutions, which convert a company's back-office
into a seamlessly functioning whole.
Recognizing how integral education is to the operation and success of any
organization, it is not surprising that e-learning is now on a parallel track with other
large-scale enterprise implementations, such as ERP, customer relationship
management (CRM) and supply chain management (SCM). As organizations
transform into e-businesses, they move from a fragmented information culture
populated with disparate legacy systems to what is known as a "contextual
collaborative" culture—a real-time, knowledge-sharing system to coordinate business
activities across the enterprise and extended supply chain (via enterprise portals,
applications or EDI, electronic data interchange).
This worldwide trend is reshaping the e-learning market as well. Instead of
implementing and supporting a hodgepodge of stand-alone projects, companies are
looking to extend the value of their e-learning resources to support their diverse
education needs across business functions.
As stated in a META Group report titled "2002/03 Trends—Web & Collaboration
Strategies" (Oct. 9, 2002), "Through 2005, high-performing enterprises will manage a
portfolio of investments that facilitate internal and external collaboration around
strategic business initiatives and organizational goals (e.g., innovation, knowledge
management, learning)."
There is a host of factors driving the implementation of LMS solutions. In today's
knowledge economy, companies must be able to distribute, manage and assess
1 of 7
2/13/2003 3:10 1
'mg got off to a shak...> </P> Chief Learning Officer Magazine
http://www.clomedia.com/contentlanmviewer.asp?a=91&print=
educational programs across practices and geographies in an efficient, fast and low-cost
manner. And, as any corporate learning officer knows, corporate learning and training
responsibilities do not abruptly end at the corporate firewalls. Ongoing educational
services also must be provided to a host of external audiences: the developer
community, partners, customers, resellers and suppliers.
Globally, organizations are faced with skilled labor shortages and limited resources. In
response, multinational companies are seeking ways to enforce consistent learning
standards and skill training across their operations. Realistically, this can only be
accomplished through a Web-based system providing 24x7 flex-time learning in
multiple languages.
Another key driver in this all-inclusive approach to education is corporate accountability.
With increased scrutiny from stakeholders, regulatory agencies and the media, public
companies and financial services providers must be able to demonstrate that their
senior management, sales force and financial consultants are fully versed in
compliance laws and business ethics. Moreover, for companies with overseas offices,
this training must drill down to regional and local statutes and ordinances.
In this article, we will examine key performance and administrative functions of a
learning management system. At the same time, we will address recommended
features, such as open standards to promote interoperability. Next, we explore the
comparative benefits of purchasing versus outsourcing options. Finally, we will provide
advice on how to assess LMS solutions.
Evaluating a Learning Management System
When you begin to evaluate learning management systems, you are going to hear a lot
of terms that end in "-ity": high availability, usability, scalability, interoperability,
stability and security. Let's quickly examine each of these issues and why they are
critical to the function of any enterprise management system.






2 o17
High availability: The LMS must be robust enough to serve the diverse needs of
thousands of learners, administrators, content builders and instructors
simultaneously.
Scalability: The infrastructure should be able to expand—or "scale"—to meet
future growth, both in terms of the volume of instruction and the size of the
student body.
Usability: To support a host of automated and personalized services, such as
self-paced and role-specific learning, the access, delivery and presentation of
material must be easy-to-use and highly intuitive—like surfing on the Web or
shopping on Amazon.com.
Interoperability: To support content from different sources and multiple vendors'
hardware/software solutions, the LMS should be based on open industry
standards for Web deployments (XML, SOAP or AQ) and support the major
learning standards (AICC, SCORM, IMS and IEEE).
Stability: The LMS infrastructure can reliably and effectively manage a large
enterprise implementation running 24x7.
Security: As with any outward-facing collaborative solution, the LMS can
selectively limit and control access to online content, resources and back-end
functions, both internally and externally, for its diverse user community.
2/13/2003 3:10 PI•
! n g got off to a shak...> /P> ~ Chief Learning Officer Magazine
http://www.clomedia.com/content/anmviewer.asp?a=91&print=:
Optimally, an LMS will consolidate mixed-media training initiatives, automate the
selection and administration of courses, assemble and deliver learning content,
measure learning effectiveness and integrate with other enterprise applications.
"One of our goals was to create a complete curriculum of online courses and develop a
certification program for employees and partners of all skill levels," said Debra Morton,
director of business systems for McDATA Corp., an open storage networking solution
company based in Broomfield, Colo. "The LMS enables us to offer self-paced,
self-service courses while providing us with the necessary assessment and reporting
capabilities to help monitor and track the learning process throughout McDATA's entire
workforce as well as our partner community," she added.
General Physics, a workforce development company focused on technical and soft
skills training, worked with Oracle iLearning to provide a Web-based training portal for
power plant personnel of all energy companies. The LMS provides the platform and the
portal, while General Physics provides the technical content. The portal enables
administrators to assign, evaluate and track both knowledge-based and hands-on
qualification tasks while giving supervisors the ability to manage and track their
employees' success.
"The advantage of using a scalable solution was the ability to deliver more than 1,400
technical training lessons to energy companies of every size," said Russ Garrity,
director of General Physics Energy Services. "Because of the LMS architecture, we
can offer cost-effective Web-based courses that can be accessed quickly and easily
by a dispersed workforce."
There are features and capabilities that become increasingly important as e-leaning
usage grows. From an operational point of view, the LMS and its key
components—content management, user administration and system
administration—should be 100 percent Web-deployable, requiring no additional client
applications. As with any enterprise application, the more the software is customized,
the harder it is to maintain, upgrade and expand. In the past, a lot of companies
over-engineered their e-learning solutions, spending a lot of money on consulting and
customizing their applications. Because the pedagogical and business models for
online learning were still in their infancy (and evolving rapidly) and the educational
goals and targets changed, the organizations were stuck with "hard-wired" solutions
with limited flexibility and scale.
This leads to a second important consideration. Choose an LMS built on an open
architecture that supports the emerging learning standards, which go by the acronyms
IMS, AICC and SCORM.
The IEEE is the overall standards accreditation body that is responsible for learning
object metadata standards. These include a set of standards based on specifications
called SCORM (sharable content object reference model), which are influenced by
industry groups such as IMS and AICC. In simple terms, these industry-standard
groups are creating technical specifications to enable and support a unified,
standardized content model for Web-based learning. When you are assembling content
from multiple sources and different networks, the biggest underlying task is to get the
software and tools to "talk" to each other (i.e., import and export data and
,ing got off to a shak...> <JP> ~ Chief Learning Officer Magazine
http://www.clomedia.com/contentlanmviewer.asp?a=9I &print=
respond to the same operational commands).
The LMS should integrate with your enterprise e-business applications. To automate
learning services and resources and support self-service learning and administration,
many departments must get involved. For example, in-house sales-force training data is
often related to information such as competency models held in human resources and
perhaps even legal departments (on compliance issues or mandatory testing).
Information regarding courses taken by customers may feed directly into a CRM
system. The scheduling and use of physical space (classrooms and auditoriums) and
other hard assets (NV equipment, a videoconferencing system) should be linked to a
resource management application. Furthermore, it is not unusual for companies to
charge customers, partners and other outside parties for courses. In those instances,
the LMS should act as a fulfillment engine to manage online orders, which requires
integration with the company's financial and e-commerce systems. As online learning
enables the use of education in many stages of the customer and employee
relationship cycle, these integration requirements will continue to grow. The key is to
avoid creating "islands" of data, or worse, duplicate data, across the enterprise.
In terms of performance, the more advanced LMS solutions offer the ability to prescribe
and personalize individual learning paths, as well as reformat, reuse and syndicate
content. Many LMS systems establish predefined roles that prove inflexible and
porous; that is, they provide almost universal access to the content and learning
events. Obviously, this can have a negative impact on the effectiveness and security of
the system, since courses offered to employees might contain proprietary or sensitive
information not intended for outside consumption. An LMS built around "fine-grained
permission" allows for the role-specific definition of what each individual learner can do
or see—"just enough" training. This flexible permission model enables multiple levels
of access, learning modes and self-service tools for a range of system users (learners,
content developers, instructors and site administrators) without compromising security.
An assessment engine with built-in testing and evaluation capabilities is critical to
monitoring, tracking and rating e-learning initiatives. The system should support time
limits for self-administered tests, limit the number of attempts allowed and impose a
time delay between attempts. By tracking the learning process, learning officers and
management can evaluate the effectiveness of the courses and accurately gauge the
knowledge and skill levels of their workforce. For example, a sales readiness group
can track whether salespeople are prepared to represent a new product or service in
the field. The fact that a geographically dispersed sales force now can be trained "en
masse" results in faster time-to-market and consistent messaging throughout the sales
organization. Assessment also is key for corporate compliance. As previously
mentioned, it provides verifiable proof that employees have been schooled in the laws
and regulations governing their business practices.
Licensing and Purchasing Options
Learning management systems also are compared to an ERP system in terms of
complexity and cost of implementation. As with any large enterprise application, the
installation requires IT expertise and ongoing support. While Fortune 500 companies
might opt to purchase, plan and implement an enterprise online learning system, many
companies are moving toward the ASP model. With this outsourcing model, a third
party, often the LMS vendor, manages the IT infrastructure, either at the
')/12Mnn1 2.1111
ung got off to a shak...> 4P> ~ Chief Learning Officer Magazine
http://www.clomedia.com/contentlanmviewer.asp?a=91&prir
customer site or remotely. While outsourcing eliminates the need for IT staff
and maintenance, keep in mind that the LMS must be designed for this kind of
deployment.
Zachry Construction Corp., a worldwide construction development and management
company with more than 15,000 widely dispersed employees doing business in
hundreds of locations, chose to work with Karta Technologies Inc. when looking to
implement an LMS. Karta is a technology-based company specializing in advanced
adult education through Web-based training. Karta completed and implemented an
entire Oracle iLearning solution for Zachry Construction. Because of the many
benefits offered by the LMS, Karta plans to offer the same model to other customers.
"We can get a site set up and customized very quickly leveraging our past experience
and the Oracle iLearning features," said David Steinke, director of Product
Development for Karta. "We can now offer our clients an e-learning solution with a
proven system, minimal disruption and low implementation costs."
Garrity of General Physics touts the outsourced model for its ability to reduce costs on
their end. "Using an LMS, we were able to build a low-cost training portal to help
supervisors manage and track employee progress," said Garrity. "By implementing the
LMS as an outsourced model, we eliminated the headache of updating our system and
significantly reduced our costs."
Basically, there are four licensing models now available in the e -learning market:
1. The software is purchased and installed and managed in-house.
2. The company buys the software, but it is housed and managed remotely by a
third party. Administrators, content builders, instructors and learners access the
system over the Internet.
3. The company buys the software and installs it on its own data platform, but the
maintenance and upgrades are managed by the LMS vendor or another t hird
party (e.g., a system integrator).
4. For the cost-conscious, there is yet a fourth option. Some LMS vendors
operating their own enterprise e-learning system offer to share access to their
system. Basically, you lease "space" on the host data site.
In determining which model is right for your company, you must get the IT
professionals involved to help map the company's technical capabilities and needs.
There is no one-size-fits-all LMS solution, and vendors should offer choices.
Ultimately, it is the company's decision whether or not to have the enabling
infrastructure hosted or installed. Equally important, once you make that choice, you
should still have the flexibility to make it again. As your e-learning needs change and
grow, you don't want to be locked into what originally fit. If you installed an LMS, it
should be flexible, extensible and based on open standards. You don't want to lose
functionality, archival information and coursework from earlier learning initiatives. The
LMS should be able to scale and support new and emerging learning modes,
technologies and tools.
Getting Started
The general consensus among analysts tracking the e-learning market is that
company success is tied to real business objectives. An LMS implementation should
5of7
2/13/20033:101
!ping got off to a shak...> </P> Chief Learning Officer Magazine
http://www.clomedia.comlcontentlanmviewer.asp?a=91&print
be approached with the discipline, hard goals, timeline and performance benchmarks of
any asset management project. Don't get confused by a laundry list of features; figure
out what you want to accomplish from a business and an educational point of view and
then get technical advice on what functionality you'll need.
In evaluating an LMS, you should also take a hard look at the vendor. Whether
outsourcing or purchasing a solution, you want a software partner with the financial
resources and global market reach to support your evolving needs. Also, seek a
partner that knows and understands the educational space (by serving on industry
standard organizations or having education specialists guide product development).
"When we looked at several LMS vendors, we required a partner who was dynamic,
financially stable and could grow with us," said G.P. Singh, president and CEO of Karta
Technologies. "We offer e-learning solutions that provide corporate education to large
multinational corporations and smaller firms. The LMS solution had to meet the needs
and demands of our diverse client base so our partner had to have advanced
technology and significant resources."
In an interview last year, Ann Boland, president of the Digital Learning Organization, a
trade association of content and platform suppliers to the business learning industry,
commented on a disturbing trend that she personally had noted: Few of the people who
sell and manage e-learning have actually completed an online course. This leads to a
recommendation that should be self-evident, but apparently is not: Consider whether
the LMS vendor actually uses its own e-learning solution to train and update its own
employees and constituents. If a company uses its own product, it is doubly motivated
to constantly improve and innovate its functionality and performance.
In addition to soliciting advice from technical specialists, I recommend you ask
prospective vendors for customer references. System users can tell you both the
strengths and shortcomings of the technology, as well as what they would do
differently if starting from scratch. You also should talk to peers who have
implemented learning management systems to learn about their expectations and
goals and whether they've been met.
One way to test the system is to work with a vendor on a pilot program. You should be
able to start at a size that is comfortable to you, with minimal commitment of finances or
staff. You might consider leasing space on a vendor's e-learning system. This way, you
can test a fully operational and market-proven system on a low-cost trial basis.
Conclusion
In summary, a learning management system optimally should:









6 nf 7
Consolidate training initiatives on a scalable, low-cost Web-based platform,
Assemble and deliver learning content rapidly in multiple languages.
Measure the effectiveness of training initiatives.
Mix classroom and online learning.
Integrate with other enterprise application solutions.
Centralize and automate administration.
Use self-service and self-guided services as much as possible.
Support portability and standards: AICC, IMS and SCORM.
Personalize content and enable knowledge re-use.
')/l-;nnnl 1 .ln I
,rning got off to a shak...> </P> ~ Chief Learning Officer Magazine
http://www.clomedia.com/content/anmviewer.asp?a=91&print
John L. Hall has been the senior vice president of Oracle University, the education
and training division of Oracle Corp., since 1999. During his tenure, Hall has seen
Oracle University grow into the largest software training business in the world, with
more than 600,000 students trained annually.
copyright 2002 - CLOmedia.com / Chief Learning Officer Magazine - MediaTec Publishing
Chief Learning Officer Magazine
httpSwww.clomedia.com/
7 of 7
2/13/2003 3:10