27th Annual Conference on Distance Teaching & Learning
For more resources: http://www.uwex.edu/disted/conference
Making the Most of Your LMS: User-Centered Design Strategies
Lisa Torrescano
Senior Instructional Design Consultant
DePaul University
Dee Schmidgall
Interaction and Multimedia Designer
DePaul University
Background
SNL Online is the online degree program of the School for New Learning at DePaul University, Chicago.
We offer Bachelor of Arts degrees in accelerated degree completion and competence-based programs, and
MA or MS degrees in our graduate program. Our students are adult learners with a wide variance of
technical ability and typically juggle work, family, and academic responsibilities. We offer about 100
distinct courses with 80+ sections available per quarter, and with a course cap of 26 . Courses are
developed with and taught by both our resident and adjunct faculty; many of the latter live outside the
Chicago region . Our program is growing to both create opportunity and satisfy demand. SNL Online
includes a director of online learning, 3 instructional designers, 3 full-time program administrators, and 13 student and/or part-time workers to assist with production, administration and support.
By mandate our program is student-centered. We seek to create an online learning environment that
supports student learning by being effective, efficient, and enjoyable. In the past we had relied on
heuristics or best-practices from web design and instructional design to create our course templates. To
achieve a consistent, easy-to-use, and scalable learning environment in Blackboard we formatted our
course content pages with an external CSS. These content pages were not editable by faculty or
studentswhich gave us the ability to ensure a consistent user experience, but also required significant
resources to build, maintain, and support courses.
This design was largely successful. However, when faced with requests or complaints about usability
from students or faculty we had to respond in the absence of real data. We simply did not know if a
complaint was an indication of a universal problem or just an outlier. With our development and
maintenance load, we did not have the resources to survey students and faculty, analyze the data, or make
and implement recommendations. All we had to go on were anecdotes and course evaluations, which
were neither reliable nor appropriate measures.
In early spring of 2010, DePaul University decided to change its learning management system (LMS)
from Blackboard to Desire2Learn (D2L). The move presented both an opportunity and challenge to start
fresh and create a better design template for SNL Online courses that addressed past deficiencies and
leveraged D2L’s customization abilities. To ensure success we decided to follow user-centered design
principles.
User-Centered Design
Interaction Design, Beyond Human Computer Interaction (Sharp, Rogers, & Preece, 2007) described
user-centered design (under the umbrella term of interaction design) as a way to “creat[e] user
experiences that enhance and augment the way people work, communicate, and interact” (p. 9). UCD is a
collaborative holistic approach to designing interactive products concerned with creating quality user
experiences, meaning those that are easy to learn, effective to use, and enjoyable to use.
Copyright 2011 The Board of Regents of the University of Wisconsin System
1
27th Annual Conference on Distance Teaching & Learning
For more resources: http://www.uwex.edu/disted/conference
Though this process is widely embraced in the consumer market because it is effective, efficient, and
cost-saving, it is seldom mentioned in the domain of higher education. Because we had limited resources
of time, money, and peoplepower, we knew we could not make guesses—however well informed—about
what would work. We decided to adopt user-centered design strategies to make the most of our LMS.
UCD Process
Sharp, et al. stated that user-centered design involves four basic activities:
1.
2.
3.
4.
Indentifying needs and establishing requirements for the user experience
Developing alternative designs that meet those requirements
Building interactive versions of the designs so that they can be communicated and assessed.
Evaluating what is being built throughout the process and the user experience it offers. (p. 17)
Requirements-gathering is a crucial stage of interaction or user-centered design. It’s absolutely necessary
to understand who your users are, what abilities they have, what they want to do and how they want to do
it. Students and faculty are obvious users, but users include all stakeholders from the dean through your
production team; anyone who interacts with or is impacted by your design. It is imperative to understand
the constraints of the system you will work with, including the LMS and your IT department. It is also
critical to identify and plan for communication, training, and administration needs at the beginning of the
project.
Developing alternative designs to meet requirements is neither as difficult as you might fear nor as easy
as you might hope. It is important to get a conceptual model of what the interaction should do, rather than
what it should look like. For example, we decided our design should allow for a fully integrated learning
experience with easy, contextual access to readings, multimedia, assignments, discussions, and other
course tools. With that in mind, we could begin sketching out alternate page designs and wireframes.
The next step is to build interactive models of the designs so they can be tested and assessed. This can be
as simple as a paper prototype, where each stage of an interaction is visualized on a separate sheet of
paper. This allows for faster prototyping and iteration, and facilitates truthful user feedback during
testing. However, in some cases functional models will need to be built to fully assess their usability.
Finally, it is important to evaluate the design and user experience throughout the development process and
use that information to refine and iterate. A premise of UCD is that you don’t get it right the first time.
There is a wide variety of ways to evaluate a design including observation, questionnaires and surveys,
user tests, heuristics, and cognitive walkthroughs. The goal is always the same; to learn how users really
interact with your design and how you can improve it to provide for a better user experience.
User Test Design
The goals of the test were determined by input from the SNL online director, multimedia designer, and
instructional designer. The results of the test were evaluated, interpreted, and integrated into the next
iteration of the D2L SNL course content page layout and summary grid design.
The test was designed to exercise the course design and summary grid under controlled test conditions
with representative users. The user experience test objectives were to:


Determine the key course design elements to be included in the content page of the D2L SNL
Online course
Determine the most user friendly layout of the course content page
Copyright 2011 The Board of Regents of the University of Wisconsin System
2
27th Annual Conference on Distance Teaching & Learning




For more resources: http://www.uwex.edu/disted/conference
Determine design inconsistencies and usability problems, which may include:
o Navigation errors – failure to locate functions, failure to follow recommended screen
flow.
o Presentation errors – failure to locate and properly act upon desired information in
screens, selection errors due to labeling ambiguities.
o Execution errors – failure to interpret and understand what is expected.
Determine the most effective and usable summary grid layout
Determine the best design to integrate competence-specific assignment instructions
Establish baseline user performance and user-satisfaction levels of the interface for future
evaluations.
Participants
The participant group was composed of 6 students and 4 faculty members. The students had taken 2 or
more SNL Online courses within the last two years. The faculty had taught at least 2 SNL Online
courses. Students and faculty participants performed a number of tasks to interact with various versions
of course design templates and grids.
Methodology
Over a 3-day period, 6 students and 4 faculty participated in the user experience test. Each test session
was 1–1.5 hours. The participant was given an orientation prior to the test and was instructed to “think
aloud” during each task scenario. Participants were also advised that they would be videoed and/or timed
during each task.
Task Flow
The participants' responsibilities were to complete a set of representative task scenarios as efficiently and
timely as possible, and to provide candid feedback regarding the usability and acceptability of the course
design and grid layout. The participants were directed to provide honest opinions regarding their user
experience and participated in post-session subjective debriefing.
The roles involved in the user experience test were as follows. An individual often played multiple roles.
Facilitator
 Provide training overview prior to usability testing
 Provide overview of study to participants
 Define usability and purpose of user testing to participants
 Assist in conduct of participant and observer debriefing sessions
 Respond to participant's requests for assistance
Test Observers
 Silent observer
 Identify problems, concerns, and procedural errors
 Serve as note takers.
 Record participant’s actions and comments
To capture data free from bias or influence from prior learning, the task design was different for each
student, and each student was presented the prototypes in a different order.
Test ethics were also discussed and agreed upon. All persons involved with the user experience test were
required to adhere to the following ethical guidelines:
Copyright 2011 The Board of Regents of the University of Wisconsin System
3
27th Annual Conference on Distance Teaching & Learning


For more resources: http://www.uwex.edu/disted/conference
The performance of any test participant must not be individually attributable. A
participants name should not be used in reference outside the testing session.
A description of the participant's performance should not be reported to his or her manager
Reporting Results
The user-experience test report consisted of video clip presentations of the key results, analysis of the
usability metrics against the pre-approved goals, subjective evaluations, documentation of specific
usability problems, and recommendations for resolution.
Life After the Results
After the test results and recommendations were presented, the project moved into a phase of negotiation
with key stakeholders and iteration based on these discussions. This is where the “ideal course” suggested
by the usability test met the real world of implementation. A design document was created based on our
analysis of test results and used to create the first in a series of course templates.
We hired and trained a team of six student workers to handle the labor-intensive task of migrating 80+
courses into the new design template. A series of unexpected system shortcomings and changes to
requirements by key stakeholders forced us to adjust the design numerous times; those adjustments had to
be retrofitted in the courses that were already converted. After the 10th round of iteration, a final course
design template was approved and all courses were aligned with it.
On December 15, 2011, we finished the last course conversion of the Winter 2011 online curriculum and
opened up all the courses to the students and faculty in D2L. Though we were prepared for significant
comments and questions from our users, both students and faculty, we were gratified that most found our
new design easy to use. We credit this success to leveraging the best practices of user-centered design
when designing within the constraints of a LMS.
About the Presenters
Lisa Torrescano is an Instructional Designer with DePaul University, School for New Learning,
SNL Online. She has over 10 years experience working in educational, nonprofit, internet start-up and
publishing organizations. Her focus has been in leadership positions in program development,
instructional design and evaluating user experience for classroom-based and online education.
Address: 990 Fullerton Avenue, Suite 3000
Chicago, IL 60614
Phone: 847-912-4091
Email: eramosto@depaul.edu
Dee Schmidgall (MFA, School of the Art Institute of Chicago) is an Interaction Designer with DePaul
University, School for New Learning, SNL Online. He designs and manages SNL Online's course and
departmental web presence and multimedia. He also researches and advocates for emerging technologies
and best practices in online learning.
Address: 1 E. Jackson, Suite 1500
Chicago, IL 60604
Phone: 312-362-8164
Email: dschmidg@depaul.edu
Copyright 2011 The Board of Regents of the University of Wisconsin System
4