Insights on UNM Learn
from the
2016 Faculty Technology Survey
Stephen Burd (burd@unm.edu)
Academic Technology Liaison
Further information available at:
http://averia.unm.edu
Last revised: 7/17/2016 3:37 AM
Overview

UNM Learn is a system that many faculty love to
hate:
 Is that a problem?
 Why is it a problem?
 How can UNM fix the problem?

Gathering faculty and student perspectives:
 Faculty technology survey (completed in April)
 Student technology survey (completed but results not
yet analyzed)
LMS Faculty Survey Questions

LMS defined as:
A learning management system (LMS) is a web application that
provides an instructor with a way to create and deliver educational
content/experiences, support and monitor student participation, post
and receive student assignments.


I use an LMS for class sizes of _____ students
Agree or disagree - The LMS






is critical to my teaching
is very useful as a tool to enhance my teaching
is very useful as a tool to enhance student learning
creates efficiencies for my instruction
Years of experience using LMS technology?
Do you currently use UNM Learn?
Survey Questions - Continued

For UNM Learn Non-Users
 What factors influence your decision not to use UNM
Learn?

For UNM Learn Users
 Describe how you use an LMS to support your
instructional activities
 Indicate your satisfaction with the following aspects of
UNM Learn
 Do you have experience with an LMS that you prefer
over UNM Learn?
Survey Questions - Continued

For UNM Learn Users who’ve used and prefer
another LMS
 What is your preferred LMS?
 Why is do you prefer that LMS?
 If my preferred LMS was available for use at UNM, I
would invest time in moving my content to that LMS
from UNM Learn
 Provide additional comments comparing your
experience with the other LMS to UNM Learn
Survey Stats

Distributed to 1828 full-time faculty
 Main campus but not north campus
 Branches

387 responses (21%) – 328 complete – typical of
past surveys

Individual URLs make it possible to slice/dice by
Banner info such as campus, school, department,
rank, and years of service
Respondent Characteristics

43% claim 5 or more years of LMS experience
 Respondents are probably better informed about and more
experienced with the LMS than the faculty as a whole

77% currently use UNM Learn
 Large but possibly biased sample from which to gather
responses to the detailed UNM Learn questions
 No way to know how (un)representative of the faculty as a
whole are those respondents

16% have experience with an LMS that they prefer to
UNM Learn
 Large enough sample to gather some useful information
about comparisons between UNM Learn and other LMSs
 Too small for high confidence in the accuracy of those
comparisons nor whether they would reflect the attitudes of
the entire faculty should UNM switch to a different LMS
What UNM Learn Features are Used?

Frequently-used:






Distributing documents and learning materials (95%)
Gradebook (74%)
Online assignment/paper submission (63%)
Quizzes/exams (52%)
Class-related communication between the instructor and
students (50%)
Seldom-used:





Graded discussions (22%)
Group collaboration (20%)
Analytics (17%)
Student-to-student messaging (14%)
Web conferencing (8%)
Class Characteristics

No clear pattern based on class size or delivery
method/modality
Overall Satisfaction

Overall satisfaction – current users only
 Satisfied or very satisfied (59%)
 Neutral (21%)
 Dissatisfied or very dissatisfied (19%)

If users of other LMSs are presumed to be nonusers of and dissatisfied with UNM Learn them
satisfaction level is 49%
Reasons For Not Using UNM Learn

Canned responses (select all that apply)








Ease of use for faculty (18.6%)
Ease of use for students (10%)
Usefulness (lack thereof) of LMS tools (10%)
Ease of maintaining personal websites (9%)
Availability of training (9%)
Flexibility of the system (9%)
Open-ended responses were largely consistent with
the canned responses
Some responses praised the utility and ease-of-use of
alternatives such as:
 Dropbox
 Google Classroom
 Traditional web sites
Faculty Experience With Other LMSs


49 responses – top alternative was Canvas (16) – all
others in single digits
Respondents were asked why they preferred another
LMS to UNM Learn.
 Ease-of-use (lack thereof) was the most frequently cited
reason
 Other responses included LMS tool usefulness, flexibility,
and reliability/stability.


Open-ended responses gave many anecdotes
describing UNM Learn’s poor ease-of-use and other
shortcomings and praising the relative strengths of a
chosen alternative
80% said they would invest the time to transfer
content from UNM Learn to their preferred LMS
Survey Take-Aways

If you’re looking for a mandate for change it’s not
in the survey results

If you’re looking for a vote of confidence in UNM
Learn its not there either

What is there:
 UNM Learn is underutilized
 Problem areas (inter-related)
Ease-of-use
 Complexity
 Training

UNM Learn Under-Utilization

74% of survey respondents use UNM Learn but
that probably over-represents use among the
entire faculty – we don’t know by how much

Users are using a subset of the features
 Greatest use is for the “convenience” features
 Underuse of features with the greatest promise for
improved learning outcomes

Non-users have voted with their feet for other
LMSs and combinations of simpler tools
 Number and distribution of chosen alternatives provides
little clarity about a “better alternative” to UNM Learn
Costs of Under-Utilization

To students:
 A labyrinth of alternate tools and alternate ways of
using them
 Negative learning outcomes? (not a proven result)

To UNM
 Inability to tightly and uniformly integrate other systems
with the LMS (e.g., LoboAchieve, UNM-wide analytics)
 Loss of learning outcomes that might or could be
achieved through better LMS use (e.g., group
collaboration, programmed or self-paced learning,
within-course analytics)
 More complex support for faculty and students
Ease-Of-Use, Training, and Support
A Gordian Knot?




Poor ease-of-use combined with high complexity
creates a greater and more complex need for
training and support
Users, especially faculty, resist using tools that
require up-front training
Avoiding up-front training creates more failures
when faculty do try to use the LMS – those stories
feed continued underuse and resistance to use
How can this cycle be broken?
 Replacing UNM Learn with another LMS or a tool set
that’s less complex and easier to use?
 Something else?
Where To From Here?

Possibilities for discussion
 A drastically revised or completely new approach to
training and motivating LMS use?
 Providing and supporting a simpler tool set in addition
to the LMS?
 Controlled experiments with other LMSs or tool sets?
 Start an RFP process for the next generation of UNM
Learn?
 All/most of the above?
 Others?