Usability Evaluation of a
Website Designed to Guide
and Support Best Practices for
Online Teachers
Stephanie Barry
University of Hawaii
barrys@hawaii.edu
Purpose
To evaluate the content and usability of a
site intended to support online teachers.
Purpose
To evaluate the content and usability of a
site intended to support online teachers.
Why?
To help other institutions decide how to
support faculty in online teaching, and
provide a one-stop shop for best practices
information and real-world examples.
Background of TDP
• Technology & Distance
Programs
Background of TDP
• Technology & Distance
Programs
• Supports instructional
technology
Background of TDP
• Technology & Distance
Programs
• Supports instructional
technology
• Serves the College of
Education at the
University of Hawaii
Background of TDP
• Technology & Distance
Programs
• Supports instructional
technology
• Serves the College of
Education at the
University of Hawaii
• Provides workshops
and consulting
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
What is
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
What is (frustration)
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
What is (frustration)
and
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
What is (frustration)
What should be:
and
Rationale for Study
• Data analysis
• Gap in Online Teaching between:
What is (frustration)
What should be:
and
Rationale for Study
• Data analysis
• Gap in Online Teaching
What should be:
• WASC Accreditation
(CRAC, 2009)
Rationale for Study
• Data analysis
• Gap in Online Teaching
What should be:
• WASC Accreditation
(C-RAC, 2009)
– Professional
development
Rationale for Study
• Data analysis
• Gap in Online Teaching
What should be:
• WASC Accreditation
(C-RAC, 2009)
– Professional
development
– Technical support
Review of the Literature
• Key Topics
– Online teaching
– Best Practices
– Principles
Review of the Literature
• Key Topics
– Online teaching
– Best Practices
– Principles
• Combined the Lists
Table 1
Principles of Effective Online Classes
Table 1
Principles of Effective Online Classes
Table 1
Principles of Effective Online Classes
Table 1
Principles of Effective Online Classes
Principles of Effective Online Classes
Principles of Effective Online Classes
These principles guided the site content
Methods
• Usability testing
Methods
• Usability testing
• Look at the site
• What do you notice first?
•
•
•
•
Methods
Usability testing
Look at the site
What do you notice first?
I need a volunteer
•
•
•
•
Methods
Usability testing
Look at the site
What do you notice first?
I need a volunteer
Click on this
button
to raise
your hand.
The Site
• Resources for Online Teachers
Methods
• Usability testing
Methods
• Usability testing
– First impression
Methods
• Usability testing
– First impression
– Scenarios
• Scenarios
Methods
– Video example
• Scenarios
Methods
– Video example
– http://tinyurl.com/usability-eval
• Scenarios
Methods
– Video example
– http://tinyurl.com/usability-eval
• Watch video then return here (2 min)
• Scenarios
Methods
– Video example
– http://tinyurl.com/usability-eval
• Watch video then return here (2 min)
• What would you have looked for?
– (type in the chat box)
Results
“There is no average user.” (Krug, 2005)
Results
• Content is great
• Navigation can be improved
• Site tour
Discussion
The Site
• Resources for Online Teachers
The Site
• Resources for Online Teachers
The Site
• Resources for Online Teachers
Multiple points of entry =
The Site
• Resources for Online Teachers
Multiple points of entry = confusing
The Site
• Resources for Online Teachers
The Site
• Resources for Online Teachers
The Site
• Resources for Online Teachers
The Site
• Resources for Online Teachers
What’s different?
The Site
• Resources for Online Teachers
Needs dropdown
The Site
• Resources for Online Teachers
What does “21st Century
Course Planning Mean?
•
•
•
•
Discussion
Usability finds many problems
Rhythm of the site (linking)
Do “the least you can do” (Krug, 2009)
Future: Google Sites and Analytics
Questions?
barrys@hawaii.edu
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
References
Bloom, B. S. & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational
goals, by a committee of college and university examiners. Handbook 1: Cognitive domain. New York, NY:
Longmans.
Center for Applied Special Technology. (2008). Universal design for learning guidelines version 1.0. Wakefield, MA:
Author.
Chang, C. S., Wong, W. T., Cheng Y. C. & Huang, Y. P. (2009). Usability evaluation in an exploratory design-based
research approach of convenience store new employee training e-Learning material. Pervasive Computing
(JCPC), 2009 Joint Conferences, 687 – 692. doi: 10.1109/JCPC.2009.5420096
Council of Regional Accrediting Commissions. (2009) Guidelines for the Evaluation of Distance Education (On-line
Learning). Retrieved from www.wascsenior.org/
Gagne, R. M., Wager, W. W., Golas, K. C. & Keller, J. M. (2005). Defining performance objectives. In Principles of
Instructional Design (pp. 132 – 150). Belmont, CA: Wadsworth.
Graham, C., Cagiltay, K., Lim, B.-R., Craner, J. & Duffy, T. M. (2001). Seven principles of effective teaching: A
practical lens for evaluating online courses. The Technology Source. Retrieved from
http://ts.mivu.org/default.asp?show=article&id=839
Krug, S. (2005). Don’t make me think: A common sense guide to web usability (2nd ed.). Berkeley, CA: New
Riders Press.
Krug, S. (2009) Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems.
Berkeley, CA: New Riders Press.
LaRose, R. & Whitten, P. (2000) Re-thinking instructional immediacy for web courses: A social cognitive
exploration. Communication Education, 49(4), 320 – 338.
doi: 10.1080/03634520009379221
Paloff, R. M. & Pratt, K. (2007). Building online learning communities (2nd ed.). San Francisco, CA: Jossey-Bass.
Ragan, L. (2005). 10 principles of effective online teaching: Best practices in distance education. Christopher Hill,
Ed. Distance Education Report. Madison, WI: Magna.
Resnick, P.J., Janney, A.W., Buis, L.R. & Richardson, C.R. (2010). Adding an online community to an internetmediated walking program. Part 2: Strategies for encouraging community participation. Journal of Medical Internet
Research, 12(4). Retrieved from http://www.jmir.org/2010/4/e72/
Van Duzer, J. (2002) Instructional design tips for online learning. Retrieved from www.csuchico.edu/celt/roi
Vesely, P., Bloom, L., & Sherlock, J. (2007). Key elements of building online community: Comparing faculty and
student perceptions. MERLOT Journal of Online Learning and Teaching, 3(3). Retrieved from
http://jolt.merlot.org/vol3no3/vesely.pdf