C ARD S ORTING E XERCISE
T URNITIN AND M Y LO
February 2009
University Web Services, IT Resources
Web Usability Testing
ii
Contents
CONTENTS .................................................................................................................. II
OVERVIEW: ................................................................................................................ 2
RECOMMENDATIONS ................................................................................................... 2
ABOUT THE TEST ........................................................................................................ 2
CARD SORTING METHODOLOGY: ................................................................................. 3
ANALYSIS: ................................................................................................................. 3
RESULTS OF CARD SORTING: ...................................................................................... 4
Group 1, Test 1: ......................................................................................................................4
Group 1, Test 2: ......................................................................................................................6
APPENDIX A: LIST OF LABELS USED IN CARD-SORTING EXERCISE 2.............................. 8
APPENDIX B: COMMENTS MADE DURING CARD SORTING EXERCISE 1 .......................... 13
APPENDIX C: RESULTS OF SURVEY ........................................................................... 14
REFERENCES............................................................................................................ 14
ABOUT THIS DOCUMENT ............................................................................................ 14
Revision History ................................................................................................................... 14
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
-2-
Overview:
Information used in the test was derived primarily from the Teaching and Learning
site with some terms used from related sites.
This evaluation is not an exhaustive analysis of the suitability and quality of the
Teaching and Learning site, but seeks to investigate the type of information that
academics need to support their work.
Recommendations
About the Test
Test Facilitator
Maria Moore
Client Group
Prospective students, staff who are also current students
Number of participants
11
Location
CALT meeting Room 1.1, Morris Miller Library
Conducted
12th February 2009 from 11am – 3.00pm
Information architecture is a user-map of the information space, or ‘what goes where’,
as defined by Nielsen 2004. Card sorting (Rosenfeld and Moreville 2002) was used to
establish the information architecture in two tests:
Test 1: information relating to Turnitin – open card sorting
Test 2: information relating to couses and units in the MYLO interface – closed card
sorting
Two groups were used, recruited as described below:
Group 1 - prospective students from Unistart
Group 2 – staff who are also students
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
-3-
Each group was given both tests. At the conclusion of each test, participants were
given a brief questionnaire, in which they were asked to please name up to 5 of the
most important features, to them, and to state why this was important.
Card Sorting Methodology:
A combination of closed and open card sorting was used, in which some cards had
labels (closed) or participants could write their own (open) (Rosenfeld and Moreville
2002). In Test 2 participants were presented with labels or terms relating to items they
might see when logged into MyLO.
The participants were directed to:
o Sort the cards into piles of related material
o Name each pile
o Sort the cards in each pile in order of importance
o Rank or order each pile in order of usefulness to them
o They could:
o Rename cards
o Write new cards
o Remove cards to the ‘Trash’ pile
While the participants worked, their comments were recorded using an IPod.
All participants were also given a short questionnaire at the conclusion of card work.
Analysis:
The audio recording was reviewed and interpretive (This is…) or interrogative (What
is …) comments were noted down, as an indication of where the categories or terms
used in the testing either did or didn’t make sense. Any information that was referred
to as ‘missing’ was also noted. Comments related to planning and timing were
prevalent.
These comments were sorted into general headings. Individuals are not identified. A
list of these comments appears in Appendix B.
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
-4-
At the conclusion of testing each pile and its cards were then given numbers
according to their placement by the academics. These ranking were then transferred to
the documents listed in the original audit of the website.
The comments from the questionnaire were recorded with their frequency. The results
are shown in Appendix C.
Results:
Group 1, Test 1:
Preliminary group perceptions on purpose of system:
 Citation system helper

Essay helper

Discussion groups

System to not commit plagiarism

Essay checker

Database

Ref. check

Sentence matcher

Spellcheck

Reference guide

Proof reader/”double check”

Plagiarism specialist
Structure, from card sorting:
Referencing:

Glossary of Useful Terms
Referencing:

Explaining referencing systems and uses

Which referencing system to use
o Harvard
o Number and note
o …
Definitions:
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

Reference

Avoiding plagiarism

Examples:

Reference
-5-
Essay helper
Citation helper
Other languages:

Referencing

Plagiarism
Ask a tutor for help:
Discussion group: find someone to work out the problems with
Useful resources
What to do next
Vote for different answers
Site Management

Site maintenance

Security

How your work is protected

How information is regulated daily: new material, essays, documents

Contact for person overseeing the site

Credentials

Access for Alumni

Advertising
Helper:

Definition of what the system will do

Tutorials

Info sessions on how to use it
Definitions (including examples and pictures):

What it needs

Database of phrases and sentences

Dictionary: terms that faculties use

Spell check
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

-6-
Terms
o Citation
o Essay
o Plagiarism
Frequently Asked Questions:

How do I get into it?

Can I use it from work/home?

How is the information protected?

Why should I use it?

How soon can I get results?

Will Uni staff see my work?

When is it updated?

What is unique about this system?
Where would you expect to find this system?
 Uni homepage under quick links

Library, on computer lab machines

Library home page

MyLO

Faculty notice boards

Bookshop, in the shop
Group 1, Test 2:
Structure from Card Sorting:
Only available under each unit:
Course Links

Glossary of Terms

Submit web links

Electronic discussion

Web links
Course Info

Theoretical background

Lecture notes
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

Powerpoint slides

Recorded lectures
-7-
Tutorial

Tutorial topics

Tutorial group selection

Tutorial readings

What happens in tutorials?
Both, although unit information may be specific to unit:
Submission Guidelines

Guidelines for preparing assignments

Essay Submission

Assignment submission

Assignment submission via Tunitin

Lab report submission

Lab report submission via Turnitin

Group work submissions

Mid-semester quiz
Samples

Unit outline

Sample recorded lectures

Sample submission of work

Sample self test

Self test

Sample quiz

Sample submission via Turnitin
Web Links

Web link to Library

Library catalogue for self help

Web link to IT Support

Web link to Student Support
Outside Unit, everyone can see this:
How To…
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

What is Lectopia? (new)

What is Turnitin? (new)

What is MyLO? (new)

How to write essays

How to avoid plagiarism

How to reference

How to manage your time

How to prepare for exams

Tutorials about using computers, for self help

How to use Turnitin

How to use MyLO

How to use Lectopia

Turnitin, for self help
-8-
Group 2 Test 1:
Preliminary group perceptions on purpose of system:
 Feedback

Plagiarism

Reference validation

More information about Turnitin

Submitting

Optional feedback before final submission

Secure

Privacy/ethical

Grammar check

Academic English checking

Indicate extra advantages

Simplicity, format for printing

Pre-submission check

Grammar/spelling

Basic construction check

Simple reports and options for more details
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

Plagiarism

Feedback

Checking

Referencing

Help?

Legitimisation

One stop shop for assignment writing

At the moment anti-student (need to change this)
-9-
Structure from Card Sorting:
Purpose

University’s purpose (Transparency)

Communication and Education
o UTAS
o Lecturer
Wish List

Formatting

What does it do?

Simplifying use

Score % match

Notification of assignment being received by staff member

Progress of assignment
o Viewed
o What’s happening
o Feedback

Your submission has been successful

Grammar/spelling check

Reminder of due date for assignment

Link for Current Students on UTAS web site

Big buttons – small are too hard to read

Simple terms

Basic examples of plagiarism

Flexible for each school but consistent for student use across faculties
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

- 10 -
Referral for help where do you find help?
Second chance function

Pre-submission check

Submit – feedback – resubmit – (you have paraphrased here – reference? –
you have plagiarised here – revise)
Referencing help

Input – which referencing system? – referencing check

Referencing – specific examples according to ref. style

What if you do your own referencing instead of using Endnote?
Introduction

Background information

Features

Benefits

Explanation

Why use it?

What’s it for?

Virtual tour (tutorial)One click what features and why, protect yourself

Lecturer feedback pre final submission

Integrated into current systems – eg MyLO
Where would you expect to find this system?
 MyLO

Current students

School home pages

Student learning site

International students home page

Library home page
What made you hate it?
 Policing – big brother

Guilty until proven innocent

Lack of feedback
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
- 11 -
o No indication of submission, disappeared
o Unclear feedback – just sentences, direct quotes or standard phrases

Even though cleared of plagiarism, still feel guilty

Not for students, more for the uni

No second chance

What is an acceptable amount? No parameters

Should not be an assignment writing tool

Should have multiple uses/options
Group 2, Test 2:
Structure from Card Sorting:
Only available under each unit:

Unit outline

Theoretical background

Powerpoint slides

Lecture notes

Recorded lectures

What happens in tutorials

Tutorial group selection

Tutorial topics

Tutorial readings

Electronic discussion

Mid-semester quiz

Sample submission of work

Sample self test

Sample quiz

Self test

Glossary of terms
Both, although unit information may be specific to unit:
MyLO

How to use MyLO

How to use Lectopia
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing

Tutorials about using computers, for self help

Sample recorded lecture
- 12 -
Assignment

Assignment submission

Essay submission

Assignment submission via Tunitin

Sample assignment via Turnitin

How to avoid plagiarism

How to use Turnitin

Group work submission

Guidelines for preparing assignments

Turnitin, for self help

How to write essays

How to reference

Lab report submission via Turnitin

Lab report submission
Self help

Library catalogue, for self help

How to prepare for exams

How to manage your time
Web links

Web links

Web link to IT support

Web link to student support

Web link to library

Submit web links

Random listing of links – like advertisements in Facebook (new)
Appendix A: List of Labels used in Card-Sorting Test 2
Unit outline
Recorded lectures
Tutorial readings
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
- 13 -
Theoretical background
Assignment submission
Assignment submission via Turnitin
Electronic discussion
Group work submission
Tutorial group selection
Powerpoint slides
Web links
Submit web links
Glossary of terms
Self test
Mid-semester quiz
Lab report submission
Lab report submission via Turnitin
Lecture notes
Guidelines for preparing assignments
Tutorial topics
Essay submission
Web link to Student Support
Web link to Library
Web link to IT Support
How to use MyLO
How to use Turnitin
How to use Lectopia
How to manage your time
What happens in tutorials
How to prepare for exams
How to avoid plagiarism
How to reference
How to write essays
Sample quiz
Sample submission via Turnitin
Sample submission of work
Sample self test
Sample recorded lecture
Turnitin, for self help
Library catalog, for self help
Tutorials about using computers, for self help
Appendix B: Comments made during Card Sorting Exercise 1
Interpretive or Definite Comments
 University of Tasmania,
University Web Services, IT Resources
February 2009
Web Usability Testing
- 14 -
Interrogative:
Comments Relating to Planning or Timing:
Information currently absent:
Appendix C: Results of Survey
Results show the total number of each comment that was made.
References
McCracken, Daniel, Rosalee J. Wolfe (2004) User-Centred Website Development
Pearson Prentice Hall
Nielsen, Jakob, 2004 Card Sorting: How Many Users to Test
(http://www.useit.com/alertbox/20040719.html)
Rosenfeld, Louis and Peter Morville (2002) Information Architecture for the World
Wide Web O’Reilly Media Inc, 2nd Edition
About this Document
Title
Web Usability Testing – Card Sorting Exercise – Turnitin and
MyLO
Author
Maria Moore
Version
1.0
Version Date
13th February 2009
Status
Draft
Summary
First run report
Purpose
Contribution to improvement of CALT Online Resources
Document Location
Sent to clients
Revision History
Version Number
Release
Comments
Date
Draft 1.0
 University of Tasmania,
University Web Services, IT Resources
Released to CALT staff
February 2009