SIMPLE Platform (version 1.1.2)
Heuristic Evaluation
of the User Interface
Gavin Maxwell
10th November, 2010
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Contents
Overview ....................................................................................................................................................... 3
Purpose ..................................................................................................................................................... 3
Method ..................................................................................................................................................... 3
Summary of Results .................................................................................................................................. 3
Conclusions/Results in Full ........................................................................................................................... 5
Scenario 1 – Student ................................................................................................................................. 5
Scenario 2 - Tutor ...................................................................................................................................... 7
Scenario 3 - Administrator ........................................................................................................................ 7
Appendix – Usability Heuristics..................................................................................................................... 9
2
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Overview
Purpose
Version 2 of the SIMPLE Platform should aim to make the software more usable in order to increase user
uptake. The ultimate aim is that users should be able to pick up SIMPLE without any instruction or
documentation.
This document measures the SIMPLE user interface against certain key usability principles. It is intended
to complement the user testing and pilot studies that have taken place, as well as the feedback from the
setup and running of live simulations. Although the contents are primarily the opinion of the author, the
observations of these varied SIMPLE users have been heavily drawn upon.
Method
Jakob Nielsen’s well-established usability heuristics1 form the basis of the evaluation (see appendix for
definitions). These are:
-
visibility of system status
match between system and real-world
user control and freedom
consistency and standards
error prevention
recognition rather than recall
flexibility and efficiency of use
aesthetic and minimalist design
help users recognize, diagnose, and recover from errors
help and documentation
These have been used to test against various scenarios, based on common interactions with the system:
-
Student coming to SIMPLE for the first time, and carrying out basic tasks
Tutor logging in to check players’ work
Administrator installing simulations and managing users, groups, and permissions
Summary of Results
Whilst some of the functionality of SIMPLE is found in other commonly used software, other
functionality is unique and novel to users. This poses a challenge for usability. Where standards have
already been established elsewhere, these should be followed. Where standards do not yet exist,
metaphors need to be clear and intuitive, and workflows made explicit, in order that users can grasp the
functionality easily and without recourse to support.
1
http://www.useit.com/papers/heuristic/heuristic_list.html
3
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
There are several important concepts in SIMPLE that are not obvious, but are currently treated as
assumed knowledge. These are listed below, along with some other areas where the user interface
could be improved:
1) A user is not necessarily the same thing as a Player. A Player is often a firm of multiple users.
This is required knowledge for setting up simulations.
2) Consequently, users may share a ‘Firm Environment.’ There is potential for confusion here, as
users may assume the environment is their own.
3) Players must upload drafts of documents which they can then send. This differs from common
applications such as email or blogs, where there would be a compose button next to the folder
hierarchy. We need a corresponding button/label to point users to this first step.
4) When setting up simulations, the administrator has to type unique IDs in the admin user
interface. This should always be hidden behind dropdown menus/checkboxes/etc. so that the
user knows what information is required, and to prevent incorrect data entry. We should never
assume that the user is comfortable assigning unique IDs.
5) The onus is currently on tutors to check simulation environments for student activity. This
differs from the modern paradigm of updates dynamically appearing within a central page/area
(e.g. ‘New Mail Received’), so the user expects to be made aware of new correspondence, and is
disappointed to find that they need to actively check each simulation.
6) Linked to this is the need for user home pages. This gives a central starting-point where users
know they can return to if they get lost (via the Home button). In addition to notifications being
automatically presented here, home pages would also provide a single area for all ‘out-ofsimulation’ information (i.e. any User-specific information, rather than Player information).
Finally, through the addition of home pages, the system becomes more user-centered, in line
with social networking trends, providing users with a sense of familiarity.
7) Setting up simulations is a two step process (first create in Tools, and then deploy on Platform).
We should try and give the administrator some flexibility so that making a simple mistake
doesn’t necessitate a cumbersome redeployment.
8) There is still confusing terminology. E.g. users, players, groups, firms, simulations, blueprints,
projects - most of these concepts are necessary because of the flexibility of the software, but
concise definitions should always be a click away in order to mitigate the usability costs of
system complexity.
9) Consider renaming ‘groups’ to ‘firms’, which is more concrete, and makes simulations easier to
conceptualize during creation. This also ties in with properly labeling the ‘firm environment’ so
that users are aware that they are in a shared environment.
10) We need some sort of overview or pointers to be built directly into the interface (NOT
documentation) through naming of buttons, breadcrumbs, etc. so that users know what the
basic functionality is, and how to get started.
11) The aesthetic will become more minimalist, with unnecessary information, such as credits, less
prominently displayed and important buttons made bigger.
12) Be wary of adding future functionality where it will impact negatively on usability. New features
must be integrated seamlessly into the user interface and not just tagged on.
4
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
These proposals should be implemented in version 2 of the platform. Though involving a substantial
interface redesign, the existing backend codebase is modular and does not require to be rewritten.
Conclusions/Results in Full
Scenario 1 – Student
Issue 1.1: No help option. When user gets stuck, they don’t know where to look for what may be a
straightforward solution. This becomes even more important for tutors and administrators, who are
carrying out more complicated tasks.
Principles violated: help and documentation
Solution: Add help option to main menu bar. This could lead to an externally hosted wiki, or to a selfcontained help file in the system.
Issue 1.2: The Home link does not always go to the same place. On initial login, the user is presented
with a list of simulations. If a simulation has been selected, Home leads to the firm environment for that
simulation. If no simulation has been selected, the user is taken to the News page. This is inconsistent
and confusing. If this is a simulation home button then it should not appear if no simulation is selected.
Principles violated: consistency and standards
Solution: The Home link should always go to the same place. Suggest creating home pages for users,
which would provide a central starting-point and place to display updates (see below).
Issue 1.3: Most popular platforms now have a user home page, for example the ‘dashboard’ in
Wordpress. Users expect to have a home area, to aid navigation and receive updates.
Principles violated: aesthetic and minimalist design, visibility of system status
Solution: Create user home pages. Conceptually, this would be a personal ‘user area’, as distinct from
the ‘firm environment’ within a simulation. Move news feeds/alerts into this page, and have them autorefresh using AJAX (this would be especially useful for administrators to receive updates, rather than
having to manually check).
Issue 1.4: The Firm Environment is not labeled as such. Users may be accustomed to a single-user
paradigm. They may not realize that they are logging into a shared firm environment, rather than a
personal environment. Indeed the term ‘Firm Environment’ is not used in the software. SIMPLE relies on
the administrator to title simulations descriptively.
Principles violated: visibility of system status, recognition rather than recall
Solution: Do we need to explicitly use the term ‘firm environment’ in the user interface? See also issue
3.7 below (renaming ‘groups’ to ‘firms’)
Issue 1.5: The credits/version info and news on boxes on the right of the page take up space which could
be better utilized.
Principles violated: aesthetic and minimalist design
5
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Solution: Get rid of credits box on right (could be in footer instead). Alerts could go into user’s home
page instead. Getting rid of right-hand column would allow both the left column and the main area to be
broader, preventing multi-line folder names, etc.
Issue 1.6: The simulations label for the dropdown is on a different line.
Principles violated: aesthetic and minimalist design
Solution: Redesign page in order that title appears alongside dropdown list. See 1.7B below.
Issue 1.7: Main dropdown button says ‘Go.’ It could be made more explicit to user that they are about
to enter a simulation.
Principles violated: match between system and real-world
Solution: Should be ‘enter simulation’ or something similar. See 1.7B below.
Issue 1.7B: Use of the word ‘simulation’ shatters the illusion by reminding users that they are in a
simulation.
Principles violated: match between system and real-world
Solution: Have the default value as ‘transaction’ and allow the administrator to customize the value to
their choice, e.g. ‘client interaction’
Issue 1.8: When people see Sent and Received folders, the first thing they look for is a Compose button.
It's not immediately obvious that you should upload a draft and then send, as this convention hasn't
been established elsewhere. Users have become accustomed to email and to messaging (on Facebook,
etc.) “The most important consistency is consistency with user expectations. “
Principles violated: match between system and real-world, consistency and standards
Solution: Have a button which corresponds to the Compose button (i.e. upload a draft), to provide user
with an obvious starting point. Also, we need to make the letter/envelope analogy more explicit, partly
by allowing grouping of documents in an envelope. See below.
Issue 1.9: Our letter metaphor is open to misinterpretation/misuse. The document transfer system is
intended to replicate a paper system, in order to give students experience of drafting in standard word
processors, in specific formatting, etc. It is not intended to replace email.
Principles violated: match between system and real-world, flexibility and efficiency of use, aesthetic and
minimal design
Solution: The existing plugin capability of SIMPLE should be utilized in order to provide a dedicated
‘email client’ for those simulations wishing to use it. A Voice-Over-IP plugin should also be investigated,
as well as other plugins relevant to simulated professional learning, such as time management for
billing. This would allow administrators to make available only the functionality that they require,
thereby de-cluttering the interface. A plugins download page should be promoted on SIMPLE
Community, along with docs for those wishing to write and/or submit plugins.
Issue 1.10: There is no auto-save functionality as in Gmail, etc.
6
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Principles violated: consistency and standards
Solution: Investigate where such a feature would be appropriate.
Issue 1.11: The buttons used throughout, for function such as edit and send are small
Principles violated: Fitts's law (takes longer to navigate when using small buttons)2
Solution: Commonly used buttons should be larger.
Issue 1.12: During document upload, the (important) reminder is given – “Note: If uploading a Word
document, ensure that it is 97-2003 compatible.” This is perhaps insufficient, as the same problem could
arise for different document formats.
Principles violated: help and documentation
Solution: Perhaps make this warning more general, whilst still pointing out common incompatibilities,
such as this. Ultimately, the recipient must have the appropriate software to read the document being
sent. This will be a common problem, and deserves a prominent help link.
Scenario 2 - Tutor
Issue 2.1: Tutor has to go into each simulation to check progress. Especially onerous when managing
many simulations.
Principles violated: visibility of system status, consistency and standards
Solution: As noted earlier, auto-refresh needed, similar to Gmail. System should give frequent
notifications to the tutor, and home page is sensible place for this.
Scenario 3 - Administrator
Issue 3.1: Adversarial simulations – it’s not obvious how to set these up, yet this is required knowledge
at the design stage.
Principles violated: error prevention
Solution: The structure of adversarial simulations is probably fine as it is, but the user must be guided
through the process of setting them up (in the Tools), perhaps by a wizard. This may be useful for other
aspects of creating simulations too (e.g. ‘you have no uploaded documents, click here to get started’, or
‘how many players will there be?’). In addition, adversarial simulations should either be instantiated
together (on the Platform), or the process of ‘linking’ should be made straightforward, by removing the
need to type IDs.
Issue 3.2: Setting up simulations is a two step process (first create in Tools, and then deploy on
Platform). There is no way to edit simulations once deployed, which is frustrating and time consuming.
Principles violated: user control and freedom, error prevention
Solution: We should try and give the administrator some flexibility to edit blueprints/simulations so that
making a simple mistake doesn’t necessitate a cumbersome redeployment.
2
http://en.wikipedia.org/wiki/Fitts%27s_law
7
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Issue 3.3: player ID or group ID can be used when setting up simulations – user currently needs to get ID
from URL. Unacceptable – user should never have to type numbers (IDs).
Principles violated: error prevention
Solution: should be automatically generated or popup/dropdown/checkboxes, so that the user knows
what information is required, and to prevent incorrect data entry.
Issue 3.4: A counter has been added to the Create Simulation page, to help the administrator keep track
of how many simulations they have created. Whilst this is useful to an administrator, familiar with the
interface, who is creating large amounts of simulations, it is confusing to the average user.
Principles violated: error prevention, help and documentation
Solution: If such functionality is required it should be available as a plugin, not as core functionality.
Addressing the issue of having to enter IDs should negate the need for such a counter in any case.
Issue 3.5: CSV upload (for users and groups) has no link to format details
Principles violated: error prevention, help and documentation
Solution: Format should be prominently displayed on the page, or a link to a sample CSV file could be
provided.
Issue 3.6: Some actions could be streamlined using AJAX in order to cut down the number of clicks
required
Principles violated: flexibility and efficiency of use
Solution: Examine common tasks to see if AJAX components could help.
Issue 3.7: Should ‘Groups’ be renamed to ‘Firms’? Is there too much flexibility in how users are assigned
to simulations? ’ Users often speak about firms when they are setting up and running their simulations,
but the software refers to groups, which is an abstraction. Terminology within the software should be
real-world based, not tech-speak or abstractions.
Principles violated: match between system and real-world
Solution: The word ‘firm’ is more real-world based than ‘group’, which is just an abstract concept used
for the purposes of grouping users as a single ‘player.’ It’s also just as neutral as ‘group’ since most
‘professional’ organizations can be meaningfully described as firms (law, accountancy, consultancy, etc.)
Simply renaming ‘groups’ to ‘firms’, would remove the need for the extra conceptual step of creating a
group (which is really imagined by the user as a firm). So you would simply create a firm and add users
to it. In addition, the term ‘Firm Environment’ then becomes more meaningful, as it’s obvious that this is
a shared area.
Issue 3.8: inconsistent naming in documentation.
Principles violated: consistency and standards
Solution: Revise - get rid of SIMPLE: platform and SIMPLE: Tools. All docs should refer to SIMPLE
Platform or SIMPLE Tools (or SIMPLE, which refers to the platform).
8
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Issue 3.9: SIMPLE is intended to be flexible, and is somewhat complex by design. Whilst this is not
necessarily a problem (with correct interface design), it is something to be aware of for future
development.
Principles violated: aesthetic and minimalist design, error prevention
Solution: Be wary of adding future functionality where it will impact negatively on usability. New
features must be integrated seamlessly into the user interface and existing workflows (licenses is an
example of something which has been added recently but caused problems for users).
Appendix – Usability Heuristics
Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback
within reasonable time.
Match between system and real-world
The system should speak the users' language, with words, phrases and concepts familiar to the user,
rather than system-oriented terms. Follow real-world conventions, making information appear in a
natural and logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave
the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing.
Follow platform conventions.
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the
first place. Either eliminate error-prone conditions or check for them and present users with a
confirmation option before they commit to the action.
Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not
have to remember information from one part of the dialogue to another. Instructions for use of the
system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such
that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent
actions.
9
SIMPLE Platform (version 1.1.2) – Heuristic Evaluation of the User Interface
Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of
information in a dialogue competes with the relevant units of information and diminishes their relative
visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and
constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide
help and documentation. Any such information should be easy to search, focused on the user's task, list
concrete steps to be carried out, and not be too large.
10