COM531 Multimedia
Technologies
Lecture 10 – HCI Guidelines
Evaluation
The User Interface

All those parts of the system we come into
contact with…

Physically we might interact with a device by pressing
buttons or moving levers and the interactive device might
respond by providing feedback through the pressure of
the button or lever.

Perceptually the device displays things on a screen, or
makes noises which we can see and hear.

Conceptually we interact with a device by trying to work
out what it does and what we should be doing. The
device provides messages and other displays which are
designed to help us do this.
The User Interface

Input




Methods are needed to enter commands (tell the
system what we want it to do)
We also need to be able to navigate through the
commands and the content of the system
We need to enter data or other content into the
system
Output


So the system can tell us what is happening - provide
feedback
So the system can display the content to us.
Key Issues




Accessibility
Usability
Acceptability
Engagement
Accessibility
Removal of the barriers that would otherwise
exclude some people from using the system
at all
 Legislation requires software to be
accessible.



Web Accessibility Initiative


UK’s Equality Act 2010 and Section 508 in the US
The Web Content Accessibility Guidelines (WCAG)
2.0, explain how to make Web content accessible for
people with disabilities.
ISO 9241 Part 171 Guidance on Software
Accessibility
Principles of Universal Design




Equitable Use: The design does not disadvantage or
stigmatize any group of users.
Flexibility in Use: The design accommodates a wide
range of individual preferences and abilities.
Simple, Intuitive Use: Use of the design is easy to
understand, regardless of the user's experience,
knowledge, language skills, or current concentration
level.
Perceptible Information: The design communicates
necessary information effectively to the user, regardless
of ambient conditions or the user's sensory abilities.
Principles of Universal Design



Tolerance for Error: The design minimizes hazards and
the adverse consequences of accidental or unintended
actions.
Low Physical Effort: The design can be used efficiently
and comfortably, and with a minimum of fatigue.
Size and Space for Approach & Use: Appropriate size
and space is provided for approach, reach, manipulation,
and use, regardless of the user's body size, posture, or
mobility.
Acceptability and Engagement

Acceptability refers to fitness for purpose in the context of
use
 Unacceptable use of mobile phones

Engagement is concerned with all the qualities of an
experience that really pull people in
 A sense of immersion that one feels
Design Principles, Guidelines
and Rules
International Standards
Design Principles
Universally
applicable high level
design goals, based
on International
Standards
Open to board
interpretation
‘Design for Human
Cognitive Limitations’
Design Guideline
Principles are
interpreted;
guidelines produced
to assist with design
situations
Must be interpreted
within the context of
the task. Usability
must include taskcontext dependent
feature
‘Recognition rather
than recall’
‘Make it obvious
which menu items
are/are not active at
any time’
Design Rule
Highly specific low
level design rules
Found in corporate
style guides and
design manuals.
In menu design,
‘Max of 10 items per
panel; inactive items
should be greyed out’
HCI Guidelines




“Broad brush” design rules
Useful check list for good design
Better design using these than using nothing!
Different collections:



Benyon and Turner’s 12 Principles
Nielsen’s 10 Heuristics
Shneiderman’s 8 Golden Rules
Benyon and Turner’s 12
Principles
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Visibility
Consistency
Familiarity
Affordance
Navigation
Control
Feedback
Recovery
Constraints
Flexibility
Style
Conviviality
1. Visibility






Computers are good at remembering, people are not!
Try to ensure that things are visible so that people can
see what functions are available and what the system
is currently doing
This is an important part of a psychological principle
that it is easier to recognize things than to have to
recall them
If it is not possible to make it visible, make it
observable
Consider making things ‘visible’ through the use of
sound and touch
Use menus, icons, dialog boxes vs commands
1. Visibility
The common
commands
and defaults
are made
visible
Visibility and
sensible
grouping
makes people
aware of other
options
Other
commands are
observable by
using the drop
down menus
2. Consistency





Be consistent in the use of design features
Be consistent with similar systems and standard
ways of working
Consistent use of commands, sequence of tasks,
use of terminology, layout and structure
Internal consistency within the system; interaction
style and presentation
External consistency between packages; easier to
move from one application to another; ‘look and feel’
(but don’t guarantee usability)
3. Familiarity



Use language and symbols that users are familiar
with
Where this is not possible because the concepts are
quite different from those people know about,
provide a suitable metaphor
To help them transfer similar and related knowledge
from a more familiar domain
User:
Designer:
User:
Designer:
User:
Designer:
User:
“I just got a message Rstrd Info. What does it mean?”
“That’s restricted information.”
“But surely you can tell me!!!”
“No, no… Rsdrd Info stands for “Restricted Information.”
“Hmm… but what does it mean???”
“It means the program is too busy to let you log on.”
“Ok. I’m taking a break.”
4. Affordance


Design things so that it is clear what they are
for
The properties of objects and how these
relate to how objects are used


For example make buttons look like buttons so
people will press them
Use textboxes for data entry, labels for displaying
output
5. Navigation



Provide support to enable people to move
around the system
Maps, directional signs, informational signs
Menus are often used for navigation, signs
(labels) indicate where else you can go in the
system
6. Control





Make it clear who or what is in control
Allow user to take control of the system which
responds to actions
Interleaving modes
 Normal View, Print Preview
 Design View, Code View
Tailor system to individual needs (Accessibility)
 Window Size, Font Size, Colour, Toolbars
Good help and documentation
 Tooltips, Context Sensitive Help
 User Guides
 Online help
7. Feedback




Feed back information from the system to user so that
they know what effect their actions have had
Continuously inform the user about
 what it is doing
 how it is interpreting the user’s input
Constant and consistent feedback will enhance the
feeling of control
 E.g. Cursor style, Status bar
User should know when completion is successful
 Direct view as changes happen
 Message box
8. Recovery





Enable recovery from actions and errors quickly and
effectively
Users know that they can return to the previous state by
providing Undo and Cancel options
If they make a mistake, offer them clear and informative
instructions to enable them to recover
 E.g. Message boxes giving instructions
Users don’t like to feel trapped by the computer!
Provide clearly marked exits - should offer an easy way
out of as many situations as possible
9. Constraints





Provide constraints so that people do not try to
do things that are inappropriate
Have allowable actions
Give confirmation of dangerous operations
Users are prevented from making mistakes by
limiting the amount of typing required
Disable menu commands (grey = inactive)
10. Flexibility


Multiple ways of doing things to
accommodate different levels and needs of a
range of users
Provide Shortcuts through use of Hot Keys



E.g. Ctrl C for ‘Copy’ , Ctrl V for ‘Paste’
Perform regular, familiar actions more quickly
Personalise the system by viewing/removing
toolbars as needed
11. Style


Stylish designs
Attractive
12. Conviviality

Polite, friendly and pleasant designs
Nielsen’s 10 Heuristics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Visibility of system status
Match between system and the real world
User control and freedom
Consistency and standards
Error prevention
Helping users recognise, diagnose and recover from
errors
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help and documentation
Shneiderman’s 8 Golden Rules
1.
2.
3.
4.
5.
6.
7.
8.
Strive for consistency
Enable frequent users to use shortcuts
Offer informative feedback
Design dialogs to yield closure
Offer error prevention and simple error
handling
Permit easy reversal of actions
Support internal locus of control
Reduce short-term memory load
International Standards




ISO 9126 Software Engineering
International standard for the evaluation of the
quality of software
4 Parts
Part 1 Software Quality






Functionality
Reliability
Usability
Efficiency
Maintainability
Portability
International Standards




ISO 14915-1:2002 Software ergonomics for
multimedia user interfaces
 Part 1: Design principles and framework
 Part 2: Multimedia navigation and control
 Part 3: Media selection and combination
ISO 20282-1:2006 Ease of operation of everyday
products
 Part 1: Design requirements for context of use and
user characteristics
ISO 6385:2004 Ergonomic principles in the design of
work systems
ISO 9355-2:1999 Parts 1 and 2 "Ergonomic
requirements for the design of displays and control
actuators"
International Standards

ISO/IEC 25051:2005 Software engineering -- Software
product Quality Requirements and Evaluation (SQuaRE)
 Requirements for quality of Commercial Off-The-Shelf
(COTS) software product and instructions for testing

ISO/IEC 25062:2006 Software engineering -- Software
product Quality Requirements and Evaluation (SQuaRE)
 Common Industry Format (CIF) for usability test
reports

ISO/TR 16982:2002 Ergonomics of human-system
interaction
 Usability methods supporting human-centred design
International Standards
Specific for Icons








ISO 11581- 1 Icon Symbols & Functions - General
ISO 11581- 2 Object Icons
ISO 11581- 3 Pointer Icons
ISO 11581- 4 Control Icons
ISO 11581- 5 Tool Icons
ISO 11581- 6 Action Icons
ISO 18035 Icons for Controlling Multimedia Software
ISO 18036 Icons for World Wide Browser Toolbars
International Standards

ISO 9241 Ergonomic Requirements for Office Work
with Visual Display Terminals (VDT).
 32 Parts covering all aspects of Usability
(hardware, software, processes)
 Part 11 Guidance on Usability



Read ISO 9241 Bluffer Guide
For more information on standards:
http://www.procontext.com/en/guidelines/standards.
html
Evaluation
User-Centred System Design
Problem
Statement
Usability Guidelines
& Heuristics
Task
Analysis
Observation of
existing systems
Requirements
Gathering
Requirements Statement –
Functional and non-functional
Design &
Storyboarding
Storyboard
Prototype
Implementation
Evaluation
Installation
Prototype
Transcript & Evaluation Report
Final Implementation
The Star Method
Evaluation




‘Evaluation is concerned with gathering data about the
usability of a design or product by a specified group of
users for a particular activity within a specified
environment or work context’ (Preece, 1994)
Evaluation is central to designing interactive systems
Everything gets evaluated at every step of the process
For example, requirements are evaluated, storyboards
evaluated and a prototype built. The prototype is then
evaluated and some aspect of a physical design
identified and implemented; this is then evaluated again
and so the iteration continues until a final product is
complete
Why do we Evaluate?
‘Users will evaluate your interface sooner or later.’







To suggest improvements to the design
To confirm that the software meets all of the
functional and usability specifications
To confirm acceptability of the interface and/or
supporting materials
To compare alternative designs
To ensure that it meets the expectations of users
To match or exceed the usability of competitor’s
products
To ensure that it complies with standards and any
statutory requirements
Evaluation is Often Performed Badly





Designers assume their own personal behaviour is
‘representative’ of that of an average user
Designers make assumptions about how people are
able to operate the software, but these assumptions
might well be unfounded.
Acceptance of traditional/standard interface design assume style guides ensure good software design
Evaluation may be postponed until ‘a more convenient
time’ when functionality is complete
Poor knowledge of evaluation techniques and lack of
expertise in analysing experiments
What Do We Evaluate?






Usability (Criteria)
Initial designs (pre-implementation)
Interfaces/Interaction (Heuristics)
Prototype at various stages
Final implementation of software system
Documentation
Types of Evaluation


Formative Evaluation
 Evaluation within the design process
 Produce good usability through the process of
evolution, forming/reforming the product
 Informal, Structured
Summative Evaluation
 Take the finished system & assess it for aspects of
usability
 Carry out experiments, after implementation
 Purpose is Quality Control
 Formal, Costly & Time-consuming
Formative Evaluation



Ask the Experts (No users)
1. Cognitive Walkthrough
2. Heuristic Evaluation
Ask the Users
3. Focus groups
4. Questionnaire
5. Interviews
User interaction
6. Think Aloud
7. Co-operative Think Aloud
Cheap
C
O
S
T
S
Expensive
Types of Data


Quantitative data
 ‘Objective’ measures of certain factors by direct
observation
 E.g. time to complete certain tasks, accuracy of
recall, number of errors made
 User performances or attitudes can be recorded in a
numerical form
Qualitative data
 ‘Subjective’ responses; Opinions rather than
measurements
 Reports and opinions that may be categorized in some
way but not reduced to numerical values
Recording Methods






Paper and pencil – cheap, limited to writing speed
Audio – good for think aloud, difficult to match with
other protocols
Video – accurate and realistic, needs special
equipment
Computer logging – automatic and unobtrusive, large
amounts of data difficult to analyze
User notebooks – coarse and subjective, useful
insights, good for longitudinal studies
Mixed use in practice. Audio/video transcription
difficult and requires skill. Some automatic support
tools available
1. Cognitive Walkthrough





‘Expert’ simulates user actions/thoughts and steps through
the action sequence to complete the task
Has rough plan and explores the system for possible actions
Interprets system responses and assesses if each step is or
is not good for a new user
 Are the actions appropriate and visible? Is the feedback
adequate?
Suits systems primarily learned by exploration e.g. walk-upand-use such as ATM, ticket machines
Overall question:
 How successfully does this design guide the unfamiliar
user through the performance of the task?
2. Heuristic Evaluation






A ‘heuristic’ can be defined as a ‘rule-of-thumb’ or
general rule
The idea is to assess the design against known criteria
A number of reviewers (3-5) go through product,
screen by screen, and note any problems and note
violations of these principles, with a severity rating (04)
All responses are collected and aggregated
About 5 reviewers can find about 75% of the problems
Nielsen’s 10 Heuristics
3. Focus Groups





A group of users and an evaluator
Structured group interview; flexible
Allows interaction between users
Typically for requirements gathering, not
system use…
… but can be used post-task
4. Questionnaire




Questions fixed in advance
Completed independently of the evaluator
The purpose of the questionnaire and
purpose of information must be clear
General, open ended, scalar, multi-choice,
ranked questions
5. Interviews





Asking users about their experience with a
system
General questions first, followed by more
detailed ones
Needs careful planning, structured around
some central questions
Structured, semi-structured, unstructured
Requires consistency and flexibility
6. Think Aloud


User participation
User is asked to talk through what he/she is
doing whilst being observed







describing what is happening,
why an action is taken,
what the user is trying to do,
what the user is thinking,
the goal
Evaluator documents actions and problems
found with the interface
Actions and comments are recorded by the
observer using paper notes, video or audio
recording
6. Think Aloud


To increase the quality of the research, we
must avoid the following effects during an
observational study:
Hawthorne Effect


Observer Bias


User increases performance to please observer
Observer only sees and records what they want to
see
Halo Effect

Observer’s judgement is influenced by another,
separate, positive judgement
7. Co-operative Think Aloud




A variation on think-aloud
User and evaluator co-operate in identifying
problems
Evaluator asks questions during the session
User can ask for clarification
General Points


Evaluation is relevant throughout all stages
of development
Different methods are most suited at different
stages - rule-of-thumb:




Early design - analytical
Prototype development - observational/
experimental
Late development - survey
A mix of objective and subjective measures is
desirable