HCI - Lesson 2
07
Interaction
Outline
• What is Interaction Design
– A multidisciplinary field
• Terminology
07
– Interaction “Metaphors” and “Paradigms”
– Ergonomics
– Interface “types”
• Interaction Models (theoretical approaches)
• Users and Stakeholders
• The Interaction Design Cycle and its role in the
conventional sw process
Recap
• HCI has moved beyond desktop machines
and designing interfaces for the
– See “Readings” Economist Oct 8, 2011 «Special
Report: Personal Technology»
– About extending and supporting all manner of
human activities in all manner of places
– Facilitating user experiences through designing
interactions
•
•
•
•
•
Make work effective, efficient and safer
Improve and enhance learning and training
Provide enjoyable and exciting entertainment
Enhance communication and understanding
Support new forms of creativity and expression
What is interaction design?
• Designing interactive products to support the way
people communicate and interact in their everyday
and working lives
– Sharp, Rogers and Preece (2011)
• The design of spaces for human communication and
interaction
– Winograd (1997)
4
Goals of interaction design
• Develop usable products
– Usability means easy to learn, effective to use and
provide an enjoyable experience
• Involve all stakeholders (end-users + ….) in the
design process
5
Scope of interaction design
• Number of other terms used emphasizing what is
being designed, e.g.
– user interface design, software design, user-centered design, product
design, web design, experience design (UX)
• Interaction design is the umbrella term covering all
of these aspects
– fundamental to all disciplines, fields, and approaches concerned with
researching and designing computer-based systems for people
6
HCI and interaction design
7
Relationship between ID, HCI and
other fields
• Academic disciplines contributing to ID:
– Psychology
– Social Sciences
– Computing Sciences
– Engineering
– Ergonomics
– Informatics
8
Relationship between ID, HCI and
other fields
• Design practices contributing to ID:
– Graphic design
– Product design
– Artist-design
– Industrial design
– Film industry
9
Relationship between ID, HCI and
other fields
• Interdisciplinary fields in interaction design:
–
–
–
–
–
–
–
10
HCI
Ubiquitous Computing
Human Factors
Cognitive Engineering
Cognitive Ergonomics
Computer Supported Co-operative Work
Information Systems
Working in multidisciplinary teams
• Many people from different
backgrounds involved
• Different perspectives
and ways of seeing
and talking about things
• Benefits
– more ideas and designs
generated
• Disadvantages
– difficult to communicate and
progress forward the designs being create
Interaction metaphors
• Metaphor = a literary figure of speech that uses an image,
story or tangible thing to represent a less tangible thing or
some intangible quality or idea; e.g., "Her eyes were glistening
jewels."
• In HCI:
– helpful to conceptualize what we are doing with an interactive system,
e.g. “surfing the web”
– A “conceptual model” instantiated at the interface, e.g. the desktop
metaphor
12
Interaction metaphors
• Interaction designed to be similar to the one with
physical entity but also has own properties
– e.g. desktop metaphor
• Can be based on “activities” (what you can DO
with the systems), “objects” (what you manage
during an activity) or a combination of both
• Exploit user’s familiar knowledge, helping them
to understand ‘the unfamiliar’
• Conjures up the essence of the unfamiliar activity,
enabling users to leverage of this to understand
more aspects of the unfamiliar functionality
13
Benefits of interaction metaphors
• Makes learning new systems easier
• Helps users understand the underlying
conceptual model
• Can be very innovative and enable the realm
of computers and their applications to be
made more accessible to a greater diversity of
users
14
Problems with interaction metaphors
• Break conventional and cultural rules
– e.g. recycle bin placed on desktop
• Can constrain designers in the way they
conceptualize a problem space
• Forces users to only understand the system in terms
of the metaphor
• Designers can inadvertently use bad existing designs
and transfer the bad parts over
• Limits designers’ imagination in coming up with new
conceptual models
15
Different Interaction Metaphors
• Instructing
– issuing commands and selecting options
• Conversing
– interacting with a system as if having a conversation
• Manipulating
– interacting with objects in a virtual or physical space by
manipulating them
• Exploring
– moving through a virtual environment or a physical space
16
1. Instructing
• Where users instruct a system and tell it what to
do
– e.g. tell the time, print a file, save a file
• Very common metaphor, underlying a diversity of
devices and systems
– e.g. word processors, VCRs, vending machines
• Main benefit is that instructing supports quick
and efficient interaction
– good for repetitive kinds of actions performed on
multiple objects
17
Example
18
2. Conversing
• Underlying model of having a conversation with
another human
• Range from simple voice recognition menu-driven
systems to more complex ‘natural language’
dialogs
• Examples include timetables, search engines,
advice-giving systems, help systems
• Also virtual agents, toys and pet robots designed
to converse with you
19
Example
https://areaclienti187.telecomitalia.it/auth/registrautente.do?access=portal
20
Pros and cons of conversational
metaphor
• Allows users, especially novices and
technophobes, to interact with the system in a
way that is familiar
– makes them feel comfortable, at ease and less scared
• Misunderstandings can arise when the system
does not know how to parse what the user says
21
3. Manipulating
• Involves dragging, selecting, opening, closing and zooming
actions on virtual objects
• Exploit’s users’ knowledge of how they move and
manipulate in the physical world
• Can involve actions using physical controllers (e.g. Wii) or
air gestures (e.g. Kinect) to control the movements of an
on screen avatar
• Tagged physical objects (e.g. balls) that are manipulated in
a physical world result in physical/digital events (e.g.
animation)
22
Direct Manipulation
• Shneiderman (1983) coined the term DM, came from
his fascination with computer games at the time
– Continuous representation of objects and actions of
interest
– Physical actions and button pressing instead of issuing
commands with complex syntax
– Rapid reversible actions with immediate feedback on
object of interest
23
Why is DM metaphor enjoyable?
• Novices can learn the basic functionality quickly
• Experienced users can work extremely rapidly to
carry out a wide range of tasks, even defining new
functions
• Intermittent users can retain operational concepts
over time
• Error messages rarely needed
• Users can immediately see if their actions are
furthering their goals and if not do something else
• Users experience less anxiety
24
• Users gain confidence and mastery and feel in control
What are the disadvantages with DM?
• Some people take the metaphor of direct
manipulation too literally
• Not all tasks can be described by objects and not all
actions can be done directly
• Some tasks are better achieved through delegating
– e.g. spell checking
• Moving a mouse around the screen can be slower
than pressing function keys to do same actions
25
4. Exploring
• Involves users moving
through virtual or
physical environments
– E.g., browsing the web
– Moving in a 3D virtual
space
• Physical environments
with sensing
technologies, e.g.,
– Context aware
computing
– Wearable computing
26
Which metaphor is best?
• Direct manipulation is good for ‘doing’ types of tasks,
e.g. designing, drawing, flying, driving, sizing
windows
• Issuing instructions is good for repetitive tasks, e.g.
spell-checking, file management
• Having a conversation is good for children, computerphobic, disabled users and specialised applications
(e.g. phone services)
• Hybrid conceptual models are often employed,
where different ways of carrying out the same
actions is supported at the interface - but can take
longer to learn
27
Further Terminology: “Paradigm”
• General approach adopted by a community
for carrying out research
– shared assumptions, concepts, values, and
practices
– e.g. desktop, ubiquitous computing, in the wild
28
Examples of new paradigms
•
•
•
•
•
•
Ubiquitous computing (mother of them all)
Pervasive computing
Wearable computing
Tangible bits, augmented reality
Attentive environments
Transparent computing
– and many more….
29
Further terminology: Ergonomics
the study of designing equipment and
devices that fit the human body, its
movements, and its cognitive abilities.
Ergonomics - examples
• arrangement of controls and displays
e.g. controls grouped according to function or frequency of use, or
sequentially
• surrounding environment
e.g. seating arrangements adaptable to cope with all sizes of user
• health issues
e.g. physical position, environmental conditions (temperature,
humidity), lighting, noise,
• use of colour
e.g. use of red for warning, green for okay,
awareness of colour-blindness etc.
Ergonomics and Interaction Design
• Ergonomics: Study of the physical characteristics of
interaction
• Also known as human factors – but this can also be
used to mean much of HCI!
• Ergonomics good at defining standards and
guidelines for constraining the way we design certain
aspects of systems
From Interaction to
Interface
From interaction to interface
• Interaction metaphor:
– what the user is doing when interacting with a system, e.g.
instructing, talking, browsing or other
• Interface type:
– the kind of interface used to support the metaphor, e.g.
speech, menu-based, gesture
34
Many kinds of interface types
available…
•
•
•
•
•
•
•
•
•
•
•
•
35
Command line
menus
speech and natural language
question/answer and query dialogue
data-entry & form-fills
WIMP
Three–dimensional interfaces
Multitouch ((see dedicated lesson)
Gesture (see dedicated lesson)
Tangible (see dedicated lesson)
Augmented reality
…
Command line interface
• Way of expressing instructions to the computer
directly
– function keys, single characters, short abbreviations, whole
words, or a combination
•
•
•
•
suitable for repetitive tasks
better for expert users than novices
offers direct access to system functionality
command names/abbreviations should be
meaningful!
Typical example: the Unix system
36
Menus
• Set of options displayed on the screen
• Options visible
– less recall - easier to use
– rely on recognition so names should be meaningful
• Selection by:
– numbers, letters, arrow keys, mouse
– combination (e.g. mouse plus accelerators)
• Often options hierarchically grouped
– sensible grouping is needed
• Restricted form of full WIMP system (see next slides)
37
Speech & Natural language
• Speech recognition or typed natural language
• Familiar to user
• Rapidly improving …
… but still inaccurate
• Problems (see phone services!)
– vague
– ambiguous
– hard to do well!
• Solutions
– try to understand only a subset
– pick on key words
– Provide feedbacks
e.g. airline reservation:
reliable “yes” and “no”
+ system reflects back its understanding
“you want a ticket from New York to Boston?”
38
Error and repair (in speech interaction,
and in general)
can’t always avoid errors …
… but we can put them right
make it easy to detect errors
… then the user can repair them
hello, this is the Go Faster booking system
what would you like?
(user) I want to fly from New York to London
you want a ticket from New York to Boston
(user) no
sorry, please confirm one at a time
do you want to fly from New York
(user) yes
………
39
Q&A and Query interfaces
• Question/answer interfaces
– user led through interaction via series of questions
– suitable for novice users but restricted functionality
– often used in information systems
• Query languages (e.g. SQL)
– used to retrieve information from database
– requires understanding of database structure and
language syntax, hence requires some expertise
40
Form-fills
•
•
•
•
Primarily for data entry or data retrieval
Screen like paper form.
Data put in relevant place
Requires
– good design
– obvious correction
facilities
41
Spreadsheets
• first spreadsheet VISICALC, followed by Lotus
1-2-3
MS Excel most common today
• sophisticated variation of form-filling.
– grid of cells contain a value or a formula
– formula can involve values of other cells
e.g. sum of all cells in this column
– user can enter and alter data spreadsheet
maintains consistency
42
WIMP Interface
Windows
Icons
Menus
Pointers
… or windows, icons, mice, and pull-down menus!
• default style for majority of interactive computer systems,
especially PCs and desktop machines
• Point&Click mechanism
– just click something!
• icons, text links or location on map
43
– minimal typing
elements of the WIMP interface
windows, icons, menus, pointers
+++
buttons, toolbars,
palettes, dialog boxes
44
also see supplementary material
on choosing wimp elements
WIMP: Windows
• Areas of the screen that behave as if they were
independent
– can contain text or graphics
– can be moved or resized
– can overlap and obscure each other, or can be laid out next
to one another (tiled)
• scrollbars
– allow the user to move the contents of the window up and
down or from side to side
• title bars
45
– describe the interaction context (e.g., the name of the
window)
WIMP: Icons
• small picture or image
• represents some object in the interface
– often a window or action
• windows can be closed down (iconised)
– small representation fi many accessible windows
• icons can be many and various
– highly stylized
– realistic representations.
46
WIMP: Pointers
• important component
– WIMP style relies on pointing and selecting things
• uses mouse, trackpad, joystick, trackball, cursor keys or
keyboard shortcuts
• wide variety of graphical images
47
WIMP: Menus
• Choice of operations or services offered on the screen
• Required option selected with pointer
File
Edit
Options
Font
Typewriter
Screen
Times
problem – take a lot of screen space
solution – pop-up: menu appears when needed
48
WIMP: Kinds of Menus
• Menu Bar at top of screen (normally), menu drags
down
– pull-down menu - mouse hold and drag down menu
– drop-down menu - mouse click reveals menu
– fall-down menus - mouse just moves over bar!
• Contextual menu appears where you are
– pop-up menus - actions for selected object
– pie menus - arranged in a circle
• easier to select item (larger target area)
• quicker (same distance to any option)
… but not widely used!
49
WIMP: Menus extras
• Cascading menus
– hierarchical menu structure
– menu selection opens new menu
– and so in ad infinitum
• Keyboard accelerators
– key combinations - same effect as menu item
– two kinds
• active when menu open – usually first letter
• active when menu closed – usually Ctrl + letter
usually different !!!
50
WIMP: Menus design issues
• which kind to use
• what to include in menus at all
• words to use (action or description)
• how to group items
• choice of keyboard accelerators
51
WIMP: Buttons
• individual and isolated regions within a display
that can be selected to invoke an action
• Special kinds
– radio buttons
– set of mutually exclusive choices
– check boxes
– set of non-exclusive choices
52
WIMP: Toolbars
• long lines of icons …
… but what do they do?
• fast access to common actions
• often customizable:
– choose which toolbars to see
– choose what options are on it
53
WIMP: Palettes and tear-off menus
• Problem
menu not there when you want it
• Solution
palettes – little windows of actions
– shown/hidden via menu option
e.g. available shapes in drawing package
tear-off and pin-up menus
– menu ‘tears off’ to become palette
54
WIMP: Dialogue boxes
• information windows that pop up to inform of
an important event or request information.
e.g: when saving a file, a dialogue box is displayed
to allow the user to specify the filename and
location. Once the file is saved, the box
disappears.
55
WIMP: Look and … feel
• WIMP systems have the same elements:
windows, icons., menus, pointers, buttons, etc.
• but different window systems
… behave differently
e.g. MacOS vs Windows menus
appearance + behaviour = look and feel
56
Beyond WIMP:
Three dimensional interfaces
• 3D sometimes used in ‘ordinary’ WIMP systems
– highlighting
– visual affordance
– indiscriminate use
just confusing!
• More frequently: 3D workspaces
– use for extra virtual space
– light and occlusion give depth
– distance effects
57
flat buttons …
click me!
… or sculptured
Beyond WIMP
• Multitouch, Gesture, Tangible: see dedicated lessons
• Augmented reality and wearable interaction
• And many others…
Which interaction and interface type
to choose?
• Need to determine requirements and user
needs
• Take budget and other constraints into
account
• Also will depend on suitability of technology
for activity being supported
59
“Models” of interaction: theoretical
approaches
60
Norman’s
execution/evaluation loop
goal
execution
evaluation
system
•
•
•
•
•
•
•
61
user establishes the goal
formulates intention
specifies actions at interface
executes action
perceives system state
interprets system state
evaluates system state with respect to goal
execution/evaluation loop
goal
execution
evaluation
system
•
•
•
•
•
•
•
62
user establishes the goal
formulates intention
specifies actions at interface
executes action
perceives system state
interprets system state
evaluates system state with respect to goal
execution/evaluation loop
goal
execution
• user establishes the goal system
evaluation
• formulates intention
• specifies actions at interface
• executes action
• perceives system state
• interprets system state
• evaluates system state with respect to goal
63
execution/evaluation loop
goal
execution
•
•
•
•
•
•
•
64
evaluation
system
user establishes the goal
formulates intention
specifies actions at interface
executes action
perceives system state
interprets system state
evaluates system state with respect to goal
Using Norman’s model as an
interpretative tool
Some systems are harder to use than others
Gulf of Execution
user’s formulation of actions
≠
actions allowed by the system
Gulf of Evaluation
user’s expectation of changed system state
≠
65
actual presentation of this state
Human error - slips and mistakes
Slip (=svista)
understand system and goal
correct formulation of action
incorrect action
mistake
may not even have right goal!
Fixing things?
slip – better interface design
mistake – better understanding of system
66
Abowd & Beale’s model
extension of Norman…
their interaction framework has 4 parts
– user
– input
S
– system
core
– output
O
output
U
task
I
input
each has its own unique language
interaction  translation between languages
problems in interaction = problems in translation
67
Using Abowd & Beale’s model
user intentions
 translated into actions at the interface
 translated into alterations of system state
 reflected in the output display
 interpreted by the user
general framework for understanding interaction
–
–
–
–
68
not restricted to electronic computer systems
identifies all major components involved in interaction
allows comparative assessment of systems
an abstraction
Recap
• Terminology
– Interaction Design
– Ergonomics
– Interaction metaphor
– Interface types
– Cognitive models of interaction (Norman’s, Abow
& Beale’s)
69
The Interaction Design Process
OUTLINE
• Phases
• What is involved in Interaction Design?
– Importance of involving users
– Degrees of user involvement
– What is a user-centered approach?
• Some practical issues
–
–
–
–
Who are the users?
What are ‘needs’?
Where do alternatives come from?
How do you choose among alternatives?
• The ID process in the sw development life
cycle
70
Interaction Design PROCESS:
Four basic activities
1. Establishing requirements
2. Designing alternatives
3. Prototyping
4. Evaluating
71
71
Iterative Activities:
The interaction design lifecycle model
Requirements
Management
Design
Evaluation
Prototyping
72
Interaction Design is a user-centered approach
based on:
– Early focus on users, tasks, and context of use: directly
studying cognitive, behavioral, cultural, attitudinal
characteristics of the people who (will) use the product
and their context of use
– Empirical observation: users’ reactions and performance
to scenarios, manuals, simulations & prototypes are
observed, recorded and analysed
– Iterative process: when problems are found, fix them and
carry out more tests
73
Some practical issues
• Who are the users?
• What do we mean by ‘needs’?
• How to generate alternatives
• How to choose among alternatives
• How to integrate interaction design
activities with other models?
74
Who are the “users”?
• Not as obvious as you think:
–
–
–
–
–
those who interact directly with the product
those who manage direct users
those who receive output from the product
those who make the purchasing decision
those who use competitor’s products
• Three categories of user (Eason, 1987):
– primary: frequent hands-on
– secondary: occasional or via someone else
– tertiary: affected by its introduction, or will influence its
purchase
75
Improving our terminology
Users= end users of the product (primary or
secondary users)
Stakeholders= all kind of users (primary or
secondary or tertiary): Any subject how is
interested to the interactive product
76
Who are the stakeholders?
Check-out operators
The programmers
• Suppliers
• Local shop
owners
Managers and owners
77
Customers
Importance of involving stakeholders
• Expectation management
–
–
–
–
Realistic expectations
No surprises, no disappointments
Timely training
Communication, but no hype
• Ownership
– Make the stakeholders active participants
– More likely to forgive or accept problems
– Can make a big difference to acceptance and success of
product
78
Degrees of stakeholders involvement
• Member of the design team
– When?
• During requirements management
• During design and prototyping
• During pre-release evaluation
– For how long?
•
•
•
•
Full time (constant input)
Part time (patchy input)
Short term : in specific phases only
Long term (along the whole design process, and after – see below)
• Involvement after product is released
– Evaluation, maintenance and update
79
• Combination of these approaches
What do we mean by ‘needs’?
• Stakeholders rarely know what is possible
• Stakeholders can’t tell you what they ‘need’ to
help them achieve their goals
• Instead, look at existing tasks:
– their context
– what information do they require?
– who collaborates to achieve the task?
– why is the task achieved the way it is?
• Envisioned tasks:
– can be rooted in existing behaviour
– can be described as future scenarios
80
How to generate alternatives
• Humans stick to what they know works
• But considering alternatives is important to ‘break
out of the box’
• Designers are trained to consider alternatives,
software people generally are not
• How do you generate alternatives?
—‘Flair and creativity’: research and synthesis
—Seek inspiration: look at similar products or
look at very different products
81
How to choose among
alternatives
• Evaluation with users or with peers, e.g. prototypes
• Technical feasibility: some not possible
• Quality thresholds: Usability goals lead to usability criteria set
early on and check regularly
—safety: how safe?
—utility: which functions are superfluous?
—effectiveness: appropriate support? task coverage, information
available
—efficiency: performance measurements
82
Testing prototypes to choose among
alternatives
83
How to integrate interaction design in
other models
(e.g. for sw development)
• Software engineering is the discipline for understanding the
software design process, or life cycle
• Interaction design occurs at many stages of the life cycle, not
as a single isolated activity
84
Sw process models:
The waterfall model
Requirements
specification
Architectural
design
Detailed
design
Coding and
unit testing
Integration
and testing
Operation and
maintenance
85
Activities in the sw life cycle
Requirements specification
designer and customer try capture what the system is expected to provide
can be expressed in natural language or more precise languages, such as a
task analysis would provide
Architectural design
high-level description of how the system will provide the services required
factor system into major components of the system and how they are
interrelated needs to satisfy both functional and nonfunctional
requirements
Detailed design
refinement of architectural components and interrelations to identify
modules to be implemented separately the refinement is governed by the
non-functional requirements
86
The life cycle for interactive systems
cannot assume a linear
sequence of activities
as in the waterfall model
Requirements
specification
Architectural
design
Detailed
design
Coding and
unit testing
lots of feedback!
Integration
and testing
Operation and
maintenance
87
Sw process: Bohem’s model
cctr.umkc.edu/~kennethjuwng/spiral.htm
88
Where and How Interaction Design plays?
A new, complementary
perspective in a
number of phases
Requirements: focus
on all stakeholders and
their need
Requirements
specification
Interaction
and Interface
Design
Architectural
design
Design &
Implementation:
focus on interface and
interaction
mechanisms
Testing: focus on
features directly
perceived by end-users
89
Detailed
design
Coding and
unit testing
Interaction
and Interface
Implementation
Integration
and testing
Operation and
maintenance
Recap: Interaction Design Process is
• a goal-directed problem solving activity
informed by intended use, target domain,
materials, cost, and feasibility
– a creative activity
– a decision-making activity to balance trade-offs
• a user centered process
90