PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS
(544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
SOFTWARE INTERFACES
HMI - 2
Marek Milosz
1
Agenda
2



Types of software interfaces
Material for interacting
Interaction mechanisms
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
3
Types of software interfaces
Text-based Interface, Text User Interface
Graphical User Interface
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Text User Interface
4






TUI
The characters displayed on the monitor in line or
lines (text)
Semigraphics or pseudographics
Until today: emoticons (SMS, Skype, e-mail,...)
Encoding: ASCII
ASCII art
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
TUI - Emoticons
5

:) :-) ;-)  :D :*
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
TUI - ASCII art (1)
6
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
TUI - ASCII art (2)
7
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Graphical User Interface
8




GUI
Uses: graphics objects
User interaction with graphical objects
Methods:
 Graphics
cursor + Keyboard
 Mouse
 Touch
screen
 Light pen
 ...
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
9
Material for interacting
Interface objects
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Line of text
10



Type using a keyboard (physical or virtual)
Command line or text box
Elements of I/O:
 Forms
 Questions/answers
 Command
line
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Menus
11


Set of options displayed on the screen
Options visible
less recall - easier to use
 rely on recognition so names should be meaningful


Selection by:
numbers, letters, arrow keys
 mouse
 combination (e.g. mouse plus accelerators)


Options often hierarchically grouped

sensible grouping is needed
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Forms to fill
12




Primarily for data entry or data retrieval
Screen like paper form
Data put in relevant place
Requires


good design
obvious correction
facilities
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Windows
13

Areas of the screen that behave as if they were
independent:
 can
contain text or graphics
 can be moved or resized
 can overlap and obscure each other, or can be laid out
next to one another (tiled)


Scrollbars – allow the user to move the contents of
the window up and down or from side to side
Title bar – describes the name of the window
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Icons
14




Small picture or image
Represents some object in the interface – often a
window or action
Windows can be closed down (iconised)
Icons can be many and various
 highly
stylized
 realistic representations
 standarisation in a particular environment

Hard to develop in many industrial cases
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Icons – standards
15
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Pointers – cursors
16





Shows point of input device activity
Texts vs. graphics
Important component WIMP style
relies on pointing and selecting
things
Uses mouse, trackpad, joystick,
trackball, cursor keys or keyboard
shortcuts
Wide variety of graphical images
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Menus – kinds
17

Menu Bar at top of screen (normally), menu drags down
pull-down menu - mouse hold and drag down menu
 drop-down menu - mouse click reveals menu
 fall-down menus - mouse just moves over bar!


Contextual menu appears where you are
pop-up menus - actions for selected object
 pie menus - arranged in a circle

easier to select item (larger target area)
 quicker (same distance to any option)
… but not widely used!

PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Menus - extras
18

Cascading menus
 hierarchical
menu structure
 menu selection opens new menu
 and so in ad infinitum

Keyboard accelerators
 key
combinations - same effect as menu item
 two kinds
 active
when menu open – usually first letter
 active when menu closed – usually Ctrl + letter
 usually
different in different applications
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Menus – design issues
19






Which kind to use?
What to include in menus?
Words to use (action or description)
How to group items?
How to order items?
Choice of keyboard accelerators (possible conflicts)
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Buttons
20


Individual and isolated regions within a display that
can be selected to invoke an action
Special kinds
 radio
buttons
– set of mutually exclusive choices
 check boxes
– set of non-exclusive choices

Parts of windows/forms
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Toolbars
21



Long lines of icons
Fast access to common actions
Often customizable:
 choose
which toolbars to see
 choose what options are on it
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Ribbons
22




Kind of toolbars
Form of a set of toolbars placed on several tabs
Group and speed using of icons
Mainly used in MS Office
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Palettes and tear-off menus
23


Problem:
menu not there when you want
it!
Solution:
 tear-off
and pin-up menus –
menu ‘tears off’ to become
palette
 palettes – little windows of
actions – shown/hidden via
menu option e.g. available
shapes in drawing package
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Dialogue boxes
24




Information windows that pop up to inform of an
important event or to request information
Dirrerent types: alert, error, informational
Different icons used
Modal windows
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Dialogue boxes - symbols
25
25
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Other window objects
26





Text field
Choice field
Label
List
…
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Physical objects
for interacting
27





Buttons
Devices – controllers
Accelerometer
Cameras
…..
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
28
Interaction mechanisms
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Direct and indirect interaction
29
system
real
system
interface
immediate
feedback
real feedback
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Initiative
30



Who has the initiative?
old question–answer
WIMP interface
–> computer
–> user
WIMP exceptions …
pre-emptive parts of the interface
Modal dialog boxes
come and won’t go away!
 good for errors, essential steps
 but use with care

PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Type of interaction
mechanisms
31




Instructions
Talk, conversation (usually with avatars)
Direct manipulation of objects (virtual or physical,
visual objects)
Exploration (virtual or physical environment)
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Type of interfaces
32








Command line (Command Line Interface, CLI)
WIMP (Windows - Icons - Menus - Pointer)
Natural User Interface (NUI)
Voice
Touch
Gesture
Multimodal
Augmented reality
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Interface: CLI (1)
33



Command line
Character cursor - keyboard (including special keys
F…)
Specific residues: enter the text fields or CAPCHA
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Interface: CLI (2)
34
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Interface: WIMP
35

Windows – Icons – Menus – Pointer
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Interface WIMP – precursor
36



Xerox Alto
1973
One of the first PC
using the Desktop
Metaphor
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Xerox Alto –
file manager
37
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Interface WIMP – Windows
38


W1.0 – 1985; W3.0 – 1990, W95, W2000
Graphical, 16-bit multi-tasking shell on top of an
existing MS-DOS installation
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Windows 3.0
39

1990
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Evolution of interfaces
40
CLI
• Codified
• Strict
GUI
• Metaphor
• Exploratory
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
NUI
HMI - 2
• Direct
• Intuitive
NUI – Natural User Interface
41
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
MUI – Modern User Interface
42
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Voice interface
43


Hands-free and Eyes-free
Disadvantage:
Noise
 Errors in identifying the command
 Snooping (lack of privacy of communication)


User Requirements:
Hands free
 I do not need to look
 Close to customers


Problems
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Voice interface - Avatars
44
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Touch interface
45



It uses the sense of touch
It can work two ways: input and output
Input:
 Touchpad
 Touchscreen

Output:
 Gloves

and clothes making pressure
Special I/O devices using the Braille alphabet
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Touchpad
46



Mouse replacement device
The advantage - built-in and requires no additional
work surface as a mouse
The alternative - trackball (inverted mouse) or the
trackpoint (joystick)
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Touchscreen
47
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Touchscreen – gestures (1)
48
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Touchscreen – gestures (2)
49
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Braille devices
50
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Gesture interface
51


It uses the human body to transmit commands
Devices:
 Gloves
with sensors
 2D cameras
 3D cameras
 Cameras with structured light
 Motion controllers
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Cameras with structured light
52


Use structural lighting (e.g. points or grid) to obtain
information about the environment (3D)
Examples:
Kinect (2010)
 EyeTracker


Kinect:
Cloud of points (infrared LED + infrared camera) -> image
depth
 The image -> an ordinary 2D camera
 Microphones (4)
 Head rotating by motor (vertical)
 Accelerometer

PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Kinect – cloud of points
53
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Gesture interface –
applications
54



Entertainment
Medical (no touch)
Processing of large number of visual data /
graphics / multimedia
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Augmented reality
55

Overlaping the real and virtual images
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Augmented reality - welding
technology training
56
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Groupware Interfaces
57

Provide the simultaneous interaction of a large
number of participants
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Brain-Computer interface
58




BCI
Recognition signals the brain working and using
them to transfer commands to a computer
Electroencephalograph
Threats:
 The
accuracy of reading and interpreting
 The variety of people
 Variability of states of a single man
 Ethics: reading thoughts, control his thoughts and so on
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
BCI – first experiments
59
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
Physical object design
60

Many constraints:
 ergonomic
– minimum button size
 physical – high-voltage switches are big
 legal and safety – high cooker controls
 context and environment – easy to clean
 aesthetic – must look good
 economic – … and not cost too much!

Physical vs. logical state (button: on/off)
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2
61
Questions?
Thank you for attention
PROfessional network of Master’s degrees in Informatics as a Second Competence –
PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)
HMI - 2