cs123
INTRODUCTION TO COMPUTER GRAPHICS
Steve Jobs introducing the 1984 Apple Macintosh
User Interface I
Interface Design and Philosophy
Andries van Dam©, James Foley
1 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
The User Experience


Star Trek IV: The
Voyage Home (1986)
A bad UI day
Andries van Dam©, James Foley
2 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Leading Questions


What is the most common use of computer graphics today?
What are we going to use all that ever-increasing “Moore’s
Law” compute/graphics power for?!?




Most apps now have all the power and functionality they need…
What approximate percentage of a modern app’s code is
the UI?
What differentiates apps? Platforms?
What lies beyond WIMP GUIs?
Andries van Dam©, James Foley
3 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Computer power vs. brain power
Compute
Graphics
UI
Computing Capacity
“Moore’s Law”
Human Capacity
t
t
Use compute power in UI to increase b/w to the brain
Andries van Dam©, James Foley
Courtesy of Bill Buxton
4 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Why Interface Matters…
Not just applicable to computer interfaces!
“Web sites with large liberal
followings like Mother Jones,
Slate and The Huffington Post
highlighted a video that claimed
to show a “Romney-loving“
voting machine in Pennsylvania
that was converting Obama
votes into votes for Mitt
Romney.”
http://www.youtube.com/watch
?v=QdpGd74DrBM
http://www.youtube.com/watch?v=yUdpj3gJofQ
Andries van Dam©, James Foley
5 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Why Interface Matters…
UI should protect against
obvious user error
The Mac’s 1984 technique
of graying out
inapplicable choices and
beeping when the
command didn’t make
sense was a novel
alternative to error
messages
Andries van Dam©, James Foley
6 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Why Interface Matters…


Sometimes a user interface can be a matter of life and death
Dec. 20th 1995: Ambiguity in a typed-in command caused Cali, Columbia
757 crash that took the life of CS Prof. Paris Kanellakis and his family
Andries van Dam©, James Foley
7 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
A Few More Questions

What is the most common complaint about computers?

What are some interfaces you like and dislike?

What are the identifying characteristics of current interface paradigms?

How do technical considerations affect interface considerations and vice-versa?

How could “intelligent” interfaces help or get in the way of users? (NB: Clippy!
Is Siri that much better?!?)

Is it possible to accommodate users of all levels with a single interface?

What is your ideal interface?
 For general use (operating system / work environment)?
 For specific applications?
Andries van Dam©, James Foley
Bonzi Buddy image:
https://en.wikipedia.org/wiki/BonziBuddy#/media/File:Bonzi_buddy.jpg
Bonzi Buddy
8 / 61
cs123
•
•
•
•
•
•
INTRODUCTION TO COMPUTER GRAPHICS
Is there an ideal user interface?
No! UIs are a necessary evil
Counterpoint: aesthetics of a good UI, once mastered
Want to communicate and control s/w as we do in and with the real
world so we can leverage real world knowledge (yet allow magic)
– Objects (tea cups, pens, door handles, steering wheels,…)
– Other participants (real and software agents)
Models for agents: Wodehouse’s Jeeves
• But beware of HAL-9000
– Understand context: physical, personal, social...
– Infer intent
– Knowledge Navigator as an early concept video
– Siri on iPhone as embryonic example
Future: brain-machine interfaces, “cogito ergo fac” (braingate,
much less invasive techniques)
Today: transparent, fluid UI's
automaticity
Andries van Dam©, James Foley
9 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
An Extreme

“Microsoft Word” at its worst (pre-”ribbon”)
Andries van Dam©, James Foley
10 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Improved?

Microsoft Word from MS Office 2016
At least it’s harder to clutter your screen like in prior versions, and there is no Clippy.
And you can make the ribbon disappear…
Image source:
Andries van Dam©, James Foley https://en.wikipedia.org/wiki/Microsoft_Office_2016#/media/File:Microsoft_Office_2016_Screenshots.png
11 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Tradeoffs in Design

What are the pros and cons of these interface styles?
 Command language
 Direct manipulation, e.g., WIMP GUI’s point & click, drag & drop
 Swipe, pinch-zoom, finger-paint
 Pen for select, gestures, literal input of text, math, drawing elements, etc.
 “Intelligent” interfaces, agents, social interfaces

The effectiveness of an interface is determined by the evaluation of its tradeoffs
The usability of a system too often is inversely related to its functionality
A successful interface designer must know her users and their priorities, the
computing environment, and the task domain


Andries van Dam©, James Foley
12 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
User Interface Design
Overview
 This lecture addresses primarily User-Centered Design for WIMP, i.e. Windows, Icons, Menus
and Point-and-click GUI’s using keyboard and mouse/touchpad
 This style of interface revolutionized computing and made it accessible to the masses, even
toddlers, starting with the Mac in ‘84, based on XEROX PARC’s Alto and Star workstations
 Much applies as well to non-WIMP/post-WIMP interfaces, e.g., those interfaces using
gestures (based on input via pen/stylus, multi-touch, VR data wands, computer vision (e.g.,
Kinect), …) or speech recognition. Corning Video
Outline

General observations and overview of user interfaces

User interface design methodology and principles

Summary of guidelines and main concepts in user
interface design
Andries van Dam©, James Foley
Image of Tesla Model S interior from http://www.ibtimes.com/first-ever-tesla-model-s-teardowndashboard-more-silicon-valley-less-detroit-1704803
13 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Purpose of UI Design (1/2)
The user interface is the key to productivity

Provide usable, functional, safe, efficient
systems for people

Concentrate on user-centered design:


High Level goals of UI

Make easy things easy; make hard things
possible

Optimize human factors and ergonomics
 Make your interface comfortable and
inviting, as well as attractive

Maximize speed of learning
 Including the transition from novice to
expert user

Maximize speed and ease of use

Minimize error rate

Enhance the UX (User Experience)!

Consistency with user’s expectation: “law of
least astonishment!”
Design for your user, not your hardware
UI now much more important than features

Feature bloat

Remember Pareto’s Principle:
80/20 rule: 20% of features used 80% of time
Andries van Dam©, James Foley
14 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Purpose of UI Design (2/2)
Note: These goals cannot all be fully accommodated in the same interface.
We must determine which goals are most important for the user and the
purpose of the application
Berkeley Breathed's
“Bloom County”
Andries van Dam©, James Foley
15 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Brief History of UI - Major Events and Innovations (1/3)

1963, Ivan Sutherland published the landmark graphics system
Sketchpad, which had lots of physical buttons, keys, panning, and
zooming.

1968, Douglas Engelbart shows “the Mother of All Demos” of the
NLS/Augment hypermedia document system at the Fall Joint
Computer Conference. Featured tiled windows, mouse, chord
keyboard, command line interface, remote collaborators sharing
document editing…

Above: Sutherland’s Sketchpad
Below: Engelbart during the 1968 demo
1970, Engelbart patented the mouse

Apple “borrowed” the mouse from Xerox PARC,
who “borrowed” it from Engelbart; we had a display with a mouse in
1970 (MIT design)
Andries van Dam©, James Foley
16 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Brief History of UI - Major events and innovations (2/3)

1973, Xerox PARC produced the Alto, the first personal
workstation. Based on bit-mapped (raster) display,
commercial mouse, Ethernet, and client-server
architecture. Had world’s first WYSIWYG text editor,
Smalltalk (Alan Kay, Adele Goldberg, et. al.), and WIMP
GUI’s including window managers and browsers; also
Metcalfe’s Ethernet, client-server model

1972, Alan Kay envisions the Dynabook, for kids


Xerox PARC Alto, image by Kevin Powers
Simulation and graphics-based tablet/laptop running
Smalltalk, first full OOPL
Colleague of Mathematician Seymour Pappert (Piagetinfluenced constructivist learning, Logo)
Andries van Dam©, James Foley
17 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Brief History of UI - Major events and innovations


In 1981, Xerox introduced the Star Information System – commercial flop:
too expensive, slow, too far ahead of its time
1984, Apple released the Mac as the first commercial graphics desktop
microcomputer, based on Alto and Star





(3/3)
“Messy desktop” metaphor with overlapping windows, not tiled
Pull-down menus
Icons & toolbars
Drag-and-drop file manipulation
1985, Microsoft Windows, considered a Mac imitation with
minimal improvements
Apple vs. Microsoft suit over Windows’ use of WIMP GUI
 Foley on behalf of MS, Shneiderman for Apple
 Apple lost all claims
2012, Apple vs. Samsung : Jobs’ “thermonuclear war” against Google and its Android
h/w vendors: Jury upholds Apple patents on iPhone industrial design, “bounceback”,
H/V scrolling, translucent overlays,…iPhone design features, awards 1.2B$; 2013,
Apple awarded 290M$, for a total of about 930M$; 2014, Apple awarded 120M$ in
different case, 2015, the $930M$ appealed and lowered… the battle continues…


Andries van Dam©, James Foley
18 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Characteristics of UI Design
The Nature of the Beast
 Collaborative

Iterative

Multi-disciplinary
Players in the UI design game

Hardware engineers

Devices for graphics, video, audio, force feedback…

Software engineers

Human Factors (ergonomics) engineers

Graphic designers

Linguists

Perceptual psychologists

Cognitive scientists

Adventuresome sociologists and cultural anthropologists who study the uses of technology in situ
(e.g., Dr. danah boyd, Brown CS /2001, author of “It’s Complicated”)

And UI/UX designers

Note that industrial design and UI/UX design are intertwined (e.g., iTouch, iPhone, iPad, Android phones…)
Andries van Dam©, James Foley
19 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Language Model for Command Line and WIMP UI’s – Abstraction Layers (1/3)
A. Meaning of an interface, its semantics

Conceptual level:


B. Form of an interface, its “look and feel”

Cognitive uses, models, and metaphors; application
objects and operations (e.g., messy desktop, icons
and windows for docs and apps, trashcan)
Functional level: what I can do



Each function’s semantics: including pre-, post- and
error-conditions (e.g., insert, delete, cut/copy/paste)
Interaction sequencing level: how I specify it
Ordering and interlacing of inputs and outputs, syntax
(e.g., move ptr, select, drag, drop on trash can, deselect to
delete; move ptr, double-click to open a doc) - 2 languages!!!
Binding level:

How input and output units of form are actually created from
physical hardware primitives, lexemes/tokens (e.g., cursor
keys vs. finger or pen) gestures vs. speech)
Conceptual level
Meaning
Form
Functional/semantic level
Sequencing/syntactical level
Binding/lexical level
Andries van Dam©, James Foley
Look
and
Feel
20 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Language Model for Command Line and WIMP UI’s (2/3)





Example: “Delete this file” by drag-and-drop vs. delete key: itemize the layers…
Defining lexemes is usually easy for small WIMP interfaces, harder for non-WIMP
interfaces, especially Virtual Reality (VR)
 Creating a user interface for a large application is essentially management of
complexity
 In IVR applications must also deal with the problems of latency, synchronizing
screens, tiles for compound displays (e.g., Walls, Caves)
Command syntax and semantics tend to be more complicated than lexemes
Often helpful to have a Finite State Machine (FSM) to represent a sequencing of
lexemes to make a complete command. Huge aggregate FSM for UI as a whole
This design model may seem excessively rigid, but a good UI design is a formal process
resulting in a good architecture. Constant tradeoff with rapid prototyping/user testing
after mockups, Wizard of Oz prototypes…
Andries van Dam©, James Foley
21 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Language Model for Command Line and WIMP UI’s (3/3)
Finite State Machine example (FSM is simplest in Chomsky hierarchy of
automata and their equivalent languages: phrase-structure grammars)

Example: UI FSM for blur operation in Filter
Andries van Dam©, James Foley
22 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Design Methodology (1/4)
The overall steps
 Analyze
 Formalize
 Synthesize
 Evaluate
 Implement
 Test
Note: steps are not all distinct or
sequential
(even less so than in software design)
Andries van Dam©, James Foley
For each step in design
 Consider multiple alternatives
 Choose the one which best matches
 User characteristics
 Design objectives
 Functional requirements
Rapid prototype early and often!

6 discrete iterations of TAG UI for Nobel
exhibition, tested each with both
designers and walk-up users (e.g., how to
close a pop-up on 82” screen)
23 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Design Methodology (2/4)
Process model
 “Waterfall” model, a step-by-step
approach, a pipeline originally deviced
for s/w development; for user interface
design, this model is less linear than
s/w waterfall model and more complex
because of human element

Model is not a hierarchy: has feedback
loops
Andries van Dam©, James Foley
24 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Design Methodology (3/4)
Analyze
 Requirements definition
 User(s) definition


Novice, casual user, power user…
Working environment

Office, home, school…
Formalize
 Define design goals
Andries van Dam©, James Foley
Synthesize
 Conceptual design
 Semantic design
 Dialogue design
 Syntactic design
 Lexical design
 Graphic and other design
modalities
 Documentation design (text docs
increasingly skipped – online
help? instructional videos?)
25 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Design Methodology (4/4)
Evaluate
 Design review
Implement
 Prototyping
 Implementation
 Software debugging
Test
 User interface debugging
 Usability testing/evaluation


Formative vs. summative assessment,
e.g., observation and questionnaires vs.
gathering and analyzing statistics –
“design of experiments”.
Again, steps are NOT all distinct or sequential!
Andries van Dam©, James Foley
26 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
User Definition and Work Environment (1/2)

Demographics




Age
Education
Cultural characteristics
Disabilities

Intrinsic personality factors:





Andries van Dam©, James Foley
Attitude toward computers
Secure/insecure
Bold/timid
Adaptable/rigid
Motivated/apathetic
27 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
User Definition and Work Environment (2/2)

Knowledge





Previous computer experience
Skill level (novice, intermediate,
expert)
Education level/background
Reading ability
Typing and other interaction ability

has changed dramatically since
smartphones, tablets/pads
Andries van Dam©, James Foley

“Work” environment






Frequency of computer use
Time allotted to learn system
Mental workload or overload
Stress level
Executive/management vs.
clerical/data entry vs. casual use
Ambient conditions: supermarket
vs. shop floor vs. traveling
salesperson vs. white-collar office
vs. home
28 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Conceptual Design
Identify real (or magical) world models and metaphors

Examples:

2D messy desktop, point-and-click, drag-and-drop

Menus of menus (drop-down, pop-up, radial…)

Tool bar (like tool rack, box or belt)

Color pickers and paintbrushes in art programs

Timeline editor for movie making

IVR 3D virtual world: reach out and touch (e.g. using wand or laser pointer type device), tricorder simulation
with a hand-held smartphone

Use metaphors only if and when it is appropriate: they can constrain and break too easily (e.g., 2D desktop, 3D
rooms with literal furniture, filing cabinets…)
Identify key concepts in application:

Types of objects

Relations between objects

Attributes of objects

Actions on objects, relations, attributes
Andries van Dam©, James Foley
29 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Functional/Semantic Design
Completely design units of meaning between user and computer, but not form
 List what information is used for each operation on each object
 Results, errors, edge conditions
From user to computer
 Detailed definition of commands for operating on objects and on attributes of objects
From computer to user
 Selection of what information needs to be presented to the user
Identify problems that can occur and engineer them out when possible
 Analyze and try to predict possible user actions
Structure semantics to ease learning
 Remember Pareto’s principle mentioned on slide 13


80/20 rule, 20% of the features will be used by the user 80% of the time
Follow “law of least astonishment:” consistency /predictability is vital
Andries van Dam©, James Foley
30 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Dialogue Design (1/2)
Precursor to syntactic and lexical design
 An overview of both syntactic and lexical bindings
Some dialogue styles
 Question and answer
 Form fill in
 Command/Scripting language
 Menu selection (fixed, pop-up, radial, etc…)
 Direct manipulation
 Gestural specification
 Natural language (subset) – typed and/or spoken
 Hybrid style: e.g., finger/pen gestures + gaze tracking + voice
Andries van Dam©, James Foley
31 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Dialogue Design Example

This graphic is from Microsoft’s Call flow
design in their Speech Application SDK

In this example, the user's task is to schedule
a meeting with a number of attendees. The
call flow steps through all the dialogs that are
necessary to gather the primary information
needed to schedule the meeting. At the end of
that subtask, the application confirms the
individual details before checking the schedule
for conflicts. If there is a conflict, the
application asks which of the variables the
user wants to revise—the date, start time,
duration or subject.
Andries van Dam©, James Foley
Text and Image from http://msdn.microsoft.com/en-us/library/ms991088.aspx
32 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Dialogue Design (2/2)
Issues in dialogue design
 Prospective user and workspace
 Factory floor, business office, plane, car…
 Initiative: Who has control?
 Training requirements
 Learning time
 To accomplish trivial tasks
 To become proficient
 Speed of use
 For novice
 For expert/”power user”
Andries van Dam©, James Foley
33 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Direct Manipulation (1/2)
An industry standard and the heart of WIMP interfaces

Direct Manipulation interfaces provide visual metaphors for commands, e.g., drag-and-drop
But is direct manipulation WIMP really better?

“Now, the abortion that happened after PARC was the misunderstanding of the user interface
that we did for children, which was the overlapping window interface which we made as naïve as
absolutely we possibly could to the point of not having any work flow ideas in it and that was
taken over uncritically out into the outside world…. I characterize what we have today as a
wonderful bike with training wheels on that nobody knows they are on so nobody is trying to
take them off. I just feel like we’re way, way behind where we could have been if it weren’t for
the way commercialization turned out.”


Alan Kay, a chief researcher at Xerox PARC during the 70’s, inventor of Smalltalk with its window manager, the Dynabook
vision…50th Anniversary symposium for Vannevar Bush’ “As We May Think” in the Atlantic Monthly – the vision of an
associative memory “Memex,” anticipating today’s WWW http://www.cs.brown.edu/memex/Bush_Symposium_Panels.html
Multi-point iPhone/iPad, and “touch first” Windows 8 w/ Metro blended into Win10
permit even more direct “direct manipulation”! (Thu: post-WIMP “Natural” UI’s)
Andries van Dam©, James Foley
34 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Direct Manipulation (2/2)
Pros
 Novices can learn the system relatively quickly
 Less time required of user to learn command syntax
 Visually appealing and enjoyable
 Easier to retain command set/??
Cons
 Some action commands seem awkward or impossible, e.g., how do you execute a
“rm -f n*” in a visual file manager? Is dragging your disk to the trash really intuitive?
Emacs users are amazingly efficient…fixed menus on large screens don’t work…commands
with many options may be more rapidly done via a pop-up form
 In some cases, consumes more system resources
 Visually impaired or disabled may have more difficulty with the interface


accessibility design, “universal design”
Ben Shneiderman’s view: direct manipulation empowers users, autonomous agents do not,
but consider the efficiency of “please set the table”
Andries van Dam©, James Foley
35 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Syntactic Design
Design of how lexemes are
arranged
 Placement (this is a graphical
UI!)
 Sequencing
From user to computer
 Sequence of commands and
parameter specification
 Where commands and
parameters are specified
Andries van Dam©, James Foley
From computer to user
 When computer tells user something
 Positioning and appearance of
information
Prefix, infix, postfix
 Order of commands and arguments
can lead to excessive use of modes
(e.g., input vs. edit mode)
Minimize modes!
36 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Modes
A state in which just a subset of user-interaction tasks can be performed
Harmful mode
 Lasts for a period of time
 Not be visible to the user (transparent)
 Annoying examples: insert mode, dialogs that force OK’s
Useful mode (typically temporary)
 Narrows the choices of what to do next

Enables better context information (e.g., help)
Provide feedback
 Include obvious and fast means for exiting
 The mode is apparent at the locus of attention
Good examples
 Window managers
 “Button-down-dynamic feedback-button-up” interaction techniques often use temporary modes (drag & drop)


Muscle tension makes mode apparent
Andries van Dam©, James Foley
37 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Lexical Design
Bindings
 Bind hardware capabilities to primitives (lexemes) of input and output languages
 Usually done by window system
 Define how primitives (lexemes) combine to form tokens
 Tokens combined by syntax
From user to computer
 Input devices and interaction techniques
From computer to user
 Output primitives and attributes
 May include sound and haptic “display”
Andries van Dam©, James Foley
38 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Lexical Consistency
Coding consistent with common usage (for a given culture)
 Red = bad, green = good
 Left = less, right = more
 Dual coding always best
Readable text
 Consistent abbreviation rule
 Equal length, or first set of unambiguous characters
 Mnemonic names instead of cryptic codes
Devices used same way in all phases of interaction
 Delete key is always the same
 Function key meanings are consistent throughout application
 Menu command placement is consistent
Andries van Dam©, James Foley
39 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (1/7)
Visual clarity --- Gestalt Laws for perception/layout

Gestalt principle: the human eye sees the whole
(the gestalt) before seeing detail, the whole is
greater than the sum of the parts
Minimize

Eye movements

Hand movements

Visual “noise” (non-informative
decoration)

Rule of similarity

Rule of proximity

Rule of common fate
Visual codings

Rule of good continuation

Difficult to design intuitively obvious visual
symbols – icons are notoriously hard

E.g., restrooms, Olympic sports
… see more online if interested
Be aware of contextual and cultural baggage

Andries van Dam©, James Foley
40 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (2/7)
Visual Consistency
 Differences in appearance imply differences in
functionality or information content

Principle of Similarity, human perception tends to
see stimuli that physically resemble each other as
part of the same object
Layout principles
 Place related controls and information together



Minimizes physical and cognitive distance between
widget and application objects (e.g., pop-up
menus)
Principle of Proximity , human perception tends to
group stimuli that are close together as part of the
same object
Grids (e.g., “Swiss grids”) and proportion of
elements in your interface
An example of a Swiss Grid layout from
http://www.graphics.com/article-old/better-design-swiss-grid-system
Andries van Dam©, James Foley
41 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design: Affordances to Allow Disclosing/Discovering




Affordance: perceived or actual properties of an object, primarily
those that determine just how object could be used. A chair
affords (“is for”) support and therefore “affords” sitting.
Doorknobs invite turning, sliders sliding, etc..
Icons are not self-disclosing; neither are sliders unless marked
with values/scales. Use DUAL CODING!!!! (mouse-over or label
icons/buttons)
Making modes “show” themselves is especially tricky
Pen, touch and “in air” gestures don’t have any
discoverability…bigger initial learning curve; tool tips and
Andrew Bragdon’s “gesture bar”1, like a tool bar but shows
animation of each gesture and lets you practice
1 Bragdon et al. “GestureBar: Improving the Approachability
Andries van Dam©, James Foley
of Gesture-Based Interfaces”
42 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (3/7)
What are the strengths and weaknesses of this interface?
Andries van Dam©, James Foley
43 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (4/7)

Radial menu image courtesy of Emanuel Zgraggen
See the PanoramicData website for more information
Radial/Pie Menus http://en.wikipedia.org/wiki/Pie_menu


Pop up menus that use muscle memory, which is
more orientated towards direction than vertical
distance (like in the menu on the previous slide)
Research has been done on the advantages of
people remembering the “mark” they make on a
radial menu to do something


Kurtenbach, G. & Buxton, W. “User learning and performance with
marking menus” Proceedings of CHI '94, 258-264
This menu can also be hierarchic, for an increased
performance boost

Kurtenbach, G. & Buxton, W. “The limits of expert performance using
hierarchic marking menus” Proceedings of InterCHI '93, 482-487.
Andries van Dam©, James Foley
Radial Win 8 example from
http://circledock.wikidot.com/
44 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (5/7)
Why is the graphic design of these user interfaces poor?
Andries van Dam©, James Foley
45 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (6/7)

Print Settings view from Epson’s iPrint
app for the iPhone:



No BACK button to navigate back! Stuck in
settings forever.
The gray areas resemble buttons, even
though they are not.
Too much empty space, unpolished look
and feel.
Andries van Dam©, James Foley
46 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Graphic Design (7/7)


Bad design is not limited to computer interfaces
http://www.baddesigns.com/examples.html
Can you tell how fast are you going in this car?
Andries van Dam©, James Foley
Ever tried to use one of these?
Entire UI post on them here
47 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Hall of Shame
What’s the problem again?
Some never get old…
Windows 7 added printer window, but you
cannot resize to show more or search for the
printer model you want to add!
Andries van Dam©, James Foley
48 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Documentation Design (Does anyone still use doc?)
Several types
 Guided tour
 “Cheat sheet”
 Tutorial video and manual
 Reference manual
 FAQ
 Interactive help (how well does it actually work?!?)
 Andrew Bragdon’s “gesture bar” (slide 41), a tool bar that animates simple pen/touch gestures
Should be available online
 Context-sensitive
 Hypermedia
 e.g., Links from reference manual to tutorial to show functionality in context
Doing documentation well is hard – plan it from the outset
Andries van Dam©, James Foley
49 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Design Review
Review user interface design before
implementing

Using prototype

Using detailed, formal design

Example of TAG design document is covered
on the next three slides
Evaluate
 Against design objectives and guidelines
 Observation, questionnaires, timed
tasks…
 With measured metrics, such as mouse
movements/action counts, time/tasks,
error rate, etc.

Analogous to structural walk-through

Walk-through scenarios examine:

what user does

what user sees

what user must know
Andries van Dam©, James Foley


The time to acquire a target is a function of
the distance to and size of the target.
Easy to define metrics for canonical tasks;
(e.g., select or track an object)
More difficult to define metrics for less
mechanical tasks such as visualization
which are context-sensitive and require
gaining insight
50 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
TAG Design Document (1/3) – UI Wireframe
Andries van Dam©, James Foley
51 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
TAG Design Document (2/3) – UI Design and Redlines
Andries van Dam©, James Foley
52 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
TAG Design Document (3/3) – “Final” UI
Andries van Dam©, James Foley
53 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Prototyping the Design
Objectives and plans for prototype should be understood
 By designers
 By users
 By prototypers
Expect many revisions
 Have software tools available so that designers can respond quickly to user requests
Use prototyping early and often!
 When user requirements are unclear – they evolve based on user testing!
 When implementation requirements are unclear
 To user-test the design before too much effort is spent on it
Andries van Dam©, James Foley
54 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Prototyping Hints (1/2)
Manage and control prototyping
 Plan in advance
 Cost
 Effort
 Time limits
 Set and enforce explicit procedures
 Deadlines
 Documentation
 Testing criteria
 Combine prototype with life-cycle approach,
if possible
Andries van Dam©, James Foley
55 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Prototyping Hints (2/2)
Perceived problems of prototyping

Unrealistic expectations
 User
 Management
 Marketing

Enthusiasm wanes with time spent working on a prototype

Difficult to manage and control

No rules for where and how to set boundaries for prototyping large systems
Andries van Dam©, James Foley
56 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Implementation
Structure for change
 Use good CASE (Computer Aided Software Engineering) tools
 Stress modularity
 Object-orientation is well-proven in building user-interfaces
 Many Object Oriented interface toolkits around



Mac/iOS: Cocoa and Cocoa Touch, both part of Xcode
Platform Agnostic: QT, balsamiq, HTML5, Illustrator, Fireworks
Others can be found here: http://webdesignledger.com/tools/13-super-useful-uiwireframe-tools or here : http://en.wikipedia.org/wiki/List_of_widget_toolkits
Make skeletal user interface available for next step as soon as possible
 UI code is hard to write
 Reuse as much as you can
 Toolkits are your friends
Good software engineering is a must!
Andries van Dam©, James Foley
57 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
User Interface Evaluation
Involve real users
 Both new and experienced users
 Designers or implementers do not make good testers
 “It works fine when I use it.”
Designers present at usability testing
 But gagged in background, or at least can’t use their hands
 To listen, observe, learn, and sweat!
Experimental design methodology (stats-based)
 What to test and how to test it takes ingenuity and experimentation


Granularity spectrum: individual interaction techniques to task performance – task may be quite complex
User Interface/User Experience labs – formal studies have a “design of experiments” methodology
Plan on major investments
 Particularly for commercial products – UI may cost as much as the rest of the system!
Andries van Dam©, James Foley
58 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Lecture Summary (1/3)




UI design is the skill of developing a dialogue between the user and the computer -2 languages
UI design is a collaborative, multi-disciplinary art
“Know thy user!”, user-centered design
The study of computer interfaces is still a relatively new discipline, and finally considered part of CS


Jeff Huang teaches cs1300 (User Interfaces) and cs2300 (HCI Seminar)
For those interested in computer user interface design some universities have established separate
groups for this study, often under the title of Human Computer Interaction (HCI)
 University of Washington; HCDE and dub group (former home of our own Jeff Huang)
 UC Berkeley; Berkeley Institute of Design
 UC San Diego; Distributed Cognition and HCI Lab
 Carnegie Mellon; Human Computer Interaction Institute
 Georgia Tech; Graphics, Visualization, and Usability Center
 Stanford; HCI group
 University of Maryland; HCI Lab
 MIT; Fluid Interfaces Group
Andries van Dam©, James Foley
59 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Lecture Summary (2/3)



UI is the component most critical to user satisfaction, and probably the
largest component of today's interactive apps
Rapid prototyping and usability testing is paramount
There are linguistic “abstraction layers” just like for programming
languages, and each layer has design choices to be made – ecapsulation!
Conceptual level
Meaning
Form
Functional/semantic level
Sequencing/syntactical level
Binding/lexical level
Andries van Dam©, James Foley
Look
and
Feel
60 / 61
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Lecture Summary (3/3)



WIMP interfaces are dominant, but more human-centered post-WIMP styles
are gaining in importance (e.g., voice- and gesture-recognition-based, multimodal and “natural user interfaces” (NUIs))
WIMP's "One size fits all" doesn't work for all circumstances and form factors
(including the need for "universal design" principles to allow computers and
info appliances to be used by those with disabilities or those who are illiterate;
e.g., designing for visually impaired, the aged, or villagers in developing
countries
UI/UX design is a separate field that overlaps with software engineering and
human studies and art/design and has its own design discipline and literature –
it can be seen as a key specialty within computer graphics
Andries van Dam©, James Foley
61 / 61