Creativity Support Tools:
A Grand Challenge for Interface Designers
Ben Shneiderman (ben@cs.umd.edu)
Founding Director (1983-2000), Human-Computer Interaction Laboratory
Professor, Department of Computer Science
Member, Institute for Advanced Computer Studies &
Institute for Systems Research
University of Maryland
College Park, MD 20742
UIST Vancouver November 3, 2003
Human-Computer Interaction Laboratory
Interdisciplinary research community
- Computer Science & Psychology
- Information Studies & Education
www.cs.umd.edu/hcil
User Interface Design Goals

Cognitively comprehensible:
Consistent, predictable & controllable

Affectively acceptable:
Mastery, satisfaction & responsibility
Design philosophy:
Direct Manipulation
NOT:
Adaptive, autonomous & anthropomorphic
Scientific Approach (beyond user friendly)
Specify users and tasks
 Predict and measure






time to learn
speed of performance
rate of human errors
human retention over time
Assess subjective satisfaction
(Questionnaire for User Interface Satisfaction 7.0)
Accommodate individual differences
 Consider social, organizational & cultural context

Scientific Approach (beyond user friendly)
Specify users and tasks
 Predict and measure






time to learn
speed of performance
rate of human errors
human retention over time
Assess subjective satisfaction
(Questionnaire for User Interface Satisfaction 7.0)
www.awl.com/DTUI
Accommodate individual differences
 Consider social, organizational & cultural context

Library of Congress
Scholars,
Journalists, Citizens
Teachers,
Students
Visible Human Explorer (NLM)
Doctors
Surgeons
Researchers
Students
NASA Environmental Data
Scientists
Farmers
Land
planners
Students
Bureau of Census
Economists,
Policy
makers, Journalists
Teachers,
Students
NSF Digital Government Initiative
 Find
what you need
 Understand what you Find
UMd & UNC
www.ils.unc.edu/govstat/
Creativity Support Tools
Creativity Support Tools
Creativity Support Tools
Creativity Support Tools: Goals


More people, more creative, more of the time

Revolutionary breakthroughs, paradigm shifts,
H-creativity

Evolutionary, normal science, music & art,
creative knowledge work

Impromptu everyday creativity
Raised expectations for professionals


Tailored solutions
Cheaper, faster, better
Structuralists: A plan, method, process

Polya's four steps in How to Solve It (1957):





1) Understanding the problem
2) Devising a plan
3) Carrying out the plan
4) Looking back
Couger (1996) reviews 22 "creative problem
solving methodologies"




Preparation
Incubation
Illumination
Verification
Structuralists: A plan, method, process

Atman's design steps:









Problem definition – identify need
Gather information
Generate ideas – brainstorm & list alternatives
Modeling – describe how to build
Feasibility Analysis
Evaluation – compare alternatives
Decision – select one solution
Communication – write or present to others
Implementation
(Atman et al., Design Thinking Research Symposium2003)
Inspirationalists: Aha, Aha, Aha!

Free associations




Breaking set
Getting away to different locations
 Working on other problems
 Meditating, sleeping, walking
Visualization
 2-d networks of ideas
 Sketching


Brainstorming
Thesauri, photo collages
Random stimuli, inkblots
Situationalists: context, community, collaboration

Personal history




Consultation




Family history, parents, siblings
Challenging teachers, inspirational mentors
Supportive peers and partners
Peers and mentors
Early, middle and late stages
Information and empathic support
Motivations


Fame, legacy, admiration
Competition
Csikszentmihalyi’s book Creativity (1993)

1) Domain: e.g. mathematics or biology
"consists of a set of symbols, rules and procedures”

2) Field: "the individuals who act as gatekeepers to the
domain...decide whether a new idea, performance, or product
should be included”

3) Individual: creativity is "when a person... has a new idea
or sees a new pattern, and when this novelty is selected by the
appropriate field for inclusion in the relevant domain"
Genex (Generator of Excellence) Framework

Collect


Relate


Consult with peers & mentors, early, mid & late stages
Create


Learn from previous works in digital libraries, the web, etc.
Explore, discover, compose, evaluate possible solutions
Donate

Disseminate refined results and contribute to the digital
libraries, the web, etc.
(Codex, memex, genex: The pursuit of transformational technologies
IJHCI, 1998)
Genex: Some potential down sides

Collect


Relate



Could mentors discourage exotic ideas?
Could peers rip-off your innovation?
Create


Will knowledge of previous work limit imagination?
Will using standard tools limit creativity?
Donate

Could the desire for intellectual property protection
limit dissemination?
Eight Activities in Genex








1) Searching & browsing digital libraries
2) Consulting with peers & mentors
3) Visualizing data & processes
4) Thinking by free associations
5) Exploring solutions - What if tools
6) Composing artifacts & performances
7) Reviewing & replaying session histories
8) Disseminating results
(Creating creativity: User interfaces for supporting innovation
ACM TOCHI, 3/2000)
1) Searching & Browsing Digital Libraries

Effective search: Basic Google Search

Improved multimedia search
Overviews & previews
Result set categorization & visualization
Multiple session searches



(Clarifying Search, CACM 4/98)
1) Advanced Search: Google
1) Search: Overviews & Previews


Faceted search
Preview cardinality
of result sets
www.endeca.com
www.epicurious.com
2) Consulting with Peers & Mentors



Early, middle and late stages
Information and empathic support
Build trust by negotiated expectations





Email, listservs, newsgroups, discussion boards
Chat rooms, instant messaging, audio/video conferencing
Comprehensive online communities
Tele-medicine, tele-meeting, tele-democracy
Collaboratories
2) Components of Negotiated
Expectations

Clearly identify and refine through dialog



who I am
what I want to do
Declare understandings



Why I think you can help
How you can help
(specific request with time period)
How much is in it for you
(payment, shared honor, appreciation)
2) Example Request for Consultation

Poor: Dear Prof. Shneiderman: Attached is my PhD
proposal, please tell me what you think.

Better: Dear Prof. Shneiderman: I am a Computer
Science PhD student at Imperial College. My advisor,
Prof. Spence, knows you and I have read your papers.
My work extends your concept of dynamic queries.
I’ve attached my 2-page doctoral proposal in the hopes
that you can give me your comments before I defend it
in two weeks. Your input would be acknowledged and
Dr. Spence would be glad to pay for your plane ticket to
join the committee meeting.
3) Visualizing Data & Processes

Visual bandwidth is enormous



Human perceptual skills are remarkable
 Trend, cluster, gap, outlier...
 Color, size, shape, proximity...
Human image storage is fast and vast
Opportunities





Spatial layouts & window coordination
Information visualization
Scientific visualization and simulation
Telepresence and augmented reality
Virtual Environments
3) Visualizing Data & Processes

Visual bandwidth is enormous



Human perceptual skills are remarkable
 Trend, cluster, gap, outlier...
 Color, size, shape, proximity...
Human image storage is fast and vast
Opportunities





Spatial layouts & window coordination
Information visualization
Scientific visualization and simulation
Telepresence and augmented reality
Virtual Environments
Treemap Implementations
3) TimeSearcher

Time series






Stocks
Diseases
Weather
Genes
User-specified
patterns
Rapid search
3) Information Visualization: Mantra










Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
Overview, zoom & filter, details-on-demand
3) Information Visualization: Data Types
Document Lens, SeeSoft, Info Mural, Value Bars

1-D Linear
2-D Map
3-D World
Multi-Dim

Temporal
Perspective Wall, LifeLines, Lifestreams,
Project Managers, DataSpiral

Tree
Network
Cone/Cam/Hyperbolic, TreeBrowser, Treemap




GIS, ArcView, PageMaker, Medical imagery
CAD, Medical, Molecules, Architecture
Parallel Coordinates, Spotfire, XGobi, Visage,
Influence Explorer, TableLens, DEVise
Netmap, netViz, SeeNet, Butterfly, Multi-trees
(Online Library of Information Visualization Environments)
otal.umd.edu/Olive
4) Thinking by Free Associations

Free associations




Breaking set




Brainstorming
Thesauri, photo collages
Random stimuli, inkblots
Getting away to different locations
Working on other problems
Meditating, sleeping, walking
Visualization


2-d networks of ideas
Sketching
An Idea Visualization Tool
The human vision is by far the most
developed and powerful faculty. The Idea
Processor exploits visual attributes such as:
color, shape, size, scale, position, depth,
link, icon, etc. Visual cues facilitate recall,
association, and discovery. Diagrams and
pictures help you to represent and solve
complex problems. Visualization reinforces
your short term memory.
Towards Higher Abstractions
Ideas and diagrams are the basic
abstractions of the Axon Idea Processor.
Ideas are shown as graphical objects and its
relationship shown as links. You get the big
picture at all times, and details can be
hidden from view.
Stimulate Recall & Creativity
The Idea Processor has an integrated
Checklist Management System and a library
of Checklists on problem-solving strategies,
words of wisdom, etc. Checklists are
effective means of capturing and
transferring knowledge, and it effectively
amplifies your intelligence.
5) Exploring Solutions - What If Tools

State space exploration



“Combinationist theory”
Spreadsheets
Simulation as a third paradigm of science



SimCity & Flight Simulator
Economic models
Weather forecasts
5) Exploring Solutions - What If Tools

Terry & Mynatt:
Previews
6) Composing Artifacts & Performances

Initiate a new composition




Revise at multiple levels




Exemplars
Templates
Processes
Low
Middle
High
Evaluate and refine


Feedback about problems
Measurement
(Composition, Hawaii Int’l Conf. on Systems Science, January 2000)
6) Composing Artifacts & Performances

Initiate a new composition
7) Reviewing & Replaying Session Histories



Record compact histories
Allow users to review & annotate history
Disseminate histories (histories as first class objects)






Send by email
Post to website
Consult synchronously & asynchronously
Edit, extract, combine, search
Replay: slow, fast, reverse
Macros to automate exploration (Photoshop)
(Learning Histories,
Computer Supported Collaborative Learning Conf., December 1999)
7) Imaginary dialysis procedure exploration
7) Adobe Photoshop History tools
8) Disseminating Results

Disseminate refined solution to gatekeepers




Facilitate web publishing & focused advertising
Reach subscribers & organizational gatekeepers
Ensure quality by editors & reviewers
Contribute to digital library communities


Journals, books, resources, libraries, communities
Science fairs, student projects, galleries, performances
Genex: Integrated Framework for Software
Modular design to allow components
 Common file formats to ease data movement
 Consistent commands to reduce cognitive load
 Shortcuts for experts & immediate feedback
 Direct manipulation for rapid learning, powerful
actions, low error rates, high retention

Genex Phases and Activities
Genex Phase
Primary Activity (but can be used at any phase)
Collect
Searching and browsing digital libraries
Visualizing data and processes
Relate
Consulting with peers and mentors
Create
Thinking by free associations
Exploring solutions - What if tools
Composing artifacts and performances
Reviewing and replaying session histories
Donate
Disseminating results
Figure 1: Genex phases and their related primary activities
Challenges for Creativity Researchers
Domain knowledge is vital
 Creative work may take years
 Individuals have highly varied approaches
 Evaluation is difficult




Controlled studies are unrealistic
Case studies are not replicable
Theories are shallow
An Inspirational Muse:
Leonardo da Vinci (1452-1519)
Renaissance Man




Combined
science & art
Integrated
engineering & esthetics
Balanced
technology advances
& human values
Merged
visionary & practical
(MIT Press, Oct 2002)
Human-Computer Interaction Laboratory
21th Annual Symposium
June 3-4, 2004
www.cs.umd.edu/hcil
References
Shneiderman, Ben and Plaisant, Catherine, Designing the User
Interface: Strategies for Effective Human-Computer Interaction:
Fourth Edition, Addison Wesley Longman, Boston, MA (2004)
Shneiderman, Ben, Relate-Create-Donate: A teaching philosophy for
the cyber-generation, Computers & Education 31, 1 (1998), 25-39.
Shneiderman, B., Creating Creativity: User interfaces for supporting
innovation, ACM Transactions on Computer Human Interaction 7,
1 (2000), 114-138.
Shneiderman, B., Supporting creativity with powerful
composition tools for artifacts and performances, Proc. 33rd
Hawaii International Conference On System Sciences
(HICSS) (1/2000).
www.cs.umd.edu/hcil