Identification and Validation of Cognitive Design
Principles for Automated Generation of
Assembly Instructions
Julie Heiser, Doantam Phan, Maneesh Agrawala,
Barbara Tversky and Pat Hanrahan
Stanford University and Microsoft Research
1
2
3
4
5
6
Cognitive Design Principles
Identify

How people conceive of 3D assemblies
 How people comprehend visual instructions
Validate

Build automated instruction design system
 Evaluate usability of resulting instructions
Prior Work: Identification


Diagrams better for showing context
Diagrams and text equivalent for showing actions
[Bieger 85] [Bieger 86]


High spatial  prefer diagrams only
Low spatial  prefer diagrams and text together
[Winn 80] [Mayer 94] [Hegarty 94] [Kalyuga 98] [Trafton 2001]

Step-by-step better than showing only end state
[Novick 00]

Arrows and lines convey actions and sequences
[Tversky 00]
Prior Work: Validation
IBIS

AWI
CATHI
Automated presentation design systems
COMET [Feiner 89], IBIS [Seligmann 91], WIP [Andre 93], AWI [Rist 94],
CATHI [Butz 97], [Strothotte 98], [Tang 03]
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
• Step-by-step, one diagram for each major step
• Clear and explicit order
• Parts added in each step should be visible
• Mode of attachment should be visible
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
• Step-by-step, one diagram for each major step
• Clear and explicit order
• Parts added in each step should be visible
• Mode of attachment should be visible
Our Approach
Identification

Production

Preference

Comprehension
Validation

Instantiation

Usability
• Step-by-step, one diagram for each major step
• Clear and explicit order
• Parts added in each step should be visible
• Mode of attachment should be visible
TV Stand




Contains several parts and actions
Ordering constraints
One person can assemble
Representative of other furniture
Spatial Ability Tests
Mental Rotation [Vandenburg 78]
Navigation [Money 78]
Separate high and low spatial ability
Stage 1: Production
Stage 1: Production



43 Participants
Assemble TV Stand without instructions
Write instructions for novice assembler
Stage 1: Mean Completion Time
16
Time to
assemble
(min)
14
12
12.76
10
8
7.29
6
4
2
0
low spatial
Low spatial
high spatial
High spatial
Stage 1: Instructions Produced


Almost all contained diagrams
Text redundant with diagrams
98%
62%
Stage 1: Errors in Instructions


Errors in low spatial instructions
Errors in high spatial instructions
86%
12%
Stage 1: Errors in Instructions
support board
sides


Errors in low spatial instructions
Errors in high spatial instructions
86%
12%
Stage 1: Classes of Diagrams
Structural diagrams
Parts menu



Action diagrams
Parts menu to differentiate parts
Structural diagrams depict completed step
Action diagrams show assembly action/operation
Stage 1: Action Diagrams
3.5
Mean
number
per set
3
2.5
2.67
2
1.5
1
0.5
0
0.64
low spatial
Low spatial

High spatial



More action diagrams
More 3D diagrams
Less text
high spatial
High spatial
Stage 2: Preference
Stage 2: Preference



21 Participants
Assemble TV Stand without instructions
Rated 39 sets of redrawn instructions
Stage 2: Highest Rated


Ratings similar across all participants
Spatial ability does not affect preference
Stage 3: Comprehension
Stage 3: Comprehension
Set 1: Text + Action



Set 3: Parts menu + Structural + Action
44 Participants
Given 1 of 4 instruction sets from Stage 2
Assemble TV stand using instructions
Stage 3: Results

No difference in assembly time by condition
 Instruction consultations: Low 8.9 High 7.1
 Box picture consultations: Low 9.1 High 3.4
Comments



Should show relevant parts and attachments
Structural diagrams and exploded view hard to use
Text not very useful
Cognitive Design Principles



Sequence assembly operations
Ensure visibility of parts
Illustrate assembly operations
Sequence Assembly Operations
Single exploded view diagram
Step-by-step diagrams


Step-by-step, one diagram per major step
Clear and explicit order
Ensure Visibility of Parts




Show parts added in each step
Show mode and location of attachment
Avoid changing viewpoint
Use physically stable orientation
Illustrate Assembly Operations
Structural diagrams
Action diagrams


Use action diagrams rather than structural
Use arrows and guidelines to indicate attachment
Stage 4: Instantiation
Stage 4: Instantiation [Agrawala et al. SIGGRAPH 03]
All parts
Search
Leftover parts
Best subset
of parts
Structural Diagrams
Sequence Parts
Action Diagrams
Reorientation
Sequence of assembly steps
Stage 4: LEGO
Stage 4: Bookcase
Stage 5: Usability
Stage 5: Usability



30 Participants
Given 1 of 3 instruction sets: hand-drawn, factory, computer
Assemble TV stand using instructions
Stage 5: Hand-drawn
Stage 5: Factory Produced
Stage 5: Computer Generated
Stage 5: Instructions Tested
Hand-drawn
Computer generated
Factory
Stage 5: Results
25
Time to
assemble
(min)
20
15
18.9
16.0
10
10.2
5
0
Hand-drawn

Factory
Computer
Errors: Hand-drawn 1.6 Factory 0.6 Computer 0.5
 Users rated task as easiest in computer condition
Summary and Conclusions

5-Stage approach can give insight into effective
design of assembly instructions

Cognitive design principles




Clear sequence of operations
Maintain good visibility
Use action diagrams
Extend approach beyond assembly instructions
Acknowledgements
Boris Yamrom
Christina Vincent
John Haymaker
Jeff Klingner
Generalization

Approach extends beyond assembly instructions
Hand-drawn route map
LineDrive route map [Agrawala 01]