Tangible User Interfaces (TUI’s)
What are Tangible User Interfaces?
TUI’s
Physical World
Digital world
2
GUI
vs
TUI
(Ishii 2008)
3
Precursors
• E.g. Marble answering machine, Durrell
Bishop, Royal College of Art, Interaction
Design, 1992. http://vimeo.com/19930744
4
Early Work
• Fitzmaurice et al. 1995 – Graspable user
interfaces
http://www.youtube.com/watch?v=V-TGEeImro
• Ishii et al. 1997 - Tangible bits
5
Recent work
• E.g. Lumino, Baudish et
al, 2010
http://www.youtube.co
m/watch?v=tyBbLqViX7g
• E.g. Portico, Avrahami et
al, 2011
http://vimeo.com/29359
319
6
Benefits of TUI’s (Ishii 2008)
1. Double interaction loop immediate tactile feedback
2. Persistency of tangibles
3. Coupled input/output space
4. Special vs generic purpose
5. Space-multiplexed vs timemultiplexed input
Also fun +
engaging!
7
Can we connect these physical
drawing tools to the digital space?
• Sense position &
orientation on touch
technology
• Add intelligent drawing
support
• How can they be best
combined with multi-touch
surfaces for enjoyable and
productive interaction?
8
Our Approach
Capacitive Touch (CapTUI)
– Ryan Tan & Bryan Chen
– Rachel Blagojevic
Infrared Touch (TanGeo)
– Jacky Zhen
• Design
– Tangible hardware
• Implementation
– Recognizer
– Drawing application
• Usability evaluation
9
CapTUI Technology – Capacitive
• Small touch screens e.g. iPad, smart phones
etc…
• Touch detection via electrical pulse from
fingers/conductive material
10
Design: Tangible Hardware
11
Final Design
12
Implementation: Tangible Recognition
• Tangible ID
– 3 point (min) unique patterns
Valid patterns
Invalid patterns
13
Implementation: Tangible Recognition
• Learning phase
• Recognition phase
– Touch point detection
– Match point distances to saved tangible ID’s
– No way of knowing which part of the touch point
is in contact (+/- error)
14
Implementation: Drawing Application
• Beautification
Ink-to-edge snapping
& Length visualization
Corner snapping
15
Implementation: Drawing Application
• Visual drawing guides
Tangible outline
Angle visualization
16
• First iteration: Video
• Second iteration: Demo
17
Evaluation
• First iteration: usability
– Simple drawing tasks
• Second iteration: comparative study
– Recognizable vs non recognizable drawing tools on
screen
18
Usability Evaluation
• Can users construct simple drawings using the
tangibles? Is the system usable?
• 10 participants
• 5 simple drawing tasks
19
Usability Evaluation: Results
• First exploration
– Technology works
– Is usable for simple drawings
• Tangible detection problems
–
–
–
–
–
Stability
Consistent circuit
Finger to tangible contact
Friction with screen
Comfortable drawing
• Tangible outline helpful – recognition indicator
• Drawing guides needed
20
Comparative Study
• Does CapTUI assist users to easily draw precise
geometric drawings
• Recognizable vs non recognizable drawing tools on
screen
• 12 Participants
21
Comparative Study: Results
• CapTUI rated significantly higher than Paint overall
• Visual guides helpful for precise drawing
– significantly lower average angle error.
• Participants enjoyed using CapTUI significantly more
• Participants believed that CapTUI produces significantly
more tidy drawings than Paint.
• Making fine grained movements with the tangibles
difficult
• Tangible design still needs work
– consistent detection
– accurate positioning
22
Tangeo Technology – Infrared
• Table tops e.g. Microsoft Surface 2.0
• PixelSense - Touch detection via infrared
reflection for each pixel
• Image processing on detected pixels
• Identifies finger/blobs/tags
23
Design: Tangible Hardware
24
Final Design
25
Implementation: Tangible Recognition
• Learning phase
• Recognition phase
– Detection via
custom tags
– Use tag location to
get tangible outline
– Use thresholds for
blob sizes
26
Implementation: Drawing Application
27
Implementation: Drawing Application
• Visual Guides
– Tangible outline
– Angle visualisation
– Length visualisation
– Ink beautification
• corner snapping
• ink-to-edge snapping
28
Usability Evaluation
• Can users construct simple geometric drawings
using Tangeo? Is the system usable?
• 2 phase cycle
• 8 participants
• 4 drawing tasks
29
Usability Evaluation: Results
•
•
•
•
•
Enjoyable / easy to use the tangibles
Good recognition
Visual guides helpful and easy to understand
Drawing accuracy – less positive perception
Add stylus for drawing
30
References
•
•
•
•
•
•
•
•
•
•
Marble answering machine - Crampton Smith, G. The Hand That Rocks the Cradle. I.D., May/June 1995, pp. 60-65.
Fitzmaurice G. W., H. Ishii, and W. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), ACM Press/AddisonWesley Publishing Co., New York, NY, USA, 442-449.
Ullmer B. and H. Ishii. 1997. The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of
the 10th annual ACM symposium on User interface software and technology (UIST '97). ACM, New York, NY, USA,
223-232.
Ishii H., B. Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the
SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia,
United States
Ishii H., 2008. Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and
embedded interaction (TEI '08). ACM, New York, NY, USA, xv-xxv.
Baudisch P., T. Becker, and F. Rudeck. 2010. Lumino: tangible building blocks based on glass fiber bundles. In ACM
SIGGRAPH 2010 Emerging Technologies (SIGGRAPH '10). ACM, New York, NY, USA, Article 16 , 1 pages.
Avrahami D., J. Wobbrock, and S. Izadi. 2011. Portico: tangible interaction on and around a tablet. In Proceedings
of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA,
347-356.
Blagojevic R., X. Chen, R. Tan, R. Sheehan, and B. Plimmer. 2012. Using tangible drawing tools on a capacitive
multi-touch display. In Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and
Computers (BCS-HCI '12). British Computer Society, Swinton, UK, UK, 315-320.
Zhen, J. S., R. Blagojevic and B. Plimmer (2013). Tangeo: Geometric Drawing with Tangibles on an Interactive TableTop. CHI 2013. Paris France, ACM. WIP: in press.
Shaer O. and E. Hornecker (2010) "Tangible User Interfaces: Past, Present and Future Directions",
Foundations and Trends® in Human-Computer Interaction: Vol. 3: No 1-2, pp 1-137.
31