i-Care Project
Dietary-Aware Dining Table:
Observing Dietary Behavior over
Tabletop Surface
Keng-hao Chang, Shih-yen Liu,Toung Lin, Hao Chu,
Jane Hsu, Polly Huang, (Cheryl Chen)
i-space Laboratory
National Taiwan University
1
What is it?
• A dietary-tracker built into an everyday dining table
– Track what & how much you eat over tabletop surface
• Motivation
– We are what we eat
– Food choices affect long-term & short-term health
• Show a demo video
2
Smart Everyday Object
• Digital-enhanced everyday objects
– Provide digital services
• Support natural human interactions
– Natural human interactions = inputs to digital services
• Goals
– Providing digital services without (users) operating digital
devices → better usability
– Human-centric computing: technology adapting to users rather
than users adapting & learning about technology
3
Outline for Reminder of Talk
•
•
•
•
•
•
Related work
Approach
Assumptions & Limitations
Design & Implementation
Experimental Evaluation
Future work
4
RelatedWork
• Dietary trackers
– Shopping receipt scanner (GaTech)
– Chewing Sound (ETH)
– My food phone (startup)
• Intelligent surfaces
– Load sensing table (Lancester)
– Smart floor (GaTech, NTU)
– Posture Chair (MIT)
• What’s new here?
– Accuracy
– Fine-grained tracking
– Simultaneous concurrent interactions
5
Contribution claims
• It is a fine-granularity (automated) dietary tracker.
– It can track multiple concurrent interactions from multiple
individuals over the same tabletop surface.
• People usually don’t eat alone
• It is an enhanced loading sensing table.
6
General Approach
• RFID tags on food containers
• Two sensor surfaces on table
– Each surface is made of cells
– RFID reader surface
• Detect RFID(s) in each cell
– Weighting surface (load cells)
• Measure weight change in each cell
• Track the food path from container(s) → container(s) → mouth
using these two sensor surfaces
7
Assumptions (Limitations)
• Closed system rather than open system.
– Food transfers among tabletop objects and mouths, no external objects and
food sources
•
•
•
•
•
Users identified by personal containers (personal plates and cups)
Food containers tagged with RFID tags
No cross-cell objects
No leaning their hands on the table
Not a mobile tracker
8
Single Interaction Example
• Bob pours tea from the tea pot to his personal cup, and
drinks it
• Detect tea transfer from one container to another
container
1) Identify the presence & absence of containers
• RFID tags on containers
• tag-food mapping
2) Track tea transfer
• Weight change detection
• Weight matching algorithm
9
Single Interaction Example
• Bob pours tea from the tea pot to personal cup, and drinks
Put on tea pot.
it
•RFID tag appears
•Weight increases ∆w3
Pour tea!
• |∆w3 - ∆w1 | ≈ ∆w2
Pick up tea pot.
Pour tea?
•Weight increases ∆w2.
• RFID tag disappears
•Weight decreases ∆w1
10
Single Interaction Example
• Bob pours tea from the tea pot to personal cup, and drinks
it
Put on cup.
•RFID tag appears.
•Weight increases ∆w2.
Drink tea? (only if no match)
• Amount | ∆w2 - ∆w1 |
Pick up cup.
• RFID tag disappears.
•Weight decreases ∆w1.
11
Concurrent Interactions Example
• Bob pours tea & Alice cuts cake
Cut cake
•Weight decreases ∆w2
Pour tea?
Cut cake?
•Weight change ∆w
Pour tea
•Weight increases ∆ w1
12
Concurrent Interactions Example
• Multiple, concurrent person-object interactions
– The larger the cell, the higher the possibility of concurrent
interactions over a cell
– Cell size = average size of container
– Reduce the possibility of concurrent interactions over one cell
13
Design Architecture
Applications (Dietary-aware Dining Table)
Dietary Behaviors
Behavior Inference Engine
Intermediate Events
Event Interpreter
Sensor Events
Weight Change Detector
Object Presence Detector
Weighing surface (weighing
sensors)
RFID Surface (readers)
Tag-object
mappings
Common sense
semantics
14
Inference Rule
Dietary behaviors
Behavior Inference Rules
Transfer(u, w, type)
Weight-Change(Object-i1, Δw1) ∩ (Δw1< 0) ∩
Weight- Change (Object-i2, Δw2) ∩ (Δw2 > 0) ∩
Contains(Object-i1, type) ∩ Owner(Object-i2, u)
∩(|Δw1 +Δ w2 |< ε) →Transfer (u, Δw2, type)
Eat(u, w, type)
Weight-Change(Object-i, Δw) ∩ (Δw<0) ∩
Contains(Object-i, type) ∩ Owner(Object-i, u)→ Eat(u,
-Δw, type)
15
Experimental setup
• 2 Dining settings
– Afternoon tea
– Chinese-style dinner
• 2 Parameters
– # of participants
– Predefined vs. Random
Keng-hao
Sequence
A
Willy
16
Experimental Results
Scenarios
Dining
#
Activity
Scenarios Users Sequence
#1 Afternoon
tea
#2 Afternoon
tea
#3 Afternoon
tea
#4 Chinesestyle dinner
1
Event Statistics
Time
# of
Average
Duration Dietary Activity
(seconds) Behavior Interval
Predefined
73
12
6.08
Results
Behavior
Recognition
Accuracy
2
Predefined
162
24
6.75
100%
2
Random
913
78
11.71
79.49%
3
Random
1811
162
11.18
85.8%
100%
17
Predefined Activity Sequence
Afternoon Tea (Single User)
1.
2.
3.
4.
5.
6.
cut a piece of cake and transfer it to the
personal plate;
pour tea from the tea pot to the
personal cup;
add milk to the personal cup from the
creamer;
eat the piece of cake from the personal
plate;
drink tea from the personal cup;
add sugar to the personal cup from the
sugar jar.
Afternoon Tea (Multi-users)
1. A cuts cake and transfers it to A’s personal
plate;
2. B pours tea from the tea pot to B’s
personal cup;
3. A pours tea to A’s personal cup while B
cuts a piece of cake and transfers it to B’s
personal plate;
4. A adds sugar from the sugar jar to A’s
personal cup while B adds milk from the
creamer to B’s personal up;
5. A eats cake and B drinks tea;
6. B eats cake from B’s personal plate while A
drinks tea from A’s personal cup;
7. A pours tea from the tea pot to both A’s
and B’s personal cups.
18
Activity Recognition Accuracy in
Scenario #3
Dietary Behavior
Transfer event
Eat event
# of Actual Events
41
37
Recognition Accuracy
70.73%
89.19%
19
Causes of Misses in Scenario #3
Causes of misses
# of misses of
transfer events
6
# of misses of eat
events
2
Total
Weight matching threshold
Slow RFID sample rate
Touching table
2
3
1
0
0
2
2
3
3
Total of misses
12
4
16
Event interference within the
weighing cell’s weight
stabilization time
8
20
Activity Recognition Accuracy in
Scenario #4
Dietary Behavior
Transfer dish A events
Transfer dish B events
Transfer dish C events
Transfer rice events
Transfer soup events
Eat events
# of
times
19
29
23
12
19
60
Recognition Accuracy
Weight Accuracy
73.68%
79.31%
82.61%
68.42%
78.75%
79.19%
83.33%
84.21%
88.33%
81.88%
80.16%
91.23%
21
Causes of Misses in Scenario #4
Causes of misses
Segmented weight-change events
Eating before transferring food to
personal containers
Weight matching ambiguity
Touching table
Slow RFID sample rate
Total of misses
# of misses of
transfer events
5
5
# of misses of
eat events
0
5
Total
7
3
3
0
2
0
7
5
3
23
7
30
5
10
22
Conclusion
• It is a smart object and a smart surface
• It supports natural user interface
• It supports fine-grained dietary tracking at individual
level
• It is about human-centric computing
• Accuracy can be improved further
23
FutureWork
• Improving recognition accuracy
• Removing constraints (assumptions)
• Persuasive computing
– Encourage balanced diet
– Encourage proper amount of diet
24
Questions & Answers
ThankYou
25