Donnie H. Kim, Nicolai M. Petersen, Mohammad Rahimi, Jeff Burke,
Deborh Estrin
Center for Embedded Networked Sensing, UCLA
Lenore Arab
David Geffen School of Medicine, UCLA
Presented by Reg Arvidson
 Phones
are everywhere!
 Increasingly carry imaging and location
capabilities
 Creation of assisted recall systems
• Record aspects of the environment for later
playback
 Rewind
supports data collection on
specific behaviors and situations
• As opposed to life blogging systems
 Scalable
system of everyday mobile
phones and supporting web services
 Explore how client/server-side image
processing can…
• lower bandwidth needs
• streamline user navigation
 Original
pilot was to assist in recall of
dietary intake
 Other short/exploratory trials developed
 Take
advantage of the application’s
constraints in…
• capturing images
• presenting images
• processing images
• and uploading images
 Manages
data flow
among system
components
• Interacts using
SSL/TLS encryption
and authenticated
transmission
 Provides
a user
interface to view
captured images
• Password protected
web interface
 Web
Interface – communication over
HTTPS through URLs, stored in secure file
system and relation database
 Image Handling Services – checks if
processing is required for each image
• Resizing – generates thumbnail for web interface
• Reaping – deletes images marked by user or
filters
• Image processing – pushes images to IPS and
retrieves results
 Can
potentially capture significant
personal data
• Family members, computer screen contents…
• Even occasional image inside a restroom
 Privacy
addressed at earliest phases of
prototyping
 Secure HTTP over SSL with a X.509 public
key certificate for the web server
 Images viewable only by the individual
owner, no identifiable information stored
 Users
given authenticated access to
Image Viewer with a User ID/Password
 Shown a subset of images (thumbnails)
based on time clustering and quality
rank

Tasked by DMS to…
• Process images
• Classify images
• Annotate images

Time-consuming and
application-dependent
image processing tasks
separated from core
data flow services
• Easily scalable through
addition of IPSs
 Matlab
used as computational engine
 Extended with an internal TCP/IP server
to provide an interface for external
applications
 Can handle image processing as a
scheduled task or in a FIFO manner
 IPS classifies image into four categories…
• Clear
• Blurred
• Exposed
• BlurExposed
 Class determined from four well-known
features
•
•
•
•
Mean of Intensity
Standard Deviation of Intensity
Number of Edges
Sum of High Frequency Coefs. Of Discreate Cosine
Transform (DCT)
 Nokia N80
 S60 3rd Edition
User Interface
 Symbian v9.1 Operating System
 3 megapixel camera
 Both 802.11 b/g and GSM connectivity
 Runs Campaignr
• Acquires data from hardware/software sensors
• Immediately stores to internal memory, queues for
upload to DMS
• Application-specific XML file specifies which
sensors to collect data from
Clear
Clear
Blurred
Motion of Carrier
Blurred
Motion of Subject
Exposed
Poor Lighting
BlurExposed
 FullDayDietary
pilot provided a large
and varying data set to work with
 Many images were blurred due to motion
of individual of subjects of image
 Additional images were either over- or
under-exposed
 Individuals marked the classification of
images to produce groundtruth data
83% of images correctly classified
93% of clear images correctly classified
2% of low quality images were incorrectly classified
 Latency
dominated by image processing
computation
 Very small deviation in processing
latency per image
• Roughly predetermine time to process images
 DietaryRecall
• 10 users – 11,090 images uploaded
• Device turned on only during meals
• Experience kept simple, 35 images max per episode
 FullDayDietary
• 14 users – 14,958 images released (6 users)
• Ran while outside home, 6 images/minute
 PosterSessionCapture
• 15 users – visitors to a research conference poster
session
• Tested system with many simultaneous users
 Pilots
showed large number of filtered
images
 Wireless upload channel became
congested with presence of co-located
users
 Image processing became nonnegligible with many users
 Added
extensions to the system to support
local image processing
 Early pilots resulted in many low quality
images and congestion on the upload link
• 33% of DietaryRecall images were marked low qual
 Filter
out extremely low quality images that
would be filtered by back-end server
anyways
 Also resulted in interesting requirements for
prioritized and real-time upload
•Stores image to database upon capture with other sensed data
•Images annotation - fetches image, extracts selected feature, stores results
•Classification – read computed features, classifies image using decision tree
•Clustering – generates cluster ID using the capture time
•Upload Ranker – ranks images based on configured upload policy
 DCT
ignored due to high cost on IPS
 Edge count not implemented
 File size used due to a high correlation with
number of edges AND normalized sum of DCT
coefs.
DCT on IPS vs size on phone
Edge count on IPS vs size on phone
 82%
of images correctly classified
 Only 14% of low quality images incorrectly
classified
 Phones
collecting multiple images per
second can easily congest narrow upload
channels
 Extended periods of disconnect can result
in a sizable backlog
 Prioritizing upload order can improve
usability of the system
• Reverse Chronological Order – uploads the most
recently captured image first
• Prioritized Upload – ranked by quality and cluster
 Not
an issue if users do not attempt to
view images for a duration of time
 In event of rapid review of images,
selective uploading can enhance
interactivity and responsiveness
 Can show best images throughout the
day (best of clusters) or reverse
chronological (reverse order) as
opposed to chronological order
Waiting time before
viewing the most recently
captured image following
different disconnection
times.
Completion delay
comparison following
different disconnection
times.
 Within
a cluster, prioritized method uploads
high quality image first
 Performance gain insignificant when every
image is of low quality
 Rewind: Leveraging
Everyday Mobile
Phones for Targeted Assisted Recall
(UCLA Technical Report 2008)
 Urban Sensing – CENS/UCLA
• http://urban.cens.ucla.edu/projects/dietsense/