Virtual Contact Sheets
Wolfgang Richter
Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan
Harkes, Lin Zhong*, Mahadev Satyanarayanan
PARALLEL DATA LABORATORY
Carnegie Mellon University
*Rice University
Crowdsourcing the News [1]
“When every cell phone can record
video and take pictures, everyone
is a potential news source.”
Who killed
Ian Tomlinson?
http://www.pdl.cmu.edu/
2
Wolfgang Richter © July 16
CNN: Caught Red Handed [2]
I’m a sneaky
thief.
http://www.pdl.cmu.edu/
3
Wolfgang Richter © July 16
Problem
Older,
uploaded
Cloud
Recent,
not uploaded
http://www.pdl.cmu.edu/
4
Wolfgang Richter © July 16
Fundamental Mobile Challenges
• Real-time sync not possible:
How
far
can
we
degrade
the
• Energy
objects
in life
a desired
search before it
• Infinite battery
loses meaning?
• Bandwidth
• Increasingly metered and charged
http://www.pdl.cmu.edu/
5
Wolfgang Richter © July 16
Just-in-Time Search
1. Check virtual contact sheet
• Collection of fidelity-reduced images in the
cloud
2. Pull full fidelity image on hit
• Search full fidelity; present full fidelity results
to searcher
http://www.pdl.cmu.edu/
6
Wolfgang Richter © July 16
Near Real-Time Sync
(1) Search hit
(2) Retrieve
http://www.pdl.cmu.edu/
Cloud
7
Wolfgang Richter © July 16
Contact Sheet Analogy
http://www.pdl.cmu.edu/
8
Wolfgang Richter © July 16
Evaluating Feasibility [3]
Benchmark
Face Detection,
RGB Histogram,
Brightness Filter
Varying Resolution
and JPEG Quality
Factor
Using
http://www.pdl.cmu.edu/
9
Wolfgang Richter © July 16
Energy Savings
10 images =
5 seconds call time (Droid)
http://www.pdl.cmu.edu/
10
Wolfgang Richter © July 16
Bandwidth Savings
Orders of magnitude
savings in bandwidth.
http://www.pdl.cmu.edu/
11
Wolfgang Richter © July 16
What about Search Accuracy?
Face Detection
No one-size-fits-all sweet spot.
http://www.pdl.cmu.edu/
12
Wolfgang Richter © July 16
Conclusion & Future Work
• Searches retain meaning
• No one-size-fits-all, but
• Orders of magnitude bandwidth savings
• Significant energy savings
• More search primitives?
• Perceptual hashing
• Face recognition
• Texture Filters
• Better fidelity reduction?
• Keep faces at high fidelity
http://www.pdl.cmu.edu/
13
Wolfgang Richter © July 16
Further Reading
[1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.
[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.
[3] SimpleCV. http://simplecv.org/.
Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach
to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.
Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and
Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report
TR0627-2011.
Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and
Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.
Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.
Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for
wireless transmission. ICME 2009.
http://www.pdl.cmu.edu/
14
Wolfgang Richter © July 16
Full Fidelity
Average photo size: 374 KB
12,000+ Flickr dataset
http://www.pdl.cmu.edu/
15
Wolfgang Richter © July 16
Reduce Resolution
Average photo size: 221 KB
12,000+ Flickr dataset
http://www.pdl.cmu.edu/
16
Wolfgang Richter © July 16
JPEG Quality Factor
Average photo size: 6 KB
12,000+ Flickr dataset
http://www.pdl.cmu.edu/
17
Wolfgang Richter © July 16
Bandwidth Savings Table
Resolution
100
95
75
50
25
5
640x480
221 KB
114 KB
47 KB
31 KB
19 KB
6 KB
320x240
61 KB
33 KB
14 KB
10 KB
6 KB
2 KB
160x120
17 KB
10 KB
5 KB
3 KB
2 KB
826 B
80x60
5 KB
3 KB
2 KB
1 KB
892 B
460 B
40x30
2 KB
1 KB
725 B
588 B
484 B
345 B
20x20
1000 B
727 B
501 B
436 B
385 B
316 B
http://www.pdl.cmu.edu/
18
Wolfgang Richter © July 16