The BASiCS Group
Berkeley Audio-visual Signal processing and Communication Systems
http://www.basics.eecs.berkeley.edu
Distributed signal processing:
compression: challenges and
opportunities
Kannan Ramchandran
Jan. 28, 2004
UCB Sensor Nets Day
Towards a System Theory for Robust
Large-Scale Sensor Networks
NSF Sensors (Ramchandran, Sastry, Tse, Vetterli, Poolla)
Design guidelines for
robust large-scale networks:
Channel Physics, Percolation Theory
Representation and Data-Acquisition:
Distributed Sampling Theory
f(x)
Information Dissemination:
Routing, Compressing, Mobility
Closing the Loop:
Inference & Adaptive Control
Jan. 28, 2004
UCB Sensor Nets Day
Sensor networks: a systems view
Systems tasks:
•
•
•
•
•
Data acquisition
Distributed compression and communication
Networking and routing
Distributed inference and decision (classification / estimation)
Closing the loop (control)
Guiding principles:
•
•
•
•
Statistical models for sensor-fields
Scaling laws for dense networks
Information and coding theory
Learning theory and adaptive signal processing
Jan. 28, 2004
UCB Sensor Nets Day
Distributed SP (DSP): “low-hanging fruit”
 Revisit many classical SP problems (estimation, inference, detection, fusion)
under
constraints of:
 bandwidth (compression)
 noisy transmission medium (coding + MAC)
 total system energy (communication + processing)
 highly unreliable system components (robust design)
Voila  you get a “distributed signal processing” recipe!
•Constraints force robust distributed solutions – sampling, processing,
routing, compressing, coding, controlling.
 Architectures should reflect and exploit computational diversity in wireless
devices (TV’s, cell phones, laptops, cheap sensors)
 Asymmetric complexities
 In-built robustness & fault-tolerant designs:
 Diversity in representation & communication
 Rehaul “deterministic” frameworks (e.g. prediction-based) with
“probabilistic” ones
Jan. 28, 2004
UCB Sensor Nets Day
Sampling sensor fields
– Many physical signals e.g., pressure, temperature, are approximately BL
– Physical propagation laws often provide a natural smoothing effect
bad
A/D converters
(sensors)
Sensor network constraints
• Low-precision A/D
• Limited power and bandwidth
good
good
“Central unit”
Sampling a 1-D spatio-temporal field
T
Jan. 28, 2004
2T
UCB Sensor Nets Day
3T
time
Motivation: Acquisition & reconstruction of sensor fields
 Is there an “information” scaling law ?
• [Gupta-Kumar’00]: In ad-hoc networks, with
f(x)
independent data sources, throughput/sensor  0
as 1/sqrt(N).
• In sensor nets, data correlation increases with density.
• Can information-rate/sensor and reconstruction distortion
go to zero with density?
 Tradeoffs between sensor precision and # of sensors?
• Can we overcome low precision sensors by throwing
scale at the problem?
• Is there an underlying “conservation of bits” principle?
Jan. 28, 2004
UCB Sensor Nets Day
Sensor-Field Reconstruction: ‘Distributed’ Sampling Theory
•
“Conservation of bits” principle We can trade off A/D precision for
oversampling rate (quality  bits per Nyquist interval).
1 bit/sample, T/2
2 bits/sample, T
similar
D=c
2-k
D
k
•
•
•
Bit-budget 
Distortion ~ O(1/N)
RNyquist ~ O(log N)
RJan.
~ O(log N / N)
28, 2004
sensor
log (# of sensors)
Error 
accuracy
(1,k)
0
(2,k-1)
(k-2,3)
(k-1,2)
(k,0)
A/D precision b-bit
• Need concept of “dithering” and “distributed coding”
Ishwar, Kumar & Ramchandran (IPSN ’03)
UCB Sensor Nets Day
Overcoming Unreliable Radios
•
Narrowband Radios
–
–
Crystal Oscillator (precise but expensive)
MEMS Resonator (less precise & less expensive)
On-chip LC-Resonator (cheap, low-power, imprecise)
P(fcarrier)
•
•
•
Simple, used by all sensor nodes today [Motes, PicoRadio, Ember,
SmartDust]
How to get fcarrier?
•Can we overcome cheap radios
by throwing scale at the
problem?
Signal BW
3s variation
Jan. 28, 2004
fcarrier
UCB Sensor Nets Day
• Can we devise clever
probabilistic distributed
algorithms for routing &
network coding that exploit the
randomness in the
manufacturing process?
(Picoradio)
Distributed compression
Y
X
Encoder
Decoder
^
X
X
Y
Dense, low-power
sensor-networks
•The encoder needs to compress the source X.
•The decoder has access to correlated side
information Y.
•Can we compress X to H(X|Y)?
Information theory: X can be theoretically compressed at a rate
equal to that when the encoder too has access to Y
Can design practical distr. source coding framework to approach this.
Jan. 28, 2004
UCB Sensor Nets Day
Integrating learning: correlation tracking
• Many sensors report to controller
• Correlation tracking
– Controller keeps track of correlation
– Specifies how much compression
– Sensors blindly encode readings
• Minimal processing at sensor nodes
– Complexity at controller
– Cheap sensors
• Probabilistic reference to side
information allows for robustness to
packet loss
X
Source
Jan. 28, 2004
UCB Sensor Nets Day
R
R
Channel
•
•
•
R
•
•
•
Collector
Collaborative processing:
compressing raw-data versus local estimates
Several scenarios:
• Sensor-clusters (groups of sensors that can collaborative)
• Multiple antennas per sensor
• Multimodal sensors
Jan. 28, 2004
UCB Sensor Nets Day
Result
• If collaborative processing is (MSE) optimal when R is infinity, …
~ 2
• Here, R = infinity and E || X  X ||
Jan. 28, 2004

UCB Sensor Nets Day
E || X  Xˆ ||2
Result
• … then it is also optimal for any finite R.
Suggests that distributed estimation and compression tasks can be
“de-coupled”, i.e., one can design & adapt network topology by
ignoring bandwidth requirements
a number of scenarios.
Jan. 28, 2004
UCB Sensor Netsin
Day
Opportunities: architecture rehauls
 Architectures should reflect and exploit computational
diversity in wireless devices (TV’s, cell phones, laptops, cheap
sensors)
 Asymmetric complexities
 In-built robustness & fault-tolerant designs:
 Diversity in representation & communication
 Rehaul “deterministic” frameworks (e.g.predictionbased frameworks: LP, DPCM, etc.) with “probabilistic”
ones
Jan. 28, 2004
UCB Sensor Nets Day
Rethinking video-over-wireless

Today’s video architectures shaped by downlink broadcast model:
Complex
Light

encoder
Motion estimation task dominates (up to 90%)
decoder
Changing landscape: “uplink” heavy applications
Wireless and surveillance cameras
• Ultra-low-power video sensors
•
•
•
•
Network
Multimedia-enabled cellphones & PDA’s
High-resolution wireless digital video cameras
Wireless-video teleconferencing systems
Home-entertainment and home-networking systems
Video is not just a downlink broadcast experience any more!
Jan. 28, 2004
UCB Sensor Nets Day
New class of video codecs: requirements
 Light codec complexity in order to
 Maximize battery-life.
 Satisfy complexity constraints at encoding device.
 High compression efficiency to match
 Available bandwidth/storage constraints.
 Low transmission power constraints.
 Robustness to packet/frame drops to
 Combat harsh wireless transmission medium.
Jan. 28, 2004
UCB Sensor Nets Day
Rethinking the division of labor
Decoder
Encoder
Transcoding
proxy
light
Heavy encoder
Light decoder
Under reasonable signal models, it is possible to transfer (motion search) complexity
to decoder
loss of compression
(Ishwar, Prabhakaran, & Ramchandran, 2003)
Jan. 28,without
2004
UCBefficiency
Sensor Nets Day
PRISM video simulation results
•
Sequence used: Football (14 frames, 352x240)
•
Comparison: H.263+ (free version from UBC, Vancouver)
•
Frame rate: 30fps, Encoding rate: 10kB per frame
•
Compression: Performance is visually competitive with respect to fullmotion complex inter-frame codecs such as MPEG-4 & H.263+.
(For pure compression, H.263+ outperforms PRISM
dB on our tests on the Football sequence)
•
by about 1.3
Robustness: Much more robust than current solutions. Can recover from
frame losses.
-
Test for robustness: second frame was removed from frame memory after
decoding. third frame was decoded off the first frame in both cases.
Jan. 28, 2004
UCB Sensor Nets Day
Qualcomm’s simulator for “CDMA-2000 1X”
• At packet error rate
6%:
• At packet error rate
11%:
• H.263+ at packet
error rate of 3% and
PRISM at 16%:
Jan. 28, 2004
UCB Sensor Nets Day
PRISM is 4-8 dB better than
H.263+ for the loss rates
investigated.