Consensus-Based Distributed
Least-Mean Square Algorithm
Using Wireless Ad Hoc Networks
Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis
ECE Department, University of Minnesota
Acknowledgment: ARL/CTA grant no. DAAD19-01-2-0011
1
Motivation
 Estimation using ad hoc WSNs raises exciting challenges
 Communication constraints
Single-hop communications
 Limited power budget
 Lack of hierarchy / decentralized processing
Consensus
 Unique features
 Environment is constantly changing (e.g., WSN topology)
 Lack of statistical information at sensor-level
 Bottom line: algorithms are required to be
 Resource efficient
 Simple and flexible
 Adaptive and robust to changes
2
Prior Works
 Single-shot distributed estimation algorithms
 Consensus averaging [Xiao-Boyd ’05, Tsitsiklis-Bertsekas ’86, ’97]
 Incremental strategies [Rabbat-Nowak etal ’05]
 Deterministic and random parameter estimation [Schizas etal ’06]
 Consensus-based Kalman tracking using ad hoc WSNs
 MSE optimal filtering and smoothing [Schizas etal ’07]
 Suboptimal approaches [Olfati-Saber ’05], [Spanos etal ’05]
 Distributed adaptive estimation and filtering
 LMS and RLS learning rules [Lopes-Sayed ’06 ’07]
3
Problem Statement
 Ad hoc WSN with
sensors
 Single-hop communications only. Sensor ‘s neighborhood
 Connectivity information captured in
 Zero-mean additive (e.g., Rx, quantization) noise
 Each sensor
, at time instant
 Acquires a regressor
and scalar observation
 Both zero-mean w.l.o.g and spatially uncorrelated
 Least-mean squares (LMS) estimation problem of interest
4
Centralized Approaches
 If
,
jointly stationary
Wiener solution
 If global (cross-) covariance matrices
,
available
Steepest-descent converges avoiding matrix inversion
 If (cross-) covariance info. not available or time-varying
Low complexity suggests (C-) LMS adaptation
Goal: develop a distributed (D-) LMS algorithm for ad hoc WSNs
5
A Useful Reformulation

Introduce the bridge sensor subset
1)
2)

For all sensors
,
For
, there must
such that
such that
Consider the convex, constrained optimization
Proposition [Schizas etal’06]: For
WSN is connected, then
satisfying 1)-2) and the
6
Algorithm Construction

Problem of interest

Two key steps in deriving D-LMS
1)
Resort to the alternating-direction method of multipliers
Gain desired degree of parallelization
2)
Apply stochastic approximation ideas
Cope with unavailability of statistical information
7
Derivation of Recursions

Associated augmented Lagrangian

Alternating-direction method of Lagrange multipliers
Three-step iterative update process
Step 1: Multipliers
Step 2: Local estimates
Step 3: Bridge variables
Dual iteration
Minimize
Minimize
w.r.t.
w.r.t.
8
Multiplier Updates

Recall the constraints

Use standard method of multipliers type of update

Requires
from the bridge neighborhood
9
Local Estimate Updates

Given by the local optimization

First order optimality condition

Proposed recursion inspired by Robbins-Monro algorithm
1)
2)

is the local prior error
is a constant step-size
Requires


Already acquired bridge variables
Updated local multipliers
10
Bridge Variable Updates

Similarly,

Requires


from the neighborhood
from the neighborhood in a startup phase
11
D-LMS Recap and Operation

In the presence of communication noise, for
Step 1:
Step 2:
Step 3:
Steps 1,2:
Rx
from
Sensor

Step 3:
Tx
Rx
to
from
Tx
to
Bridge sensor
Simple, fully distributed, only single-hop exchanges needed
12
Further Insights
 Manipulating the recursions for
and
yields
 Introduce the instantaneous consensus error at sensor
 The update of
becomes
 Superposition of two learning mechanisms
 Purely local LMS-type of adaptation
 PI consesus loop
tracks the consensus set-point
13
D-LMS Processor
Sensor j
Local LMS
Algorithm
Consensus Loop
PI Regulator
To
 Network-wide information enters through the set-point
 Expect increased performance with
Flexibility
14
Mean Analysis

Independence setting
(As1)
signal assumptions for
is a zero-mean white random vector
, with spectral radius
(As2) Observations obey a linear model
where
is a zero-mean white noise
(As3)

Define
and
are statistically independent
and
Goal: derive sufficient conditions under which
15
Dynamics of the Mean
Lemma: Under (As1)-(As3), consider the D-LMS algorithm
initialized with
Then for
,
.
is given by the second-order recursion
with
and
, where
 Equivalent first-order system by state concatenation
16
First-Order Stability Result
Proposition: Under (As1)-(As3), the D-LMS algorithm whose
positive step-sizes
such that
sense i.e.,
and relevant parameters are chosen
, achieves consensus in the mean
 Step-size selection based on local information only
 Local regressor statistics
 Bridge neighborhood size
17
Simulations
node WSN,
Regressors:
Observations:
D-LMS:
True time-varying weight:
i.i.d.
,
18
Loop Tuning
 Adequately selecting
actually does make a difference
 Compared figures of merit:
 MSE (Learning curve):
 MSD (Normalized estimation error):
19
Concluding Summary
 Developed a distributed LMS algorithm for general ad hoc WSNs
 Intuitive sensor-level processing
 Local LMS adaptation
 Tunable PI loop driving local estimate to consensus
 Mean analysis under independence assumptions
step-size selection rules based on local information
 Simulations validate mss convergence and tracking capabilities
 Ongoing research
 Stability and performance analysis under general settings
 Optimality: selection of bridge sensors,
 D-RLS. Estimation/Learning performance Vs complexity tradeoff
20