Estimation of Clock Parameters and
Performance Benchmarks for
Synchronization in Wireless Sensor
Networks
Qasim M. Chaudhari and Dr. Erchin Serpedin
Department of Electrical and Computer Engineering
Texas A&M University, College Station, TX.
1
Outline
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Wireless sensor networks
Related work
Clock model
A Sender-Receiver protocol
Clock offset estimation
Clock offset and skew estimation
Simplified schemes
Best Linear Unbiased Estimation – Order Statistics
Minimum Variance Unbiased Estimation
Minimum Mean Square Error estimation
2
11.
12.
13.
14.
Clock synchronization of inactive nodes
Clock offset and skew estimation in a Receiver-Receiver
protocol
Conclusions
Future research directions
3
Wireless Sensor Networks
S
S
Source
D
Destination
S
Server
Internet
D
Gateway
Wireless Terminal
4
Introduction







Small scale sensor nodes
Limited power
Harsh environmental conditions
Communication failures
Node failures
Dynamic network topology
Mobility of nodes
5
Applications

Monitoring





Environment and habitat
Military surveillance
Security
Traffic
Controlling and tracking



Industrial processes
Fire Detection
Object tracking
6
Main Challenges
7
Importance of time
synchronization

Time synchronization in WSNs is important for






Efficient duty cycling
Localization and location-based monitoring
Data fusion
Distributed beamforming and target tracking
Security protocols
Network scheduling and routing, TDMA
8
Constraints

Limited hardware



Limited energy



Communication vs. computation
RF energy required to transmit 1 bit over 100 meters is equivalent to execution
of 3 million instructions [Pottie 00]
Traditional clock synchronization techniques



Reduced computational power
Low memory
Communication comes for free
Computational resources are powerful
Examples: NTP is energy expensive, GPS is cost expensive
9
Related Work

Reference Broadcast Synchronization (RBS) [Elson 02]




Timing synch Protocol for Sensor Networks (TPSN)
[Ganeriwal 03]



Conventional receiver-receiver protocol
Reduces nondeterministic delays
Conserves energy via post facto synchronization
Conventional sender-receiver protocol
Two operational phases: Level Discovery and Synchronization
Time Diffusion Protocol (TDP) [Su 05]

Achieves a network-wide equilibrium time using an iterative, weighted
averaging technique based on diffusion of timing messages
10
Related Work

Analysis of a sender-receiver model [Ghaffar 02]




For known fixed delays, maximum likelihood estimator for clock offset
does not exist
Five algorithms: median round delay, minimum round delay, minimum
link delay, median phase, average phase.
Minimum link delay algorithm has the lowest variance
Maximum likelihood clock offset estimator for unknown fixed
delays [Jeske 05]
11
Clock Model

A computer clock consists of two components





Frequency source
Means of accumulating timing events
Practical clocks are set with limited precision
Frequency sources run at slightly different rates
Frequency of a crystal oscillator varies due to



Initial manufacturing tolerance
Temperature, pressure
Aging
12
Clock Model

A general clock model can be represented by

where is the clock offset, is the clock skew and
clock drift
Clock synchronization problem


Given the logical clock
is the
for a node k in the network, then
is a function of



Target synchronization accuracy
Amount of energy the network is willing to pay
13
A Sender-Receiver Protocol
T2,k
T3,k
B
Sources of error (time uncertainty) associated with message exchanges

2. Node
sends
an ACK
(Level ofa Node
B, T1, T2, and T3) to Node A at T3.
o Send
time:Btime
spent
to construct
message
1. Node A sends a timing message (Level of Node A and T1) to Node B at T1.
With this,
A calculates
the clock
offset.
o Access
time:Node
delays
at MAC layer
before
actual transmission
o Propagation time: time of flight from one node to another
o Receive time: time needed for the receiver to receive the message and process it
T1,k
(T2  T1 )  (T4  T3 ) U  V


2
2
T1 , T4 : Local time stamps at Node A
T2 , T3 : Local time stamps at Node B
T4,k
A
 : Fixed portion of delays
 : Clock offset
14
Observations




Fixed clock offset model is not sufficient in practice
Clock skew correction results in long term synchronization
and hence more energy savings
Network delays being asymmetric is a more realistic scenario
Even for the symmetric clock offset only model, better
estimation schemes achieving are possible



Minimum Variance Unbiased Estimation (MVUE)
Minimum Mean Square Error Estimation (MMSE)
Lack of analytical performance bounds and metrics

Average RBS error:
[Elson 02] or
[Ganeriwal 03]?
15
Clock Offset



Gaussian Noise Assumption
One motivation comes from experimental basis [Elson 02]
In case of unknown delay distribution, we can evoke Central
Limit theorem
Example: for uniform delays, the sum of even two of them
starts resembling a Gaussian curve
16
Clock Offset
T2,k
T3,k
B

T1,k

The likelihood function can be written as

And the clock offset estimate and the CRLB are
T4,k
A
17
Clock Offset



Exponential Delay Assumption
Random delays often modeled as exponential
Several traces of delay measurements on Internet collected by
[Moon 99] fitting an exponential distribution
Conformation of experimental observations with mathematical
results

Experimental observations



Minimum link delay algorithm [Paxson 98]
Clock Filter algorithm in NTP [Mills 91]
Mathematical results


Best performance by Minimum link delay algorithm [Ghaffar 02]
ML estimate based on minimum order statistics [Jeske 05]
18
Clock Offset

Likelihood function is given as

ML clock offset estimate is

CRLB is derived as
19
Clock Offset and Skew
 : clock offset
 : clock skew
(  1) T1,k    X k 
(  1) T4, k    Yk 
Node B
T2,1 T3,1
T2,k
T2,N
T3,k
T3,N
(  1)(T4, N  T1,1 )
T2,2 T3,2

Node A
T1,1
T4,1 T1,2
T1,k
T4,2
T1,N
T4,N
  X k   Yk
T1,k
T1,1  0
T4,k
T4,k
T2,k  (T1,k    X k )  
T3,k  (T4,k    Yk )  
20
Clock Offset and Skew
Gaussian

Likelihood function with
is

Joint ML estimate for clock offset is shown to be
where
21
Clock Offset and Skew
Gaussian

And for the clock skew

Computationally quite complex
Fixed delay must be known
Open problem: Recursive implementation/update?


22
Clock Offset and Skew
Gaussian

Cramer-Rao Lower Bound is expressed as
where


Proportional to clock skew squared
Not only dependent on number of synchronization messages
but also on the synchronization period
23
Clock Offset and Skew
Exponential

The likelihood function in this case is

Four different cases need to be considered
Case I
Known
Known
Case II
Known
Unknown
Case III
Unknown
Known
Case IV
Unknown
Unknown
24
Clock Offset and Skew
Exponential
Case I:
known,

Constraints

ML estimator
known
25
Clock Offset and Skew
Exponential
26
Clock Offset and Skew
Exponential
Case II:

known,
unknown
Constraints
Lemma 1:
lies on one of
the following curves

27
Clock Offset and Skew
Exponential
Lemma 2:
lies either on point A or to the left of it (B,C,…)
 Lemma 3: To the left of A,
boundary of support region is
formed by a sequence of
curves with decreasing slopes
 Lemma 4:
is unique
and is given by one of

28
Clock Offset and Skew
Exponential
29
Clock Offset and Skew
Exponential
30
Clock Offset and Skew
Exponential
Case III:

unknown,
known
Constraints
31
Clock Offset and Skew
Exponential


Lemma 5: Only two points satisfy the constraints
ML estimator has the closed-form expression
32
Clock Offset and Skew
Exponential
Case IV:
unknown,
unknown

Constraints

Curves intersect on the line

Over this line,
is constrained by
33
Clock Offset and Skew
Exponential


Problem can be solved by the application of four lemmas
Final form of the ML estimator is
34
Clock Offset and Skew
Exponential
35
Clock Offset and Skew
Exponential
36
Simplified Schemes



Fixed delay must be known in Gaussian case
Computational complexity
Further simplification within the same framework is possible
suitable for WSNs in case




Synchronization accuracy constraints are not stringent
Energy conservation constraints are strict
One simple scheme is independent of delay distribution
involved
Cost paid is slight degradation in estimation quality
37
Utilizing Data Samples 1,N




Better skew estimation for large synchronization period
Utilize only 1st and last sample differences for eliminating the
clock offset
Define
Simplified new model
where
and
are either Gaussian
or Laplacian distributed depending on original delay
distribution
38
Utilizing Data Samples 1,N

Gaussian delays
Likelihood function for highly reduced data set is

ML-Like clock skew estimator is expressed as

CRLB-Like lower bound is
Depends on timestamping “distances”

39
Utilizing Data Samples 1,N

Exponential delays
The reduced likelihood function is

ML-Like clock skew estimator can be derived as

CRLB-Like lower bound
40
Utilizing Data Samples 1,N

Simulation results
41
Two Minimum Order Statistics

Motivation


Unknown delay distribution
Small synchronization period

Opening the model equations as

Choose two points as
42
Two Minimum Order Statistics




Joint the two points to obtain the estimate
slope and intercept
The form of the estimator is
through its
Almost as simple as the clock offset only case
Knowledge of is not required
43
Two Minimum Order Statistics
44
Two Minimum Order Statistics

Simulations results
45
Two Minimum Order Statistics

Computational complexity comparison with the MLE
46
Summary
Gaussian
Exponential
Offset Model
MLE + CRLB
CRLB
Offset + Skew Model
MLE + CRLB
MLE + Algorithms
Offset + Skew Model Using First and Last Using First and Last
sample
sample
ML-Like + LB
ML-Like + LB
Offset + Skew Model
Two minimum order statistics
Algorithm + Computational Complexity
47
Best Linear Unbiased
Estimation – Order Statistics




Limited power resources in WSN implies better estimation
techniques should be utilized
Results derived so far correspond to symmetric delays,
although asymmetry is a more realistic scenario
Best Linear Unbiased Estimation (BLUE) is suboptimal in
general due to linearity constraint
What if the linearity constraints are on the order statistics of
observed data, instead of the raw observations?
48
Best Linear Unbiased
Estimation – Order Statistics

Transforming the data as

Following relations hold for ordered data
49
Best Linear Unbiased
Estimation – Order Statistics

The covariance matrix

Its inverse can be found by Gauss-Jordan elimination
Let the ordered observations be represented as

for
can be derived as
50
Best Linear Unbiased
Estimation – Order Statistics

Asymmetric Link Delays
The asymmetric linear model can be written as

And the Gauss-Markov theorem implies
51
Best Linear Unbiased
Estimation – Order Statistics

The covariance matrix for

The final expression for
is
is
52
Best Linear Unbiased
Estimation – Order Statistics

Symmetric Link Delays
The symmetric linear model can be written as

The Gauss-Markov theorem yields the solution
53
Best Linear Unbiased
Estimation – Order Statistics

Covariance for
is

The expression for

BLUE-OS for
is
same as the MLE for symmetric link delays
54
Minimum Variance Unbiased
Estimation



Asymmetric Link Delays
Found by the application of Rao-Blackwell-Lehmann-Scheffe
theorem
Likelihood function can be expressed as
According to Neyman-Fisher factorization theorem, the
sufficient statistics is
55
Minimum Variance Unbiased
Estimation



Notice that
Find
such that
Applicable only if is a complete sufficient statistic
Only function of is unbiased
56
Minimum Variance Unbiased
Estimation

Unbiased estimator of
?
Note that BLUE-OS is unbiased and hence MVUE

Compensation for asymmetry through

Variance of the clock offset

!
57
Minimum Variance Unbiased
Estimation



Symmetric Link Delays
Again applying the Rao-Blackwell-Lehmann-Scheffe theorem,
the likelihood function is
More than one unbiased functions of complete statistic?
Through Neyman-Fisher factorization theorem, the actual
sufficient statistics is
58
Minimum Variance Unbiased
Estimation



is proved to be complete
Unbiased estimator of
?
BLUE-OS is unbiased and hence the MVUE

In symmetric case, the MVUE and BLUE-OS of
with MLE

Its variance is
coincide
59
Summary
Clock Offset
MVUE
Symmetric Delays
MSE
Remarks
Asymmetric Delays MVUE
MSE
Remarks
60
Explanatory Remarks




Does this discontinuity in clock offset estimates performance
make sense?
Which estimator is better when the network delays are slightly
symmetric?
MVUE is the best in unbiased class of estimators, not all.
For asymmetric case,
61
Explanatory Remarks

The MLE outperforms the MVUE under the condition

Estimator could be chosen according to the number of
synchronization messages if knowledge of
is available
Around the point
, MLE attains lesser MSE as the link
asymmetry decreases, i.e.,

62
Explanatory Remarks

Simulations results
63
Explanatory Remarks



Apparently, adapting between the two estimators a good idea
according to
since
have been obtained too.
Despite the fact that MLE is functionally invariant,
considerable amplification of errors occurs due to repeated
nonlinear processing
Results are even applicable to Internet time synchronization
64
Explanatory Remarks




As a byproduct, the MVUE of the fixed and mean variable link
delays are obtained
Endd-to-end delay measurements are helpful in analyzing
network performance
Very useful for applications behaving adaptively based on
observed network performance
Continuous media applications, such as audio and video,
absorb the delay jitter perceived at receiver for smooth playout
of media stream
65
Minimum Mean Square Error
Estimation




In general, the MMSE estimator is not realizable due to the
dependence of MSE on the unknown parameter
MSE is a sum of variance and bias squared
This dependence usually comes from the bias
Setting the MSE proportional to inverse of the scale parameter
cancels the dependent factors
66
Minimum Mean Square Error
Estimation

The MMSE estimator comes out to be a function of MVUE
Closed-form expression for

And for mean link delays

MSE of clock offset

67
Minimum Mean Square Error
Estimation

The MMSE estimator comes out to be a function of MVUE
Closed-form expression for

And for mean link delays

MSE of clock offset

68
Clock Synchronization for
Inactive Nodes

Packet synchronization protocols



Receiver-Receiver (R-R)
Sender-Receiver (S-R)
For WSNs implementing any sender-receiver protocol, the
inactive nodes can exploit the timing messages received
q
m
q
p
m
p
69
Clock Synchronization for
Inactive Nodes

The model can be represented as
p
m
rjp m
r1p m s mj
s1m
q
rjm p s jp m
r1m p s1p m
r1mq
r1p q
rjmq
rJm p sJp m
rJp m
sJm
rjp q
rJmq
rJp q
70
Clock Synchronization for
Inactive Nodes

Likelihood function, assuming symmetric delays, is

The maximum likelihood estimator is derived as
71
Clock Synchronization for
Inactive Nodes
72
Clock Synchronization for
Inactive Nodes

The pdf of
is obtained as

Cramer-Rao Lower Bound is

Is the ML estimator efficient?
73
Clock Synchronization for
Inactive Nodes


An efficient estimator does not exist due to the rule “if an
efficient estimator exists, the ML procedure will produce it”.
Simulation results
74
Clock Synchronization for
Inactive Nodes



Symmetric delay assumption was less realistic
Better estimation techniques can be employed
Using the transformed data,
the linear model can be written as
75
Clock Synchronization for
Inactive Nodes
and

Hence, the covariance matrix is

Final form of estimator
76
Clock Synchronization for
Inactive Nodes

The likelihood function in symmetric case is

MVUE for the clock offset
is derived and shown to
coincide with BLUE-OS and MLE

Its variance is give by
77
Clock Synchronization for
Inactive Nodes

Similarly, for asymmetric link delays, the BLUE-OS is

which is also the MVUE
Its variance is given by
78
Clock Synchronization for
Inactive Nodes

MMSE estimator can be derived as a function of MVUE
Closed-form expression for

And for mean link delays

MSE of clock offset

79
Clock Offset and Skew in a
Receiver-Receiver Protocol

Main sources of errors – send time and channel access time –
are removed

A receiver-receiver model can be represented as
80
Clock Offset and Skew in a
Receiver-Receiver Protocol

The likelihood function can be expressed as
where

Objective function to be maximized is
over the constraints
81
Clock Offset and Skew in a
Receiver-Receiver Protocol

JML estimator
82
Clock Offset and Skew in a
Receiver-Receiver Protocol

Why Gibbs Sampler?





Joint ML estimator is biased
MVUE does not exist since the sufficient statistics depend on
unknown parameters
Posterior distribution can be found and clock parameters can be
estimated by its mean for better results
Straightforward extension to additional unknown parameters, e.g.,
clock drift
Posterior distribution involves complex integrations, hence the
Markov-Chain Monte Carlo (MCMC) methods
83
Clock Offset and Skew in a
Receiver-Receiver Protocol

Algorithm for Gibbs Sampling is to iterate the following with
initial values
:

After a threshold value , the set
sample values from the joint posterior
behaves as the
84
Clock Offset and Skew in a
Receiver-Receiver Protocol

In the current scenario, the Gibbs Sampler is implemented as
follows

Performs better than the JML estimator
Simulated with



MVUE as the lower bound with one parameter known
BLUE as the upper bound due to linearity constraints
85
Clock Offset and Skew in a
Receiver-Receiver Protocol

Simulation results
86
Conclusions



General exponential family model
Accumulated error analysis for multihop protocols
Effect of mobility on time synchronization
87
Future Research Directions



General exponential family model
Accumulated error analysis for multihop protocols
Effect of mobility on time synchronization
88