Cyclone Time Technology
Deriving Consistent Time Base Using Local Clock Information
Ashok Agrawala
Moustafa Youssef
Bao Trinh
University of Maryland
College Park, MD 20742
Some Common Characteristics
• Peer-to-Peer Architecture
– The scheme relies only on local information or
what they can obtain from their immediate
neighbors
– No central/master clock
• Fast convergence
Clock Model
• Each node has a local clock which ticks at a constant rate
• The clock is stable in that its drift rate does not change rapidly
 Local time can be recorded at any instant by reading the clock which is
an integer register incremented every tick time
• Local time at any node A is represented as
   (  t )
A
A
A
• Where
A is the current local time at node A at time instant
•

•
•
 A is the drift rate of the clock at node A, and
 A is the offset
Two Nodes
 AA (n)
Node A
 AA (n)
 AA (n  1)
Node B
 AB (n)
 AB (n  1)
Assumptions
•
•
•
•
•
All nodes are connected in that there is a path from
any node to every other node.
The transit time for a message from Node A to Node B,
ΔAB, is fixed ( relaxed later).
Each node is capable of timestamping an incoming
message with its local clock time to within a clock tick.
Each node is capable of sending a message at a
defined local time to within a clock tick
If there is a variability in timestamping operation, this
gets lumped into the variability in the transit time
Outline
•
•
•
•
•
Introduction
Drift correction scheme (Cyclone)
Results
Virtual Clock Scheme
Conclusions
Scheme 1
Drift Correction (Cyclone)
• Goal : Correct drift at all nodes
• Each node sends a beat message at times
it determines from its local information
• This message is only a time marker with
no other information bits
• Each node uses a common constant
number  whose value is fixed at design
time
Two Nodes
 AA (0)
A(0)
A(1)
A(2)
…
Node A
Node B
 BB (0)
B(0)
B(1)
B(2)
…
Algorithm
1. Initially each node sends the 0th beat at
 AA (0)  BB (0)
2. Each node sends the 1st beat at
 (1)   (0)   (0) where  (0)  
A
A
A
A
A
A
A
A
Algorithm
3. For subsequent beats Node B calculates
 (n)   (n)   (n  1)
B
A
B
A
B
A
Kb
and
1
 (n  1) 
kb
B
B

πx2B
B
x
( n)
πx1B
x
πxkbB
B
πBB
4. It sends the (n+1)st beat at
B
B
B
 B (n  1)   B (n)   B (n  1)
Analysis
 AA (n)   AA (n)   AA (n  1)
  A [ A  t A (n)]   A [ A  t A (n  1)]
  A [t A (n)  t A (n  1)]
  A A (n)
where
 A (n)  t A (n)  t A (n  1)
Similarly
 AB (n)   AB (n)   AB (n  1)
  B [ B  t A (n)   AB ]   B [ B  t A (n  1)   AB ]
  B [t A (n)  t A (n  1)]
  B A ( n )
Therefore
B
B 1
 B (n  1)   [
kB
kB

x 1
x
(n)]
Analysis
Matrix Notation
A(i, j )  1/ k j
Thus at each step we carry out a distributed calculation of
 (n  1)  A  (n)
As A is a stochastic matrix, this iteration converges with all items in the
vector taking the same value.
Convergence rate is determined by the second dominant eigen value.
Practical Considerations
•
Transit delay
–
–
•
We assume it to be a constant. If it has some variability, it will
have to be treated as a random variable. In that case the
degree of clock synchronization depends on the jitter in the
transit delay.
When transit time is much larger than the cycle time, special
steps are required in the beginning of the operations
Finite precision
–
–
–
The development above assumed an infinite precision
arithmetic and infinite resolution clocks.
These are small perturbations to the calculations but have to
make sure that their affects do not accumulate.
Require phase locking.
Outline
•
•
•
•
•
Introduction
Drift correction scheme (Cyclone)
Results
Virtual Clock Scheme
Conclusions
Simulation Parameters
•
•
•
•
Clock Tick Time – 100 ps (10 GHz)
Cycle Time – 125 ms (8000/sec)
Latencies – Random 0 and 80 cycles
Topologies
– Chain
– Star
– Random
• Drift rate - ±100 ppm
Simulation Results
• Convergence achieved every time
• On convergence, jitter 1-2 clock ticks
• Long Term Stability
– Similar jitter and no drift after 12 hours of
simulation time.
Convergence Time for Different Network Topologies
Convergence time in Seconds
# Nodes
Star
Chain
Bidirectional Random
20
2
5
5
4
50
15
62.5
62.5
31.125
100
15
62.5
62.5
31.125
200
N/A
N/A
N/A
31.125
500
N/A
N/A
N/A
31.125
1000
N/A
N/A
N/A
31.125
Effects of Perturbations
• Transit time
– Varied by 10%
– No more than the transit time change
– Stabilizes very fast after that
• Clock Drift
– Varied again by 10%
• Again, a step change results in immediate jitter
determined by the change in clock drift
• Stabilizes very fast.
Transit Delay and Convergence
Convergence Rate (Secs)
250
200
150
100
50
0
0
200
400
600
Max Latency (ms)
800
1000
Latency Perturbations
Node
ID
0.01%
0.10%
1%
CTJ
CTJ
CTJ
1
0.000%
0.000%
0.000%
2
0.001%
0.008%
0.055%
3
0.001%
0.007%
0.064%
4
0.001%
0.008%
0.074%
5
0.001%
0.008%
0.072%
6
0.001%
0.007%
0.083%
7
0.001%
0.008%
0.073%
8
0.001%
0.007%
0.072%
9
0.001%
0.008%
0.083%
Drift Rate Perturbation
Node
ID
100 PPM
10 PPM
1 PPM
CTJ
CTJ
CTJ
1
0.000%
0.000%
0.000%
2
0.014%
0.001%
0.000%
3
0.004%
0.001%
0.000%
4
0.008%
0.001%
0.000%
5
0.007%
0.002%
0.000%
6
0.011%
0.001%
0.000%
7
0.012%
0.002%
0.000%
8
0.015%
0.001%
0.000%
9
0.010%
0.001%
0.000%
10
0.009%
0.001%
0.000%
Implications
• This scheme achieves clock synchrony in
that all nodes achieve a common cycle
value of *
• The local value A is different at each node
• The beat instants are not synchronized
– They do not drift
How to achieve a common clock reading ??
Outline
•
•
•
•
•
Introduction
Drift correction scheme (Cyclone)
Results
Virtual Clock Scheme
Conclusions
Virtual Global Clock
• A clock with parameters
 * and *
We define a scheme such that based only on
local information any node can convert its
local time to the time at this Virtual Global
Clock.
Key idea is to use a distributed consensus
technique
Assumptions
• For the discussion right now we add two
additional assumptions:
– All connections are bidirectional
– Transit time in two directions are the same
 AB   BA
Approach
• Carry out the first scheme and reach
convergence
• At convergence we note that
 A   A
• The time when node A sends its nath beat
 AA (na )   A ( A  t A (0)  na )
  A [ A  t A (0)]  na A   AA (0)  na A
Two Node Case
• As a part of the beat message node A also
sends
–
–
–
–
–
–
Its converged cycle length
n
Current cycle number
a
A
Time  A ( na )
A
Time  B (nb  1)
A value Offset A (na )
A value
Drift A (na )

• Node B sends similar values
A
Calculations
 /    /
B
A
B
A
1 A B
B
A
A
A
B
A
[ B { B (n)   A (n)}   A (n)   B (n)]   (   )
2 
Similar calculations are done by node B
Node A can convert its local time to the time at Node B as
 
B
B
A
 
A
B
A
 (   )
A
B
A
Multinode Operations
• When this phase starts
– For each of its incoming links node A calculates
B C
,
,...
A A
and
 A ( B   A ),  A ( C   A ),...
It initializes
Offset A (0)  Average[  A ( x   A )]
and
x
A
Drift (0)  Average[ ]
A
Multinode Operations
• For each subsequent cycle
– It calculates the new values of A and B as
averages of incoming values of A and B
adjusted to the local scale.
x

Drift A (k )  Average[ Drift x (k  1) A ]

and
A

Offset A (k )  Average[ x Offset x (k  1)   A ( x   A )]

On Convergence
• Node A has values
*

Drift A*  A

and
Offset A*   A ( *   A )
It can convert its local clock values to Virtual global clock as
  Drift   Drift Offest
*
A* A
A*
A*
Current Status
• Simulation Results confirm the claims
• Working on prototype implementations
using standard NICs
Comparisons
CTT
IEEE-1588
NTP
GPS
TTP
SERCOS
Spatial extent
General
A few subnets
Wide area
Wide area
Local bus
Local bus
Communication
s
General
Network
Internet
Satellite
Bus or star
Bus
Target accuracy
Sub-microsecond
Sub-microsecond
Few milliseconds
Sub-microsecond
Sub-microsecond
Sub-microsecond
Style
Distributed
Master/slave
Peer ensemble
Client/server
Distributed
Master/Slave
Moderate
Moderate
Resources
Small network
message
and
computatio
n footprint
Small network
message
and
computatio
n footprint
Moderate
network
and
computatio
n footprint
Moderate
computatio
n footprint
Latency
correction
Yes
Yes
Yes
Yes
Configured
No
Drift Correction
Yes
Yes
No
No
No
No
Protocol
specifies
security
No (V2 may
include
security)
No (V2 may
include
security)
Yes
No
No
No
Administration
Self organizing
Self organizing
Configured
N/A
Configured
Configured
For highest
accuracy
For highest
accuracy
No
RF receiver and
processor
Yes
Yes
Configured
~2 seconds
Varies, nominally
seconds
~1 second
Hardware?
Update interval
Every TDMA
cycle, ~ms
Every TDMA
cycle, ~ms
Concluding Remarks
• Use of consensus approach simplifies the clock synchronization
• As the scheme only depends on local information it is highly
scalable
• Primary results to date
– Analytic
– Simulation
– Implementations ?
– Appropriate estimators/filters
– Practical considerations
• Node Failure
• Node Joining
• …