Reliable and Real-time Communication in
Industrial Wireless Mesh Networks
Song Han, Xiuming Zhu, Al Mok
University of Texas at Austin
Deji Chen, Mark Nixon
Emerson Process Management
Outline
• Introduction
• Network Management Techniques
– Reliable graph routing
– Schedule construction and channel management
• Performance Evaluation
• Implementation and Deployment
• Future Work
2
Introduction
• WirelessHART network
– Secure and TDMA-based wireless mesh networking technology
– Centralized network architecture
– Stringent timing and reliability requirements
• Previous work
– Full-blown WirelessHART stack (RTAS’08)
– Compliance test suite (RTAS’09)
– Data quality maintenance techniques in wireless network
(RTSS’05, TC’08, RTSS’09)
3
Introduction (cont.)
• Challenge
– The complexity of network management is pushed to the
centralized manager but engineering decisions can have large
performance impact.
• Goals
– Achieve reliable graph routing in WirelessHART network
– Achieve real-time communication by deterministic link and
channel assignment
– Evaluate their performance in industrial environments
4
Reliable Graph Routing
• Reliable Broadcast Graph (GB)
– GB is a graph connecting Gateway (GW) downward to all DEVs
– Broadcasts common configuration and control messages
– Each DEV has at least two parents in GB
5
Reliable Graph Routing (Cont.)
• Reliable Uplink Graph (GU)
–
–
–
–
GU is a graph connecting all DEVs upward to the Gateway
DEVs propagate periodic process data
Each DEV has at least two children in GU
Both GB and GU have no fewer than 2 Access Points
6
Reliable Graph Routing (Cont.)
• Reliable Downlink Graph (Gv)
–
–
–
–
The graph from the Gateway to DEV v
Transmit unicast messages from the GW and NM to v
Each intermediate DEV has at least two children in Gv
There exists at least one directed cycle in Gv
To avoid infinite forwarding loop:
1) Only one cycle of length 2 in Gv
2) Each DEV on the cycle has direct edges to v
7
Constructing GB
• Drop the links with low Receive
Signal Strength (RSS) in the
original network topology G
G
A
A
1
2
3
4
5
8
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
S
A
1
A
2
3
4
5
9
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
A
A
• Grow S according to
1
2
S = {G, Aps, 1}
3
4
5
10
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
A
A
S
• Grow S according to
1
2
S = {G, Aps, 1, 2}
3
4
5
11
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
A
A
• Grow S according to
1
2
S = {G, Aps, 1, 2, 4}
3
4
5
12
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
A
A
• Grow S according to
1
2
S = {G, Aps, 1, 2, 4, 5}
3
4
5
13
Constructing GB
• Drop the links with low RSS in the
original network topology G
G
• Maintain a set of explored node
S, initially S = {G, APs}
A
A
• Grow S according to
1
2
S = {G, Aps, 1, 2, 4, 5, 3}
3
4
5
14
Construct Gv
• More complicated than GB and GU:
– Only involves part of the nodes in G
– The existence of cycle
– Restrictions: One cycle (length 2) between the parents of
destination node v
• Standard Reliable Downlink Graph
– Construct a completely new graph from GW to DEV v
– Configuration in intermediate nodes cannot be reused
– High configuration cost and poor scalability
15
Sequential Reliable Downlink Routing
(SRDR)
• Key Principles
– Each node only keep a small local graph
– Local graphs are reusable building blocks for constructing
reliable downlink graph for multiple destinations
Low configuration cost
High Scalability
High Reliability
16
An example of SRDR
G
A1
1
G
A2
A1
2
4
3
1
5
G
A2
2
4
A1
3
5
2
4
(b) Downlink graph: g2
Sequential route for Dev 2: g2
(a) Original network topology
1
A2
3
5
(c) Downlink graph: g3
Avoid
node
at DEV2
Sequential
routefailure
for Dev 3: g3
(b)
G
A1
G
A2
A1
G
A2
A1
A2
Local graph
1
2
4
3
5
(d) Downlink graph: g1
Sequential route for Dev 1: g2, g1
1
2
4
3
5
(e) Downlink graph: g4
Sequential route for Dev 4: g2, g1, g4
1
2
4
3
5
(f) Downlink graph: g5
Sequential route for Dev 5: g2, g5
17
SRDR vs. Standard Downlink Graph
Configure cost is reduced by 3 links
G
A1
1
G
A2
2
4
A1
3
5
(a) Downlink graph for Dev 2
1
G
A2
2
4
A1
3
5
(b) Downlink graph: g5
Sequential route for Dev 5: g2, g5
1
A2
2
4
3
5
(c) Standard downlink graph for Dev 5
More significant improvement in large scale networks
18
Sequential Reliable Downlink Routing
(SRDR) Extensions
• Network layer header extension:
19
SRDR Extensions
• Routing module extension:
– retrieve the earliest graph ID in the graph list and route the
packet on this graph
– If current node is the sink of the graph, remove this graph ID
and route the packet on the next earliest graph.
– If routing is failed, remove this graph ID and try the next
earliest graph ID if it has the corresponding edges.
20
Optimization on SRDR
• In SRDR, routing is performed strictly according
to the sequence in the ordered graph list.
• SRDR-OPT
– Observation: each node can keep graph info to multiple
destination.
– Have chance to take the “shortcut”
– Principle: Search the ordered graph list backward and route
the packet on the first graph ID that is stored in its table
21
An example of the SRDR-OPT
S
2
g2
4
g3
g4
1
3
S
2
2
2
4
g2
g4
g3
1
1
3
3
22
Communication Schedule and Channel
Management
• Key Principles:
– Spread out the channel usage in the network
– Apply Fastest Sample Rate First policy (FSRF)
– Allocate the links iteratively from Src to Dest
– Split traffic (bandwidth) among all successors
23
Schedule Construction (An Example)
Ch Offset
16
.
.
.
2
Global Channel-Time Slot Matrix
1
0
Device Schedule
Slot
100
200
300
400
G
Dev 1
A
B
AP A
Dev 2
1
1 sec
2
3
24
2 sec
1 sec
Schedule Construction (An Example)
Ch Offset
16
.
.
.
2
1
0
Slot
100
200
300
400
G
Dev 2
A
B
AP A
AP B
1
1 sec
2
3
25
2 sec
1 sec
Schedule Construction (An Example)
Ch Offset
16
.
.
.
2
1
0
Slot
100
200
300
400
G
Dev 3
A
B
AP B
Dev 2
1
1 sec
2
3
26
2 sec
1 sec
Schedule Construction (An Example)
Ch Offset
16
.
.
.
2
1
0
Slot
100
200
300
400
G
Dev 2
A
B
AP A
AP B
1
1 sec
2
3
27
2 sec
1 sec
Schedule Construction (An Example)
Ch Offset
16
.
.
.
2
Channel offset will be converted into
practical channel number in the runtime
1
0
Slot
100
200
300
400
G
Dev 2
A
B
AP A
AP B
1
1 sec
2
3
28
2 sec
1 sec
Performance Evaluation
Configuration overhead in broadcast graphs
Recovery overhead to regain reliability
Reachability in broadcast graphs
Reachability in downlink graph
29
Performance Evaluation
Configuration overhead in downlink graphs
Average latency vs. Network size
Reusable local graph makes the difference
Shortcut makes the difference
Success ratio vs. Sample rate
Network utilization vs. Sample rate
30
WirelessHART Prototype System
Major Components in the prototype :
•
•
•
•
•
•
Network Manager
Gateway
Host Application
Access Point
Device
Sniffer
PC Side
Embedded Side
31
Overall Design of the System
32
Overview of the Network Manager
Network Topology
Routing Graphs
Device Configuration
Global
Schedule
Device
DeviceBandwidth
Schedule
33
Overview of the System
• 10 devices and 1 AP in the system
• Devices publish data to GW with
different sampling rates (1sec – 8sec)
• Retry happens but no packet loss is
detected
34
Deployment (Work-in-progress)
Network Manager, Gateway
and Access Point
UT Pickle Research Center
Petroleum Engineering Department
35
Future Work
• A general and adjustable framework
– Applications have different requirements on timing, security, …
– Building an adjustable MAC state machine
• Collaborative wireless system
– Wifi, WirelessHART, ZigBee,…
– Competition -> Collaboration
36