An Experimental Study on Multi-channel
Multi-radio Multi-hop Wireless Networks
Yunxin Liu, Yongqiang Xiong, Yang Yang and Pengzhi Xu1, and Qian Zhang
{yunliu, yqx, qianz} @microsoft.com, Microsoft Research Asia
5F, Beijing Sigma Center, No.49, Zhichun Road, Beijing 100080, P.R.C.
Abstract1—In this paper, we present experimental studies on
Multi-channel Multi-radio Multi-hop (M3) wireless networks.
The interference of multiple channels and multiple radios is studied. The result may help to understand effect of multi-channel
and multi-radio in real system and is useful as reference for network performance optimization. We evaluate the coexistence of
real-time traffic and best-effort traffic in multi-hop wireless networks. Our experiments show that coordination among wireless
nodes is critical for optimal network performance, especially in
the scenarios with Quality of Service (QoS) requirements. To
perform the experiments, an M3 testbed with 32 nodes is built.
The testbed is based on Windows operating system and provides
convenient programming interface and management tools. The
architecture and key modules of the testbed design are described.
I.
INTRODUCTION
Due to their low costs, ease of deployment, and increased
coverage, multi-hop wireless networks such as mesh networks
that utilize inexpensive and readily available 802.11 wireless
interfaces are touted as the new frontier of wireless networking. More and more researchers are studying the performance
of these networks. Most of them use simulations while some
others are based on real systems [9, 10].
One main problem of multi-hop wireless networks is the
reduction in total capacity due to interference between multiple simultaneous transmissions. Previous studies have shown
that overall network capacity decreases rapidly as node density and the number of hops increases [1, 17, 18].
To improve the capacity of multi-hop wireless networks, a
promising way is equipping each node with multiple radios.
By transmitting and receiving simultaneously, the capacity of
relay nodes is possibly improved. In addition, with multiple
radios, a node may transmit or receive on multiple channels
simultaneously which potentially improves the capacity.
In theory, by utilizing two orthogonal channels, the capacity may be doubled. However, in the real world, the performance of wireless network interface card (NIC) is constrained
by the available manufacture procedure and may not operate
as expected. Actually, the extent of the interference between
radios appears to be dependent on the specific hardware chosen. For example, though 802.11b provides three orthogonal
channels, A. Adya et al. [16] point out that Netgear cards provide only one orthogonal 802.11b channel. Therefore, studying the interference between multiple radios and multiple
channels with real wireless NICs is critical to achieve optimal
performance in multi-hop wireless networks using multichannel multi-radio.
1
This work was performed when the authors were interns at Microsoft Research Asia.
In this paper, we present our experiment results on M3
wireless networks. Using the “off-the-shelf” commercial wireless NICs, we evaluate the interference between multiple radios and multiple channels. We find that even using orthogonal channels the performance of two radios on the same node
is much poorer than expected. In addition, we also study the
performance of coexisting real-time traffic and best-effort
traffic in multi-hop wireless networks. We find that interference between flows is significant and leads to unacceptable
performance, especially for real-time flows. Based on these
results, we conclude that coordination among all wireless
nodes is critical for the performance of M3 wireless networks,
and schemes to enable nodes to collaboratively work together
are required for optimal network performance.
To perform the experiments, we build an M3 testbed with
32 nodes based on Windows operating system, which is located on one floor of a fairly typical office building. The testbed is designed to handle multiple radios/channels and provides interface to develop customized components such as
routing protocols in user mode. We also develop a set of tools
to control, manage, and measure the whole testbed centrally.
The rest of the paper is organized as follows. In Section II,
we describe the related work. Section III presents the software
design and Section IV introduces the testbed deployment. Experiments and results are shown in Section V, and finally,
Section VI concludes the paper.
II. RELATED WORK
There are several real implementations for multi-hop ad-hoc
wireless networks in previous and related work. Most of them
focus on ad-hoc routing protocol implementation and are
based on Linux/FreeBSD operating systems.
AODV-UCSB [2] and AODV-UU [3] are two implementations of Ad-hoc On-Demand Distance Vector (AODV) routing protocol [13] on Linux operating system. These implementations use Netfilter [14] to pass all packets from kernel
mode to user mode and implement AODV routing protocol in
user mode. Instead, Kernel-AODV [4] implements AODV
routing protocol totally in Linux kernel. Another AODV implementation [6] is done by extending Address Resolution
Protocol (ARP). The implementation modifies the ARP code
in Linux kernel and essentially the ARP table acts as the routing table.
In Monarch project [7], Rice University implements Dynamic Source Routing (DSR) protocol [12] on FreeBSD operating system. The implementation extensively modifies the
kernel Internet Protocol (IP) stack, which makes maintenance
difficult.
V. Kawadia et al. [8] propose architecture and generic API
to augment the current routing architecture. The API is implemented on Linux and provided as a shared user mode library called the Ad-hoc Support Library (ASL). To prove the
viability of the framework, they provide a full-fledged implementation of AODV protocol using ASL and a design for the
DSR protocol.
MIT Roofnet [10] is a 38-node urban multi-hop 802.11b
network deployed on Linux system. Based on link-level measurements, D. Aguayo et al. [15] analyze the causes of packet
loss.
R. Draves et al. develop the Mesh Connectivity Layer
(MCL) [9] based on Windows operating system. Architecturally, MCL is a loadable Windows driver. In MCL driver, they
implement Multi-Radio Link-Quality Source Routing (MRLQSR) protocol and analyze its performance [11].
In addition, Intel provides an implementation of AODV for
Windows system [5].
Our M3 testbed is based on Windows operating system.
Besides two ad-hoc routing protocol implementations, DSR
and AODV, the testbed also provides interface to develop new
components such as security modules and other routing protocols. To make development easy, the programming interface
is provided in user mode. In addition, we also develop a set of
tools to conveniently control, manage, and measure the whole
testbed on a central node.
As for multi-radio, A. Adya et al. [16] study the interference of orthogonal 802.11 channels by experiments and propose the Multi-radio Unification Protocol (MUP) to coordinate the operation of multiple wireless NICs on single node. R.
Draves et al. [11] study the performance of MR-LQSR protocol in multi-radio case. Our experiments focus on the interference between multiple radios/channels themselves, and the
performance of coexisting real-time traffic and best-effort
traffic in multi-hop wireless networks.
III. SOFTWARE DESIGN
This section describes the software design. All the software
is developed and tested on Windows XP operating system.
A. The Software Architecture
To communicate with each other, each node installs software whose architecture is shown in Figure 1. The architecture consists of two parts, the kernel mode part and the user
mode part.
The kernel mode part is implemented as a Windows Network Interface Specification (NDIS) driver, called M3 driver.
Similar to MCL driver, M3 driver is implemented in layer 2.5
between layer 2 (the link layer) and layer 3 (the network
layer). From upper layer point of view, M3 driver implements
a virtual network adapter. By binding on underlying physical
NICs, it transmits and receives packets in a tunnel over one or
multiple physical links including Ethernet links. Thus, M3
driver is able to handle multiple radios by design.
The main tasks of M3 driver is to capture packets sent to
and received from the bound NICs and to interact with Windows kernel. Rather than implementing routing protocol in
driver level as MCL, M3 driver passes all captured packets
into user mode and all modules including routing protocol are
implemented in user mode. This is because we want to pro-
vide a framework for further development instead of just implementing a routing protocol. Generally developing kernel
program requires more skills than user mode programming.
In the user mode part, a Dynamic Link Library (DLL) is
implemented to interact with M3 driver and to provide a programming interface to upper layer. Using an event-driven
mechanism, packets captured by M3 driver are passed into the
DLL from kernel. By registering a callback function, upper
layer modules are able to process the captured packets. After
being processed, these packets may be passed back to M3
driver by calling the DLL’s function, and then be sent out by
M3 driver. In addition, the DLL also provides some common
utility functions such as timer management.
Based on the programming interface provided by the DLL,
developers are able to implement their modules including
routing protocols and security components. By hiding the
driver details and providing programming interface in user
mode, the DLL makes development possible with little
knowledge on Window kernel/driver.
Currently we implement two ad-hoc routing protocols, DSR
and AODV. In addition, we implement a control port by
which we control the behaviors of the driver and the routing
protocols locally or remotely. We also monitor the status of
each module through the control port.
Figure 1. Software Architecture.
B.
The Watchdog tools
Because generally all nodes of ad-hoc wireless networks
are widely distributed in space, it is quite hard to individually
configure each node, especially for large scale networks.
Therefore, we develop a set of tools, called Watchdog, to efficiently manage, control and measure the testbed. The main
tools are Watchdog Agent and Watchdog Server. Each wireless node runs a Watchdog Agent to act as a proxy while the
Watchdog server runs on a control node and communicates
with all Watchdog Agents.
Figure 2 shows the user interface of the Watchdog Server.
The main window in the middle shows the current topology of
the testbed. Each node is shown according to its physical location and various lines are used to indicate the connectivity
among nodes, the routing paths, the traffic paths and so on.
The two windows (may be more) at the left show the traffic
throughput between given nodes in real-time. The windows at
the right are used for scripts composing and performing.
Scripts may be saved as commands and to be performed later.
With scripts, we perform various tasks, for example, starting/stopping the Watchdog Agents, disabling/enabling M3
drivers, showing routing path between give nodes, showing
the throughput and paths of given traffic flows. With Watchdog tools, we are able to conveniently and visually manage
the whole testbed in central way.
Figure 2. Watchdog Server.
Figure 3. Physical Deployment.
IV. TESTBED DEPLOYMENT
With the software introduced in Section III, we build our
M3 testbed on one floor of a fairly typical office building as
shown in Figure 3. There are totally 32 nodes in the testbed.
Each node is equipped with one or two Linksys Dual-Band
a/b/g wireless NICs for testing and a 100Mbps Ethernet NIC
for control and monitoring. All these nodes install Windows
XP operating system. 27 nodes are Dell desktop PC with different hardware configurations. In specific, 12 nodes are Dell
GX280 with 3GHz CPU and 1GB memory, 3 nodes are Dell
GX270 with 2.8GHz CPU and 512MB memory, 2 nodes are
Dell GX260 with 2.4GHz CPU and 512MB memory, 4 nodes
are Dell GX240 with 1.8GHz CPU and 512MB memory, 4
nodes are Dell GX1p with 500MHz – 1.3GHz CPU and
256MB or 384MB memory, 1 node is Dell Precision 330 with
1.4GHz CPU and 256MB memory, and 1 node is Dell Precision 330 with 866MHz CPU and 256MB memory. The other
5 nodes are Dell laptops with 700MHz – 1.8GHz CPU and
256MB or 512MB memory. These laptops may act as mobile
nodes.
V. EXPERIMENTS AND RESULTS
Using our testbed, we perform several experiments to study
the interference between multiple radios, multiple channels,
and coexistence of real-time traffic and best-effort traffic in
multi-hop case. Each experiment is repeated for 5 times and
the result is the average of the 5 trials.
A. Interference between multi-channel
As shown by the left picture in Figure 4, 4 nodes form two
ad-hoc networks operating on different channels. Node 1 and
node 2 communicate in ad-hoc network 1 while node 3 and
node 4 communicate in ad-hoc network 2. Two UDP flows
are added from node 1 to node 2 and from node 3 to node 4,
respectively. The UDP senders are aggressive and try their
best to send out as more data as possible. The UDP socket
sending buffer size is 8K bytes and 40M bytes data is sent out
in each experiment. By fixing the channel of network 1 and
changing the channel of network 2, we observe the goodput
between node 1 and node 2 while the UDP traffic between
node 3 and node 4 acts as interference source. We measure the
goodput between node1 and node 2 when there is no traffic
between node 3 and node 4 as the baseline.
Figure 5 shows the result when network 1 uses 802.11a
channel 36 while network 2 uses 802.11a channel 36 to 64.
Figure 6 shows the result when network 1 uses 802.11a channel 52 while network 2 uses 802.11a channel 36 to 64. From
these results, we see the interference is significant if the two
ad-hoc networks use the same channel or two adjacent channels. The capacity decreases more than 50% when the two
networks operate on the same channel. If the two channels are
not the same or adjacent, for example, channel 36 and channel
44, no significant interference observed because they are nonoverlapping or almost non-overlapping in spectrum utilization
according to IEEE 802.11a specification.
B. Interference between multi-radio
The right picture in Figure 4 shows the topology for multiradio experiments. Node 1 is a Dell Latitude C640 laptop with
512MB memory, 1.8GHz CPU, and Windows XP operating
system. Node 1 has two wireless NICs installed in its
PCMCIA slots (the two slots are directly adjacent in physical
position). One is Linksys Dual-Band a/b/g Combo card, and
the other one is Proxim ORiNoCO a/b/g Combo card. The
reason why we don’t use two Linksys cards is because two
such cards can’t be inserted into the laptop due to their physical size. Node 1 communicates with node 2 through card 1 on
801.11a channel 36, and with node 3 through card 2 on
802.11a channel 64, all in ad-hoc mode.
We measure the goodput on the two channels in three cases:
both card 1 and card 2 send data, card 1 receives data while
card 2 sends data, and both card 1 and card 2 receive data.
Figure 7 shows the results.
Based on these results, we see that, using two radios, the total capacity is not as high as expected. Though 802.11a channel 36 (operating at 5.18GHz) and channel 64 (operating at
5.32GHz) are non-overlapping in spectrum utilization, interference between the two radios is significant. In case 1, rather
than doubled capacity, the total goodput is even lower than the
Figure 4. Left: Topology of Multi-channel Experiments.
Right: Topology of Multi-Radio Experiments.
25000
Goodput (Kbps)
20000
15000
10000
5000
Goodput
Baseline
0
36
40
44
48
52
56
60
64
Interference Channel
Figure 5. Goodput of Channel 36.
25000
Goodput (Kbps)
20000
15000
10000
5000
Goodput
Baseline
0
36
40
44
48
52
56
60
64
Interference Channel
Figure 6. Goodput of Channel 52.
25000
21933
Channel 36
Channel 64
21638
22086
20000
Goodput (Kbps)
one using single radio. In case 2, the sending card almost
starves the receiving card because generally the power in
sending mode is much higher than the one in receiving mode.
Due to signal power leakage, theoretically non-overlapping
channels aren’t exactly orthogonal. When two radios locate
very closely, interference between channels simultaneously
sending data becomes significant. However, in case 3, both
cards are able to simultaneously receive data at full speed,
which shows significant gain is possible using multi-radio. All
these affects should be considered for optimal performance in
multi-radio multi-channel wireless networks.
Figure 8 shows the result when card 1 operates on 802.11a
channel 48 while card 2 operates on 802.11b channel 6. The
result demonstrates that there isn’t significant interference
between 802.11a radio and 802.11b radio and high gain is
achieved using two such radios.
C. Coexistence of real-time traffic and best-effort traffic
In this sub-section, we study the performance of coexisting
real-time traffic and best-effort traffic in multi-hop wireless
networks. Specifically, we evaluate the performance of Voice
over IP (VoIP) in coexisting with best-effort traffic.
We develop a VoIP application which uses GSM 06.10 codec. The frame interval is 20 milliseconds and the payload of
each packet is 33 bytes. The VoIP data is transmitted using
UDP packets. With 20 bytes IP header and 8 bytes UDP
header, the total packet length is 61 bytes.
Figure 9 shows the network topology for this experiment.
All the nodes operate on 802.11a channel 56. We use a 4-hop
chain as the path of VoIP and the sixth node as interference
source. We use DSR protocol and fixed route. A VoIP connection between node 1 and node 5 is established on path 1-23-4-5, and an aggressive UDP/TCP traffic is added from node
6 to node 4 as best-effort traffic. We measure the goodput and
packet loss rate of the VoIP traffic on node 5 in cases of with
UDP and TCP traffic, respectively. We also measure the performance of the VoIP traffic without any best-effort traffic as
the baseline. Figure 10 and 11 show the results.
From the results, best-effort traffic may have significant
impact on real-time traffic. Without best-effort traffic, the
VoIP gets constant bit rate goodput and zero packet loss rate.
With the interference of UDP or TCP traffic, the goodput of
VoIP decreases to less than 1Kbps while the packet loss rate is
consistently higher than 80 percents, which leads to totally
unacceptable voice quality.
The reason why VoIP has such a poor performance is because there is not any coordination scheme among nodes in
multi-hop wireless networks. If there is any large volume local
background traffic on the path of the VoIP, the VoIP traffic is
significantly interfered and even starved. To meet the QoS
requirements of real-time traffic, schemes to coordinate all
nodes are needed. One of our on-going works is to provide
QoS support for real-time applications in M3 wireless networks by coordinate all the nodes at layer 2.5 MAC.
15000
10054
10000
8444
5000
787
0
S/S
R/S
R/R
Figure 7. Interference between Two 802.11a Radios. Case 1:
both cards send data. Case 2: card 1 receives data while card 2
sends data. Case 3: both cards receive data.
30000
802.11a channel 48
802.11b channel 6
Goodput (Kbps)
25000
25987
24945
23190
22982
20000
15000
10000
5480
5832
5760
5369
5000
0
S/S
S/R
R/S
R/R
Figure 8. Interference between 802.11a and 802.11b Radios.
Case 1: both cards send data. Case 2: card 1 sends data while
card 2 receives data, Case 3: card 1 receives data while card 2
sends data. Case 4: both cards receive data.
VI. CONCLUSIONS
In this paper, we describe our real testbed with 32 nodes for
multi-channel, multi-radio, multi-hop wireless networks. The
testbed is implemented on Windows operating system and
provides convenient programming interface in user mode for
further development. We also develop Watchdog tools to efficiently mange, control and measure the testbed. With our testbed, we study the interference between multiple channels and
multiple radios by experiments. Our results are useful for understanding real multi-channel multi-radio wireless networks
and may be referred to optimize the performance of such networks. We also evaluate the performance of coexisting realtime traffic and best-effort traffic in multi-hop wireless networks. We conclude that coordination among nodes is critical
for optimal performance of M3 wireless networks and
schemes to coordinate all the nodes are required.
VII. REFERENCES
[1]
[2]
[3]
[4]
[5]
Figure 9. Topology of Experiments on Coexistence of Realtime Traffic and Best-effort Traffic.
[7]
30000
[8]
25000
20000
Goodput (bps)
[6]
[9]
15000
[10]
[11]
with UDP best-effort traffic
baseline
with TCP best-effort traffic
10000
[12]
5000
0
0
10
20
30
40
50
60
70
80
90
Time (s)
[13]
Figure 10. Goodput of VoIP.
[14]
[15]
1.2
Packet Loss Rate
1
[16]
0.8
0.6
[17]
0.4
with UDP best-effort traffic
baseline
with TCP best-effort traffic
0.2
[18]
0
0
10
20
30
40
50
60
70
Time (s)
Figure 11. Packet Loss Rate of VoIP.
80
90
J. Li, C. Blake, D. S. J. De Couto, H. I. Lee, and R.Morris.
“Capacity of ad hoc wireless networks”. In Mobicom, 2001.
AODV-UCSB, http://moment.cs.ucsb.edu/AODV/aodv.html.
AODV-UU, http://www.docs.uu.se/henrikl/aodv.
Kernel-AODV, http://w3.antd.nist.gov/wctg/aodv-kernel.
AODV for Windows Operating System by Intel Corporation.
http://moment.cs.ucsb.edu/AODV/aodv-windows.html.
S. Desilva, and S. Das. “Experimental evaluation of a wireless
ad-hoc network”. In Proceedings of the 9th International Conference on Computer Communications and Networks, 2000.
Implementation of DSR. http://www.monarch.cs.cmu.edu/dsrimpl.thml.
V. Kawadia, Y.G. Zhang, and B. Gupta. “System services for
ad-hoc routing: architecture, implementation and experiences”.
In MobiSys, 2003.
Microsoft Mesh Connectivity Layer (MCL) Software.
http://research.microsoft.com/mesh.
MIT roofnet. http://www.pdos.lcs.mit.edu/roofnet.
R. Draves, J. Padhye, and B. Zill. “Routing in multi-radio,
multi-hop wireless mesh networks”, In Mobicom, 2004.
D.B. Johnson, D.A. Maltz, and Y. Hu. “The Dynamic Source
Routing protocol for mobile ad hoc networks (DSR)”. Internet
draft, April 2003. http://www.ietf.org/ internet-drafts/draft-ietfmanet-dsr-09.txt.
C. Perkins, E. Belding-Royer, and S. Das. “Ad-hoc OnDemand Distance Vector (AODV) Routing”. RFC3561, 2003.
http://www.ietf.org/rfc/rfc3561.txt?number=3561.
Netfilter homepage. http://www.netfilter.org.
D. Aguayo, J. Bicket, S. Biswas, G. Judd, and R. Morris.
“Link-level measurements from an 802.11b mesh network”. In
SigComm, 2004.
A. Adya, P. Bahl, J. Padhye, A. Wolman, and L. Zhou. ” A
multi-radio unification protocol for IEEE 802.11 wireless
networks”. In Broadnets, 2004.
P. Gupta and P. R. Kummar. “the capacity of wireless
networks”. IEEE Trans. On Information Theory, 46 (2), March
2000.
S. Xu and T. Saadwi. “Does the IEEE 802.11 MAC protocol
work well in multi-hop wireless ad-hoc netowrks”. IEEE
Communications Magazine, 39 (6), June 2001.