An emulation infrastructure for multi-hop wireless
communication networks ∗
Vincent Guffensa
Georges Bastina
Olivier Bonaventureb
guffens@auto.ucl.ac.be bastin@auto.ucl.ac.be bonaventure@info.ucl.ac.be
a Centre for Systems Engineering and Applied Mechanics (CESAME)
b Department of Computing Science and Engineering
Université Catholique de Louvain
B-1348 Louvain-La-Neuve, Belgium
In this paper, a novel approach for wireless network emulation is proposed. The emulation
infrastructure is built on a virtualized Linux kernel (User Mode Linux) which allows for
highly realistic emulations. The virtualized kernel is extended with software based wireless
cards which are plugged in a generic physical layer emulator. The emulator proposed in
this paper tries to reduce the gap between simulations and implementations by allowing
a developer to test his/her implementation in an almost real, but entirely software-based
environment. The advantage of such an approach is that its cost, in terms of required
infrastructure and learning curve for the developer is reduced to a minimum.
I. Introduction
Once a new protocol has been designed or when a
given modification has been proposed for an existing
standard, it is usually desirable to test it in an environment as close as possible to one encountered in a
real life scenario. Development and testing are usually
carried on in a network laboratory but equipment and
maintenance costs may render this situation impractical. Furthermore, to test wireless protocols, equipments should be located at a suitable distance from
each other and their mobility should be such as to
allow for intermittent transmission. Maintaining this
setup for days may be impossible.
Obviously, new protocols may be tested on high
level simulators such as the famous NS2 where it is
easy to create a situation such as one described above.
Unfortunately, factors such as interoperability with
existing implementation, performance of integration
which existing code might be overlooked.
This tradeoff between simulation and laboratory
testing is however blurred by the increasing use of virtualization. In [13], Keshav et al. propose a protocol
testing environment which uses virtualized networking kernel. In this system, the BSD kernel API has
been ported to User Space letting applications such as
mrouted, gated or ping running without modification.
∗
This paper presents research results of the Belgian Programme on Interuniversity Attraction Poles, initiated by the Belgian Federal Science Policy Office. The scientific responsibility
rests with its author(s).
The Linux kernel has also been ported to user space
by J. Dike who has implemented the User Mode Linux
architecture [5] referred to as UML. Multiple virtualized Linux machines may run on a single host and be
interconnected via a software switch running on the
host. UML has been used to implement web hosting solution, honeypot and its use has been reported
in [7] for network protocol testing with the VNUML
project. A mobile ad-hoc network simulator based on
VNUML is also proposed in [11].
In this paper, we extend the idea of virtualization
to the kernel driver level and describe how an existing
driver for a wireless 802.11 card may be used with
UML. UML machines running the wireless driver are
interconnected through a physical layer emulator and
represented in a graphical environment where they can
be freely moved.
This system therefore allows for a highly realistic
emulation environment for multi-hop wireless communication networks where every tool already running in Linux may be used without modification. To
the belief of the authors, this framework is of great
practical interest for network software development,
interoperability testing and educational purposes.To
our knowledge, the emulator that we propose is the
only software-based solution that allows to support
wireless and mobile nodes in a virtual environment.
Our simulation framework is described in Section II to
VI and a practical illustration of the various possibilities of this system is given in Section VII. The chosen
example is the implementation of the manet name resolution rfc draft [6] which relies on existing software
Mobile Computing and Communications Review, Volume 1, Number 2
1
and already implemented protocols. Some related
works are presented in Section VIII. The network emulator described in this paper is freely available and
can be found at http://www.auto.ucl.ac.be/uml-wifi/.
Virtual machine
App.
syscal
Linux
hostap
kernel exported
symbols
driver
code
II. Components used in the emulator
As described in the introduction, the main components used in the emulator are the User Mode Linux
kernel code and a wireless driver. For this work, we
choose the hostap driver [10] because it supports different kind of hardware and its hardware dependent
code is therefore neatly separated. Furthermore, the
hostap driver may be setup as an access point and
supports software-based encryption. The advantages
of the Linux kernel are, among others, availability of
the source code, availability of advanced networking
features, increasing use in embedded network devices
and huge amount of networking related softwares. In
the context of this paper, the availability of a stable
User Mode port is of course essential.
Linking the hostap driver with the User Mode kernel requires the resolution of all symbols from the kernel’s exported symbol table which is not possible as
User Mode Linux does not contain any bus implementation. Unresolved symbols, whose names are selfexplanatory are for instance : pci enable device,
writew and readw.
The first component that had to be implemented
was therefore a simple bus suitable to export the required symbols for the hostap driver to be inserted in
the kernel. The modification of the hostap driver are
minimum and consists in replacing unresolved symbols with the new ones, along with some minor modifications due to the simpler implementation of our bus
compared to a standard PCI bus. We refer to this bus
as “netbus” as its primary use is to connect a network
device to UML. The functionalities of this bus, sending and receiving data as well as transmitting interrupt
signals are implemented with TCP connections. Once
the hostap driver is successfully linked into the kernel, it must of course act on a device which also has
to be emulated.
As any other device, our emulated wireless card has
to be connected or “virtually plugged” into the UML
via netbus. This situation is depicted in Fig. 1 showing the different interconnections existing between the
components introduced above. The emulated wireless
card runs as a different process, which is the emulator
itself. This core program is written in QT/C++ and
also implements the physical layer emulation as well
as a visualisation system.
netbus
tcp
separated
processes
new
exported
symbols
virtual external device
Figure 1: Interconnection of the driver with the UML
kernel
As mentioned above, netbus is built with TCP connections and the core emulator is therefore written as
a TCP server.
The architecture of the emulator is depicted in
Fig. 2 where it can be seen that the hostap driver
is depicted as a TCP client which is implemented on
top of netbus. When the driver tries to register itself,
it calls a function netbus register device which
tries to bind to the emulator server socket. If the operation is successful, a new object implementing the
virtual wireless card is instantiated in the emulator.
The visualisation system displays an icon representing the mobile node with a surrounding circle which
represents the reachability zone of the card.
Each component is described in more details in the
following sections.
III. Netbus implementation
As mentioned previously, the connection between the
emulated card and the driver is realised with a TCP
connection. Using TCP makes it possible to run multiple UML machines on different hosts. The simulation
can therefore be distributed among multiple CPU’s.
The TCP protocol has been chosen for its reliability.
Indeed, it is not desirable to introduce uncertainty in
the delivery of interrupt and data as one would like to
control in a deterministic fashion if packet drops occur
or not.
In our prototype, two different TCP connections are
used : one connection is used for data transmission
while the other is only used for interrupt emission.
Mobile Computing and Communications Review, Volume 1, Number 2
2
http://www.unik.no/~paalee/research.htm
UML machine
hostap driver
tcp client1
machine2
0x0
0x1
UML machine
hostap driver
tcp clientN
netbus
netbus
virtual
wireless card
virtual
wireless card
simulation server
Figure 2: Architecture of our wireless emulator
The client socket corresponding to the interrupt line
at the UML side is configured in ASYNC mode and
the associated SIGIO signal is registered in UML as
an interrupt signal associated with the wireless card.
The data line operates in blocking mode. This is necessary as, from the UML point of view, the read/write
operation has to be “atomic” and must be completed
entirely before the UML may continue to execute. Every command sent on the data line is therefore followed by a blocking recv call which waits for an acknowledgement from the card and synchronises the
kernel with the card.
As packets are transmitted by the hostap driver by
a series of writew command (programmed IO), the
synchronisation mode described above induces a high
latency in packet transmission. Therefore, an extra
feature has been implemented in netbus which permits
to send a block of “len” bytes of data in a single blocking operation. In some sense, this feature is analogous
to a DMA in a real hardware.
IV.
Implementation of the emulated
wireless card
The virtual wireless card is implemented so as to react, from the driver perspective, as a real hardware
card would have reacted. To that end, several registers are implemented such as the command register and commands issued by the driver are interpreted. Among the most important implemented com-
0xfff
fid_entry
0xabc
Wireless network
Physical layer emulation
Network
visualisation
window
0xcba
"on board memory"
machine1
0x0
0x1
0xfff
{ char * ptr;
short int len;
short int refcnt;
}
len
[ data...]
Figure 3: Memory management for the “on-board
memory” showing a data segment shared between two
different machines.
mands, we describe (see Sec. IV.A and IV.B) the
COMMAND CODE ALLOC which allocates memory segments and CMD CODE TRANSMIT which
instructs the card to transmit a frame previously
copied into the memory through netbus.
IV.A.
Memory management
The memory available in each virtual wireless card is
managed using a simple segmentation scheme [4] as
shown in Fig. 3. Each machine may allocate 0xfff
segments of length smaller than 4kB (0xfff). During memory initialisation, memory zone accessed by
the driver are preallocated and formatted so that the
driver finds the expected information. For instance,
the segment 0xfc0 is preallocated and the hardware
address of the virtual card is copied at memory address 0xfc0100 where the driver expects to find it.
Memory may also be dynamically allocated by the
driver to be used for packet transmission as detailed
in the next subsection.
IV.B.
Packet transmission
During initialisation, the driver allocates memory segments large enough to accommodate a packet length
equal to the configured MTU and remembers the segment and the offset necessary to access these memory zones. In fact, only a fid entry structure (See
Fig. 3) is allocated as the pointer toward the actual
data is set later on. For packet transmission and reception, the driver and the card use a mechanism referred to as “Buffer Access Path”: to access a given
buffer, the driver first issues a setup bap command
by writing the corresponding code into a command
register. This command consists in writing the previously stored segment and offset in some specified reg-
Mobile Computing and Communications Review, Volume 1, Number 2
3
isters. The frame to be transmitted is then sent through
the data connection. The ptr entry of the preallocated
fid entry structure is modified to point toward the
data received by the server data socket. This operation as the effect of copying the received frame at the
right place in the virtual machine memory.
A loop iterates over every virtual machine and allocates a fid entry whose ptr entry points toward
the same shared data segment increasing the refcnt
entry by one. In fact, the packet is transmitted between two UML machines according to some physical models described in Section V and may therefore
be dropped or corrupted at that stage.
Each virtual wireless card issues a receive interrupt
after having setup the buffer access path registers to
point toward their received packet. The driver may
now issue a from bap command with the corresponding segment and offset value. The fid entry is unallocated when the receive interrupt is acknowledged
and the data segment is freed if the rfcnt has reached
zero.
V.
In order to ease protocol testing, it might be convenient to adopt a much simpler view on the physical layer. Results presented in this paper therefore use a simple threshold based propagation model.
The transmission power configured at each node (using the Linux command iwconfig wlan0 txpower
[power]) is linearly converted to a suitable distance
(in pixels) and machines falling in that range receive
transmitted packets with probability one. A yellow
circle (see Fig. 4) is depicted around each machine
so that the operator of the simulation may easily move
machines from a reachable zone to an unreachable one
and easily create an arbitrary topology.
Obviously, the choice of a physical model will depend upon the type of experiment to be carried on.
Physical layer emulation
As described previously, packet transmission is realized in the server code by pointer passing between
virtual machines. The physical layer emulation essentially consists in determining if this operation is
to be realized successfully or if this operation should
fail and result in packet loss or packet mangling. In
other term, the physical layer emulation is equivalent
to the determination of a “Packet Error Rate” (P ER)
between a pair of machines. The “Bit Error Rate”
(BER) and the P ER are linked through the following formula :
P ER = 1 − (1 − BER)len
Where len is the length of the packet being transmitted. The BER mainly depends on the type of digital
modulation being used and on the Signal To Noise Ratio (SN R) at the receiver. Theoretical formulas that
give the BER as a function of the SN R for a given
modulation scheme are widely available in the literature and usually rely on the assumption of an Additive White Gaussian Noise (AWGN) which may be
entirely specified by its variance. A global variable
therefore exists in the emulator to specify the variance
of a background thermal noise.
The signal power at the receiver obviously depends
on the distance between the emitter and the receiver
and is estimated via a Path Lost function P L(d) where
the distance d is calculated via the distance in pixels between two virtual machines. Once again, Path
4
Lost functions are widely available in the literature
[12] and may be easily plugged in the emulator. If
the SNR at the receiver is below a given sensitivity
threshold, the packet is dropped, if the SNR is greater
than the sensitivity, a random bit in the packet may be
modified before transmission with a probability given
by the PER. Note that packet mangling requires a hard
copy of the buffer being transmitted.
VI.
Packets visualisation
In order to obtain a visualisation of the traffic through
the network, packets emitted by each machine are represented by moving dots in the visualisation window.
This graphical representation is meant to give an impression of how many packets are being transmitted.
The moving dots are implemented with animated canvas object available in QT. Whenever a packet is to
be transmitted from one node to another, an animated
canvas item is released from a recycle bin and is displayed on the visualisation screen. The QT animated
canvas system takes care of the correct displacement
of the item. Once this animated item has reached its
destination, the corresponding object is put back into
the recycle bin.
This system is illustrated in Fig. 4 showing the visualisation window with five UML machines. The
machines uml01,uml02 and uml03 are running the
AODV routing protocol and uml01 is pinging uml03.
Meanwhile, uml04 is flooding uml05 with ping requests (ping -f). It is indeed straightforward to determine from this figure that the number of packets
exchanged between uml04 and uml05 is greater than
the rate of packets between uml01 and uml03.
Mobile Computing and Communications Review, Volume 1, Number 2
Figure 4: The network visualisation window with packets flowing between virtual machines. Packets representation allows the operator to see where the traffic is flowing in the network.
VII. Example: name resolution in adhoc network
In a recent IETF draft [6], an extension to the AODV
protocol was proposed to resolve name in multihop
ad-hoc networks. As an example, we used our emulation framework to develop and test a prototype implementation of this draft in a short time frame.
To simplify our implementation, we rely modify an
existing DNS proxy [1]. The proposed name resolution protocol has been added to the NIST implementation of the AODV protocol [9] which runs as a kernel
module in Linux. Fig. 5 illustrates the different stages
needed to resolve a name in a multi-hop wireless environment, as described in the name resolution draft
mentioned above. Fig. 5 shows three machines called
uml01.air, uml02.air and uml03.air.
Whenever an application wants to resolve a name
(1), a DNS request is sent to a DNS server (2). In
our implementation, each machine runs a DNS proxy
which translates this request into an AODV route request query (RREQ) to which a Name Request extension (NREQ EXT) is appended (3).
Using virtualized kernel allows to test the system
with unmodified applications and actual DNS queries
which gives a strong confidence that the system would
work well in a real scenario.
The RREQ is forwarded (4,5) by intermediary
nodes because it contains a request for an address
255.255.255.255 that may not be in the routing table. The RREQ eventually reaches the node whose
name matches the query in the NREQ EXT (6). A
Route Reply (RREP) with a Name Reply extension
(NREP EXT) is sent back to the originator of the
query (7). The proxy finally forges a DNS reply to
answer the DNS query.
This draft implementation has been developed using UML and tested successfully with our emulator in a network made of 10 nodes. Although this
implementation could have been tested with existing
UML Ethernet interfaces, the wireless environment
has some unique characteristics such as unidirectional
links and multiple broadcast domains on a single interface. Furthermore, machines may be moved and
the topology may be changed easily by simply moving the nodes with the mouse. This can be done, for
instance, while a TCP connection is active between
two machines and the effect on the connectivity may
be appreciated in real time. More accurate measurement may of course be obtained using packet capture
programs. Profiling tools such as gprof may also be
used to profile the execution of the driver and of the
implemented protocol.
Mobile Computing and Communications Review, Volume 1, Number 2
5
8
dns reply
2
DNS protocol
AODV protocol
dns query
DNS
PROXY
app.
lo
uml01.air
6 this is my name
1 ping uml03.air
7
RREP
NREP_EXT
uml03.air
uml02.air
4 not my name
3
RREQ
NREQ_EXT
5
RREQ
NREQ_EXT
Figure 5: Resolution of a name in a multi-hop wireless network.
VIII. Related work
When the performance or stability of new protocols
and mechanisms must be evaluated, researchers usually rely on three types of tools : mathematical models, simulations and implementation. Mathematical
models are usually very abstract and allow to understand the key parameters of the proposed mechanism.
When mathematical models are not possible or too
complex, researchers often use simulators such as NS2 [3], J-Sim [14], OPnet [2] or others. All those simulators contain simulation models of the main protocols
and mechanism and can be easily enhanced to support
new protocols. The advantage of testing a new idea in
a simulator is that it can be evaluated in a completely
controlled environment. However, as simulations are
based on models, they do not always take into account
the complexity of adding a new mechanism or protocol to an existing implementation. Developing an implementation is often very complex and time consuming. Furthermore, deploying the proposed solution in
a real network requires a lot of infrastructure which is
not always available.
The emulator proposed in this paper tries to reduce
the gap between simulations and implementations by
allowing a developer to test his/her implementation in
an almost real, but entirely software-based environment. The cost is thus limited to one or a few workstations for large networks.
To our knowledge, the emulator that we propose is
6
the only software-based solution that allows to support wireless and mobile nodes in a virtual environment. Several types of emulators have been proposed
in the literature. In [13], Keshav et al. virtualised the
networking stack of a FreeBSD kernel and allowed
to run routing protocols and other networking applications in an emulated environment. Another similar
approach is IMUNES proposed in [15]. IMUNES allows a FreeBSD kernel to maintain several networking stacks that are used to support different applications. However, those two solutions do not support
wireless interfaces.
In [8], Judd and Steenkiste propose a hardwarebased emulator to reproduce the characteristics
of wireless links.
Compared to our emulator, this solution provides a more accurate emulation of the physical layer, but its cost is much
higher. Another approach is the emulab testbed
(http://www.emulab.net) that was recently upgraded with several wireless nodes. As mentioned
in the introduction, VNUML can be used to simulate
wireless ad-hoc networks. However, this approach is
less flexible as it requires an explicit description of
the topology. Furthermore, communication links are
bidirectional and symmetrical. In contrast, our physical model allows for a more realistic description of
the transmission medium.
Mobile Computing and Communications Review, Volume 1, Number 2
IX.
Conclusion and future work
In this paper, we have described a UML-based emulator for wireless networks. In our emulator, each emulated node is implemented as a UML Linux kernel.
The wireless interfaces of all the emulated nodes are
implemented by using modified hostap driver that
interact via TCP connections with the physical layer
emulation. By changing the physical layer emulation,
it is possible to test various scenarios for packet losses
or bit errors. Finally, a visualisation window allows to
track in realtime the behaviour of the emulator.
We have shown how the emulator could be used
as a simple testbed to implement extensions to existing protocols. In practice, our emulator is not limited
to such usages. It could be used as a testbed to implement new routing or transport protocols in a completely controlled and low-cost environment. It could
also be very useful as a teaching aid for students willing to better understand the behaviour of protocols in
wireless networks.
[10] Jouni Malinen. Host ap driver for intersil
prism2/2.5/3,http://hostap.epitest.fi/.
[11] Luca Mottola.
How to simulate mobile ad hoc networks using vnuml,
http://jungla.dit.upm.es/ vnuml/doc/1.3/examples/.
[12] J.G. Proakis. Digital communication. McGraw
Hill, fourth edition, 2001.
[13] X.W.Huang, R. Sharma, and S. Keshav. The
entrapid protocol development environment. In
Proc. of Infocom, March 1999.
[14] Hung ying Tyan. Design, realization and evaluation of a component-based compositional software architecture for network simulation. PhD
thesis, Ohio State University, 2002.
[15] Marko Zec. Implementing a clonable network
stack in the freebsd kernel,. In Proceedings
of the 2003. USENIX Annual Technical Conference, San Antonio, Texas, June 2003.
References
[1] Dnrd - domain name relay daemon.
http://users.zoominternet.net/%7Egarsh/dnrd/.
[2] Opnet simulator, http://www.opnet.com.
[3] Ucb/lbl/vint, network simulator
2),http://www.isi.edu/nsnam/ns/.
(version
[4] D. P. Bovet and M. Cesati. Understanding the
Linux Kernel. O’Reilly, 2001.
[5] J. Dike. User mode linux. In 5th Annual Linux
Showcase & conf., Oakland CA, 2001.
[6] Paal E. Engelstad and al. Name resolution in ondemand manets and over external ip networks,
internet draft, January 2004.
[7] D. Fernandez, Tomas de Miguel, and F. Galan.
Study and emulation of ipv6 internet-exchangebased addressing models. IEEE Communication
Magazine, pages 105–112, January 2004.
[8] Glenn Judd and Peter Steenkiste. Repeatable and
realistic wireless experimentation through physical emulation. SIGCOMM Comput. Commun.
Rev., 34(1):63–68, 2004.
[9] Luke Klein-Berndt.
Kernel aodv,national
institue of standards and technology ,
http://w3.antd.nist.gov/wctg/aodv kernel/.
Mobile Computing and Communications Review, Volume 1, Number 2
7