Simulator requirements
- a comparative evaluation of tools
Editor
Henrik Christiansen
COM, Technical University of Denmark
Contributors:
Gerben Kuijpers, Ericsson Telebit
Hiroyuki Yomo, WING, Aalborg University
Hanane Fathi, WING, Aalborg University
Version 2.1
September 8th, 2003
Copyright © 2003 CNTK, All rights reserved.
Executive summary
This document has compared a total of ?? simulation tools….
Version 2.1, September 8th 2003
Page 2 of 52
Table of Contents
1.
Introduction 8
1.1. Document overview ................................................................................................... 8
2.
Simulation tools in general ................................................................................................ 9
2.1. A classification of simulation tools............................................................................ 9
2.2. The tasks of the simulation process ........................................................................... 9
2.3. Using general-purpose languages .............................................................................. 9
2.4. Using a general-purpose simulation tools ................................................................ 10
2.5. Special-purpose simulation tools ............................................................................. 10
2.6. Comments ................................................................................................................ 10
3.
List of requirements / evaluation criteria ......................................................................... 12
3.1. Goals / overall requirements .................................................................................... 12
3.2. Functional Requirements ......................................................................................... 12
3.3. Statistics generation ................................................................................................. 13
3.4. Performance/Scalability ........................................................................................... 13
3.5. Extensibility ............................................................................................................. 13
3.6. Usability ................................................................................................................... 14
3.7. Cost / Availability .................................................................................................... 14
3.8. Verifiability / Correctness ........................................................................................ 14
4.
Description of existing tools ............................................................................................ 15
4.1. Description / comparison procedure ........................................................................ 15
4.2. Candidates ................................................................................................................ 15
4.3. WIPSIM ................................................................................................................... 16
4.3.1.
Overall description ........................................................................................... 16
4.3.2.
Pros and cons ................................................................................................... 18
4.4. OPNET..................................................................................................................... 19
4.4.1.
Overall description ........................................................................................... 19
4.4.2.
Pros and cons ................................................................................................... 20
4.5. NS2 .......................................................................................................................... 21
4.5.1.
Overall description ........................................................................................... 21
4.5.2.
Pros and cons ................................................................................................... 21
4.6. GlomoSim ................................................................................................................ 22
4.6.1.
Overall description ........................................................................................... 22
4.6.2.
Pros and cons ................................................................................................... 22
5.
Comparison 23
5.1. Candidates ................................................................................................................ 23
5.2. Functional Requirements ......................................................................................... 23
5.2.1.
WIPSIM ........................................................................................................... 23
5.2.2.
OPNET............................................................................................................. 25
5.2.3.
NS2 .................................................................................................................. 26
5.2.4.
Glomosim......................................................................................................... 27
5.2.5.
A new tool ........................................................................................................ 27
5.2.6.
A combinations of a number of existing tools ................................................. 27
5.2.7.
Summary .......................................................................................................... 27
5.2.8.
List of protocols supported .............................................................................. 29
5.2.9.
List of configurable parameters ....................................................................... 30
5.3. Statistics generation ................................................................................................. 31
Version 2.1, September 8th 2003
Page 3 of 52
5.3.1.
WIPSIM ........................................................................................................... 31
5.3.2.
OPNET............................................................................................................. 32
5.3.3.
NS2 .................................................................................................................. 32
5.3.4.
Glomosim......................................................................................................... 33
5.3.5.
A new tool ........................................................................................................ 33
5.3.6.
A combinations of a number of existing tools ................................................. 33
5.3.7.
Summary .......................................................................................................... 33
5.4. Performance/Scalability ........................................................................................... 34
5.4.1.
WIPSIM ........................................................................................................... 34
5.4.2.
OPNET............................................................................................................. 35
5.4.3.
NS2 .................................................................................................................. 35
5.4.4.
Glomosim......................................................................................................... 36
5.4.5.
A new tool ........................................................................................................ 36
5.4.6.
A combinations of a number of existing tools ................................................. 36
5.4.7.
Summary .......................................................................................................... 36
5.5. Extensibility ............................................................................................................. 37
5.5.1.
WIPSIM ........................................................................................................... 37
5.5.2.
OPNET............................................................................................................. 38
5.5.3.
NS2 .................................................................................................................. 38
5.5.4.
GloMoSim........................................................................................................ 39
5.5.5.
A new tool ........................................................................................................ 39
5.5.6.
A combinations of a number of existing tools ................................................. 39
5.5.7.
Summary .......................................................................................................... 39
5.6. Usability ................................................................................................................... 40
5.6.1.
WIPSIM ........................................................................................................... 40
5.6.2.
OPNET............................................................................................................. 41
5.6.3.
NS2 .................................................................................................................. 41
5.6.4.
GlomoSIm ........................................................................................................ 42
5.6.5.
A new tool ........................................................................................................ 42
5.6.6.
A combinations of a number of existing tools ................................................. 42
5.6.7.
Summary .......................................................................................................... 42
5.7. Cost / Availability .................................................................................................... 43
5.7.1.
WIPSIM ........................................................................................................... 43
5.7.2.
OPNET............................................................................................................. 43
5.7.3.
NS2 .................................................................................................................. 43
5.7.4.
Glomosim......................................................................................................... 43
5.7.5.
A new tool ........................................................................................................ 43
5.7.6.
A combinations of a number of existing tools ................................................. 43
5.7.7.
Summary .......................................................................................................... 43
5.8. Verifiability / Correctness ........................................................................................ 44
5.8.1.
WIPSIM ........................................................................................................... 44
5.8.2.
OPNET............................................................................................................. 44
5.8.3.
NS2 .................................................................................................................. 44
5.8.4.
Glomosim......................................................................................................... 44
5.8.5.
A new tool ........................................................................................................ 44
5.8.6.
A combinations of a number of existing tools ................................................. 44
5.8.7.
Summary .......................................................................................................... 44
6.
Conclusion 46
7.
References 47
Version 2.1, September 8th 2003
Page 4 of 52
Appendix A.
Simulation tools under evaluation ............................................................... 48
Appendix B.
Tool requirements ........................................................................................ 50
Version 2.1, September 8th 2003
Page 5 of 52
Document history
Version Date
Author
Comments
0,1
06-08-2003
Henrik Christiansen
Document created…
0,3
12-08-2003
Henrik Christiansen
Added contribution on general properties of simulation tools
1.0
19-08-2003
Henrik Christiansen
Added list of contributors + minor
changes
1.5
23-08-2003
Henrik Christiansen
Simulator requirements merged with
requirements from the “use cases”
document. Document changed as
agreed upon at the meeting August
22th.
2.0
05-09-2003
Henrik Christiansen
Gerben Kuijpers
Hiroyuki Yomo
Hanane Fathi
Contributions from all authors merged
into one document.
2.1
08-09-2003
Henrik Christiansen
Minor improvements and corrections
2.2
15-09-2003
Henrik Christiansen
Document finalized
Version 2.1, September 8th 2003
Page 6 of 52
Abbreviations and acronyms
Acronym
Meaning
ATM
Asynchronous Transfer Mode
CNTK
Center for Netværks TjenesteKonvergens
MIT
NS2
Network Simulator 2
OLSR
Optimized Link State Routing protocol
OPNET
Optimized Performance Network Engineering Tool
WIPSIM
Wireless IP SIMulator
AODV
Ad hoc On-demand Distance Vector routing protocol
Version 2.1, September 8th 2003
Page 7 of 52
1. Introduction
Within the CNTK framework there has been identified a need for a common simulation tool.
[Cntk1]
1.1. Document overview
The structure of the document is as follows: firstly a set of requirements is set up. Secondly, a
number of existing tools are described. Thirdly, a comparison of these tools is made. Finally,
the conclusion and a list of references are given.
Version 2.1, September 8th 2003
Page 8 of 52
2. Simulation tools in general
Contributors: Henrik Christiansen (COM, Denmark), Gérard Hébuterne (INT, France)
This section gives an overview of some general properties of simulation tools and tries to categorize a number of existing tools based on their properties.
2.1. A classification of simulation tools
By “simulation tool” it is meant today not only the basic tool which runs the simulation model, but also all related utility programs attached to any simulation project, such as graphical
aids and development tools – these ones becoming of growing importance. Comparing simulation tools is to some extend impossible as they have different goals, different fields of application, offer different tools and possibly address different populations of users.
A first step towards the classification is in the nature of the language the simulation makes
use of. To better explain it, and to understand the implication of the choice, one has to think
about the tasks involved in running a simulated model of any system.
2.2. The tasks of the simulation process
Naturally, the system and the way it works have to be described. Each building block of the
whole system may be described with a variable level of detail, which depends first on the
precision the study is to provide (think of a system involving communication between subsystems: the description may choose to incorporate the details of the communication protocol, or may adopt a more global view). But this depends too on the basic tools the language
provides. For instance, the language may provide some capability as “put object X in queue”;
“extract an object from queue”, etc. Or the complete set of operations corresponding to queuing, etc., has to be described.
A second set of tasks to be taken account of is the whole set of operations that must be performed in the framework of any simulation study. The simplest example is “draw a random
variable according to a given probability distribution”. Such variables represent the duration
of a task, the time interval between events such as node failures, etc. These operations (draw
random variables, manage the event list, perform basic statistics,…) are repeated for each
simulation experiment.
2.3. Using general-purpose languages
First, the development of the simulation study may be done using general-purpose languages
– such as C, C++, Fortran, etc. As they are not oriented towards simulation tasks, they offer
no help for that goal: the developer has to perform the whole set of actions described above:
whole description of the system in its finest details, and description of all “simulationrelated” tasks. As it allows building a simulation program perfectly tailored to the needs of
the study, the product obtained will have the highest possible performance level (e.g. in terms
of consumed run time). Usually, the choice of this approach is motivated by the need of extensive use of a program that would be otherwise prohibitively slow. However, the gain in the
exploitation phase is balanced by an increased effort in the development phase (both in terms
of analyzing the model, of coding it, and last but not least, of debugging). The task of simulation development consists often in building (or assembling) a library of basic routines, with
which the final package is constituted. One finds in the literature numerous examples of such
libraries – see e.g. [Feldman].
Version 2.1, September 8th 2003
Page 9 of 52
2.4. Using a general-purpose simulation tools
Here, the developer makes use of a simulation language, which is a computer language aiming at easing to describe the model, by providing a high-level instruction set by which the
system is much more easily described than using general-purpose languages. Typical instructions allow to enter or extract “customers” from queues, to choose a service discipline, to
synchronize tasks, to draw random variables, etc.
First simulation tools were proposed in the 60’s, and look like general-purpose languages of
the same period. Examples of such tools are Simula, GPSS, Simscript (but many others have
been elaborated). Some of them have had a quite long career and have been continuously improved. However, most of today’s users prefer tools of the following generation, characterized by a more or less sophisticated graphical interface. The simulation model may be built
from the interface – through a few “mouse clicks”, and the tool often provides utilities to visualize the results, and even to produce the final report. The trend is also to provide a larger
and larger library containing built-in sub-models. OPNET is perhaps the most widely known
example of this category, but many others exist (Bones, SES-Workbench, see next section).
In fact, the difference between these two categories tends to vanish. First, even if using a
“genuine graphic” simulation tool, the study of any elaborated model (in fact, any model of
real size, apart from toy cases) asks the developer to “open” the basic building blocks and to
write down pieces of code (most frequently using C, or C++). Second, most of the languages
in text form of the 60’s, which are still in use, have been greatly improved and provide most
of the functionality as true graphic tools. This is especially the case for Simscript II.5 (the latest version of the popular language), but other ones have evolved the same way.
2.5. Special-purpose simulation tools
While general-purpose simulation languages are not specific of an application, the third category offers languages through which the user simply describes the system by specifying the
topology, the kind of equipment, the numerical figures of traffics, etc. The simulator is tailored to the study of a quite specific application, such as a data network, and is of no help
outside of this application. The effort of development is minimum and restricted to the definition of the simulation experiment and the analysis of the results. Examples of such tools are
COMNET, SIMFACTORY (simulation of manufacturing applications), NETWORK II.5
(from CACI), etc. These tools are sometimes referred to as simulators (as opposed to simulation languages of the previous section) – see e.g. [Law, Kelton]
However, such a tool is of limited help, in that it can only be used for studying existing and
well-documented technologies. It is thus poorly suited as soon as new equipment, new protocols, new networking paradigms are concerned. Rather, its field of application is to be found
on pure network planning and dimensioning, in the operational phase of the technology. In
the domain of networking, COMNET is a typical example of this category.
2.6. Comments
The above classification appears rather “rough”. The frontier between general-purpose and
special-purpose simulation tools is somehow fuzzy. For instance, one may build network
models using OPNET in a way much like a special-purpose language (using the specific libraries it provides), nevertheless it has to be seen as a general-purpose simulation tool, as
new network devices may be freely developed.
However, the classification emphasizes a major difference between languages.
Version 2.1, September 8th 2003
Page 10 of 52
There is however a class of special-purpose tools which may be of more general help: free
software tools have been devised, mostly by U.S. universities. Examples are NETSIM (MIT),
NIST (for ATM networks, and based upon the previous one), INSANE (Berkeley), NS (project VINT), etc. They offer the user the possibility to develop new modules or alter the code
already produced, allowing thus to enlarge the scope of the tool. NS is probably the most
known example of this class.
Other classifications could be proposed, e.g. emphasizing the technical aspects of the simulation kernel. For instance, some tools use parallel simulation. Other tools are presented as
based on an “object oriented” approach – but this is not, however, a sound criterion, as most
simulation languages use these concepts naturally.
Version 2.1, September 8th 2003
Page 11 of 52
3. List of requirements / evaluation criteria
This chapter defines a set of criteria, which will be used when comparing different simulation
tools. The criteria list is an extended version of the requirement list set up in [cntk1]. Requirements from [cntk1] are not repeated here, but directly referenced as Rx referring to the
tags from chapter 5 in [cntk1] (requirements are also listed in appendix B of this document).
The requirements from [cntk1] have been subdivided into a number of categories, namely
(some requirements appear in more than one category):

Performance evaluation capabilities (R1, R12)

Modeling capabilities (R2, R3, R6, R13, R14, R17, R18, R19, R20, R21)

User interface

–
Configuration: (R5, R7, R11, R12, R17, R22)
–
Presentation: (R15)
General properties of the tool: (R8, R9, R10, R17, R23)
The total list of criteria has been subdivided into groups, which are: functional requirements,
statistics generation, performance/scalability, extensibility, Usability, cost / availability and
correctness / Verifiability. A more precise definition of these requirements are given hereafter..
3.1. Goals / overall requirements

TIP integration

Evaluation of ad-hoc routing protocols under constraints of devices' power supply;

(2nd priority) QoS in such scenarios
3.2. Functional Requirements
Definition:
This criterion is related to the specific use of the simulation tool in the CNTK project.
More elaborated list of requirements:

Performance evaluation capabilities (R1, R12)

Modelling capabilities (R2, R3, R6, R13, R14, R17, R18, R19, R20, R21)

ad-hoc connectivity

mobility of nodes (mobility model)

L1: abstract channel model (e.g. described by bit error patterns, throughput, delay,
depending on # active sources in some geographical region)

Model of Power consumption for data transmission

L2: Complete model of MAC behavior; at least for 802.11 & Bluetooth

L3: support of as many different routing protocols as possible (mainly for ad-hoc
routing but also 'traditional' protocols, e.g. OSPF, for comparison); extensibility for
self-developed protocols/protocol modifications (see section 3.3)
2nd priority:
Version 2.1, September 8th 2003
Page 12 of 52

DiffServ support (scheduling/marking)

L4-7: Traffic models:
–
CBR (UDP based)
–
Bursty ON/OFF (UDP based)

Extensibility (see section 3.3)

TCP traffic models, traffic generated by actual code of sensor applications

Support for node-failure modeling

Support for modeling of power-consumption due to processing in nodes

Support for modeling processing times
3.3. Statistics generation
Definition:
The ability of the tool to do measurements on the models while the simulation is running and
present them afterwards.
More elaborated list of requirements:

User interface, Presentation: (R15)

Dropped packets, packet delay/jitter, throughput

Statistics for protocol overhead

Fairness

Allow for on-line/integrated computation of statistics

Simulation run-time
3.4. Performance/Scalability
Definition:
The performance of the simulator impacts the relation between the real-time (the time the
computer uses to do the simulation) and the simulated time (the timeframe that are being
modeled). The scalability is the impact the size of the model has on the performance.
More elaborated list of requirements:

Real-time simulation of 200-500 nodes under high traffic load

Support of distributed simulation
3.5. Extensibility
Definition:
Extensibility is a measure of whether a tool is built so that it can be expanded with e.g. new
features by the user. An important factor here is the amount of work / time needed to extend
the tool.
More elaborated list of requirements:

General properties of the tool: (R8, R9, R10, R17, R23)

Integration of TIP stack in simulator
Version 2.1, September 8th 2003
Page 13 of 52

Integration of new routing protocols

Definition of new mobility models

Processing in real-time

Definition of new traffic models (including possibly the use of real sensor applications for traffic generation)

Definition of measurement procedures for new statistics

Good development/source-code documentation

Simulator source code including compilation environment needs to be available
3.6. Usability
Definition:
The “user-friendlyness” of a system. Is it easy to use for an experienced / inexperienced user? How it the daily workwith the tool and how easy is it to learn.
More elaborated list of requirements:

User interface
–
Configuration: (R5, R7, R11, R12, R17, R22)
–
Presentation: (R15)

Good documentation of the simulator (users' manual)

Short 'learning period' for new users

existing experience with simulator within CNTK
3.7. Cost / Availability
Definition:
The cost of the tool is simply the price of acquiring the product. Availability is whether the
tool is available to the general public, to CNTK project members only or limited to employees
of a certain company.
More elaborated list of requirements:

Either costs for simulator (source code + compilation environment) as low as possible, or simulator already available for all CNTK partners
3.8. Verifiability / Correctness
Definition:
The correctness of a tool defines how confident one can be that a given implementation of,
say, a protocol has been proven to conform to e.g., the standards or otherwise works correctly. Verifiability is the ability of the user to, by using the tool, test whether a new protocols
implementation works as it is supposed to.
More elaborated list of requirements:

used simulator features should be known 'to work correctly'

results should be reproducible by other scientific organizations
Version 2.1, September 8th 2003
Page 14 of 52
4. Description of existing tools
This document is not the only one trying to compare different simulation tools - a number of
comparisons of existing tools have been carried out [Bragg2000, Ince2002].
4.1. Description / comparison procedure
Each new tool under consideration must undergo the following screening procedure:
1. Answer the following question: “is it possible to extend the tool with new protocols?”
If no then stop evaluating this tool, otherwise continue.
2. Answer the following question: “is it possible to extend the tool with new statistics?”
If no then stop evaluating this tool, otherwise continue.
3. Answer the following question: “is the tool able to carry out simulations on large
network models?” If no then stop evaluating this tool, otherwise continue.
4. Create a list of built-in protocols for this tool –
a. Is the number of built-in protocols enough for our interest? If no stop evaluating this tool, otherwise continue
b. if the list is huge the limit to protocols relevant to this project. Integrate the
list into the appropriate table in the summary part of section 5.2
5. Create a list of configurable parameters for this tool – might be limited to parameters
relevant to the list of protocols created in step 4. Integrate the list into the appropriate
table in the summary part of section 5.2
6. Generate a sub-section for section four of this document (i.e. section 4.x) giving an
overall description of the tool. The description must begin with the name of the contributor and a definition of his / her experience with the tool. At the end of the section
a summary of the tool’s pro and cons should be given.
7. Generate a sub-section for each criteria treated in section five of this document (i.e.
section 5.x.y, where y is a unique number for the tool under study). In each subsection it must be specified whether the requirements from section 3 are met or not.
In addition the comparison table in the summary part of each sub-section must be
filled out.
4.2. Candidates
The following tools will be evaluated:

WIPSIM [WIPSIM]

OPNET [OPNET]

NS2 [NS2]

Extensible and High-Fidelity TCPIP Network Simulator [Exten]

MPLS Network Simulator [MPLSSim]

Bluehoc [Bluehoc]

CDMA Wireless Network Simulator [CDMASim]

GloMoSim [GLOMOSIM]
Version 2.1, September 8th 2003
Page 15 of 52

QualNet [QUALNET]

CNET [CNET]

Real [REAL]

NetSIM [NetSIM]

FLAN [FLAN]

NCTUns [NCTUns]

SimMan 1.0 [SimMAN]

VENUS [VENUS]

AnSIM [AnSIM]

NIST [NIST]

INSANE [INSANE]
The initial screening procedure as outlined in section 4.1 yields the following set of simulation tool for further comparison, the complete outcome of the screening can be seen in appendix 1:

WIPSIM

OPNET

NS2

GloMoSim

…
4.3. WIPSIM
Contributor: Gerben Kuijpers; Experience with the tool: ???
4.3.1. Overall description
Development of the Wireless IP Simulator (WIPSIM) was started in 2000 in the WINGgroup of CPK at Aalborg University. In July 2001, the framework of the simulator was redesigned to allow for easy addition of protocols and mechanisms by multiple developers. Several reasons to start writing a new simulator instead of using an existing tool:

No (affordable) simulator available that could fulfill the needs of the WING-group

Increased learning effect by implementing protocols from scratch

Create tool that could aid in education
WIPSIM is an event-driven simulator, written in ANSI-C++ and is open source. Input- and
output is handled using text-files:

Scenario-file: movement of the nodes and the traffic in the network

Network-file: nodes, interface, protocols and their parameters

Config-file: default parameters for protocols, aliases
Version 2.1, September 8th 2003
Page 16 of 52
config file
output file
network file
scenario file
WIPSIM
Figure 1Input- and output of WIPSIM

Output-file: selected simulation events
The framework of the simulator connects the different layers of the protocol stack. Each part
of the framework is implemented as a base-class, which provides the interface for access to
other parts of the simulator and to other layers. Each specific protocol implementation can
then be derived from this base-class. The framework for the protocol layers for a single node
is shown in figure 2.
Version 2.1, September 8th 2003
Page 17 of 52
A p p lic a tio n
P in g
C B R s o u rc e
Socket M anager
Socket
N e t w o r k l a y e r - IP v 6
R o u tin g
p ro to c o l
L o n g e s t-p re fix m a tc h in g
AODV
In te rfa c e
D a ta l i n k l a y e r
B lu e to o th
8 0 2 .2
M A C la y e r
B lu e to o th
IS M A
8 0 2 .11 D C F
M e d iu m la y e r
B lu e to o th
IS M A
8 0 2 .11 D C F
Figure 2Structure of a node in WIPSIM
At the physical layer, interfaces of nodes are either in-range or out-of-range of each other.
Currently, no detailed radio propagation models are used, a frame transmitted at the physical
layer either reaches other nodes (if the interface of that node is in-range) or not (if the interface is not in-range). Optionally, a frame error rate can be set. The radio propagation is kept
this simple to reduce simulation overhead and because the focus of the simulator is the IP and
MAC layer.
Movement of nodes is described in the scenario-file, where event are listed where nodes
move in- and out-of-range of each other. Any tool can be used to provide the input for the
movement of nodes to the scenario-file based on the desired mobility model. As such, mobility is pre-determined before the simulation starts. This allows for comparison of different protocols under the exact same mobility conditions. Moreover, it reduces the amount of calculations that needs to be done in case a number of simulations are carried out with the same mobility.
4.3.2. Pros and cons
Main pros of WIPSIM:

Development knowledge available within CNTK

Simulator + source code freely available
Version 2.1, September 8th 2003
Page 18 of 52

Design of the general framework of the simulator makes it easy to add new protocols/mechanisms.

Source code is well documented
Main cons of WIPSIM:

User/developer base is still small, therefore there is some uncertainty about the correctness of the implementation of protocols.

The number of implemented protocols is not so large yet.

Development and user documentation is still under development.
4.4. OPNET
Contributor: Henrik Christiansen; Experience with the tool: app. four years.
4.4.1. Overall description
OPNET modeler is a commercially available product from OPNET technologies [OPNET].
OPNET is a GUI with a (high) number of various editors for creating/modifying/verifying
models and for running simulations and displaying/analyzing results. OPNET runs on top of a
C compiler. Models in OPNET are built in a hierarchical fashion. Models can be built either
top-down or bottom-up and each level represents the internal structure and functionality of
the level above. The levels are:

Network level (not related to OSI layer 3!)
Modeling of network topologies and overall configuration takes place at this level of
modeling. Network elements such as communication links and node devices are used
to build the model. In addition, node/link failure/recovery can be modeled.

Node level
At this level the internal structure of network level devices are modeled. Elements
used for modeling includes: generic processor modules, queue modules, receivers and
transmitters. These are interconnected by streams or statistic wires.

Process level
Node level device functionality are modeled at the process level. By means of finitestate-machines (FSMs) any functionality can be modeled quite efficiently.

Proto-C level
The lowermost modeling level is called the proto-C level. Proto-C is an extension of
the C (or C++, depending on the underlying compiler) programming language. A
large number of kernel procedures are available. All built-in models are available at
this level, i.e., as source code.
Version 2.1, September 8th 2003
Page 19 of 52
Figure 3: The hierarchical structure of OPNET models
Models can be edited at each level. This allows modeling every single detail from layer 1 to
layer 7 and on top of that, configuration and simulation sequences.
A vast number of protocol models as well as device models are available. These can be used
as is or be used as a basis for model development.
Simulation is carried out from the GUI or from the command line. Before running s simulation, the desired statistics to be collected are selected. During simulation the statistics are
written to files – either scalar files or vector files depending on the type of data.
The simulation kernel itself is extremely efficient – the new version 10.0 of the product has
been optimized and runs on multi-processor computers.
4.4.2. Pros and cons
Main pros of OPNET

Large customer base.

Professional support

Very well documented

Ships with a large number of built-in protocols
Main cons of OPNET

Relatively high price – but cheap for universities…

Complex, takes time to learn
Version 2.1, September 8th 2003
Page 20 of 52
4.5. NS2
Contributors: Hiroyuki Yomo (HY), Hanane Fathi (HF).
4.5.1. Overall description
Ns is a discrete event simulator targeted at networking research provided by USC/ISI [NS2].
It models system as events, which the simulator has, list of. The process is made such way:
“take next one, run it, until done”. Each event happens in an instant of virtual (simulated)
time, but takes an arbitrary amount of real time. The design of the simulator is separating the
“data” from the control: C++ for “data” (per packet processing, core of ns, fast to run, detailed, complete control); and OTcl for control (simulation scenario configurations, periodic
or triggered action, manipulating existing C++ objects, fast to write and change).
C++
C++/OTcl
split objects
otcl
“Ns” Components are Ns the simulator itself and Nam the network animator that permits to
visualize ns output and that provides a GUI interface to generate ns scripts.
Ns focuses on modelling network protocols:
 Wired, wireless, satellite networks,
 TCP, UDP, multicast, unicast,
 Web, telnet, ftp,
 Ad hoc routing, sensor networks,
 Infrastructure: statistics, tracing, error models, etc…
Ns functionalities comprise routing, transportation (TCP and UDP), traffic sources (web, ftp,
telnet…), queuing disciplines, QoS (IntServ and Diffserv) and emulation for the wired systems. For the wireless part, ns provides ad hoc routing and mobile IP. Ns provides also traffic
and topology generators and simple trace analysis.
4.5.2. Pros and cons
Pros

Easy-configurable and fast simulation by using two different programming languages (OTcl and C++)

Many protocols already implemented

It has been well-known
Version 2.1, September 8th 2003
Page 21 of 52

Well-documented manual

Big user-groups

Open-source code
Cons

Long time to get used to using it

Badly documented source code

Difficulty to do quick evaluation of small idea (you have to know all the simulation
structures even if you want to simulate a part of the protocol stacks)
4.6. GlomoSim
Contributors: Hiroyuki Yomo (HY), Hanane Fathi (HF).
4.6.1. Overall description
In GloMoSim [GLOMOSIM], a scalable simulation environment for wireless and wired network systems is built. It is being designed using the parallel discrete-event simulation capability provided by Parsec (http://pcl.cs.ucla.edu/projects/parsec/). GloMoSim currently supports protocols for a purely wireless network. In the future, it is planned to add functionality
to simulate a wired as well as a hybrid network with both wired and wireless capabilities.
Most network systems are currently built using a layered approach that is similar to the OSI
seven layer network architecture. The plan is to build GloMoSim using a similar layered approach. Standard APIs will be used between the different simulation layers. This will allow
the rapid integration of models developed at different layers by different people.
To run GloMoSim, the latest Parsec compiler is required (now included with the GloMoSim
distribution). If we want to develop our protocols in GloMoSim, we should have some familiarity with Parsec, but we don't need to be an expert. Most protocol developers will be writing
purely C code with some Parsec functions for time management. Parsec code is used extensively in the GloMoSim kernel. Most users do not need to know how the kernel works. If we
are interested in the GloMoSim kernel, we need to have extensive knowledge about Parsec.
4.6.2. Pros and cons
Version 2.1, September 8th 2003
Page 22 of 52
5. Comparison
5.1. Candidates
Existing tools (passing the screening test)

WIPSIM

OPNET

NS2

GloMoSim
Other options

Development of a new tool from scratch

A combination of a number of existing tools
5.2. Functional Requirements
5.2.1. WIPSIM
5.2.1.1. Performance evaluation capabilities
To evaluate the performance of simulated networks, it is possible to select a wide variety of
events that are recorded in the output file (.snf-file). Any tool can then be used to process the
output file and calculate all the desired performance metrics. The simulator also includes an
analyzer module, which currently calculates statistics such as average packet delay, throughput and packet delivery ratios. It is very easy to extend the simulator with more recorded
events and to add statistics for the built-in analyzer.
5.2.1.2. Modeling capabilities
The focus of the simulator is on layer 2 and 3 (MAC and IP layer), which are modeled in
considerable detail. A simple model is used for the physical layer.
5.2.1.3. Ad hoc connectivity
WIPSIM includes ad hoc routing protocols and thus provides ad hoc connectivity.
5.2.1.4. Mobility of nodes (mobility model)
Mobility of nodes is supported by specifying the times at which nodes move in range and out
of range of each other in the scenario-file (.scn-file). A separate mobility generator tool is
used to generate the movement-statements for the scenario-file according to a certain mobility model. Calculating node mobility before the simulation allows using the exact same node
movement in multiple simulations. This is very important when trying to evaluate the performance between different protocols or extensions to protocols.
5.2.1.5. L1: Abstract channel model
A simple physical layer model is used that includes the bit-rate of the channel and a propagation delay. The bit-rate together with the size of the frames results in the correct throughput
and delay. Furthermore, a frame error rate is modeled at the physical layer.
Version 2.1, September 8th 2003
Page 23 of 52
5.2.1.6. L2: Complete model of MAC behavior; at least for 802.11 & Bluetooth
The following MAC layers are supported:

802.11 Distributed Coordination Function (DCF)

Bluetooth Baseband, L2CAP, BNEP for piconets

ISMA MAC
5.2.1.7.
L3: Support of as many different routing protocols as possible; extensibility for self-developed protocols/protocol modifications
The following routing protocols are currently supported:

Ad hoc On-demand Distance Vector routing protocol (AODV)

Multipath-AODV

Optimized Link State Routing protocol (OLSR)

Simple shortest-path routing (Dijkstra algorithm) for fixed network, where no exchange of routing protocol messages is modeled
Self-developed protocols can be easily added to the simulator.
5.2.1.8. DiffServ Support (scheduling/marking)
DiffServ priority queues are supported.
5.2.1.9. L4-7: Traffic models
WIPSIM contains the following traffic models:

CBR UDP traffic model

MPEG-4 video source model
5.2.1.10. Extensibility
The simulator has been developed with easy extensibility as one of the mayor goals in mind.
Due to the modular framework, new protocols and mechanisms can be easily added.
5.2.1.11. TCP traffic models, traffic generated by actual code of sensor applications
Currently, TCP traffic models are not implemented; neither is the possibility to plug-in real
traffic traces.
5.2.1.12. Support for node-failure modeling
Currently, node-failure is not modeled. However, modeling full node failure (nodes switching
off or at least losing communication abilities) would require only a trivial extension and
could be added in case the need arises.
5.2.1.13. Support for modeling of power-consumption due to processing in
nodes
This is currently not implemented.
Version 2.1, September 8th 2003
Page 24 of 52
5.2.1.14. Support for modeling processing times
WIPSIM models the time it takes for ad hoc network nodes to parse and route (find appropriate next hop) received packets.
5.2.2. OPNET
5.2.2.1. Performance evaluation capabilities
OPNET has strong support for performance evaluation. It is possible to specify sets of simulation and thus sweep over a range of input parameters / traffic. The simulations can either
use the built-in random number generators or one can provide / specify another. In terms of
output analysis, either the built-in features for statistics can be used or all data can be exported to an external analysis tool.
5.2.2.2. Modeling capabilities
All layers of from layer 1 to layer 7 can be modeled. Everything is customizable and configurable.
5.2.2.3. Ad hoc connectivity
supported
5.2.2.4. Mobility of nodes (mobility model)
Node mobility can be fully modeled. A trajectory for each mobile node can be specified or a
node can be set to move randomly.
5.2.2.5. L1: Abstract channel model
Layer 1 properties are modeled as a set of so-called pipeline stages that allows for a detailed
specification of transmission properties. As well wired as wireless media can be modeled.
5.2.2.6. L2: Complete model of MAC behavior; at least for 802.11 & Bluetooth
OPNET comes with a number of MAC protocols for wired and wireless protocols. Currently,
Bluetooth does not ship with OPNET, but a number of user contributed models are available
on their web-site. 802.11 is available as well as UMTS MACs.
5.2.2.7.
L3: Support of as many different routing protocols as possible; extensibility for self-developed protocols/protocol modifications
OPNET ships with support for a number of routing protocols (OSPF, RIP, EIGRP, BGP,
IGRP, DSR, TORA, IS-IS, PNNI). OPNET is fully extensible sp that
5.2.2.8. DiffServ Support (scheduling/marking)
There is a full implementation of Diffserv in OPNET.
5.2.2.9. L4-7: Traffic models
OPNET provides a number of application models. In addition fully customized can be easily
developed.
5.2.2.10. Extensibility
OPNET is inherently extendible because it runs on top of a C-compiler. All provided models
are available as source code, and can thus form a basis for further model development.
Version 2.1, September 8th 2003
Page 25 of 52
5.2.2.11. TCP traffic models, traffic generated by actual code of sensor applications
OPNET has built-in TCP models. Every single detail of TCP is modeled and OPNET comes
with typical parameter setting for most major operating systems so that clien/server behavior
can be modeled.
5.2.2.12. Support for node-failure modeling
OPNET contains full support for modeling of node/link failure/recovery. Nodes / links can be
set to fail / recover at a specified time during simulation.
5.2.2.13. Support for modeling of power-consumption due to processing in
nodes
5.2.2.14. Support for modeling processing times
OPNET has built-in full-featured models for modeling CPU usage and utilization for all
kinds of server architectures.
5.2.3. NS2
5.2.3.1. Performance Evaluation Capabilities
NS2 supports many performance measures such as packet loss ratio and end-to-end delay for
evaluating routing protocols, and also supports some trace for the layer-3 packets to get some
statistics of transferred packets specific to layer 3 (R12). However, it does not include the
measure of the power efficiency or modeling of power consumption models, so in order to
compare the routing protocols from power consumption point of view, further modification is
required (R1).
5.2.3.2. Modeling Capabilities
Ns-2 has many ad hoc routing protocols and data link layer protocols already implemented
(R2, R3, R14). It has some modeling for BER pattern such as multi-state Markov model in
the physical layer (R6). QoS mechanisms such as DiffServ have been already supported
(R13). NS2 supports Network configuration can be changed in each simulation, but not dynamically during the simulation (R17). Not so many packet impairment control has been implemented (R18). The modification of the packet structure and the introduction of the new
packet type into the simulator are easy (R19). The asymmetric link can be supported (R20).
IPv6 has not been completely supported (R21).
5.2.3.3. Ad-hoc connectivity
Ad hoc connectivity is supported by using wireless technologies such as IEEE 802.11 and
Bluetooth..
5.2.3.4. Mobility of nodes (mobility model)
Several mobility models are supported. We can also input pre-generated mobility patterns.
5.2.3.5.
L1: abstract channel model (e.g. described by bit error patterns,
throughput, delay, depending on # active sources in some geographical region)
An arbitrary multi-state Markov models is supported.
Version 2.1, September 8th 2003
Page 26 of 52
5.2.3.6. Model of Power consumption for data transmission
Power consumption model has not been included.
5.2.3.7. L2: Complete model of MAC behavior; at least for 802.11 & Bluetooth
IEEE 802.11 MAC has been partly included, and there is a simulator called BlueHoc that is
the extension of NS2.

L3: support of as many different routing protocols as possible (mainly for ad-hoc
routing but also 'traditional' protocols, e.g. OSPF, for comparison); extensibility for
self-developed protocols/protocol modifications
OSPF is supported. For Ad hoc routing, DSDV, DSR, AODV, TORA, and ZRP have been
implemented.
2nd priority:
5.2.3.8. DiffServ support (scheduling/marking)
DiffServ is supported.
5.2.3.9. L4-7: Traffic models:
CBR (UDP based) is supported.
Bursty ON/OFF (UDP based) is supported.
5.2.3.10. Extensibility (see section 3.3)
5.2.3.11. TCP traffic models, traffic generated by actual code of sensor applications
No.
5.2.3.12. Support for node-failure modeling
No.
5.2.3.13. Support for modeling of power-consumption due to processing in
nodes
No.
5.2.3.14. Support for modeling processing times
No
5.2.4. Glomosim
5.2.5. A new tool
5.2.6. A combinations of a number of existing tools
5.2.7. Summary
Functional requirements
Version 2.1, September 8th 2003
Page 27 of 52
Tool # 8
Tool # 7
Tool # 6
Tool # 5
GloMoSim
NS2
OPNET
WIPSIM
Requirement
Tools
Performance Evalua- XX
tion Capabilities R1
Performance Evalua- XX
tion Capabilities R12
X
Ad hoc connectivity
+
X
Mobility of nodes
+
X
L1: Abstract channel 0
model
X
L2: Complete model +
of MAC behavior; at
least for 802.11 &
Bluetooth
X
L3: Support of as -; +
many different routing protocols as possible;
extensibility
for
self-developed
protocols/protocol
modifications
X
DiffServ
Support +
(scheduling/marking)
L4-7: Traffic models
+
Extensibility
++
TCP traffic models, -traffic generated by
actual code of sensor
applications
X
X
Support for node- -failure modeling
Support for modeling -of
powerconsumption due to
processing in nodes
Version 2.1, September 8th 2003
Page 28 of 52
Support for modeling +
processing times
5.2.8. List of protocols supported
NE = not evaluated
IPv4
X
IPv6
X
X
X
X
X
X
X
TCP
X
IEEE
802.3
CSMA/CD MAC
X
X
X
X
IEEE 802.11 (b)
CSMA/CA MAC,
X
X
X
X
learning bridge
protocol,
NE
spanning
protocol,
NE
X
Bluetooth radio
X
NE
X
Bluetooth Baseband
X
NE
X
Bluetooth LMP
X
NE
X
Bluetooth L2CAP
X
NE
X
RIP
NE
X
OSPF
NE
X
X
X
UDP
X
X
X
X
HTTP
NE
X
X
X
X
X
FTP
NE
X
X
X
X
X
CBR
NE
X
X
Fisheye
NE
NE
X
LARScheme 1
NE
NE
X
ODMRP
NE
NE
X
tree
Tool # 8
GlomoSim
Bluehoc
NCTUns
NS2
OPNET
WIPSIM
Protocol
Tools
X
X
Version 2.1, September 8th 2003
X
X
X
X
Page 29 of 52
WRP
NE
NE
X
Bellman-Ford ??
NE
NE
X
DSR
NE
X
X
MACA
NE
NE
OLSR
X
NE
NE
NE
NE
NE
NE
ISMA
X
NE
NE
NE
NE
NE
NE
Telnet
NE
NE
X
X
X
X
AODV
X
NE
X
X
Multipath
AODV
X
NE
NE
NE
X
X
NE
NE
NE
5.2.9. List of configurable parameters
Scheduling
X
Queuing
X
Protocol
modules
the node
X
X
X
X
to-
X
X
TCP parameters
(maximum congestion window
size,
max
segment
size)
X
Tool # 8
GlomoSim
X
X
in
Mobility
model
Network
pologies
Bluehoc
NCTUns
NS2
OPNET
Parameter
WIPSIM
Tools
Version 2.1, September 8th 2003
X
X
Page 30 of 52
MAC parameters
(max
contention
window size,
payload size)
X
X
Simulation
Time
X
X
Power range
of the nodes
X
Packet reception model
X
Bandwidth of
links
X
X
Propagation
model
X
X
Node
ment
place-
X
X
Traffic source
X
X
Simulated
Area
X
5.3. Statistics generation
5.3.1. WIPSIM
5.3.1.1. User interface, presentation
Input and output is done using text-files. WIPSIM runs as a non-graphical command-line
program. The main reason for this is to maintain very easy portability across different computer platforms.
5.3.1.2. Dropped packets, packet delay/jitter, throughput
All of these statistics can either be found by processing the output-file, or by using the builtin analyzer, which includes these statistics.
5.3.1.3. Statistics for protocol overhead
The protocol overhead can be found by processing the individual events in the output file, or
by using the built-in analyzer.
5.3.1.4. Fairness
Fairness is currently not implemented as a statistic in the analyzer module, but could be added if needed.
Version 2.1, September 8th 2003
Page 31 of 52
5.3.1.5. Allow for on-line/integrated computation of statistics
WIPSIM allows for integrated computation of statistics using its integrated output analyzer
module.
5.3.1.6. Simulation run-time
Depends on system and the specifics of the simulation scenario.
Examples will be added soon.
5.3.2. OPNET
5.3.2.1.
User interface, Presentation
5.3.2.2.
Dropped packets, packet delay/jitter, throughput
5.3.2.3.
Statistics for protocol overhead
5.3.2.4.
Fairness
5.3.2.5.
Allow for on-line/integrated computation of statistics
5.3.2.6.
Simulation run-time
5.3.3. NS2
5.3.3.1. User interface, Presentation
In Post-processing, simple trace analysis can be done often in Awk, Perl, or Tcl. R15 is
supported.
5.3.3.2. Dropped packets, packet delay/jitter, throughput
This can be done by parsing the trace file that is obtained as an output when tracing has
been turned on at the beginning of the simulation. On the Ns2 website, there are tools already implemented downloadable.
5.3.3.3. Statistics for protocol overhead.
Possible.
5.3.3.4.
No
Fairness
5.3.3.5.
No
Allow for on-line/integrated computation of statistics
5.3.3.6.
?
Simulation run-time
Version 2.1, September 8th 2003
Page 32 of 52
5.3.4. Glomosim
5.3.4.1.
User interface, Presentation
5.3.4.2.
Dropped packets, packet delay/jitter, throughput:
5.3.4.3.
Statistics for protocol overhead.
5.3.4.4.
Fairness
5.3.4.5.
Allow for on-line/integrated computation of statistics
5.3.4.6.
Simulation run-time
5.3.5. A new tool
5.3.6. A combinations of a number of existing tools
5.3.7. Summary
Statistics generation
Dropped pack- XX
ets, packet delay/jitter,
throughput
X
Statistics for pro- XX
tocol overhead.
X
Fairness
Tool # 8
Tool # 7
Tool # 6
X
Tool # 5
Good
GloMoSim
NS2
User interface, none
presentation
OPNET
WIPSIM
Requirement
Tools
--
onXX
line/integrated
computation of
statistics
Simulation run- X
time
Version 2.1, September 8th 2003
?
Page 33 of 52
5.4. Performance/Scalability
5.4.1. WIPSIM
5.4.1.1. Real-time simulation of 200-500 nodes under high traffic load
A number of simulations has been carried out to investigate the speed of WIPSIM simulations for networks with varying number of nodes, different mobility and different traffic load.
All simulations are run on a desktop computer: Pentium III, 600 MHz, 256 MB RAM (1192.0
BogoMips).
The following protocols are used in each scenario:

AODV routing protocol

IPv6

802.2 Datalink layer

802.11 DCF MAC layer
In the first set of simulations, the nodes are all fixed at a certain random initial position (no
mobility) and the number of nodes is varied from 25-500. For each scenario 10 traffic flows
are active between a random source and a random destination, where each traffic flow consists of 500-bytes packets that are send at a speed of 4 packets/s. The radio transmission
range of each node is 100 m and the nodes are placed in a square area with a size that results
in a node density of approx. 150 nodes/km^2. Total simulation time is 100 seconds. The results of the first simulation set can be found in the table below:
Number of nodes
Simulation time [s]
Real-time / Simulation time
25
3.10
0.031
50
3.80
0.038
100
15.90
0.159
200
18.30
0.183
500
27.60
0.276
For a second set of simulations, the mobility is varied. Each network consists of 200 nodes,
where each node moves according to the random waypoint mobility model with zero pausetime and a constant speed. The network traffic is the same as for the previous set of simulations. The results are shown in the table below:
Node speed [m/s]
Simulation time [s]
Real-time / Simulation time
0
18.3
0.183
1
36.5
0.365
Version 2.1, September 8th 2003
Page 34 of 52
2
41.1
0.411
5
67
0.67
Finally, the number of traffic flows is varied, where each flow still consists of 500-bytes
packets that are sent at a rate of 4 packets/s. The network consists of 200 nodes that are moving according to the random waypoint mobility model with zero pause-time and a constant
speed of 1 m/s. The results are shown in the table below:
Number of traffic flows
Simulation time
Real-time / Simulation time
10
36.5
0.365
20
70.8
0.708
30
105.3
1.053
50
141.0
1.41
In almost all simulation scenarios, WIPSIM runs faster than real-time. Of course, the results
depend highly on the speed of the computer that is used for the simulations and the one that is
used for these simulations is certainly not one of todays fastest.
5.4.1.2. Support of distributed simulation
Currently, WIPSIM does not support the possibility of distributed simulation
5.4.2. OPNET
5.4.2.1.
Real-time simulation of 200-500 nodes under high traffic load
5.4.2.2.
Support of distributed simulation
5.4.3. NS2
More elaborated list of requirements:
5.4.3.1. Real-time simulation of 200-500 nodes under high traffic load
The time taken before the actual simulation can start is called the start-up time. On Ns2 website, the start up time for 1000 nodes is 7 minutes. The experiment was done in a relatively
old single-CPU, Pentium-II 448 MHz machine with a large memory of 1GB running Linux
redhat-7.0.
5.4.3.2. Support of distributed simulation
The design of ns is such that simulation of very large networks is difficult, if not impossible,
due to excessive memory and CPU time requirements. A research group at Georgia Tech has
developed extensions and enhancements to the ns simulator to allow a network simulation to
be run in a parallel and distributed fashion, on a network of workstations.
Ns users can distribute their simulation on several (e.g. 8-16) workstations connected via a
standard Ethernet network using the TCP/IP protocol stack.
Version 2.1, September 8th 2003
Page 35 of 52
5.4.4. Glomosim
5.4.4.1.
Real-time simulation of 200-500 nodes under high traffic load
5.4.4.2.
Support of distributed simulation
5.4.5. A new tool
5.4.6. A combinations of a number of existing tools
5.4.7. Summary
Performance / scalabilty
Real-time simu- 0
lation of 200-500
nodes under high
traffic load
X
Support of dis- -tributed simulation
X
Version 2.1, September 8th 2003
Tool # 8
Tool # 7
Tool # 6
Tool # 5
GloMoSim
NS2
OPNET
WIPSIM
Requirement
Tools
Page 36 of 52
5.5. Extensibility
5.5.1. WIPSIM
5.5.1.1. General properties of the tool
If needed, any general additions to the simulator can be made rather easily, since WIPSIM is
relatively new and therefore not too extensive. Moreover, development experience is available within CNTK.
5.5.1.2. Integration of TIP stack in simulator
Even though this integration can be quite a challenge, WIPSIM development experience
within CNTK/Ericsson Telebit is expected to ease the task.
5.5.1.3. Integration of new routing protocols
The framework of WIPSIM has defined a separate object for the routing protocol. Specific
implementations of new routing protocols can therefore be developed independent of the rest
of the simulator.
5.5.1.4. Definition of new mobility models
The movement of nodes is described by events in the scenario-file. Any tool can be used to
generate these input-events based on a specific mobility model. The mobility generator tool
that is used by the WIPSIM developers can be easily extended with new mobility models.
5.5.1.5. Processing in real-time
See section 5.3.1.6
5.5.1.6. Definition of new traffic models
Can be added to the simulator.
5.5.1.7. Definition of measurement procedures for new statistics
New statistics can either be added by writing new scripts for processing the simulation output
or by extending the analyzer module.
5.5.1.8. Good development/source-code documentation
Source code documentation is rather extensive; development documentation separate from
the source code is under development, but not yet available.
5.5.1.9.
Simulator source code including compilation environment needs to
be available
Source code is open source, thus available. The simulator compiles both with Borland C++builder on Windows (compiler is freely downloadable, but not the development tools) and
with gcc on Linux.
Version 2.1, September 8th 2003
Page 37 of 52
5.5.2. OPNET
5.5.2.1.
General properties of the tool
5.5.2.2.
Integration of TIP stack in simulator
5.5.2.3.
Integration of new routing protocols
5.5.2.4.
Definition of new mobility models
5.5.2.5.
Processing in real-time
5.5.2.6.
Definition of new traffic models
5.5.2.7.
Definition of measurement procedures for new statistics
5.5.2.8.
Good development/source-code documentation
5.5.2.9.
Simulator source code including compilation environment needs to
be available
5.5.3. NS2
More elaborated list of requirements:
5.5.3.1. General properties of the tool
Ns2 source code is available freely. The extensibility R8 should be possible. But one should
probably become fairly familiar with ns before he tries this himself, and some C++
knowledge is definitely necessary. One should also have a good understanding of the
C++/Otcl linkage of Ns2. Therefore R8 and R9 are supported. R10 has nothing to do with
extensibility.
R17 is possible if the mobility model is configured in the configuration Tcl script. R23 is
supported because it is open source.
5.5.3.2. Integration of TIP stack in simulator
Ask Ericsson.
5.5.3.3. Integration of new routing protocols:
It is possible to add routing protocols.
5.5.3.4. Definition of new mobility models:
Some mobility models are available like random mobility model. But the design of a new
mobility model is possible by using the node-movement generator.
5.5.3.5. Processing in real-time:
Ns is processing in real-time.
Version 2.1, September 8th 2003
Page 38 of 52
5.5.3.6.
Definition of new traffic models (including possibly the use of real
sensor applications for traffic generation)
This is possible thanks to a traffic generator.
5.5.3.7. Definition of measurement procedures for new statistics:
This is possible by changing the trace file or parsing different things from the trace file.
5.5.3.8. Good development/source-code documentation:
Ns have a good user manual, but the source code is not well documented.
5.5.3.9.
Simulator source code including compilation environment needs to
be available:
This is requirement R23.
5.5.4. GloMoSim
5.5.4.1.
General properties of the tool
5.5.4.2.
Integration of TIP stack in simulator
5.5.4.3.
Integration of new routing protocols
5.5.4.4.
Definition of new mobility models
5.5.4.5.
Processing in real-time
5.5.4.6.
Definition of new traffic models
5.5.4.7.
Definition of measurement procedures for new statistics
5.5.4.8.
Good development/source-code documentation
5.5.4.9.
Simulator source code including compilation environment needs to
be available
5.5.5. A new tool
5.5.6. A combinations of a number of existing tools
5.5.7. Summary
Extensibility
Version 2.1, September 8th 2003
Page 39 of 52
R8
X
R9
X
Tool # 8
Tool # 7
Tool # 6
Tool # 5
GloMoSim
NS2
OPNET
WIPSIM
Requirement
Tools
R10
R17
X
R23
X
Integration of TIP
stack in simulator
?
New statistics
X
Processing in realtime
X
Good
development/source-code
documentation
New routing protocols:
X
New mobility models:
X
New traffic models
X
5.6. Usability
5.6.1. WIPSIM
5.6.1.1. User interface
See section 5.3.1.1
5.6.1.2. Good documentation of the simulator (users’ manual)
A WIPSIM tutorial document exists, describing the basic use of the simulator. This document
is going to be extended. Moreover, questions can be asked directly to the authors of the simulator, also regarding use of the simulator.
Version 2.1, September 8th 2003
Page 40 of 52
5.6.1.3. Short ‘learning period’ for new users
Experience from students that learned to use the simulator has not revealed any problems on
this issue.
5.6.1.4. Existing experience with simulator within CNTK
Extensive existing experience with WIPSIM is available within CNTK (one person, coauthor of the simulator).
5.6.2. OPNET
5.6.2.1.
User interface
5.6.2.2.
Good documentation of the simulator (users’ manual)
5.6.2.3.
Short ‘learning period’ for new users
5.6.2.4.
Existing experience with simulator within CNTK
5.6.3. NS2
More elaborated list of requirements:
5.6.3.1.
User interface
– Configuration: (R5, R7, R11, R12, R17, R22)
The simulations are easily configurable in the Tcl script, a large range of parameters is configurable: Scheduling
–

Queuing

Protocol modules in the node

Mobility model

Network topologies

TCP parameters (maximum congestion window size, max segment
size)

MAC parameters (max contention window size, payload size)

Traffic source
Presentation: (R15)
See section 5.3.3. With nam, protocols can be visualized as animations
5.6.3.2. Good documentation of the simulator (users' manual):
Ns2 has been well documenter for users.
5.6.3.3. Short 'learning period' for new users:
Quite short learning period is needed for the new users (2 weeks), excluding testing and installing. Also ns2 can be used for educational purposes like laboratory exercises for undergraduates.
Version 2.1, September 8th 2003
Page 41 of 52
5.6.3.4. Existing experience with simulator within CNTK:
AUC have been using it for a relatively short time.
5.6.4. GlomoSIm
5.6.4.1.
User interface
5.6.4.2.
Good documentation of the simulator (users’ manual)
5.6.4.3.
Short ‘learning period’ for new users
5.6.4.4.
Existing experience with simulator within CNTK
5.6.5. A new tool
5.6.6. A combinations of a number of existing tools
5.6.7. Summary
Usability
R5
X
R7
X
R11
X
R12
X
R17
X
R22
X
R15
X
Good documentation of the
simulator (users'
manual)
X
Short 'learning
period' for new
users
X
Version 2.1, September 8th 2003
Tool # 8
Tool # 7
Tool # 6
Tool # 5
GloMoSim
NS2
OPNET
WIPSIM
Requirement
Tools
Page 42 of 52
Existing experience with simulator
within
CNTK
X
5.7. Cost / Availability
5.7.1. WIPSIM
The simulator is freely available, including the compilation environment
5.7.2. OPNET
5.7.3. NS2
Simulator already available for all CNTK partners, Open source
5.7.4. Glomosim
5.7.5. A new tool
5.7.6. A combinations of a number of existing tools
5.7.7. Summary
Cost / availability
X
Version 2.1, September 8th 2003
Tool # 8
(X)
Yes
Tool # 7
Open source?
Tool # 6
X
Tool # 5
No
GloMoSim
NS2
Simulator
al- No
ready available
for all CNTK
partners ?
Requirement
OPNET
WIPSIM
Tools
Page 43 of 52
5.8. Verifiability / Correctness
5.8.1. WIPSIM
5.8.1.1. Used simulator features should be known ‘to work correctly’
The implementer of each protocol has tested that protocol extensively. Moreover, the source
code is freely available to everyone, which allows the implementations to be verified by every user that is interested. However, since the user base of WIPSIM is rather small yet, there
can be some uncertainty about the correctness of the protocols.
5.8.1.2. Results should be reproducible by other scientific organizations
Results that are obtained from simulations with WIPSIM are reproducible by others, since the
simulator is freely available. Of course, to be able to reproduce the results, either the simulation scenario needs to be described in detail, or the input files need to be made available for
the other organizations.
5.8.2. OPNET
5.8.3. NS2

used simulator features should be known 'to work correctly'
Ns2 developers tried best to validate ns with regression tests. However, abstraction of the real
world is necessary for a simulator.

results should be reproducible by other scientific organizations:
Ns2 is widely used by researchers in many networking areas such as: TCP, DiffServ/IntServ,
Queuing models, scheduling, multimedia, multicast, traffic modeling, web caching, wireless
sensor networks, satellite networks, and wireless networked-mobile robots.
5.8.4. Glomosim
5.8.5. A new tool
5.8.6. A combinations of a number of existing tools
5.8.7. Summary
Verifiability / correctness
Version 2.1, September 8th 2003
Page 44 of 52
Used simulator
features should
be known 'to
work correctly'
X
Results should
be reproducible
by other scientific
organizations
X
Version 2.1, September 8th 2003
Tool # 8
Tool # 7
Tool # 6
Tool # 5
GloMoSim
NS2
OPNET
WIPSIM
Requirement
Tools
Page 45 of 52
6. Conclusion
Version 2.1, September 8th 2003
Page 46 of 52
7. References
[Bragg2000]
Arnold W.Bragg, “Which network Design Tool is Right for You?”, ITProfessional , IEEE
Computer Society, September/October 2000.
[Cntk1]
Rolf Christensen et al, “Link Layer Simulator – use cases and requirement” , CNTK, 2003
[Feldman]
Philip Feldman: Discrete-Event Simulation for Performance Evaluation Systems With Algorithms and Example in C and C++ , John Wiley & Sons, 2000.
[Ince2002]
A.Nejat Ince, Ed., "Modeling and Simulation Environment for Satellite and Terrestrial Communication Networks ", Kluwer Academic Publishers, 2002.
[Law, Kelton]
Averill M. Law, W. David Kelton: Simulation Modeling and Analysis, second edition, McGraw Hill, 1991.
[OPNET]
[WIPSIM]
[GLOMOSIM]
[QUALNET]
[CNET]
[REAL]
[FLAN]
[NetSIM]
[Exten]
[MPLSSim]
[Bluehoc]
[CDMASim]
[NCTUns]
[SimMAN]
[NS2]
[VENUS]
[AnSIM]
[NIST]
[INSANE]
www.opnet.com
http://www.wipsim.net/
http://pcl.cs.ucla.edu/projects/glomosim/
http://www.scalable-networks.com/
http://www.cs.uwa.edu.au/cnet/
http://www.cs.cornell.edu/skeshav/real/overview.html
http://picolibre.enst-bretagne.fr/projects/flan/
http://eewww.eng.ohio-state.edu/drcl/grants/middleware97/netsimQ.html
http://www.eecs.harvard.edu/networking/simulator.html
http://flower.ce.cnu.ac.kr/~fog1/mns/
http://www-124.ibm.com/developerworks/opensource/bluehoc/
http://watt.icu.ac.kr/wins.asp
http://nsl10.csie.nctu.edu.tw/
http://www.ifak.fhg.de/kommunik/englisch/SimMan.htm
http://www.isi.edu/nsnam/ns/
http://www.bell-labs.com/project/venus/
http://www.i-u.de/schools/hellbrueck/ansim/
http://w3.antd.nist.gov/Hsntg/prd_atm-sim.html
http://www.employees.org/~bmah/Software/Insane/
Version 2.1, September 8th 2003
Page 47 of 52
Version 2.1, September 8th 2003
CDMA Wireless Network Simulator
GloMoSim
QualNet
Real
CNet
NetSim
FLAN
NCTUns
VENUS
yes
Bluetooth extension of ns2
yes
No information
Yes
a commercial version of Glomosim
Origin of ns2
Yes
No
Yes
yes
The VENUS simulation tool simulates large
networks running pure OSPF and OSPF with
Traffic Engineering extensions. It provides inYes
formation about network convergence times,
network load and other performance parameters
Yes
in an operational network in the presence of
network
Source failures
NIST simulator
ANSim
OPNET
WIPSIM
Bluehoc
MPLS extension of ns2
1
Extensible and High-Fidelity TCPIP
Network Simulators
MPLS Network Simulator
Yes
Appendix A. Simulation tools under evaluation
The following table is the results of the initial screening procedure.
The numbers refer to the following questions:
1 “is it possible to extend the tool with new protocols?”
2 “is it possible to extend the tool with new statistics?”
3 “is the tool able to carry out simulations on large network models?”
Page 48 of 52
Version 2.1, September 8th 2003
Yes
Yes
High speed
Yes
No built-in wireless protocols, so not of our interest
No built-in wireless protocols, so not of our interest
Yes
Yes
Yes
Yes
see table in the document
Yes
Yes
7 piconets containing 7 nodes
Yes
Yes
Yes
Yes
yes
yes
yes
yes
Yes but 20 times slower than ns
3
no
Other comments
2
Page 49 of 52
Appendix B. Tool requirements
The list of requirements from [cntk1] is repeated here for completeness (and convenience):
Tag
R-1
Text
Possibility to model power consumption of wireless nodes
Source
AUC
Comment
Tag
R-2
Text
Simulation of ad hoc routing protocols
Source
AUC
Comment
Tag
R-3
Text
Simulation and configuration of a range of Data Link layer protocols must be supported.
Source
AUC and B2NCW
Comment
Examples: Bluetooth, IEEE 802.11, GSM, GPRS, UMTS, etc.
Tag
R-4
Text
Cross-layer information accessible
Source
AUC
Comment
Tag
R-5
Text
Easily modifiable simulation event tracing
Source
AUC
Version 2.1, September 8th 2003
Page 50 of 52
Tag
R-6
Text
BER/PER models or channel models should be supported
Source
AUC
Comment
Tag
R-7
Text
Graphical representation of the simulation results
Source
AUC
Comment
Tag
R-8
Text
Simulator should be easy to extend with new protocols or additions/changes to existing protocols
Source
AUC
Comment
In order to combine protocols under development/investigation with protocols on
different layers the simulator should contain many different protocols/mechanisms.
Tag
R-9
Text
Run real applications on top of the simulated network
Source
AUC
Comment
Tag
R-10
Text
Simulator must run in real-time when the impact of the network performance on the
user perception of the application quality is investigated
Source
AUC
Comment
Tag
R-11
Text
Easy configuration of network parameters
Source
AUC
Comment
The simulator should be configurable in terms of packet loss, transmission errors,
delays and bandwidths.
Version 2.1, September 8th 2003
Page 51 of 52
Tag
R-12
Text
Metrics to evaluate the performance of routing protocols
Source
AUC
Comment
Examples:

Which protocol performs better in selecting the best path, such that
throughput is maximized and delay minimized.

Which routing protocol is the most efficient (in terms of hop count)

How power efficient are those protocols

Packet loss ratio
Tag
R-13
Text
Quality-of-service mechanisms should be available in the simulator
Source
AUC
Comment
Tag
R-14
Text
Simulator must be able to simulate many different kinds of networks
Source
AUC
Comment
Tag
R-15
Text
Generation of statistics based on a large number of simulations for a specific network and traffic scenario
Source
AUC
Comment
Tag
R-16
Text
Simulation of multiple, freely configurable interconnected hierarchical and/or nonhierarchical networks
Source
B2NCW
Comment
Tag
R-17
Text
Dynamic reconfiguration of network topology
Source
B2NCW
Comment
Requirement for concurrent simulation of several interconnected ‘networks’.
Version 2.1, September 8th 2003
Page 52 of 52