Comparing the Accuracy of
Network Simulators for
Packet-Level Analysis using
a Network Testbed
Chaudhry Usman Ali
UNB, Fredericton
TESTING SIMULATOR ACCURACY
• Measure real network traffic.
• Compare real network traffic with the
simulator results.
Real Traffic
Types of Traffic:
CBR (Constant Bit Rate) data traffic.
(Simplicity)
FTP (File Transfer Protocol) session.
(Popularity and Dynamic behaviour)
Simulated Traffic
CBR (Few external factors influence
the model)
FTP(Complex)
HTTP(too many degrees of freedom)
METHODS OF SIMULATION
Discrete Event
Analytical Simulation
Produce predictions in the
network at a low level (packetby-packet).
Use mathematical models to
produce their results.
Accurate.
Sacrifice Accuracy !
But SLOW !
Faster speed !
NS-2 and OPNET use a hybrid approach.
SIMULATION TOOLS
• OPNET MODELLER
• NS-2
1.
2.
3.
4.
1. Finite State Machine + Analytical Model.
2. 400 special purpose modelling
functions.
3. Not open source.
4. Model parameters can be altered.
Numerous models of Internet protocols. C++ : per packet processing.
Otcl : Simulation Scenario
NAM network animator.
generation.
Animations and graphs.
Custom applications and protocols can be created.
NETWORK TESTBED
5 PCs.
Client-Server Pair.
G- Traffic Generator.
S - Traffic Sink.
Hub
10/100 Mbit links.
* KURT
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707,
2003.
NETWORK REPRESENTATION
OPNET
NS-2
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707,
2003.
EXPERIMENTS
CBR Traffic
1. Characterized by a fixed
bandwidth across the network.
2. Typically used by applications
such as video and audio.
3. CBR traffic stream can be
generated by fixing the packet
size and using the same interarrival time between packets.
FTP Session
Share, transfer and transmit
information between two
computers.
Two types of test were performed
for the FTP experiments.
1- FTP sessions using default
settings.
2- Finely tuned FTP parameters.
PARAMETER TUNING
Reno to New Reno
Fast Recovery capability + recover without retransmission.
Window Scaling
Allows advertisement of window sizes bigger than 65 kB.
Timestamp
Imitates the echoing capability of the testbed in both directions.
Link Latency
Affects RTT.
Max Segment Size
Bigger Ethernet packets.
Window Size
More range for sliding the window value.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for PacketLevel Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
TRAFFIC SCENARIOS
IP Traffic monitored using tcpdump.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707,
2003.
RESULTS
Scenario#
Client –Server Load
Traffic G. – Traffic S. Load
CBR1
2 Mb/s
0 Mb/s
ROUTER`s PERSPECTIVE
NS-2 displays a more realistic CBR behaviour.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707,
2003.
Scenario#
Client –Server Load
Traffic G. – Traffic S. Load
CBR2
2 Mb/s
2 Mb/s
CBR3
5 Mb/s
0 Mb/s
ROUTER`s PERSPECTIVE
Lower throughput for the network testbed.
Scheduling problems?
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707,
2003.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis
using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Scenario#
Client –Server Load
Traffic G. – Traffic S. Load
CBR4
5 Mb/s
6 Mb/s
CLIENT`s PERSPECTIVE
Sudden drop
SERVER`s PERSPECTIVE
ROUTER`s PERSPECTIVE
Scenario#
Client –Server Load
Traffic G. – Traffic S. Load
FTP1
10 MB File
0 Mb/s (CBR)
FTP3
10 MB File
0 Mb/s (CBR)
ROUTER`s PERSPECTIVE
Server to Client
Client to Server
Without tuning.
With tuning.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis
using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.
Scenario#
Client –Server Load
Traffic G. – Traffic S. Load
FTP2
10 MB File
6 Mb/s (CBR)
FTP4
10 MB File
6 Mb/s (CBR)
ROUTER`s PERSPECTIVE
Server to Client
Client to Server
Without tuning.
With tuning.
G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators
for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp.
700-707, 2003.
Conclusions
• Pure CBR type traffic. (NS-2 performed better).
• In one scenario OPNET performed “better”.
• FTP Experiments
• NS-2 FTP simulation model only indicated general transfer
rate rather than replicating the actual network flow.
• OPNET performed closely to the testbed results.
• Simulation speed was fast in both simulators.
• The learning curve was different and steep at times.
• NS-2: “script” generated.
• OPNET: User Interface.
Conclusions Contd..
• In FTP session, simulators did not model the
dynamic behaviour of FTP in standard form.
• FTP adapts its output to prevailing network
conditions.
• NS-2 and OPNET did not always mimic this
performance.
• However, OPNET performed better.
Future Work
• Establishing a scheme to model HTTP in the
simulators and the testbed.
REFERENCE
• G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh,
M. Fleury, and M. Reed, OPNET-Modeler and NS-2:
"Comparing the Accuracy of Network Simulators for
Packet-Level Analysis using a Network Testbed".
3rd WEAS Int. Conf. on Simulation, Modelling and
Optimization (ICOSMO 2003), Crete, vol. 2, pp.
700-707, 2003.