Progress Report
Topics
[Review of active monitoring tools]
[Network performance parameters]
By
Muhammad Ali
MS-CS 11
NUST Institute of Information Technology
National University of Science and Technology
Contents
Network Monitoring Tools ................................................................................................. 3
Introduction ..................................................................................................................... 3
Ping ................................................................................................................................. 3
Traceroute ....................................................................................................................... 4
pathChirp......................................................................................................................... 4
Pathload........................................................................................................................... 5
ABwE .............................................................................................................................. 5
Thrulay ............................................................................................................................ 5
Netperf ............................................................................................................................ 5
IPERF Features ............................................................................................................... 6
Netest .............................................................................................................................. 6
Selection of tools............................................................................................................. 6
Network performance metrics............................................................................................. 7
Available Bandwidth ...................................................................................................... 7
One-Way Delay (OWD) ................................................................................................. 7
Serialization Delay ...................................................................................................... 7
Propagation Delay....................................................................................................... 7
Queuing Delay ............................................................................................................ 7
Forwarding Delay ....................................................................................................... 7
Round-Trip Time (RTT) ............................................................................................. 8
Delay Variation (Jitter) ............................................................................................... 8
Packet Loss ................................................................................................................. 8
Congestion .................................................................................................................. 8
Errors........................................................................................................................... 8
Packet Reordering ....................................................................................................... 8
Maximum Transmission Unit (MTU) ......................................................................... 8
Bandwidth Delay Product (BDP)................................................................................ 9
References ........................................................................................................................... 9
Network Monitoring Tools
Introduction
Network monitoring can be classified into two categories: active monitoring and passive
monitoring. In active monitoring extra measurement packets are injected in real traffic
and different parameters like Round Trip Time (RTT), packet loss and available
bandwidth are calculated. In passive monitoring techniques things are watched as they
happen. Network devices (routers, switches sniffer etc.) record information like packets
passed per second, reachability, packet loss etc. in local databases known as Management
Information Base (MIB) which is retrieved by some protocol such as Simple Network
Management Protocol (SNMP).
These two techniques are complementary to one another. Passive does not inject extra
traffic but polling to gather data generates traffic, also gathers large amounts of data
which need storage space. Active provides explicit control on the generation of packets
for measurement scenarios but it injects extra artificial traffic
In the next section an overview of some publicly available active network monitoring
tools is presented with benefits and shortcomings of each highlighted.
Ping
Ping is a client/server application which uses the Internet Control Message Protocol
(ICMP) Echo mechanism. The server can send a packet of a user selected length to a
remote node and have it echoed back. Client puts timestamp in data bytes. Server
compares timestamp with time when echo comes back to get RTT.
Ping provides the measures of response time (RTT), the packet loss and reachability (no
response for a succession of pings). Figure 1 shows a sample run of ping utility. [i]
Nowadays it usually comes pre-installed on almost all platforms, so there is nothing to
install on the client or server. The server (i.e. the echo responder) runs at a high priority
(e.g. in the kernel on UNIX) and so is more likely to provide a good measure of network
performance than a user application. [ii]
Repeat count
Packet size
Remote host
RTT
syrup:/home$ ping -c 6 -s 64 thumper.bellcore.com
PING thumper.bellcore.com (128.96.41.1): 64 data bytes
72 bytes from 128.96.41.1: icmp_seq=0 ttl=240 time=641.8 ms
72 bytes from 128.96.41.1: icmp_seq=2 ttl=240 time=1072.7 ms
Missing seq #
72 bytes from 128.96.41.1: icmp_seq=3 ttl=240 time=1447.4 ms
72 bytes from 128.96.41.1: icmp_seq=4 ttl=240 time=758.5 ms
72 bytes from 128.96.41.1: icmp_seq=5 ttl=240 time=482.1 ms
Summary
--- thumper.bellcore.com ping statistics --- 6 packets transmitted, 5 packets received,
16% packet loss round-trip min/avg/max = 482.1/880.5/1447.4 ms
Figure 1: Ping example with no of packets and packet size options
Traceroute
Traceroute is used to measure hop-by-hop connectivity and RTT. It directs a packet to
each router along a path without actually knowing the path. IP packets contain a time-tolive field that is initialized by the original sender then decremented by one at each
intermediate router. If the field is decremented to zero, the packet is discarded and an
error indication packet (an ICMP “time exceeded”) is sent back to the original sender.
The source address of the ICMP “time exceeded” identifies the router that discarded the
data packet. So if packets are sent to the final destination but with the TTL set to n, the
router n hop along the path is forced to identify itself [iii]. Figure 2 shows an example run
of Traceroute with maximum hops set to 20, probs/hop set to 1 [i].
Max hops
Remote host
Probes/hop
17cottrell@flora06:~>traceroute -q 1 -m 20 lhr.comsats.net.pk
traceroute to lhr.comsats.net.pk (210.56.16.10), 20 hops max, 40 byte packets
1 RTR-CORE1.SLAC.Stanford.EDU (134.79.19.2) 0.642 ms
2 RTR-MSFC-DMZ.SLAC.Stanford.EDU (134.79.135.21) 0.616 ms
3 ESNET-A-GATEWAY.SLAC.Stanford.EDU (192.68.191.66) 0.716 ms
4 snv-slac.es.net (134.55.208.30) 1.377 ms
5 nyc-snv.es.net (134.55.205.22) 75.536 ms
6 nynap-nyc.es.net (134.55.208.146) 80.629 ms
7 gin-nyy-bbl.teleglobe.net (192.157.69.33) 154.742 ms
8 if-1-0-1.bb5.NewYork.Teleglobe.net (207.45.223.5) 137.403 ms
9 if-12-0-0.bb6.NewYork.Teleglobe.net (207.45.221.72) 135.850 ms
10 207.45.205.18 (207.45.205.18) 128.648 ms
No response:
11 210.56.31.94 (210.56.31.94) 762.150 ms
Lost packet or router
12 islamabad-gw2.comsats.net.pk (210.56.8.4) 751.851 ms
ignores
13 *
14 lhr.comsats.net.pk (210.56.16.10) 827.301 ms
Figure 2: An example run of Traceroute
pathChirp
pathChirp is used to calculate unused capacity or available bandwidth. It is based on the
concept of self-induced congestion which relies on a simple heuristic: If the probing rate
exceeds the available bandwidth over the path, then the probe packets become queued at
some router, resulting in an increased transfer time. On the other hand, if the probing rate
is below the available bandwidth, the packets face no queuing delay. The available
bandwidth can then be estimated as the probing rate at the onset of congestion [ iv].
Principle of self-induced congestion is stated in another way as:
• Probing rate < available bw  no delay increase
• Probing rate > available bw  delay increases
pathChirp features an exponential flight pattern of probes we call a chirp. Packet chirps
offer several significant advantages over current probing schemes based on packet pairs
or packet trains. By rapidly increasing the probing rate within each chirp, pathChirp
obtains a rich set of information from which to dynamically estimate the available
bandwidth. The basic idea in SLoPS is that the one-way delays of a periodic packet
stream show an increasing trend when the stream’s rate is higher than the available
bandwidth.
Pathload
Pathload is based on the technique of Self-Loading Periodic Streams (SLoPS), for
measuring available bandwidth. A periodic stream in SLoPS consists of K packets of size
L, sent to the path at a constant rate R. if the stream rate R is higher than the available
bandwidth, the one-way delays of successive packets at the receiver show an increasing
trend. This is the key idea in SLoPS. In pathload, sender uses UDP for packet stream and
TCP connection for controlling measurements. Sender timestamps each packet upon
transmission, which provides relative one-way delay at the receiver side. [v]
ABwE
ABwE is a light weight available bandwidth measurement tool based on the packet pair
dispersion technique. It uses fixed size packets with specific delay between each packet
pair. The observed packet pair delay is converted into available bandwidth calculations.
The tool can be used in continuous mode and detects all substantial bandwidth changes
caused by improper routing or by congestions. [vi]
Thrulay
Thrulay stands for THRUput and deLAY. It measures the capacity of a network by
sending a bulk TCP stream over it and measures one-way delay by sending a Poisson
stream of very precisely positioned UDP packets.[vii]
Netperf
Netperf is a benchmark that can be used to measure the performance of many different
types of networking. It provides tests for both unidirectional throughput, and end-to-end
latency. The environments currently measurable by netperf include: TCP and UDP via
BSD Sockets, DLPI, Unix Domain Sockets, Fore ATM API and HP HiPPI Link Level
Access.
Netperf is designed to use one of several (perhaps platform dependent) CPU utilization
measurement schemes. The most common use of netperf is measuring bulk data transfer
performance. This is also referred to as "stream" or "unidirectional stream" performance.
Essentially, these tests will measure how fast one system can send data to another and/or
how fast that other system can receive it.
Request/response performance is the second area that can be investigated with netperf.
Generally speaking, netperf request/response performance is quoted as "transactions/s"
for a given request and response size. A transaction is defined as the exchange of a single
request and a single response. From a transaction rate, one can infer one way and roundtrip average latency. [viii]
IPERF Features
Iperf measures the maximum TCP bandwidth and the UDP performance between two
machines. It is useful to tune the TCP window size and gives a baseline to compare
application performance against. It can also measure packet loss and variation in delay
(jitter). Many older but similar tools such as ttcp also exist. Iperf is a useful tool for nonnetwork engineers [ix]. It is widely used for end-to-end performance measurements and
has become an unofficial standard in the research networking community [x].
Iperf has two working modes: TCP mode and UDP mode. In TCP mode it is used to
measure the bandwidth. It reports MMS (Maximum Segment Size) or MTU (Maximum
Transfer Unit) size and the observed read sizes. It supports TCP window size adjustment
via socket buffers. In UDP mode packet loss and delay jitter can be measured. [xi]
Netest
Netest is a tool to measure bandwidths (physical and available) if possible, or maximum
throughput otherwise; and achievable throughput for UDP, single stream TCP, and
parallel stream TCP. It is designed for network problem analysis and diagnosis. It also
measures available bandwidth, maximum burst size, round trip delay time, and provides
information about where the bottleneck is if a protocol cannot fully utilize the available
bandwidth. In this case, netest reports the maximum throughput and tells where limits the
maximum throughput; in this case, we mean the hardware is not able to probe the
available bandwidth, so no available bandwidth information will be reported. Otherwise,
Netest reports the available bandwidth, which means the network is the bottleneck. [xii]
Selection of tools
Shriram [xiii] concludes his comparison of public End-to-End bandwidth estimation tools
on High-Speed Links with following remarks: “Pathload and Pathchirp are the most
accurate. Iperf requires maximum buffer size and is sensitive to small packet loss.”
TOPP, pathload and pathChirp are based on the concept of self-induced congestion.
Further comparisons of pathChirp with pathload and TOPP in [iv] clearly show that
pathChirp performs far batter than the other two tools in single-hop, multi-hop scenarios
and on real internet as well.
This recent research on active probing tools performance shows that pathChirp and Iperf
have better results than other tools in estimating available bandwidth and throughput
respectively. Ping utility does not require any extra installation/deployment efforts and
has become a de-facto standard for measuring RTT and packet loss.
Based on the above findings the tools selected for our project include pathChirp, Iperf,
ping and thrulay.
Network performance metrics
Available Bandwidth
Available bandwidth is defined in [iv] as: Denote the capacity of the output queue of router node i
as Ci, and the total traffic (other than probes) entering it between times a and b as Ai[a, b]. Define
the path’s available bandwidth in time interval [t -τ, t] as:
where pi is the minimum time a packet sent from the sender could take to reach router i. The delay
pi includes the speed-of-light propagation delay and packet service times1 at intermediate queues.
In reality probe packets suffer queuing delays in addition to the minimum delay pi. Thus probes
transmitted during [t- τ, t] can arrive at router i outside time interval [t- τ +pi, t+pi] and do not
exactly measure B[t- τ, t]. For large τ (>>RTT), however, the effect of queuing delay becomes
inconsequential.
One-Way Delay (OWD)
The time it takes for a packet to reach its end-to-end destination is called OWD. It can be
broken down into: per-hop one-way delays, and these in turn into: per-link and per-node delay
components. The per-link component of one-way delay consists of two sub-components:
propagation delay and serialization delay. The per-node component of one-way delay
also consists of two sub- components: forwarding delay and queuing delay.
Serialization Delay
It’s the time taken to separate a packet into sequential link transmission units (bits). It is
obtained by dividing the packet size (in bits) by the capacity of the link (in bits per
second). Nowadays, as links increasingly have a higher bit rate, serialization delay is less
relevant.
Propagation Delay
Propagation delay is the duration of time for signals to move from the transmitting to the
receiving end of a link. On simple links, this is the product of: the link's physical length
and the characteristic propagation speed of media. On high-speed wide-area network
(WAN) paths, delay is usually dominated by propagation times.
Queuing Delay
Queuing delay is defined as the time a packet has to spend inside a node such as a router
while waiting for availability of the output link. It depends on the amount of traffic
competing to send packets towards the output link and on the priorities of the packet.
Forwarding Delay
It is due to processing at the node reading forwarding-relevant information e.g.
destination address plus other headers. Another factor is forwarding decision which is
based on the routing table
Round-Trip Time (RTT)
It is the sum of the one-way delays from source to destination plus time it takes B to
formulate the response. Large RTT values can cause problems for TCP and other
window-based transport protocols. The round-trip time influences the achievable
throughput, as there can only be a window's worth of unacknowledged data in the
network.
Delay Variation (Jitter)
OWD is not constant on a real network because of competing traffic and contention for
processing resources. The difference between a given p9acket’s actual and average OWD
is termed ‘delay variation’ or jitter. It only compares the delays experienced by packets of
equal size, as OWD is dependent on packet size because of serialization delay
Packet Loss
Packet loss is determined as the probability of a packet being lost in transit from a source
A to a destination B. Applications requiring reliable transmission e.g. Bulk data transfers,
use retransmission, which reduces performance. In addition, congestion-sensitive
protocols such as standard TCP assume that packet loss is due to congestion, and respond
by reducing their transmission rate accordingly. Congestion and errors are the two main
reasons for packet loss.
Congestion
When the offered load exceeds the capacity of a part of the network, packets are buffered
in queues. Since these buffers are also of limited capacity, congestion can lead to queue
overflows, which leads to packet losses. Congestion can be caused by moderate overload
condition maintained for an extended amount of time or by the sudden arrival of a very
large amount of traffic (traffic burst).
Errors
Another reason for loss of packets is corruption, where parts of the packet are modified
in-transit. When such corruptions happen on a link (due to noisy lines etc.), this is
usually detected by a link-layer checksum at the receiving end, which then discards the
packet.
Packet Reordering
The Internet Protocol (IP) does not guarantee the packet ordering. Packet reordering
concerns packets of different sizes. Larger packets take longer to transfer and may be
overtaken by smaller packets in transit. It can be measured by: injecting the same traffic
pattern via traffic generator and calculating the reordering. To measure maximal
reordering: short burst of long packets immediately followed by a short burst of short
packets
Maximum Transmission Unit (MTU)
It describes the maximum size of an IP packet that can be transferred over the link
without fragmentation. Common MTU sizes are: 1500 bytes (Ethernet, 802.11 WLAN),
4470 bytes (FDDI, common default for POS and serial links), 9000 bytes (Internet2 and
GÉANT convention, limit of some Gigabit Ethernet adapters) and 9180 bytes (ATM,
SMDS).
Bandwidth Delay Product (BDP)
The Bandwidth Delay Product (BDP) of an end-to-end path is the product of the
bottleneck bandwidth and the delay of the path. It is often useful to think of BDP as the
"memory capacity" of a path, i.e. the amount of data that fits entirely into the path
between two end-systems. This relates to throughput, which is the rate at which data is
sent and received. Network paths with a large BDP are called Long Fat Networks or
LFNs. BDP is an important parameter for the performance of window-based protocols
such as TCP. [xiv]
References
i
Cottrell L., Internet Monitoring, Presented at NUST Institute of Information Technology (NIIT)
Rawalpindi, Pakistan, March 15, 2005
ii
Cottrell L., Matthews W. and Logg C., Tutorial on Internet Monitoring & PingER at SLAC
iii
Jacobson V., “pathchar — a tool to infer characteristics of Internet paths”
iv
Ribeiro V., Riedi R., Baraniuk R., Navratil J., Cottrell L. pathChirp: Efficient Available Bandwidth
Estimation for Network Paths.
v
Jain M., Dovrolis C., End-to-End Available Bandwidth: Measurement Methodology, Dynamics, and
Relation with TCP Throughput. ACM SIGCOMM 2002.
vi
Navratil J. and. Cottrell L., ABwE: A Practical Approach to Available Bandwidth Estimation.
vii
http://thrulay-hd.sourceforge.net
viii
http://www.netperf.org/netperf/training/Netperf.html
ix
Beginner's Guide to Network-Distributed Resource Usage
x
Cottrell, L., Logg, C.: Overview of IEPM-BW Bandwidth Testing of Bulk Data Transfer. In: Sc2002:
High Performance Networking and Computing. (2002)
xi
http://dast.nlanr.net/Projects/Iperf/iperfdocs_1.7.0.html
xii
http://www-didc.lbl.gov/NCS/netest/
xiii
Shriram A., Murray M., Hyun Y., Brownlee N., Broido A., Fomenkov M., claffy k., “Comparison of
Public End-to-End Bandwidth Estimation tools on High-Speed Links”
xiv
GÉANT2 Performance Enhancement and Response Team (PERT) User Guide and Best Practice Guide