Outline
¾ Performance Metrics
¾ Performance Measurements
Lecture 8
Performance Measurements and Metrics
Kurose-Ross: 1.2-1.4
(Hassan-Jain: Chapter 3 ”Performance
Measurement of TCP/IP Networks”)
2010-02-17
Sid 2
David Gundlegård, ITN
Performance Metrics
¾ Asessment of communication network
performance
¾ How good is our network?
Performance Metrics
¾ (How much better than competitors?)
¾ Classical: Internet access bandwidth
¾ What can it be used for?
¾ Online gaming with HSDPA-connection?
¾ Vehicle safety communication via LTE networks?
¾ What shall we measure? How? Which metrics are
important in different applications?
2010-02-17
Sid 4
David Gundlegård, ITN
Performance Metrics
Performance Metrics
¾ Main quantitative metrics for packet switched
networks
¾ Other aspects
¾ Fairness
¾ Packet loss
¾ CPU usage
¾ Due to congestion and bit errors
¾ Memory usage
¾ End-to-end delay
¾ Qualitative metrics
¾ Service experience: Good
sound, bad picture,
slow response…
¾ Battery consumption
¾ From sender to (final) receiver
¾ Response time
¾ Delay-variation (A.k.a. Jitter)
¾ Availability
¾ Variation in delay in a packet session
¾ Packet reordering
¾ Throughput (Bandwidth)
¾ Number of users
¾ Bits per second that can be sent through the network
¾ Blocking rate
¾ …
2010-02-17
Sid 5
2010-02-17
David Gundlegård, ITN
Switching and Delay
¾ Circuit switching
2010-02-17
¾ Store-and-forward delay
¾ Propagation delay
¾ Processing delay
¾ Processing delay
(typically small)
¾ Transmission delay
Sid 7
David Gundlegård, ITN
David Gundlegård, ITN
Switching and Delay
¾ Packet switching
¾ Connection establishment
delay
¾ Transmission delay
Sid 6
¾ Propagation delay
¾ Queuing delay
2010-02-17
Sid 8
David Gundlegård, ITN
Link Bandwidth and Delay
Link Bandwidth and Delay
¾ Left: high bandwidth
and/or propagation delay
¾ Sliding Window flow control
¾ Right: low propagation
delay and/or bandwidth
¾ Bandwidth x delay
product
¾ Link utilisation and max
throughput
¾ Rmax=W/RTT
¾ Utilisation
¾ R = datarate
¾ W = window size
¾ Stop and wait
¾ Sliding window
2010-02-17
Sid 9
David Gundlegård, ITN
2010-02-17
U=
1
1 + 2a
a=
W
(W < 2a + 1)
1 + 2a
End-to-end delay (PS)
¾ Store-and-forwarddelay
¾ Store-and-forward: d=QL/R
¾ Propagation: d=distance/velocity
¾ ~3*10^8 m/s (factor 0.7-1 depending on transmission media)
¾ Processing: time for process on every node –
proportional to the number of hops
¾ In a switch: L/R
¾ Error control, routing decisions etc.
¾ Transmission: d=L/R
¾ Route with Q links:
QL/R
¾ Time to transmit the bits on the link/path
¾ Determined by throughput (bottleneck)
¾ Compare with store-and-forward
¾ Cf. ”Cut through
switch”
Sid 11
David Gundlegård, ITN
t prop
t frame
David Gundlegård, ITN
Delay
¾ Prop. to packet size
(L)
2010-02-17
Sid 10
U=
¾ Queuing delay
2010-02-17
Sid 12
David Gundlegård, ITN
Queuing Delay
Queuing Delay
¾ Queues occur when incoming traffic to a
node (interface) is larger than the forwarding
capacity (for some period of time)
¾ Traffic intensity:
¾ L = packet length, R = datarate (service rate,
capacity)
¾ What happens when La/R>1?
¾ Number of links/interfaces on a node
A
¾ Infinite buffer size?
¾ Finite buffer size?
C
¾ Analysed with
¾ Queuing theory
1.5 Mb/s
B
2010-02-17
statistical multiplexing
¾ Simulation
queue of packets
waiting for output
link
Sid 13
David Gundlegård, ITN
La
R
¾ a = average rate that packets arrive to the queue
¾ Forwarding capacity VS link/interface capacity
100 Mb/s
Ethernet
ρ=
2010-02-17
Sid 14
David Gundlegård, ITN
Queuing Models
End-to-end Delay Metrics
¾ M/M/1 with and without priority
¾ Round trip time (RTT)
¾ The time needed to travel from source to destination, plus
the time to travel from destination back to the source.
¾ The Markov assumption
¾ One way delay
¾ The time needed to travel from source to destination, or from
destination to source
¾ Delay variation
2010-02-17
Sid 15
David Gundlegård, ITN
2010-02-17
Sid 16
David Gundlegård, ITN
Delay Variation (Jitter)
Throughput
¾ The end-to-end delay
variation over time
(between packets)
¾ Number of bits per time unit that can be pushed
through the link/network (A.k.a. bandwidth:
ambiguity…)
¾ Important in real-time
applications (buffer size
and playout delay)
¾ Data rate
¾ Defined for a session of
packets (>2)
¾ Throughput
¾ Often referred to as max transmitting rate at a link
¾ The actual data rate at the (final) receiver
¾ Max difference in delay
¾ Taken into account flow control, bottlenecks,
retransmissions, FEC, cross traffic (other users) etc.
¾ Mean difference in delay
¾ Standard deviation
¾ Throughput variation
¾ ...
2010-02-17
Sid 17
David Gundlegård, ITN
2010-02-17
Sid 18
David Gundlegård, ITN
Packet Loss
¾ Percentage of packets lost
¾ Packet loss distribution
¾ Bursty
¾ Uniform
¾ TCP VS UDP
¾ Reasons
¾ Buffer overflow (congestion)
¾ Bit errors (packet discarded)
¾ Typical in wireless networks
2010-02-17
Sid 19
David Gundlegård, ITN
Performance Measurements
Measurement Tasks
Measurement Tools
¾ Data collection
¾ Monitoring tools
¾ Typically raw data from live network
¾ Monitors existing traffic
¾ Analysis
¾ Ethereal, Tcpdump, Tcpstat…
¾ Statistical analysis of data
¾ Use collected data in simulation
¾ …
¾ Benchmarking tools
¾ Generates traffic used for analysis
¾ Presentation
¾ Often both client and server needed
¾ Visualisation through graphs and charts
¾ TPtest, Iperf, Netperf, Netpipe, DBS…
¾ Interpretation
¾ Standard tools
¾ What do the results tell us?
¾ ”We need new equipment”
¾ ”Ip telephony can/cannot be used in current network”
2010-02-17
Sid 21
David Gundlegård, ITN
¾ Ping, traceroute, netstat
2010-02-17
David Gundlegård, ITN
Analysis Approaches
Data Analysis
¾ Live network measurements
¾ Number of measurements
¾ Often difficult/expensive
¾ Does the system exist yet?
¾ Purpose of measurements?
¾ Variation in results?
¾ Lab experiments
¾ Length of every measurement
¾ Requires generalisations to more realistic conditions
¾ Transient behaviour?
¾ Simulations
¾ Variability?
¾ Time consuming/validation/verification
¾ Assumptions and configurations
¾ Analytical evaluation
¾ Generalisations possible?
¾ E.g. queuing theory
¾ Quick but often many simplifications
¾ Statistical measures
¾ Average, min, max, standard deviation, variance, confidence
intervals, hypothesis trial, correlation etc.
¾ Combination
¾ For validation purposes
2010-02-17
Sid 22
Sid 23
David Gundlegård, ITN
2010-02-17
Sid 24
David Gundlegård, ITN
Internet Example
Measurement Examples
¾ Ethereal
¾ >tracert IP/domännamn
¾ www.liu.se
¾ www.mit.edu
¾ >ping www.dn.se –n 10 –l 1000
¾ TPtest5
¾ Iperf
2010-02-17
Sid 25
David Gundlegård, ITN
Next Lecture
¾ Quality of Service
2010-02-17
Sid 27
David Gundlegård, ITN
2010-02-17
Sid 26
David Gundlegård, ITN