International Telecommunication Union
IP Performance Metrics:
Definitions and Implementation Examples
Al Morton
AT&T Labs
Workshop on End-to-End Quality of Service.What is it? How do we get it?
Geneva, 1-3 October 2003
Outline
ITU-T
o
Performance Management Framework
•
o
o
o
Relationship to the E2E QoS goal
IP Parameters/Metrics Summary
In-progress Metric Development
Implementations
Service Providers
2. Customers
3. 3rd Parties
1.
1.
1-3 October 2003
Performance for MPLS-enabled IP Nets
Workshop on End-to-End Quality of Service. What is it? How do we get it?
2
ITU-T
Network Performance
Management Framework
o Fault Monitoring -- failure detection
o Passive Info Collection (single point)
• Read MIB counters or control data
• Sample Traffic
o Active Measurements
• Synthetic Traffic Dedicated to meas.
o Customer Measurements
• Live or Synthetic traffic
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
3
ITU-T
Relationship to E2E QoS:
Provide answers to ...
o Network Provider
• Is the design meeting requirements for
various traffic classes or applications?
• How can I demonstrate the superior
performance of my service offering?
o Customer
• Is Network Performance  Agreement?
o 3rd Parties
• What does “the net” look like? Hot
spots?
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
4
ingress MP
egress MP
IPRE1
Packet Perf. Parameters
ITU-T
tTmax
IPRE2
Valid header and
error-free payload
Successful
IP packet outcome
IPRE2
Corrupted header or
errored payload
Errored
IP packet outcome
IPRE2
(Note)
Spurious
IP packet outcome
IPRE1
tTmax
IPRE1
Never delivered or
delivered to an unpermitted
egress MP
Lost
IP packet outcome
IPRE1
t >Tmax
IPRE2
(Note)
Lost
IP packet outcome
NOTE – Outcome occurs independent of IP packet contents
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
5
ITU-T
Metric/Parameter Definition
Summary
Framework
Sampling
Loss
Delay
Delay Variation
IETF IPPM RFCs
2330
2330 Poisson
3432 Periodic
2680
2679 (1-way)
2681 (Round Trip)
3393
Availability
2678
Bulk Transfer Cap 3148
Loss Patterns
3357
1-3 October 2003
ITU-T Recs.
Y.1540 cl 1 thru 5
(future work in
SG4 ?)
Y.1540 cl 5.5.6
Y.1540 cl 6.2
Y.1540 cl 6.2.2
Y.1540 cl 7
Possibly in G.IPP
Workshop on End-to-End Quality of Service. What is it? How do we get it?
6
ITU-T
Comparison of IETF and ITU-T
Delay Variation Metrics
IETF IPDV is a measure of transfer
delay variation w.r.t. previous packet.
For Packet n,
IPDV(n) = Delay(n) - Delay(n-1)
or = R(n) - R(n-1) - T(n) - T(n-1)
If the nominal transfer time is 
=10msec, and packet 2 is delayed in
transit for an additional 5 msec, then
two IPDV values will be affected.
IPDV(2) = 15 - 10 = 5 msec
IPDV(3) = 10 - 15 = -5 msec
IPDV(4) = 10 - 10 = 0 msec
Tx
Rcv
Playout
1

2
1
t
3
2
4
3
ITU-T SG 13 PDV is delay w.r.t. a
reference, usually minimum delay.
PDV(n) = Delay(n) - Min[Delay(*)]
PDV(1,3,4)=0 PDV(2)=5
Time spent in:
1-3 October 2003
Inter packet
arrival time,
longer than
send interval
4
Transit
Rcv Buffer
Workshop on End-to-End Quality of Service. What is it? How do we get it?
7
Transient Delay Variation
caused by burst traffic
ITU-T
100
90
Delay or Jitter (ms)
80
70
60
50
40
30
20
10
0
-10
-20
120
220
320
420
520
ITU PDV
delta ms
1-3 October 2003
620
720
820
920 1020 1120
Time in ms
IETF
IPDV ms
RT-delay ms
Workshop on End-to-End Quality of Service. What is it? How do we get it?
8
ITU-T
Packet Metrics for VoIP and
other voiceband applications
o *new* metrics in G.IPP
• Consecutive Packet Loss
• Degraded Seconds
• Short-term Delay Variation
• Overall VoIP Parameters
o Alan Clark’s Presentation
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
9
What is Packet Reordering?
ITU-T
Packets arrive at Dst, but not in send order.
1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering
In the “world of order” all these packets are of
interest.
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...
| Early | Late |
No reordering until Late Packets Arrive
# of Early Packets => Reordering Extent
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
10
ITU-T
Affect of Reordered Packets on
most applications
o Receivers must perform work to restore order
1, 2, 3,
7, 8, 9,10, 4, 5, 6, 11, 12,...
| Buffered ||Reordered|
Dst Time axis
1
2
1
1-3 October 2003
3
2
7
3
8
9 10
4
5
4
6
11
Higher
layers
5 6, 11
(& 7 to 10)
Workshop on End-to-End Quality of Service. What is it? How do we get it?
11
Definition of Reordered Packet
ITU-T o Packet n is designated reordered when
its sequence number is less than the
Next Expected threshold (set by the
arrival of a previous packet).
12
11
Next Expected
10
10
Send Order (s)
9
8
8
7
6
6
5
4
4
3
2
2
1
0
1
2
3
4
5
6
7
8
9
10
11
Arrival Order (i)
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
12
Failure Recovery Time
ITU-T
o When recovery was a simple outage,
characterization was simple, too.
o IETF Benchmarking Methodology WG has
identified 5 possible recovery scenarios:
Lost packets
7
6
Induced delay
2
Out-of-order packets
Duplicate packets
6 5 4 4 3
1-3 October 2003
4 3
1
2
1
7 5 6 3 4
2
1
Errored packets
2
1
7
6
Workshop on End-to-End Quality of Service. What is it? How do we get it?
2
1
13
ITU-T
Implementations: Customers,
Service Providers & 3rd Parties
$ ping R2 (or R3)
R1
or
R2
R3
or
R4
o Select Ping Target - make Round-trip
connectivity and RTT measurement
o Accuracy Issues include path through
router, path through net (asymmetries),
response time at target, sampling rates
o Compare to current perf. to “normal”
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
14
ITU-T
Beyond ping: ICMP Timestamp
or Timestamp Reply Message
0
1
2
3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Type
|
Code
|
Checksum
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Identifier
|
Sequence Number
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Originate Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Receive Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Transmit Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
R3
Originate
Code=13
R1
R2
Receive
R4
Transmit
Code=14
o Time spent processing packet at target
can be removed, for more accurate RTT.
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
15
ITU-T
Implementations:
AT&T Global IP Measurements
AT&T GLOBAL IP BACKBONE INFRASTRUCTURE
BR
BR
(City 1)
BR
(City n)
(City 2)
MEASUREMENT
COLLECTION
SERVER (MCS)
Measurement Probes
Measurement
probe
AGGREGATED
MEASUREMENT
DATA
MEASUREMENT
AGGREGATION
& REPORTING
SERVER (MRS)
1-3 October 2003
http://www.att.com/ipnetwork
WEB
CLIENTS
(for report
viewing)
Workshop on End-to-End Quality of Service. What is it? How do we get it?
16
AT&T’s IP Measurement
Design
24 hours
...
ITU-T
15 minutes
o Poisson Sequence
(RFC2330)
•
•
•
•
•
•
1-3 October 2003
15 minute duration
 = 0.3 pkts/sec
Type UDP, IPv4
278 bytes total
~300 packets sent
unbiased sample
o Periodic Sequence
(RFC3432)
•
•
•
•
•
1 minute duration
Random Start Time
20 ms packet spacing
Type UDP, IPv4
60 bytes total
• ~3000 packets sent
Workshop on End-to-End Quality of Service. What is it? How do we get it?
17
Technical Collaborators at AT&T
ITU-T
o Len Ciavattone
o George Holubec
o Madhukar Kshirsagar
o Ron Kulper
o Arvind Ramarajan
o Gomathi Ramachandran
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
18
ITU-T
New Measurement Challenges
for MPLS-enabled IP Networks
o Most (all?) IP/Packet Network challenges
o Two main categories of MPLS Domains:
• LDP-based, connection-less
• Traffic Engineering, connection oriented
o Label Switched Paths are Unidirectional
o point to point and multi-point to point
o Many options for Failure Recovery
o LSP identity optionally removed (PHP)
o Work in progress in SG 13 = Y.MPLSperf
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
19
ITU-T
New Measurement Challenges
for MPLS-enabled IP Networks
Scope of OA&M Measurements:
single Network Section or MPLS Domain
MPLS Domain
MPLS Domain
Network
section
Network
section
Exchange
link
Exchange
link
Exchange
link
Network
section
Exchange
link
Exchange
link
MPLS Edge Node, or
MPLS Ingress Node, or
LSR if both IP and
MPLS are enabled
MPLS
Node
Label Switched Paths
Network Section Ensemble (NSE)
MPLS Network
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
20
ITU-T
New Measurement Challenges
for MPLS-enabled IP Networks
o New Protocols = New Opportunities to
Blackhole Traffic
o Detect this new class of failures with
• Y.1711 MPLS OA&M Connectivity Verific.
• First version approved, adding fast failure
detection
• LSP-Ping, Like ICMP Echo Request, plus
• One-way Delay measurement possible
• LSP Traceroute possible
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
21
ITU-T
New Measurement Challenges
for MPLS-enabled IP Networks
o New Availability Definition? Crossroad:
• Connection-Oriented Transport has used
a 10 second sliding window
• Connection-Less Packet Transport has
used a 5 minute fixed window
o MPLS Networks => both transport types
• When Connection-oriented Services use
a Connection-less transport, which
precedent should the Availability
Definition follow?
1-3 October 2003
Workshop on End-to-End Quality of Service. What is it? How do we get it?
22
Summary
ITU-T
o
Performance Management Framework
•
o
o
o
Summary of existing Parameters/Metrics
In-progress Metric Development
Active Measurement Implementations
•
•
o
1-3 October 2003
Measurement Systems are a key step
toward the goal of E2E QoS
Ping for connectivity and ...
Dedicated Measurement Systems
Parameter Framework for MPLS has new
challenges
Workshop on End-to-End Quality of Service. What is it? How do we get it?
23
Resources and References
ITU-T
o
o
o
o
1-3 October 2003
L. Ciavattone, A. Morton and G.
Ramachandran, "Standardized Active
Measurements on a Tier 1 IP Backbone,"
IEEE Communications Magazine, June
2003.
Geoff Huston, “Measuring IP Network
Performance,” The Internet Protocol
Journal, vol 6, no.1, March 2003
http://www.cisco.com/ipj
X.Xiao, et al., “A Practical Approach for
Providing QoS in the Internet Backbone,”
IEEE Communications Magazine,
December 2002.
D. Meyer, et al., “Trends in Measurement
and Monitoring of Internet Backbones,”
Panel at NANOG 26, slides etc. at
http://www.nanog.org/mtg0210/measurement.html
o
o
o
o
o
ITU-T Rec. Y.1540, “Internet Protocol Data
Communication Service – IP Packet
Transfer and Availability Performance
Parameters,” 2003.
IETF IP Performance Metrics Working
Group, links to RFC 2330, other IPPM
RFCs and Internet Draft on Reordering:
http://www.ietf.cnri.reston.va.us/html.chart
ers/ippm-charter.html
Draft New Recommendation Y.MPLSperf,
“Performance and Availability Parameters
for MPLS Networks”
Draft New Recommendation G.IPP,
“Performance Parameter Definitions for
Quality of Speech and other Voiceband
Applications Utilising IP Networks”
RFC 792, “Internet Control Message
Protocol,” J. Postel, September 1981.
Workshop on End-to-End Quality of Service. What is it? How do we get it?
24