TCP Performance over Multilink PPP in
Wireless Networks:
Theory and Field Experiences
Abdul Jabbar Mohammad, Said Zaghloul*,
and Victor S. Frost
Information and Telecommunication Technology Center
Electrical Engineering & Computer Science
frost@eecs.ku.edu, 785-864-4833
*Currently working at Sprint in Network Services Gateway
in the Data Network/Network Development department for Brent Scott
A KTEC Center of Excellence
1
Outline
•
•
•
•
Motivation
Technology
Field Experiments
Analytic Prediction of TCP over MLPPP with
Call Drops
A KTEC Center of Excellence
2
Motivation
•
Polar Radar for Ice Sheet Measurements (PRISM)
•
The communication requirements of PRISM field experiments in Greenland and
Antarctica
– Data telemetry from the field to the University
– Access to University and web resources from field
– Public outreach
•
Mainstream communication system for polar science expeditions, field camps in
Arctic/Antarctic and other research purposes
•
•
Government and security use
Solution:
•
Implement a multi-link point-to-point Iridium communication system to combine
multiple links to obtain a single logical channel of sufficient aggregate bandwidth.
A KTEC Center of Excellence
3
Technologies-Iridium
Modulation Technique
QPSK
Frame Structure
FDMA/TDMA
Frequency
L-band from 1610 to 1626.5 MHz
Inter Satellite Links Frequency
Ka band from 23.18 to 23.38 GHz
Ground Segment Links
Ka band: Uplink:
29.1 – 29.3 GHz
Downlink: 19.4 – 19.6 GHz
Ground-Based Digital Switches
Siemens GSM-D900
Multiple Access Technique
FDMA / TDMA
Digital Voice and Data Rate
2.4 kbps
Error Protection
FEC 3/4
Channel Bandwidth
31.5 Khz with 41.5 Khz guard bands
Inter-Satellite Handover
9-10 min
ann
-pl
a
r
Int
ISL
er
Int
A KTEC Center of Excellence
erpla
n
ne
rI
SL
4
Technologies-Protocols
Remote System
Local System
Application
Application
HTTP, FTP, SSH
HTTP, FTP, SSH
TCP
TCP
IP
IP
PPP/MLPPP
point-to-point satellite links
PPP/MLPPP
Physical Modems
Physical Modems
A KTEC Center of Excellence
5
Technologies-Multi-Link Point-to-Point Protocol
• Multilink option are negotiated when
establishing the connection.
• Packets may be fragmented.
Network Layer
Network Layer
MLPPP PDUs
4
2
5
Link 1
1
6
6
Link 2
3
7
MLPPP
MLPPP
MLPPP PDUs
reassembled into
the original layer
3 packet
Link n
Layer 3 Packet
MLPPP
fragments layer 3
packets
MLPPP fragments have
non-decreasing
sequence
2
numbers
Receiver
Sender
A KTEC Center of Excellence
6
Technology- Network Architecture
World Wide Web
SUMMIT Camp, Greenland
or WAIS camp in Antarctica
100 Mbps
Ethernet
(Default gateway)
(Default gateway)
PPP Server
user 4
PPP Client
ITTC Default
Router
eth0
User 2
100 Mbps
Ethernet
ppp0
P-T-P Satellite link
ppp0
eth0
User 1
user 3
User 3
Camp
WI-FI
user 2
user 1
ITTC Network, University of Kansas
A KTEC Center of Excellence
7
Summer 2003 Field Experiments
Test Location
A KTEC Center of Excellence
8
2003 Results – Throughput
Method
1 Modem
2 Modems
3 Modems
4 Modems
Iperf
2.1
4.0
7.0
9.6

Tools used – TTCP, IPERF

Throughput varies to some
Iperf
1.9
3.9
7.0
9.3
extend due to RTT variation
Iperf
1.7
4.5
6.8
9.7
Ttcp
2.29
4.43
6.6
8.9
Ttcp
2.48
4.40
7.0
8.78
Average
2.1
4.25
6.88
9.26

Efficiency > 90%
Effective throughputs during large
file transfers
Throughput
(bits/sec)
File Size (MB)
Upload Time (min)
0.75
11
9091
3.2
60
7111
1.6
23
9275
2.3
45
6815
1.5
28
7143
2.5
35
9524
Throughput (K b p s)
Throughput Vs Number of Modems
10
9
8
7
6
5
4
3
2
1
0
9.26
6.88
4.25
2.1
1
A KTEC Center of Excellence
2
3
Number of modems
4
9
Applications – Uploads and Downloads
Files were downloaded to support the science and operations of the
camp. The importance of each file to the user is noted on a
subjective scale of 1-10,10 being the most valuable.
Title
Downloaded/uploaded
Size
Imp
1
Spectrum Analyzer programmers
Manual
Download from Agilent.com
7.2MB
9
2
Matlab Programs
Download from ITTC
500KB
7
3
Voltage regulator data sheet
Download from Fairchild.com
226KB
9
4
GPS software
Download
800KB
9
5
Proposal submission
Upload
600KB
8
6
Access point manager software
Download from Orinoco.com
4.66MB
7
7
Drawing of machine spares to order
Upload to University of Copenhagen
1MB
9
8
Video of core, datasheet
Upload for press release
2MB
8
9
Pictures, press release of longest core
in Greenland
Upload to Kangerlussauq for press release
500KB
6
A KTEC Center of Excellence
10
Eight Modem Iridium System: 2004/5 Field Experiments




The Modem/Computer box is a 19”
rack mount 5U equivalent
The front panel is 8.72’ tall and 19”
wide. The sides are 8.34” tall and 24”
deep.
Weight approximately 45lbs.
Reproduction cost= ~$18,000
Iridium Modems
Ethernet
USB
A KTEC Center of Excellence
11
Field Experiments – System Implementation
8-Channel system in a weather-port at
SUMMIT camp in Greenland, July 2004
A KTEC Center of Excellence
12
Field Experiments – Antenna Setup
4 ft
10 ft
8 Antenna setup at SUMMIT camp in Greenland, July 2004
A KTEC Center of Excellence
13
Results – Throughput
Throughput (Kbps)
Variation of throughput with number of modems
20
18
16
14
12
10
8
6
4
2
0
18.60
16.43
13.90
12.08
8.98
6.93
4.97
2.49
1
2
3
4
5
6
7
8
Number of modems

Average throughput efficiency was observed to be 95%

The above results are from the test cases where no call drops were experienced

In event of call drops the effective throughput of the system will be less than the above values
A KTEC Center of Excellence
14
Results – Throughput
FTP throughput observed during data transfer between the field camp and KU
Size of file in MB
Approx. Upload Time
Effective Throughput in Kbps
1.38
0:11:24
16.53
3.77
0:35:42
14.42
5.62
0:46:12
16.61
15.52
2:30:00
14.12
20.6
3:00:00
15.62
35.7
5:15:00
15.47
55.23
9:00:00
13.96

Average throughput for FTP upload of large files was observed to be 15.38 Kbps

Due to call drops, the efficiency was reduced to ~80%
A KTEC Center of Excellence
15
Results – Round Trip Time
Variation of RTT
Round trip time during different times of the day
4000
min
2500
avg
mdev
3000
1952
2000
2500
RTT in msec
RTT in msec
3500
2000
1500
1000
500
0
1495
1500
1291
1075
1000
930
710
1436
1304
1232
801
608
681
1244
1020
891
748
995
820
920
760
587
500
1
6
11
16
21
26
31
36
41
46 51 56
Time in sec
61
66
71
76
81
86
91
96
0
8:40
9:02
10:34
11:14
11:45
Time
11:56
12:45
Variation of RTT
7000
RTT in msec
6000

Average RTT = 1.4 sec

Minimum observed RTT = 608 msec

Mean deviation = 800 msec
5000
4000
3000
2000
1000
0
1
6
11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96
Time in sec
A KTEC Center of Excellence
16
Results – Reliability: 14th July 12-hr test
Uptime %
89
95
96
97
97
97
97
98

Call drop pattern during 8 Iridium – 8 Iridium DAV mode test for 12 hrs

Percentage uptime with full capacity (8 channels) is 89% and with at least one modem is 98%

Total number of primary call drops during 12 hrs = 4
A KTEC Center of Excellence
17
Results – Reliability: 22nd July 32-hr test
Uptime %
85
92
93
93
94
94
94
96

Call drop pattern during 8 Iridium – 8 Iridium DAV mode test for 32 hrs

Percentage uptime with full capacity (8 channels) is 85% and with at least one modem is 96%

Total number of primary call drops during 32 hrs = 24
A KTEC Center of Excellence
18
Results – Reliability: 19th July 6-hr test
Uptime %
67
81
85
85
85
85
85
90

Call drop pattern during 8 Iridium – 8 PSTN data mode test for 32 hrs

Percentage uptime with full capacity (8 channels) is 67% and with at least one modem is 90%

Total number of primary call drops during 6 hrs = 9
A KTEC Center of Excellence
19
Results – Mobile tests
Iridium antennas
Experiments monitored from
another vehicle through 802.11b
link
Iridium system mounted
in an autonomous vehicle
(MARVIN)
A KTEC Center of Excellence
20
Results – Mobile tests
Uptime %
65
79
82
84
84
85
87
92

Call drop pattern during 8 Iridium – 8 Iridium DAV mode test for 2 hrs

Percentage uptime with full capacity (8 channels) is 65% and with at least one modem is 92%

Average time interval between call drops is ~ 45 mins

Average throughput = 18.6 Kbps, Average RTT = 2 sec
A KTEC Center of Excellence
21
2004 Applications

Summer 2004 field experiments

Communications data upload – up to 40 MB files

Radar data uploads – up to 55 MB files

Text chat with PRISM group at KU

Video conference - real time audio/video

Individual audio or video conference works with moderate quality with the
commonly available codecs


Outreach Use

Daily Journal logs uploaded

Daily Pictures uploaded

Video clips uploaded

Held video conference with science teachers/ virtual camp tour
Wireless Internet access
A KTEC Center of Excellence
22
2005 WAIS Field Experiments
WAISWest
Antarctica
Ice Sheet
A KTEC Center of Excellence
23
2005 WAIS Field Experiments
Applications –
Uploads and
Download
Files were
downloaded to
support the
science and
operations of the
camp. The
importance of
each file to the
user is noted on a
subjective scale of
1-10,10 being the
most valuable.
Item
Size
Importance
Component data sheets
2.2 MB
8
Oscilloscope lab measurements
2 MB
9
Modified code for SAR
measurements
500 KB
10
C++ IDE
10 MB
5
GIS scripts
2 MB
7
GPS troubleshooting manual
10 MB
10
PICO editor
4 MB
4
Outreach pictures, journal and
weather data upload
500 KB/day
10
Video conference
Variable
6
Virtual dashboard application
50 MB
9
Critical data internet search by the
drilling team
Variable
10
Internet/email access to all field
personnel at WAIS camp
Variable
7
Remote ssh access to field programs
from KU
Variable
9
A KTEC Center of Excellence
24
Analytic Prediction of TCP over MLPPP
with Call Drops
• A call drop is the event of losing an
established connection suddenly
• Connections are automatically re-established
• It was observed that a call drop results in
TCP timeouts
• Various reasons that might lead to call drops,
• Low signal level
• Failure of the inter-satellite handovers
• Goal: Predict the throughput as a function of
drop rate and other system parameters
• First step: Call Drop model
A KTEC Center of Excellence
25
Call Drops - Distribution
• 394 call-drop measurements were collected
in the field
• Call drop pdf~exponential
• The single link ICTD
is a Poisson random
process with a rate b
• A N Link bundle’s
ICTD is a Poisson
process with a
dropping rate of:
l=N b
ICTD PDF based on Greenland–Kansas measurements.
Estimated exponential distribution (0.02exp(-0.02t)) passes
the chi-square goodness-of-fit test (5% significance level and 14 bins)
A KTEC Center of Excellence
26
Call Drops - Distribution
• KSGreenland
call dropping
rate per link
is
1/50 min-1
• KS-KS call
dropping rate
per link is
1/52 min-1
A KTEC Center of Excellence
27
TCP Performance Model
• TCP transfer latency for fs bytes given the
MSS is
 fs 
Td  
B

 MSS 
(sec)
• To estimate TCP throughput (B) in
packets/sec:
a. Evaluate the throughput if no timeouts take place
b. Extend the no timeout throughput using the empirical call
drops PDF to include timeouts
• Main Assumptions
•
•
Packet losses are due to ARQ failures (no timeouts)
Timeouts are caused by call drops only
A KTEC Center of Excellence
28
Methodology
• Modify exiting results to account for call drops:
J. Padhye, V. Firoiu, D. Towsley, and J. Kurose,
“Modeling TCP Reno performance: a simple model and
its empirical validation,” IEEE/ACM Trans.
Networking, vol. 8, pp. 133-145, Apr. 2000.
• For details of the modification see: Modeling TCP
Long File Transfer Latency over Long Delay Wireless
Multilink PPP, Said Zaghloul, Victor Frost, Abdul
Jabbar Mohammad; IEEE Communications Letters,
Vol. 9, No. 11; November 2005, pp. 988-990.
A KTEC Center of Excellence
29
Model Validation with Experiential data
File Transfers from Greenland to the University of Kansas
(Summer 2004),
T0 = 60s, p = 5E-4, b = 1/50 min-1, MSS = 1448, RTT =
19s, Wmax= 47.9KB
File Size (MB)
1.38
5.62
20.6
35.7
Measured Transfer Time (min)
11
46
180
315
Predicted Transfer Time (min)
12
51
187
324
Error %
13
11
4
3
Number of Links
3
4
5
6
7
8
File Size (MBytes)
4.82 0.85 1.91 1.39 3.40
1.40
Wmax (KBytes)
16.1 22.0 28.3 34.7
41
47.9
30
12
Measured Time (min)
Prediction (min)
96
15
21
13
98.1 13.4 24.1 15.4 33.2
A KTEC Center of Excellence
12.7
30
Case Study: Increased Dropping Rates
• A software module was
built and added to the
developed link
management software to
increase “real” drop rate
• The added module
generates call drops
according to a Poisson
process for any given
dropping rate
A KTEC Center of Excellence
31
Case Study: Effect of Wireless Errors
• A wireless error refers to the errors that the
physical layer ARQ could not handle
• Effect is amplified for low bandwidth-long delay
connections (ex. Iridium)
• An efficient ARQ mechanism minimizes wireless
errors




Inmarsat GEO, bundle =4
RTT = 0.61 s, BW = 128 kbps
MSS=1 KB and Wmax=40 KB
A slight increase of the packet loss
probability results in approximately 25 min
increase in the transfer time
A KTEC Center of Excellence
32
Information and Telecommunication
Technology Center
A KTEC Center of Excellence
33