Current
Network Research
on UKLIGHT
Saleem Bhatti
Networked and Distributed Systems (NDS) Research Group
Computer Science, University of St. Andrews
http://www.dcs.st-andrews.ac.uk/~saleem/
St. Andrews, Computer Science
1
46PaQ & MASTS
• Overview
• Current Status and Results
• Future Work
• Questions
St. Andrews, Computer Science
2
• Overview
• Current Status and Results
• Future Work
• Questions
St. Andrews, Computer Science
3
Technology
Innovation
Network
Science
Applications
UKLIGHT Network projects
ESLEA
Applications and the Network
MASTS
Analysis at All Scales
in Time & Space
Real-time Monitoring
Towards
Next
Generation
Networks
Protocol Innovation
Behaviour And Performance
For QoS and Control
Data Acquisition,
Storage & Archiving
Compression (Model Free & Model Dependant)
St. Andrews, Computer Science
46PaQ
Data Analysis
and
Traffic Modelling
Front End Enhancements: Advanced Protocols
SuperJANET and UKLIGHT
4
46PaQ overview [1]
• IPv4 & IPv6 Performance and QoS
 EPSRC
 01 Dec 2004 - 30 Nov 2006 (June 2007)
• Sites:
 Computer Science, St. Andrews
 Computer Science, UCL
 Computer Science, QMUL
St. Andrews, Computer Science
5
46PaQ overview [2]
• Network & transport protocols at multi-Gb/s:
 UCL & St. Andrews lead
 New protocols (e.g. DCCP, TCP ‘extensions’):
» Collaboration with ESLEA
 Engineering of the protocols and end-systems
• Traffic monitoring at mulit-Gb/s:
 QMUL leads
 Multi-protocol analysis:
» Collaboration with MASTS
 Traffic modelling
St. Andrews, Computer Science
6
MASTS overview [1]
• Meausrement at All Scales in Time and Space
 EPSRC
 01 Sep 2004 - 31 Aug 2007
• Sites:




E&E Eng, UCL
Computer Science & E&E Eng, Loughborough
Computer Science, QMUL
Computer Science, St. Andrews
St. Andrews, Computer Science
7
MASTS overview [2]
• Traffic flow statistics
• Growth and scaling of network:
 Traffic
 Topology
• Monitoring systems:
 Data capture
 Data processing
• Traffic modelling
St. Andrews, Computer Science
8
• Overview
• Current Status and Results
• Future Work
• Questions
St. Andrews, Computer Science
9
46PaQ current experiments
• Connectivity to UKLIGHT:
 UCL: 2 × 1Gb/s to Chicago, remote loopback
• 1Gb/s loopback experiments on UKLIGHT from
Q1/2005 (London-Chicago)
• High-speed tests within the lab:
 bi-directional data flows (not just TCP/FTP)
 multiple data flows (not just single flows)
• Examine end-to-end data path:
 Interaction between application, hosts & network
 Performance tuning for high-speed operation
St. Andrews, Computer Science
10
TCP Throughput, Chicago-loop [1]
St. Andrews, Computer Science
11
TCP Throughput, Chicago-loop [2]
Multiple-flows, aggregate throughput
St. Andrews, Computer Science
12
Observations
• Verification of known results
• Protocol performance very much dependant on
several parameters (UDP buffer sizes, TCP window
size and sender recv buffer size, CPU/HW speed)
 For TCP, window size is important for paths with large
bandwidth-delay products
• Tuning of protocol stack and/or application
is therefore the most important
performance factor
St. Andrews, Computer Science
13
TCP ‘Fairness’ - Very Early Results
Matched bit-rate fairness (MBF)
Chicago-loop, 16MB window
St. Andrews, Computer Science
Chicago-loop, 32MB window
#
&
%
(
Nb
(t)
Fk (t) = 10log10 % N k (, k = 1,K,N
%
(
% " bn (t) (
$ n=1
'
14
MASTS current work
• Set up network monitoring systems
• UKERNA - AUP for monitoring
• Storage of data:
 Data has to be moved to processing site
 Lots of data - summarisation and compression
• Data processing and analysis:
 How to process? What are we looking for?
• Monitoring of ESLEA links
St. Andrews, Computer Science
15
Prototype MASTS monitor
Loughborogh University:
David Parish
Iain Phillips
Konstantinos Kyriakopoulos
Mark Whithall
St. Andrews, Computer Science
16
Example: No problems
St. Andrews, Computer Science
17
Example: Some Problems
St. Andrews, Computer Science
18
Example: Serious Problems
St. Andrews, Computer Science
19
• Overview
• Current Status and Results
• Future Work
• Questions
St. Andrews, Computer Science
20
46PaQ
•
•
•
•
Starting DCCP experiments
Interaction with ECN and DIFFSERV
Performance on IPv6
Continued co-operation with other projects:
 MASTS (monitoring)
 ESLEA (protocol work, e.g. DCCP)
St. Andrews, Computer Science
21
MASTS
• Monitoring systems at UKLIGHT PoPs:
 Monitor 1Gb/s and 10Gb/s
 Data storage and archiving
• Traffic and performance analysis:
 How to monitor performnace at such high speeds?
 Traffic models at different timescales
• Access network vs. Core network
St. Andrews, Computer Science
22
• Overview
• Current Status and Results
• Future Work
• Questions
St. Andrews, Computer Science
23
Additional slides
St. Andrews, Computer Science
24
Equipment setup
• End hosts (one as source, one as sink):




Gentoo Linux (AMD64 compiled), kernel 2.6.11
2x Opteron 246, 1MB L2 Cache, 2.4GHz
2GB PC3200 DDR RAM on dual banks
Intel PCI-X 133MHz 64Bit dual 1Gb/s NIC
• Local switch:
 Cisco Catalyst 3750
 2x VLAN, one for each of the 1Gb/s links to Chicago
• Tests run for two configurations:
 Local-switch loopback 1Gb/s (RTT fraction of ms)
 Chicago loopback 1Gb/s (RTT ~177ms)
St. Andrews, Computer Science
25
TCP Baseline Tests
• The following TCP settings were used:









net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_rmem = 4096 87380 67108864
net.ipv4.tcp_wmem = 4096 87380 67108864
net.ipv4.tcp_mem = 196608 262144 67108864
net.core.rmem_default = 126976
net.core.wmem_default = 126976
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
• All experiments conducted using iperf
St. Andrews, Computer Science
26
TCP Throughput, local-loop [1]
St. Andrews, Computer Science
27
TCP Throughput, local-loop [2]
Multiple-flows, aggregate throughput
St. Andrews, Computer Science
28