Taiwan Advanced Research and
Education Network (TWAREN) Current status & Future Plan
Dr. Te-Lung Liu
Researcher
National Center for High-Performance Computing
tlliu@nchc.narl.org.tw
Outline
TWAREN Network Overview
Development and Research
Technologies
2
TWAREN Network Overview
Development and Research
Technologies
3
4
What is TWAREN
A physical network serves multiple purposes and
logical networks
TANet, connects to commodity Internet
TWAREN research network
experiment, testbed, special research
Provisioning services on multiple layers
L1 lightpaths
L2 VLAN
L3 IP
has been successfully migrated from old backbone
in Oct 2006
5
TWAREN Architecture
• 4 core nodes
• 20G backbone
• 12 GigaPops
• Connects HPC resources in
North and South Taiwan
6
Goals of TWAREN
TWAREN is part of “Challenge 2008”, a comprehensive sixyear national development plan formulated by the
government
Build a highly reliable, stable and flexible R&E network for
academic and research community in TW
Provide advanced network services to satisfy the needs of
academia field in TW.
Increase the International and domestic collaboration
Future infrastructure drives today’s research agenda
7
TWAREN GigaPoPs
8
TWAREN Services
■
■
■
■
■
■
■
■
Broadband Connection
Service
International Research
Network Transit (Internet2)
Measurement / Network
Management
Multimedia / Multicast
Lightpath provisioning
Virtual Private Network(VPN)
Native IPv6 Service
Internet access






MCU
Proxy Server
SourceForge
File Download Center
Consultation
Applications support
9
TWAREN Achivements
High reliability & availability
(99.9%  99.99%)
 fault tolerance
 automatic protection if possible
 automatic failure detection and locating
Better performance: minimum number of routers between
GigaPoPs
Flexible: can be easily and quickly to set up a logical network
per user’s request
People skills: Optical network OAM
10
Optical Backbone
ASCC
NTU
STM-64
STM-16
NIU
NCU
NDHU
TP
NCTU
TC
HC
NCHU
TN
NTHU
ONS15600
ONS15454
NSYSU
NCKU
CCU
11
Interconnecting with L2/L3 devices
STM64
STM16
10GE
GE
NTU
ASCC
NCCU
NIU
NDHU
NCU
NCNU
NHLTC
Taipei
Hsinchu
Taichung
NCTU
Tainan
NCHU
NTTU
NTHU
NSYSU
NCKU
CCU
ONS15600
7609
ONS15454
6509
GSR
3750
12
Protection Mechanism
Circuit break:
2 levels of protection
 By carriers: SDH protected
 By architecture:
Link b/w core nodes: VLAN are reconfigured with
rapid spanning-tree protocol. (5s)
 Link b/w GigaPOP and core node: the backup
SNCP lightpaths are configured for automatic failover. (50ms)

13
Protection Mechanism
Equipment protection
 Core node failure: Manually configure emergency
lightpaths to re-route traffic from affected GigaPoPs to
another core node. Emergency lightpaths need to be
designed and documented.
 GigaPoP failure: Spare line cards
14
Normal Traffic Flows
STM64
STM16
10GE
GE
NTU
ASCC
NCCU
NIU
NDHU
NCU
NCNU
NHLTC
Taipei
Hsinchu
Taichung
NCTU
Tainan
NCHU
NTTU
NTHU
NSYSU
NCKU
CCU
ONS15600
7609
ONS15454
6509
GSR
3750
15
In case of circuit break...
STM64
STM16
10GE
GE
NTU
ASCC
NCCU
NIU
NDHU
NCU
NCNU
NHLTC
Taipei
Hsinchu
Taichung
NCTU
Tainan
NCHU
NTTU
NTHU
NSYSU
NCKU
CCU
ONS15600
7609
ONS15454
6509
GSR
3750
16
In case of core node failure...
STM64
STM16
10GE
GE
NTU
ASCC
NCCU
NIU
NDHU
NCU
NCNU
NHLTC
Taipei
Hsinchu
Taichung
NCTU
Tainan
NCHU
NTTU
NTHU
NSYSU
NCKU
CCU
ONS15600
7609
ONS15454
6509
GSR
3750
17
TWAREN NOC
NOC (Network Operation Center)
Located at NCHC southern business unit in Tainan Science
Park
Goals: To ensure the 7x24 network operation
Major works:



Providing 7x24 network maintenance and operation
Enhance the security capacity
Provide network service



Peering
Light path provision
Network architecture design
TWAREN NOC
18
TANet VPN
MOEcc6509
NDHU6509
NCCU6509
NHLUE6509
NTU6509
One Subnet L2 VLAN
NCU6509
NCHU6509
TP7609C
L2 Switch
HC7609
TANet VLAN
TC7609
TC7609C
L2 Switch
HC7609C
L2 Switch
NCTU6509
TN7609C
L2 Switch
NTHU6509
NSYSU6509
CCU6509
NTTU6509
TN7609P
NCKU6509
19
TWAREN Research VPN
TANet (MOEcc6509)
TAIWANLight
TWGATE Internet
ASCC APAN
ISP Peering
TP12816R
iBGP RR
TP12816P
NDHU7609P
NIU7609P
ASCC7609P
NCTU7609P
NTU7609P
NCU7609P
TP7609C
Switch
NCHU7609P
TC12816P
TC7609C
Switch
Research
VLAN
HC12816P
HC7609C
Switch
HC12816R
TC12816R
TN7609C
Switch
NCNU7609P
HC7609P
NTHU7609P
NSYSU7609P
CCU7609P
TN7609P
TN12816R
iBGP RR
TAIWANLight
NCKU7609P
TN12816P
TAIWANLight
ISP Peering
20
VPN Services
Multipoint-to-Multipoint Layer2 VPN
(VPLS)
Multiple VPNs over single architecture
Cross-area campuses and offices can be
connected within single administrative
domain
 Provide dynamic creation of VPNs for Nationalwide integrated projects
User-based SSL VPN Access
Access to different VPN according to
login name and password authentication
Researchers and Professors could
access their own research resources from
home or outside
21
VPLS Architecture
22
User-Based SSL VPN Access
Core node@
HsinChu
Users
Org 1
SSL
VPN
Org 2
TWAREN
VPLS
Backbone
Web Browser
Org 3
SSL
VPN
。
。
。
Core node@
Tainan
Org n
23
TWAREN’s International Connections
 Pacific Crossing to USA’s west coast upgraded to 5 Gb/s
 Connections between LA, Palo Alto, Chicago, and New York are 2.5 Gb/s
 Connects to the rest of the world via the U.S.’s Abilene Network
Connection expanded to Europe in 2006 (IEEAF donated 622 Mbps of
bandwidth/fiber optic cable)
24
Combined TWAREN/TAIWANLight
Lambda Testbed
NCTU-15454 NCU-15454
NTU-15454 ASCC-15454
NTHU-15454
Chicago
15454
TP-15454
HC-15454 HC-15600
TP-15600
NIU-15454
TN-15454 TN-15600
Palo Alto
15454
NY
15454
TC-15454
NDHU-15454
NCSYSU15454
NCKU-15454
CCU-15454
TWAREN
Optical Network
NCHU-15454
LA
15454
TAIWANLight
25
TWAREN’s International Peerings
 TWAREN made peerings with international NRENs at Los Angeles,
Chicago, New York and Seattle (through Pacific Wave).
26
TWAREN’s Direct Peerings Coverage
 TWAREN's direct peering covers most area in America, Asia,
Australia and New Zealand, and will soon be expanded to Europe.
27
TWAREN/TAIWANLight and GLIF
TWAREN is a member of GLIF (Global Lambda
Integrated Facility)
TAIWANLight is an official optical exchange GOLE (GLIF Open Lightpath Exchange)
28
TWAREN Network Overview
Development and Research
Technologies
29
Future Internet Testbed
@ Taiwan
30
Future Internet
There are many serious limitations
in current Internet.
Scalability
Security
QoS
Virtualization
Future Internet is a summarizing
term for worldwide research
activities dedicated to the further
development of the original Internet.
(From Wiki)
31
Future Internet Testbed
For innovations and researches in
Future Internet, the testbed
requires some advanced concepts:
Programmability
Virtualization
End-to-end slice
32
OpenFlow
Make deployed networks programmable
Makes innovation easier
No more special purpose test-beds
Validate your experiments on production
network at full line speed
33
TWAREN OpenFlow Testbed in 2010
NOX
NCHC
OpenFlow
Switch
OpenFlow
Network
@KUAS
iCAIR
Capsulator
TWAREN
L3
Network
Capsulator
Capsulator
OpenFlow
Network
@NCKU
NCKU and KUAS are pilot universities that connected with the Testbed
The OpenFlow Testbed is extended to iGENI@iCAIR
Capsulator (Ethernet-in-IP tunnel) is used to emulate pure L2 network for OpenFlow
34
34
TWAREN OpenFlow Testbed in 2011
NCU
NCHC
OpenFlow
Switch
OpenFlow
Switch
TWAREN
VPLS
Capsulator
NCKU
KUAS
OpenFlow
Switch
OpenFlow
Switch
OpenFlow
Switch
NTUST
OpenFlow
Switch
CHT-TL
OpenFlow
Switch
Capsulator
NTUST, NCU and CHT-TL joined the Testbed.
For TWAREN connectors (NCKU, KUAS and NCU), a dedicated VPLS VLAN is
allocated for better transmission performance.
35
35
Emulab/ProtoGENI Testbed
TWISC (Taiwan Information
Security Research and Education
Center) operats 206 nodes of
Emulab Testbed in Taiwan.
 Third largest Emulab in the
world
 Testbed@TWISC is operated
by NCKU team and co-located
in NCHC
A portion of the testbed is planned
to try ProtoGENI test with
University of Utah.
 A lightpath is provisioned
between NCHC and iCAIR
shared by both OpenFlow and
Emulab/ProtoGENI
Control Network
Firewall
Firewall
Firewall
Firewall
Experiment
Network
Experiment
Network
BOSS
OPS
Experiment
Switch
36
36
Lightpath and VLAN setup
Lab
Vlan 2782
iCAIR
Trunk
Vlan 462
Vlan 2782
7609P@HC
Vlan 462
Vlan 2782
Vlan 462
Vlan 2782
Vlan 1548
Vlan 1555
7609P@TN
Vlan 462
Vlan 1548
7609V NCKU
Vlan 462
Trunk port
Emulab/ProtoGENI – Vlan 462
iCAIR OF (with NCKU) – Vlan 2782
Vlan 1548
Vlan 1555
Vlan 2782
NCKU
Emulab @NCHC
OF sw A
NCHC
OF sw B
7609V NCKU EE
Vlan 462
Vlan 1548
Emulab/ProtoGENI – Vlan 462
NCKU OF (with iCAIR) – Vlan 1548
37
37
iGENI - Taiwan Integrated Research
Network
38
38
Multi-Domain OpenFlow Management
Each network domain has its own OF Controller
Each Controller manages topology and flow
provisioning inside the domain
Inter-domain flow could be made by connecting
partial flows provisioned by controllers of each
cloud
Lack of global view for inter-domain flows
No loops allowed for inter-domain topology
Difficult to support QoS or SLA functions
across domains
Inter-domain topology auto-discovery is required
for multi-domain management
39
39
Inter-Domain Topology Discovery (I)


OpenFlow Controller just only knows its directly
connected switches.
ENVI is a useful GUI tool to show OpenFlow
topology under single controller.
OFA
OFB
OFC
OFD
Controller1
Controller2
Topology of Domain1
Topology of Domain2
Domain
UI
OFA
OFB
OFC
OFD
40
40
Inter-Domain Topology Discovery (II)


We add additional contents in LLDP packet to
let Controllers have its neighbors’ connectivity
details.
ENVI is also modified to show the whole
topology.
OFA
OFB
OFC
Controller1
OFD
Controller2
Domain
UI
Topology of Domain1 & 2
OFA
OFB
OFC
OFD
41
41
Results
Multi-Domain Network Topology
shown in GUI
Physical OpenFlow Network Topology
42
42
GLIF & SC11 Demo
Joint Demo among
NCHC/TW, iCair/US,
and CRC/Canada
43
Information Security
Activity Detection over
High-Speed Backbone
44
Security Detection over HighSpeed Backbone
Normally, we don’t install IDS/IDP in backbone for
performance issue.
 IDS/IDP are placed at user’s local sites
Backbone traffic is hard to mirroring due to its large
amount and high-speed
 It’s impossible to do packet analysis
 Packet header analysis is available with
Netflow/sFlow
Information Security Activity Detection over
High-Speed Backbone
 Integrate fast packet header analysis with attack
information from user’s local site
45
System Architecture
Invasion and attack info
from user’s local sites
Netflow Data
from Backbone/User Routers
Users’ IDS/IDP
Users’ HoneyPot
Users’ Log analyzer
Security Detection
@Backbone
Collect
Backbone’s
Netflow data
Users’
Netflow data
Search
Notify User with
Suspicious Activities
Orientation
Trace-back
Notification
Block
Backbone network,
peering partner,
User network
46
Design Concepts
Distributed Computing
 For monitoring netflow data in real-time
Fast Search
 Effective Tree-Searching algorithm
Expandable
 Simply add more machines when larger data analysis is required
Remote Backup
 Separate different computing nodes in order to provide robust
analysis service
Single Portal
 All input can be submit to single portal with Global Server LoadBalancing technology
Cooperate with Researchers/Developers
 Will design an open API for developers to contribute their own ideas
47
Design Blocks
Router1
Router2
Router3
RouterN
Netflow packet
Netflow packet
Netflow packet
Filter 1
Filter 2
Filter 3
Matched Netflow raw
Analyzer 1
Blacklist Search Tree
Distributor 2
Distributor 1
Filter N
Matched Netflow raw
Analyzer 2
Analyzer 3
Analyzer 1
P3333
Analyzer 2
P4444
Analyzer 3
P5555
Analyzer N
Update Search Tree
result
result
result
Controller 1
Controller 2
IDS/IDP
Honey...
Syslog
Update Blacklist
Blacklist
IP
Port
Type
Analyzer
Analyzer Port
A.A.A.A
1234
botnet
1
3333
B.B.B.B
4321
Fake-IP
2
4444
C.C.C.C
1122
Cracker
3
5555
……
48
Numerical Results of Tree Creation
49
Numerical Results of Real-time Matching
50
51
52
53
54
55
56
57
Thank You !
For more information, please see :
www.twaren.net
- 2011 -
58