STEVE PERRY, DIRECTOR OF NETWORKS UNM
PIYASAT NILKAEW, DIRECTOR OF NETWORKS NMSU
OVERVIEW
• WHY RESEARCH SPECIFIC NETWORKS?
• PRODUCTION NETWORK/SCIENCE_DMZ DESIGN BASICS
• SCIENCE_DMZ COMPONENTS
• CCIIE GRANT/RESEARCHERS REQUIREMENTS
• UNM DESIGN
POSSIBILITIES??
DESIGN CONSIDERATIONS
1.
TYPE OF R&E TRAFFIC – TCP –BASED, MICROBURST TRAFFIC THAT CAN QUICKLY
CONSUME ENTIRE AVAILABLE BANDWIDTH
A. SUBJECT TO TCP GLOBAL SYNCHRONIZATION
2.
3.
4.
5.
TCP TRAFFIC NEEDS DEEP BUFFER ON PORTS WHEN CONGESTION OCCURS.
NO COMMERCIALLY AVAILABLE SECURITY DEVICES CAN SIT IN-PATH WITH LINERATE PROCESS SPEED
100 GBPS BACKBONE ACROSS CONTINENTAL US
THE GENERAL RULE OF THUMB IS THAT YOU NEED 50MS OF LINE-RATE OUTPUT
QUEUE BUFFER FOR A 10G PORT, SO THERE SHOULD BE AROUND 60MB OF BUFFER.
RESEARCH NETWORK: SCIENCE DMZ
• A NETWORK OPTIMIZED FOR BUSINESS IS NOT DESIGNED OR CAPABLE OF
SUPPORTING DATA INTENSIVE SCIENCE.



Universities will always need to
support security features that protect
organizational financial and
personnel data.
Solution: create separate data
intensive science network, external
to university enterprise network
Design formalized by ESnet, based on
traditional network DMZ paradigm
BASIC SCIENCE DMZ
• SCIENCE DMZ: (1) DEDICATED ACCESS TO HIGH-PERFORMANCE WAN, (2) HIGHPERFORMANCE SWITCHING INFRASTRUCTURE (LARGE BUFFER MEMORY), (3)
DEDICATED DATA TRANSFER NODES
•
•
SCIENCE_DMZ COMPONENTS
DTN (DATA TRANSFER NODES—ORIGINATOR/RESPONDER)
•
•
HIGH CAPACITY SERVERS CAPABLE OF WIRE SPEED 10GBPS TRANSFER
GLOBUS GRIDFTP APPLICATION TUNED FOR LARGE DATA TRANSFERS
LARGE BUFFER CAPABLE SWITCHES TO SMOOTH TCP DROPS
•
•
MUST HAVE 60MB PER PORT BUFFER SPACE
MUST BE SDN CAPABLE
•
PERFSONAR MEASUREMENT NODES AT EACH LOCATION
•
BRO IDS (IDS VERSUS IPS, TO MINIMIZE DEEP PACKET INSPECTION)
•
OPEN DAYLIGHT SDN CONTROLLER
•
SUPPORTING STAFF
MANAGING BY MEASURING--PERFSONAR
• OFF CAMPUS / ON CAMPUS
• SERVICE TUNING - DEDICATED PERFSONAR
• BEYOND UNM / NMSU
•
•
HTTPS://PAS.NET.INTERNET2.EDU/MADDASH-WEBUI/
HTTP://PS-DASHBOARD.ES.NET/
HOW TO SECURE IT?
• USE BRO TO MONITOR IT OUT OF LINE
• IDS, NOT AN IPS
• REQUIRES FULL UNDERSTANDING OF BRO LIBRARIES AND EXPERTISE IN APPLICATION
STACKS
• ROUTER ACL OR SDN POLICY ON KEY SWITCHES FOR TRAFFIC ENGINEERING
• IPTABLES AT THE BOXES
CC*IIE GRANT
• NSF GRANT AWARDED TO UNM
• COLLABORATIVE AMONGST RESEARCHERS/IT
• INITIAL FUNDING TO BUILD OUT THE BASIC NETWORK
• HOPE TO APPLY FOR ADDITIONAL GRANTS AS AVAILABLE
UNM DESIGN
NMSU DESIGN
ISP 1
Smaller
Institutions
via MOE
ISP 2
Research
Network
I2 / AL2S
via
RGON
Internet
Edge1
Science DMZ
Internet
Edge 2
ASR 9010
10G
University
Campus
NMSU
Edge
1
Science DMZ
100G L2 x
2
Layer 2 EPG
Researchedge ASR9k
NMSU
Edge
2
40G
Fabric
Science DMZ
Spine
9500
EDGEFW-ICT
EDGEFW-MI
Spine
9500
SDN
SDN
CORE
1
Leaf
9300
CORE
2
Spine: N9336PQ
SDN Cluster
SDN
Leaf
9300
Leaf
9300
10G
Access/L2
Building 1
Acce ss switch
Building 1
Acce ss switch
Acce ss switch
Building 2
Acce ss switch
Building 2
Acce ss switch
Building 3
Acce ss switch
Building 3
Acce ss switch
Research
Building 1
Access switch
Research
Building 1
Access switch
Research
Building 2
Acce ss switch
Leaf:
N9396PX
Nexus or
Catalyst Access
Layer 2 EPG
Research
Building 2
Acce ss switch
Acce ss switch
SUMMARY
• WHY RESEARCH SPECIFIC NETWORKS?
• PRODUCTION NETWORK/SCIENCEDMZ DESIGN BASICS
• SCIENCEDMZ COMPONENTS
• UNM CCIIE GRANT/RESEARCHERS REQUIREMENTS
• UNM DESIGN