Network & Services Overview
June 2012
Jeff Ambern
jambern@grnoc.iu.edu
Agenda
•
•
•
•
Indiana GigaPOP Overview
Services
Commodity Internet Usage Trends
Monon100 (100Gbps to Chicago)
Indiana GigaPOP
• Established in Indianapolis in 1998
• Partnership between Indiana University and Purdue University
• Advanced high-speed, high-availability, feature-rich network
• Drives down the costs and increases the connection speeds for
Indiana’s top research colleges and universities
• Serves as an aggregation point for Indiana’s universities to
access (regional, national and international) R&E and
Commodity Internet.
GigaPOP Participants
•
•
•
•
•
Indiana University
Purdue University
I-Light
Notre Dame
RLHEC
•
•
•
•
SLIAC
ENA
CSPAN Archives
NCAA
GigaPOP Providers & Peers
Providers (Commodity)
• Cogent
• TimeWarner
• Iquest (Local only)
• Smithville (Local only)
• Wintek (Local only)
• Internet2 TRCPS
• WiscNet RPS
• Akamai (Cache)
• Google (Cache)
Peers (R&E)
• Internet2
• National LambdaRail
• MREN
• CIC
• ESnet
Network Elements
• Juniper MX960 (Core Nodes)
• Fully redundant chassis hardware (cooling, power supplies,
Routing Engines, SCB’s)
•
•
•
•
HP 5400 (Layer2 Backbone Switches)
Brocade MLXe (Layer2 Backbone Switch)
HP 3500 (Management Switches for OOB)
Mix of Cisco Routers (Terminal Servers, VPN, Mgt.)
Current Topology
GigaPOP Capabilities
• Layer3
•
•
•
•
R&E Access (National, Regional and Local)
Commodity Internet Access
L3VPN/MPLS
Multicast
• Layer2
• L2VPN/MPLS
• VLANs
• Layer1/2
• 100G Ethernet (Fiber)
• 10G Ethernet (Fiber)
• 1G Ethernet (Fiber, Copper)
Access to Research Networks
R&E (Base Service)
• National/International/Regional Networks (Internet2 (100G), NLR,CIC,
MREN, Esnet)
• Local R&E (Gigapop Participants, State Universities)
R&E Service includes IPv4 plus:
• IPv6 (we can provide IPv6 address space to members)
• Multicast
• Jumbo Frames (9000 bytes)
Commodity Internet
(Optional to Participants)
•
•
•
Full Transit
• Complete Internet Routing Table
• ~410K Routes
Settlement-Free Internet
• Partial Internet Routing Table
• ~240K Routes
• Commercial Entities agree to peering arrangements with I2
(TRCPS) or other GigaPOPs to save on upstream BW charges
Internet Caching Servers
• Similar to a Settlement-Free peering arrangement but involving only
one entity (i.e. Google, Akamai, Netflix (future peering))
• Server Farm and cached data is local to customer
• Does not consume Internet Transit bandwidth for cached data
(except during incremental updates)
Caching Services
Why is caching beneficial to Participants?
• Offloading commercial traffic to local servers
reduces our costs that we would have to pay to
our per Mbps transit providers
• Reduces the amount of Commodity traffic that
would congest our upstream links and force us
into costly upgrades (I2 TRCPS, Cogent and
TWTC)
• Reducing our costs allows us to pass savings
on to our Participants
Caching Services
Akamai server farm added late 2009
• 1Gbps 95%
• 500Mbps daily average we would have to pay to our upstream
providers or consume BW on our settlement-free links
Google Caching Services added 12/2011
• 1.3Gbps 95%
• 730Mbps average
Netflix Caching Services / Peering - Late 2012
• Currently working with Netflix on traffic analysis
• Depending on usage will either Peer at IXP or install cache node
in Indpls.
Commodity Usage - Gbps
• Cogent (10G) & TimeWarner
(10G)
• Full Internet Transit
• Fully Redundant
Upstreams
• Full IPv4 & IPv6 Transit
• Internet2 TRCPS Commodity
Peering Service (10G)
• Reduces cost through
aggregation and
settlement-free peering
• WisNet Regional Peering
Service (10G) – Via our CIC
membership
• Akamai Caching Service (10G)
• Google Caching Service (10G)
TimeWarner
1.28
1.82
Cogent
1
2.67
Wiscnet Regional Peering
Service
I2 Transitrail CPS
4.09
Akamai
3.03
Google
Bandwidth Trends
Commodity Internet Usage Trends (Aggregate Upstream)
• 40-70% increase in usage per year 2009-2012
•
•
•
•
Sept 2009 – 2.5G Avg (Cost/Gbps $10000) – 25K/Month
Sept 2010 – 6.5G Avg (Cost/Gbps $6000)
Sept 2011 - 9.5G Avg (Cost/Gbps $5000)
July 2012 – 13.0G Avg (New Pricing $1750/Gbps) – 22.27K/Month
• Able to keep member cost down due to decreases in
provider pricing and Settlement-Free Peers
Aggregate Commodity (Gbps)
$/Gbps
14
12000
12
10000
10
8000
8
6000
6
4000
4
2000
2
0
0
2009
2010
2011
2012
2009
2010
2011
2012
Commodity Bandwidth 20092012 24hr Avgs
Monon100
Monon100
•
Monon100 is named after the Monon Rail line that connected Indiana's higher
education institutions to the rest of the world through Chicago
•
The Monon Rail served six colleges and universities along its line (from Chicago
to Louisville):
o
o
o
o
o
o
Purdue University in West Lafayette, Indiana
Wabash College in Crawfordsville, Indiana
DePauw University in Greencastle, Indiana
Indiana University in Bloomington, Indiana
Butler University in Indianapolis, Indiana
St. Joseph's College in Rensselaer, Indiana
•
Links the Indiana GigaPoP to Internet2, NLR, CIC, MREN and other networks.
•
10 times faster than our previous network path to Chicago
•
Resource available to all Indiana GigaPoP members
Gigapop - Internet2 Update
• Connected at 100G to Internet2 in Chicago
• Upgrading optical nodes between Indianapolis and
Chicago to support 100G
• Temporary Internet2 100G Wave until Optical
upgrades completed
Optical Transition
• Phase 1
o Rebuild IPGrid Optical System to be Monon100 Optical System
• Phase 2
o Transition services from temp I2 DWDM to Monon100 Optical System
• Indiana
o Future growth will include 100GE to IUB via I-Light
• CIC
o Future growth will include 100GE to CIC in Chicago
Monon100 – Core Upgrades
• Installed new Brocade MLXe 16 slot switch in Chicago
• Core Juniper routers have been upgraded to 12.1R1.9
o
o
o
o
o
Power supplies upgraded to 4100W
Enhanced MX SCB (3 per router)
Added High Capacity Fan Trays 100G Cards online (3 at ICTC and 1 at LL)
MPC Type 3 - Modular Port Concentrator
CFP-100G-LR4
Monon100/Indiana GigaPOP
Ilight to Chicgo
I-light access to Chicago
Questions?