The Findings of the y
IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc
(Plus Updated Data)
Joint ITU/IEEE Workshop
Ethernet – Emerging Applications and Technologies
Geneva, Switzerland
22 September 2012
Regarding the Expressed Views
• Per IEEE‐SA Standards Board Operations Manual, January 2005:
Manual January 2005:
“At lectures, symposia, seminars, or educational courses an individual
educational courses, an individual presenting information on IEEE standards shall make it clear that his or her views
shall make it clear that his or her views should be considered the personal views of that individual rather than the formal position, explanation, or interpretation of the IEEE.”
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 2
Authors
• John D’Ambrosia, Dell, IEEE 802.3 BWA Chair
• Peter Anslow, Ciena, IEEE 802.3 BWA Editor
,
,
• Mike Bennett, LBNL / ESnet Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 3
Disclaimers
• All contributed information is solely the perspective of the respective contributors.
i
f h
i
ib
• The views expressed in the Assessment solely represent the views of the IEEE 802.3 Working Group, and do not necessarily represent a position of the IEEE, the IEEE Standards Association, or IEEE 802.
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 4
The 2006 HSSG Call‐For‐Interest
The Ethernet Ecosystem
Consumer Broadband Access
Content Providers
Broadband
Access Networks
Internet Backbone
Networks
Internet Backbone
Networks
Enterprise
Networks
Corporate Data Centers and
Enterprise
Internet eXchange and
(High Performance
Interconnection Points
Computing)
July 18, 2006
Higher Speed Study Group CFI, V 1.01
San Diego, CA
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Content
Networks
Research
Networks
Research, Education
and Government Facilities
(High Performance
Computing)
20
Page 5
Transport Network Classifications
Aggregation Node
Aggregation Node
Access Node
Core Node
Core Node
Access Node
Feeding the Networks • Consumers
• Businesses (data centers))
Note: “Aggregation Nodes” in single carrier networks are equivalent to “IXPs“ in multi carrier networks
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/huang_01_1111.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 6
The 2007 HSSG Tutorial
Why Higher Speed Ethernet?
Fundamental bottlenecks are happening
everywhere
Increased #
of users
As demonstrated
by the number of
ISPs: Comcast,
AOL, YahooBB,
NTT Cox,
NTT,
Cox
EasyNet, Rogers,
BT, ...
+
Increased
access
rates and
methods
+
EFM, xDSL,
WiMax,
xPON,
Cable, WiFi,
3G/4G
3G/4G…
Increased
services
everywhere
YouTube,
BitTorrent,
VOD,
Facebook,
Kazaa Netflix,
Kazaa,
Netflix
iTunes, 2nd
life, Gaming…
IEEE 802.3 Higher Speed Study Group - TUTORIAL
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Bandwidth
= explosion
18
Page 7
Life after IEEE P802.3ba
– 2013: 400 Gb/s
– 2015: 1 Tb/s
2015: 1 Tb/s
• Other bandwidth trends?
1,000,000
100 Gigabit Ethernet
100,000
10 Gigabit Ethernet
10,000
Gigabit Ethernet
1,000
Server I/O
Doubling ≈24 mos
• 2011 Formation of: 2011 F
ti
f
IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment Ad Hoc
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
40 Gi bi E h
40 Gigabit Ethernet
Core Networking
Doubling ≈18 mos
Rate Mb/s
R
• End‐users through the prior HSSG: The next speed of HSSG: The next speed of
Ethernet must begin when 100GbE done!
• HSSG Bandwidth Forecast for HSSG Bandwidth Forecast for
“Core Networking”
100
1995
2000
2005
2010
2015
2020
Date
Page 8
IEEE 802.3 BWA
Web & Reflector Information
•
•
•
•
Assessment ‐ http://www.ieee802.org/3/ad_hoc/bwa/BWA_Report.pdf
Tutorial ‐
Tutorial http://www.ieee802.org/802_tutorials/2012‐07/BWATutorial_D1_12_0716.pdf
Webpage ‐ http://www.ieee802.org/3/ad_hoc/bwa/index.html
Reflector ‐ http://www.ieee802.org/3/ad_hoc/bwa/reflector.html
Reflector http://www.ieee802.org/3/ad hoc/bwa/reflector.html
• Charter and Scope
– Evaluate Ethernet wireline bandwidth needs of the industry
– Reference material for a future activity
Reference material for a future activity
– The role of this ad hoc is to gather information, not make recommendations or create a CFI
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 9
Global IP Traffic by Local Access Technology
Source: nowell_01_0911.pdf citing Cisco Visual Networking Index (VNI) Global IP Traffic Forecast, 2010–2015, http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/nowell_01_0911.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 10
Global IP Traffic Growth, 2010–2015
Regional contributions to the Zettabyte journey
North America
North America
Western Europe
Western Europe
Central/Eastern Europe
Central/Eastern Europe
22.3 EB/Month by 2015
22.3 EB
/Month by 2015
26% CAGR
26% CAGR, , 3X
3X Growth
18.9 EB/Month by 2015
18.9 EB
/Month by 2015
32% CAGR, 4X
32% CAGR, 4X Growth
3.7 EB/Month by 2015
3.7 EB
/Month by 2015
39% CAGR
39% CAGR, , 5X
5X Growth
Japan
4.8 EB/Month by 2015
27% CAGR
27% CAGR, , 3X
3X Growth
Latin America
Middle East & Africa
Asia Pacific
4.7 EB/Month by 2015
4.7 EB
/Month by 2015
48% CAGR, 48% CAGR
48% CAGR, 7X
7X Growth
2.0 EB/Month by 2015
52% CAGR 8X Growth
52% CAGR, 8X
24.1 EB/Month by 2015
24.1 EB
/Month by 2015
35% CAGR 4X Growth
35% CAGR, 4X
35% CAGR, Source: nowell_01_0911.pdf citing Cisco Visual Networking Index (VNI) Global IP Traffic Forecast, 2010–2015, http://www.ieee802.org/3/ad_hoc/bwa/public/sep11/nowell_01_0911.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 11
Data Center Growth
Increased Storage
+
Networking
Compute
Storage
Increased Processing
+
Increased Bandwidth
= Bandwidth Explosion
l
Entered the 100GbE era in 2010
Individual switches have Tb/s of bandwidth
Individual switches have Tb/s of bandwidth
First petaflop supercomputers in 2011
Individual servers delivering 10s of Gb/s of I/O
PCIe 3.0 supports 2 x 40GbE NICs now
Entered the zettabyte (1 billion terabytes) era in 2010
Individual disk drives over 1 terabyte
1000 disk drive storage subsystem equals 1 Petabyte
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 12
Euro‐IX IXP Locations
132 IXPs
132 IXPs
36 Countries
140 Cities
420 Locations
Euro‐IX affiliated IXPs
Soon to be affiliated IXPs
Note: Global 321 IXP’s (100%), EU 41%, America’s 33%, Asia‐Pacific 19%, Africa 7%
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/steenman_01_1111.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 13
Tb/s
Tb/s
Five Year Peak European IXP Traffic Projections
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/steenman_01_1111.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 14
Global Annual IXP Peak Traffic Growth Rates:
By Region (for 2010)
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/nov11/steenman_01_1111.pdf
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 15
Science: Big Data Sources
CERN is “the tip of the iceberg”
Today
•
Future
CERN
–
–
•
Atlas detector in LHC (Large Hadron C llid )
Collider) generates ~1 petabyte/sec
1
b /
Trigger farm reduces to 450MB/sec
•
•
Tens of Gb/s of outbound traffic to analysis centers
Genome sequencing
–
–
–
Belle‐II
–
•
250PB of experimental data in first 5 years of operation
Square Kilometer Array (SKA) –
–
~2800 receivers in telescope array
2 petabytes/sec to central correlator
2 petabytes/sec to central correlator
• sending @ ~100 Gb/s to analysis centers
Per‐instrument data rate strongly ↗ (~10x over 5 years)
Data costs plummeting Æ vastly Data costs plummeting Æ
vastly
increased data volume
http://www.genome.gov/sequencingcos
ts/
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/dec11/dart_01_1211.pdf (updated: interview Eli Dart, August 29, 2012)
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 16
ESnet Accepted Traffic (Petabytes/month)
ESnet Accepted Traffic: Jan 1990 - Jul 2012 (Log Scale)
100.0000
10.0000
1.0000
Petabytes
0.1000
0.0100
Expecting 100 Petabytes/month of data in 2015
0.0010
0.0001
0.0000
Jul 2012
Jan 2012
Jul 2011
Jan 2011
Jul 2010
Jan 2010
Jul 2009
Jan 2009
Jul 2008
Jan 2008
Jul 2007
Jan 2007
Jul 2006
Jan 2006
Jul 2005
Jan 2005
Jul 2004
Jan 2004
Jul 2003
Jan 2003
Jul 2002
Jan 2002
Jul 2001
Jan 2001
Jul 2000
Jan 2000
Jul 1999
Jan 1999
Jul 1998
Jan 1998
Jul 1997
Jan 1997
Jul 1996
Jan 1996
Jul 1995
Jan 1995
Jul 1994
Jan 1994
Jul 1993
Jan 1993
Jul 1992
Jan 1992
Jul 1991
Jan 1991
Jul 1990
Jan 1990
Month
Source: http://www.ieee802.org/3/ad_hoc/bwa/public/dec11/dart_01_1211.pdf (updated 29 August, 2012) Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 17
Summary method
• Relative growth of the various sectors plotted on a single chart
on a single chart
– The growth of each sector was normalized to 2010 (the year IEEE Std 802.3ba was approved)
• Thi
This growth is a predictor of the future only if th i
di t
f th f t
l if
downward cost per bit trend is continued
– Ethernet
Ethernet cost per bit has to fall with time or the cost per bit has to fall with time or the
predicted exponential rise in traffic will result in unsupportable costs
• Servicing
Servicing demand with existing rates or new demand with existing rates or new
ones > 100 Gb/s will depend on the cost effectiveness of the solution
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 18
JD4
Growth Rate Trends
100
Traffic relative to 2010
0 value
Financial sector
fit to Figure 15 CAGR = 95%
CAGR 95%
Science
fit to Figure 13 ESnet 2004 to 2011
CAGR = 70%
10
1
Peering
fit t Fi
fit to Figure 39 39
HSSG tutorial CAGR = 64%
Slide 22 core
CAGR = 58%
Cable
Figure 20 CAGR 50%
CAGR = 50%
Figure 39 Euro‐IX
hi t i l d t
historical data
IP traffic
Figure 2
Figure 2 CAGR = 32%
0.1
HSSG tutorial
Slide 22 server I/O
CAGR = 36%
Figure 15 NYSE historical data
0 01
0.01
2004
2006
2008
2010
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
2012
2014
2016
2018
2020
Page 19
Slide 19
JD4
pete - somewhere in here we need to work the word "average"
JDAMBROSIA, 8/28/2012
IEEE 802.3 Higher Speed Ethernet Consensus Ad Hoc
• Charter and Scope – Focus on building consensus related to the next speed of Ethernet b ld
l d
h
d f h
for wireline applications, which will be used for the evaluation and possible development of an IEEE 802.3 Call‐For‐Interest for the next Higher Speed Study Group The requested duration for this Industry
Higher Speed Study Group. The requested duration for this Industry Connections activity is 12 months. •
•
•
•
Webpage –
p g http://www.ieee802.org/3/ad
p //
g/ / _hoc/hse/index.html
/ /
Reflector ‐ http://www.ieee802.org/3/ad_hoc/bwa/reflector.html
Participation is open and all are invited
Participation is open and all are invited.
Chair – John D’Ambrosia, Dell (jdambrosia@ieee.org) Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 20
The Ethernet Eco‐System
Rate (b/s)
1T
?
Key:
Chip-to chip/module
100G
Backplane
40G
10G
Twin-axial
Multimode Fibre
Voice grade copper
1G
Co-axial
Twisted pair
100M
Single-mode Fibre
Point to Multipoint Fibre
10M
Open Circle indicates
development effort
1M
0.1
1
10
102
103
Distance (m)
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
104
105
Based on slide used by permission from David Law.
Page 21
THANK YOU!
THANK
YOU!
QUESTIONS?
Joint ITU/IEEE Workshop, Ethernet – Emerging Applications and Technologies
22 September 2012, Geneva, Switzerland
Page 22