1 of 17
Gigabit Ethernet
Terena Summer 1999
Gullik Webjörn
Session Overview
2 of 17
 What is Gigabit Ethernet ?
– Technology
– Media
 Gigabit Ethernet Applications
– Computer rooms
– Server farms
– Cluster
 Problems and Solutions Gigabit Ethernet
– DMD, differential mode delay
– Collision domains
What is Gigabit Ethernet?
3 of 17
 Ethernet with a speed of 1 Gbit/sek.
– IEEE 802.3z
– Both CSMA/CD and Full Duplex
– Both fiber and copper ( Fiber Channel, 8b/10b )
– Follows the same development rate as 100 Mbit
Ethernet but converges much faster.
 Suggested ‘improvements’ and adaptions
– Jumbo Frames ( 9K bytes )
– VLAN
– Protocol stack parameter modifications.
1000baseFiber
 Multimode fiber, both 50/125 och 62/125.
4 of 17
– Installed fiber in buildings and campuses
– 850 nM = 1000base SX , 220 - 500 m (cost)
– 1300nM = 1000baseLX , 550 m (distance)
 Singlemode fiber 9/125
– Metropolitan networks and large campus net’s
– 5000 m with 1000baseLX optics
– 10000 m with “hot optics”
– 80000 m with “proprietary optics”
1000baseCopper
 STP / Coax - 1000baseCX - 802.3z
5 of 17
– 25 m over 2 150 ohm pairs / 2 75 ohm Coaxes
– Relatively low cost and simple
– Preferably in the Computer Room
– 8b/10b coding and simple transmission at 1250
baud
 Cat 5 UTP - 1000baseT - 802.3ab
– 100 m and uses all 4 pairs
– Complicated technology, DSP, PAM-5, FEC
– Most standardized copper media
– 250 Mbit / 125 Mbaud * 4 par
Problems and Solutions, GBE
 Pure scaling of Ethernet to 1 Gbit
6 of 17
– Slottime becomes 512 nS
– Max network diameter becomes 28 - 43 m
– Hard to achieve efficiency at half duplex
 Increased slot-time
– 4 uSek slot-time creates 200m collision domain
– Packet bursting increases efficiency
 Full Duplex
– Limited only by transmission characteristics
– Capacity = 2 * half duplex
– Wire-speed now becomes realistic
Problems and Solutions, GBE
 DMD, differential mode delay
7 of 17
– In multi-mode fiber discrete modes can dominate
– The problem increases with shorter wavelength
– The problem increases with better coherence
– Worst at 850 nM and 62/125 fiber
– Reduces distance due to jitter and ISI
 But, the problem might be overrated...
– 802.3z June 98 meeting
– 1000baseSX gets reduced distance (220/62/160)
– 1000baseLX gets 550 m over all MMF
– 1000baseLX increased from 3 to 5 km over SMF
Full Duplex and Switching
8 of 17
 More common than CSMA/CD 1998 already
– The arbitration problem goes away.
– Suitable for silicon based switches and routers
– Greater freedom to choose topology
– At least twice the performance of half duplex
– Possibility to switch ‘cut-through’
– Enables clusters with low latency and high
bandwidth.
– Full duplex is available today
– NO more HUBS!
Synergy effects
 Disks and Networks
– Both based on “fiber channel”
– Server area networks -- 250 Mbyte/sec FDX
– Storage networks -- U-SCSI and GBE
– Common components results in a nice
price/performance improvement (GBIC, t.ex.)
 Metropolitan Networks
– The technology with best price/performance
– 10000’s of users at 100 kbit/sec
– 100’s concurrent MPEG-2 video streams
– Makes multicast video possible (BC IP/TV)
9 of 17
Adapters and Server performance
 Gigabit adapters need bus bandwidth
10 of 17
– 64 bit (and 66 Mhz ) PCI
– Proprietary busses of abt. 200 Mbytes/sek
– 1518 bytes is too short a packetlength
– A full GBE @ 1518 bytes = 81000 packets/sec/HDX
 Performance examples
– High end Intel server std stack -- 270 Mbit/sec
 Tested in house with 4 clients.
– Alpha Server 4100 400-500 Mbit/sec
– Alpha Server using Driver API - wire speed
 Your mileage will vary !!!
Cluster and Gigabit Ethernet
 Redundant PHY -- media fault tolerance
11 of 17
Need or wish...
Available over FDDI but not over Fast Ethernet
Available in switches, when do we get a NIC ?
 Jumbo frames -- storage transport efficiency
Ethernet and Fast Ethernet 1500 bytes
FDDI - 4478 bytes
GBE - 9000 bytes
 Cut through -- Lock management efficiency
Efficient lock mgmt with large # of nodes.
Quality of Service
12 of 17
 As for other technologies: in the network boxes
– Not a specific characteristic of GBE
– High bandwidth reduces the need
– IEEE 802.1p and 802.1Q - frame tagging
– Networks have barely caught up with CPU’s
 Common with level 2,3,4 switching in GBE-switches.
– Not really QOS
– Allows priority for time critical services
– Or, you can route such traffic a different path
Present and Future
 Hunt groups
13 of 17
– 2-many aggregated links
– Redundancy, multi links - single logical port
 10 Gbit Ethernet
– 2.5 Gbit single pipe
– 4 * 2.5 Gbit / sec WDM
– 10 Gbit single pipe
 Aggressive cost curve - 1999 projections
– Shared Gigabit -- $450 - $700 per port (NOT !)
– Switched Gigabit -- $1000 - $1600 per port (!!)
– All switch or nic functions on single CMOS ASIC
Present and Future
 1550 nM GBIC’s
14 of 17
– 40-100 km on single mode fiber
– Will fit almost any GBE box with modular phy
 GBIC’s are getting cheap...
– 850 nm at $ 300 list
– 1300 nm at $ 900 list
– Fixed optics is going away…..
 VLAN is going WAN
– 802.1 p and Q over packet over SONET
– SONET/SDH is no more “the only way”
– Black colors are coming…with optical muxes
Compaq SW54xx Gigabit Products
Features:
• Policy-Based QoS
• Layer-3 IP Routing
(RIP, RIP2, OSPF)
•
•
•
•
256 VLANs per switch
802.1p and Q
128,000 MAC addresses
Port bonding (4 ports)
SW5422 -- 16 10/100 Base-TX and 6
Gigabit SX Ports
SW5411 -- 8 10/100 Base-TX and 3
Gigabit GBIC Ports with 1 redundant
phy
SW5406 -- 6 Gigabit SX and 2 GBIC
Ports
SW5450 -- 48 10/100 Base-TX and 2
Gigabit GBIC Ports with 2 redundant
phys
SW5425 -- 24 10/100 Base-TX and 1
Gigabit GBIC Port with 1 redundant
phy
GIGAswitch/Router
 Backbone Layer 2/3/4 switching
 8 slot chassis (16 slot coming soon)
– Over 30 Gb/s switching throughput
– Gigabit Ethernet and Fast Ethernet
connections via modules
– Future WAN interface modules
 Deploy as a backbone interconnection
between switches and as a connection to
high-performance servers.
 Industrial-strength, GIGAswitch-class
performance and reliability for high-speed
backbones.
GIGAswitch/Router
Two GIGAswitch/Router Chassis
8-Slot Chassis
16-Slot Chassis
Independently Verified
– 16 Gbps, 15 Million pps
56 10/100 ports
14 Gigabit ports
Future Support for
Independently Verified
– 32Gbps, 30 Million pps
120 10/100 ports
30 Gigabit ports
Future Support for
ATM, FDDI, SONET, WAN
ATM, FDDI, SONET, WAN
Compaq Computer Corporation
© 1999