A Survey on Optical Interconnects for
Data Centers
Speaker: Shih-Chieh Chien
Adviser: Prof Dr. Ho-Ting Wu
Outline

Introduction

Current Data Center

Network traffic characteristics

Optical technology

Architectures

Comparison

Conclusion

Reference
2
Introduction

Internet traffic

Emerging application


e.g. Stream video, Social network, Cloud computing
Data-intensive

e.g. cloud computing, search engines, etc.

High interaction(servers in the data center)

Power consumption(inside the rack)

each rack must the same → thermal constraints
3
Blade server
資料來源:wikipedia
Rack mount
4
Introduction (cont.)
5
Introduction (cont.)

IT power percentage

Server 40%, Storage 37%, Network devices 23%

Include HVAC (Heating-Ventilation and Air-Conditioning)

ICT GHG from 14% to 18%(2007 ~ 2020)

Goal

High throughput, reduced latency, low power consumption
→ Using optical network
6
Introduction (cont.)

Optical network

Opaque networks (older telecom. network)



OEO(optical-electrical-optical)
Main draw back is power hungry
all-optical networks (currently)

Device




Optical cross-connects (OXC)
Reconfigurable optical add/drop multiplexers(ROADM)
Point-to-point links( based on multi-mode fibers)
Provide 75% energe saving
7
Current DC with commodity switches

Data center

3 tiers



Core switches, Aggregate switches, and ToR
Advantage

Scaled easily

Fault tolerant
DrawBack

High power consumption

High number of links required
8
Current DC with commodity switches

Data center

3 tiers



Core switches, Aggregate switches, and ToR switches
Advantage

Scaled easily

Fault tolerant
DrawBack

High power consumption

High number of links required
9
Fat-tree
Core level
Aggregate level
資料來源:wikipedia
Access level
10
ToR switch
…
1Gbps links
資料來源:IBM
11
Current DC with commodity switches

Data center

3 tiers



Core switches, Aggregate switches, and ToR
Advantage

Scaled easily

Fault tolerant
DrawBack

High power consumption

High number of links required
12
Current DC with commodity switches

Data center

3 tiers



Core switches, Aggregate switches, and ToR
Advantage

Scaled easily

Fault tolerant
DrawBack

High power consumption

High number of links required
13
Network traffic characteristics


Three classes (categorized by Microsoft research)

University campus DC

private enterprise DC

cloud-computing DC
Model traffic

Interarrival rate distribution of the packet


Lognormal distribution (in the private DC)
Weibull distribution (in the campus DC)
14
Network traffic characteristics


Three classes (categorized by Microsoft research)

University campus DC

private enterprise DC

cloud-computing DC
Model traffic

Interarrival rate distribution of the packet


Lognormal distribution (in the private DC)
Weibull distribution (in the campus DC)
15
Network traffic characteristics (cont.)

Main empirical findings

Applications


e.g. HTTP, HTTPS, LDAP, Database。
Traffic flow locality


Inter rack traffic 10%~80%
Intra rack traffic

Traffic flow size and duration

Concurrent traffic flows

Packet size

Link utilization
16
Optical Technology

Splitter and combiner

Coupler

Arrayed-Waveguid Grating(AWG)

Wavelength Selective Switch(WSS)
17
Optical Technology (cont.)

Micro-Electro-Mechanical Systems Switches(MEMSswtch)

Semiconductor Optical Amplifier(SOA)

Tunable Wavelength Converters(TWC)
18
Architectures (C-Through)
Electrical network
Optical network
Rack servers
19
Architectures (C-Through (cont.))

Hybrid electrical-optical network

Traffic monitoring system

Optical configuration manager

Traffic in the ToR switch

Demutiplexed by VLAN-based routing


Packet based and circuit based network
Evaluation

Reduce completion time of the application

Reduce latency between two nodes
20
Comparison


Technology

All optical interconnection

Hybrid interconnection
Connectivity

Circuit based switching


Based on MEMS switch
Packet based switching


Array fixed lasers
Fast tunable transmitters
21
Comparison Hybrid & all-optical
22
Comparison


Technology

All optical interconnection

Hybrid interconnection
Connectivity

Circuit based switching


Based on MEMS switch
Packet based switching


Array fixed lasers
Fast tunable transmitters
23
Comparison(connectivity)
24
Comparison(cont.)

Scalability

Constrained by what?


Number of switch optical port
Number of wavelength channels

Capacity

Routing

Prototypes
25
Comparison(scalability)
26
Comparison(cont.)

Scalability

Constrained by what?


Number of switch optical port
Number of wavelength channels

Capacity

Routing

Prototypes
27
Comparison(capacity)
Capacity limitation technology
28
Comparison(cont.)

Scalability

Constrained by what?


Number of switch optical port
Number of wavelength channels

Capacity

Routing

Prototypes
29
Comparison(cont.)

Scalability

Constrained by what?


Number of switch optical port
Number of wavelength channels

Capacity

Routing

Prototypes
30
Comparison(prototype)
31
Cost and power consumption


Commercially available (lower price)

c-Through, Helios, and Proteus (optical modules)

Data-vortex, and DOS (SOA modules)
Intresting thing

OPEX (operation cost)

CAPEX(equipment's cost)
32
Cost and power consumption(cont.)

Simulation

Replacement of current switches

Data center with 1536 servers

Two-tier topology



512 ToR switches
16 aggregate switches (32x10 Gbps ports) →each arround $5k
Power consumption will be 77kW
33
Cost = OPEXCDCN − (CAPEXOI + OPEXOI)
where,
CDCN : CurrentDataCenterNetwork
OI : OpticalInterconnects
34
Conclusion

Optical interconnets (promising solution for DC)

High BW, low latency , and reduced energy consumption

Hybrid proposed as an upgrade to current networks

Schemes based on SOA for switching



Faster reconfiguration time than MEMS switches
Proteus shows high performance optical networks with
readily available optical componetnts
Schemes based on SOA and TWC

Provide higher capacites and better scalability
35
Reference

http://www.hirose.co.jp/cataloge_hp/e83001002.pdf

http://www.answers.com/topic/optical-switch


G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. E. Ng, M.
Kozuch, and M. Ryan, “c-Through: Part-time Optics in Data Centers,” in
Proc. ACM SIGCOMM 2010 conference on SIGCOMM, ser.
SIGCOMM ’10, 2010, pp. 327–338.
Kachris, Christoforos; Tomkos, Ioannis; , "A Survey on Optical
Interconnects for Data Centers," Communications Surveys & Tutorials,
IEEE , vol.14, no.4, pp.1021-1036, Fourth Quarter 2012
36