Acknowledgements:
J. Cunningham, R. Ho, X. Zheng,
<Insert Picture Here>
M. Asghari
J. Lexau, H. Thacker, J. Yao, Y.
Luo, G. Li, I. Shubin, F. Liu, D.
Patil, K. Raj, and J. Mitchell
T. Pinguet
“Overview of short-reach optical interconnects: from
VCSELs to silicon nanophotonics”
Ashok V. Krishnamoorthy ( Jack Cunningham)
Hardware Architect
Sun Labs, Oracle
Physical Sciences Center, San Diego, CA
This work was supported in part by DARPA under
contract NBCH30390002. The views, opinions, and/or
findings contained in this article/presentation are those
of the author/presenter and should not be interpreted as
representing the official views or policies, either
expressed or implied, of the Defense Advanced Research
Projects Agency or the Department of Defense
1
Approved for Public Release: Distribution Unlimited
Outline
Introduction
> Definitions
> Penetration of optics into communication systems
Fibers, connectors, and module packaging
> Optical product segmentation
> Some examples of systems using optical interconnects
Optics to the package/chip
Link energy efficiency metric and goals
Silicon photonics and WDM
Overview of recent optical component results
Brief introduction to the macrochip
2
Approved for Public Release: Distribution Unlimited
Optical Transceivers
> Integrated modules incorporating optical laser transmitters and photodiode receivers.
These modules convert physical signals from electrical to optical and vice-versa in a
network and couple the optical signals into (and out of) optical fiber. Transceivers have
serial electrical interfaces on the host board.
Parallel Optical Transceivers, Modules, Interconnects or “Parallel Optics”
> Integrated optical laser transmitter and receivers incorporating multiple signaling
channels in a single housing, each channel having a separate serial electrical interface to
the host board. Typical values are 12 channels, although higher numbers (24, 36) have
been developed. Parallel optical modules typically utilize an array of VCSELs and
detectors to transmit and receive optical signals traveling in multi-mode fibers over a
distance of up to 300m.
VCSEL
> Is a type of semiconductor laser diode with laser beam emission perpendicular from the
top surface, contrary to conventional edge-emitting semiconductor laser which emit inplane from surfaces formed by cleaved facets. VCSELs are today the most-efficient,
lowest-cost, and most widely used laser source for interconnects.
WDM
> Wavelength Division Multiplexing. Enables multiple data streams of varying wavelengths
(“colors”) to be combined into a single fiber, significantly increasing the overall capacity of
the fiber and of the connector. There are two types of WDM architectures: Coarse
Wavelength Division Multiplexing (CWDM), typically handling up to 8 wavelengths, and
Dense Wavelength Division Multiplexing (DWDM), supporting up to 160 wavelengths.
3
Approved for Public Release: Distribution Unlimited
Price evolution of optical links
J. Cunningham et al., SPIE Photonics West, Jan 2006
3
2
1
Cost per Gbps ($)
101
Module Price
($k)
Cost per Gbps ($)
102
Data Rate ( x 10 Gbps)
100
0
1994 1996 1998 2000 2002 2004 2006 2008 2010 2012
Time (yr)
Approaching ~$1/Gbps
Consumer application could further reduce price
4
Approved for Public Release: Distribution Unlimited
Optics in communications
1Mbps 10Mbps 100Mbps 1Gbps 10Gbps 100Gbps
1Tbps
Link Bandwidth
To the package/chip
10cm
1m
10,000
100 Gbps * meter
To the board
‘active cables”
$3
Link Distance
To the box/rack
100m
$30
Across central office, data centers
1km
$100
10km
Metro, access, cross-campus
100km
$300
$1,000
SM, WDM
1,000
100
10
Number of Links per system
$10
Transceiver Cost (per Gbps)
10m
Multi-mode, Parallel
1000km
Cross-country
Trans-oceanic
10,000km
SM or MM, Serial or Parallel
$3,000
SM, CWDM, FSO
$10,000
SM, DWDM
Krishnamoorthy, Optoelectronics Letters, May 2006
1980 1985
1990 1995 2000 2005 2010
1
2015 Year of Introduction
5
Approved for Public Release: Distribution Unlimited
Optical link market segments
Aggr e ga t e D a t a Ra t e
155Mbps
622Mbps
2.5Gbps
10Gbps
10m
50m
Re a ch
300m
670nm,
850nm,
1300nm
LED
850nm VCSEL
(MM)
40Gbps
120Gbps
120Gbps
Optical Active Cables
VCSELs(MM)
850nm
Parallel
850nm
Parallel VCSEL
VCSEL
(Ribbon)
(Ribbon)
2km
10km
1.3um Fabry Perot
(SM)
1.3um DFB (SM)
40m
1.55um DFB (SM)
1.55um DFB
Ext.Mod.
(SM)
80m
6
Approved for Public Release: Distribution Unlimited
Silicon photonic interconnects
Aggr e ga t e D a t a Ra t e
155Mbps
622Mbps
2.5Gbps
10Gbps
Optical Active Cables
10m
50m
Re a ch
300m
120Gbps
120Gbps
40Gbps
670nm,
850nm,
1300nm
LED
850nm VCSEL
(MM)
Silicon
Photonics.
(SM)
2km
10km
1.3um Fabry Perot
(SM)
1.3um DFB (SM)
40m
1.55um DFB (SM)
1.55um DFB
Ext.Mod.
(SM)
80m
7
Approved for Public Release: Distribution Unlimited
I/O for the world’s largest IB switch
Gen 2: Up to 648 QDR Infiniband (40Gbps) ports [Gen 1: 3,456 SDR ports]
11 U
http://www.oracle.com/us/products/servers-storage/networking/infiniband/031556.htm
First 12x QDR cable developed by Merge Optics
> Very high panel density requirement for Sun/Oracle QDR switch
> CXP active optical cable with three 4x10Gb/s (120Gbps per direction)
> Over 50Tbps front side I/O => Areal connection density > 1.7Tbps/sq. in
~6.8 Petabits/s of data bandwidth deployed
> Over 28,000 air-cooled VCSEL-based active cables installed
> Over 500km of these active optical cables into datacenters
8
Approved for Public Release: Distribution Unlimited
I/O for IBM’s P7-IH computing system
12 drawers, 8 MCMs per drawer, 4 P7 chips per MCM, 8 cores per P7
2U
A. Benner et al., paper OTuH1, OFC 2010
Over 35Tbps of optical I/O per drawer
Each drawer can be configured as 256-way SMP
Water-cooled VCSEL modules for drawer I/O
> areal connection density of 1.2Tbps/sq. inch
http://www.avagonow.com/Newsletters/PDFs/IEEE-MitchFields-march-021610.pdf
9
Approved for Public Release: Distribution Unlimited
VCSELs and detectors on CMOS
Areal density: demonstrated over 37Gbps/sq. mm (24Tbps/sq. in)
Many independent R&D efforts, e.g.
> Bell Labs - late ‘90s
Krishnamoorthy et al., IEEE PTL, August 2000
> AraLight/Xanoptix - 2002
C. Cook et al., IEEE JSTQE, March/April 2003
70°C
J. Trezza et al., IEEE Commun. Mag., Feb 2003
> Agilent – 2004
B. Lemoff et al., OSA/IEEE JLT, September 2004
> IBM - 2009
C. Schow et al., OSA/IEEE JLT, April 2009
V CSEL w a ve le ngt h: 8 5 0 nm
(ot he r w ork a t 9 8 0 nm )
VCSEL Pa ds
1 4 4 µm
VCSELs
Detectors
10
Approved for Public Release: Distribution Unlimited
GbE switch with VCSELs & detectors
Krishnamoorthy et al., IEEE JSTQE Spec. Issue on Green Photonics, to appear
11
Approved for Public Release: Distribution Unlimited
Multimode fiber bundle array
16
1
Fiber Bundle Front View (facing bundle)
• Hexagonal closepack (tightest geometry)
• Multimode 50micron-core fiber
• Terminated to MTP connectors at other end
• One optical channel per fiber (ultimately
limits density)
32
Krishnamoorthy et al., IEEE JSTQE Spec. Issue on Green Photonics, to appear
497
512
12
Approved for Public Release: Distribution Unlimited
Link energy efficiency vs distance
Link Energy Efficiency (pJ/bit/m)
1000
Electrical links today
100
10
VCSEL links today
1
0.1
0.01
0.001
0.01
0.1
1
10
100
1000
Link Distance (m)
13
Approved for Public Release: Distribution Unlimited
Improving link energy efficiency
System Target: <<1 mW per Gigabit/s per meter
1000
Link Energy Efficiency (pJ/bit/m)
Krishnamoorthy et al., IEEE JSTQE Spec. Issue on Green Photonics, to appear
Electrical links today
100
10
Active Optical Cables
1
Si Target
0.1
100fJ/bit (Core)
500fJ/bit (Core - L3 cache & MM)
2pJ/bit (Core –Distant Memory)
0.01
0.001
0.01
0.1
1
10
100
1000
Link Distance (m)
14
Approved for Public Release: Distribution Unlimited
Optics to the chip: CMOS photonics
C. Gunn, IEEE Micro, March/April 2003
Flip-chip laser
10G modulators
WDM optical filters
10G receivers
Fiber cable
Ceramic package
15
Approved for Public Release: Distribution Unlimited
Introduction to CMOS photonics
Multi-layer Metal Backend
Contacts
P+
PMDH
FOX
PMOD
FOX
OWELLP OWELLN NMOD
N+
NMDH
Buried Oxide
Modulator
Transistor
Silicon
Waveguide
Grating
Standard silicon process with SOI wafers (e.g. Luxtera)
High index contrast => sub-micron structures => fast, compact devices
Proven CMOS-compatible germanium waveguide detectors
C. Gunn, IEEE Micro, March/April 2003
16
Approved for Public Release: Distribution Unlimited
Optical proximity communication (OPxC)
OPxC optical
OPxC enables seamless multi-chip optical
interconnects
Zheng et al, Optics Express, September 2008,
Krishnamoorthy et al., IEEE Journal of Quantum Elec., July 2009
> Various approaches
24.0
- Grating couplers, reflecting mirrors, ball lens in pit…
> High performance
20.0
16.0
12.0
- High bandwidth density (potentially > 32Tbps/mm2)
- Passive coupling (no conversion pwr)
- Performance limited by transceivers
8.0
Broad band operation
1400nm
1600nm
OPxC demonstration
> Reflecting mirror OPxC
- 3 chips with 2 OPxC hops
> Promising optical performance
Passive alignment with etch pits and balls
Broad band coupling, >100nm
<4dB insertion loss per coupling interface
Negligible power penalty at receiver for 10Gbps transmission
Log BER
-
10G Transmission of OPxC
1.00E-03
1.00E-04
1.00E-05
1.00E-06
1.00E-07
1.00E-08
1.00E-09
1.00E-10
1.00E-11
1.00E-12
1.00E-13
1.00E-14
Tx/Rx Back-to-Back
Opxc Double Hop
-23
-22
-21
-20
-19
-18
Power at Receiver (dBm)
17
Approved for Public Release: Distribution Unlimited
Carrier depletion ring modulator
∆n/n Refractive index
-4
Depletion length (µm), Γ, loss (cm-1)
10
1
Modulator EO response (dBe)
0
-1
-2
-3
1V
-4
-5
-6
4
1
confinement factor
10
Frequency (GHz)
15
breakdown
voltage
loss
0.1
depletion length
bandwidth
0.1
5
10
Q=10
-7
0
-3
10
1
Breakdown voltage ( V), Bandwidth (x10GHz)
-5
1
Doping concentration x1018cm-3
20
• Carrier depletion ring for low power, high speed modulation
• Free-scale 130nm SOI CMOS process
• Relatively low Q design (<105)
• >15GHz small-signal bandwidth with 1V reverse bias
• Stable large-signal operation (no feedback control or dynamic tuning)
X. Zheng et al., Optics Express, Feb 2010
18
Approved for Public Release: Distribution Unlimited
400fJ/bit all-CMOS Tx (circuits + device)
Performance Summary:
• 5Gbps, digitally clocked
• 2V, 1.95mW or 395fJ/bit
• ER 3dB; IL 6dB
• Error free transmission for
over 1.5 peta bits of data
• Better than 10-15 BER
X. Zheng et al., Optics Express, Feb 2010
BER< 10-15
BER better than 10-15 with clocked digital Tx using ring modulator
19
Approved for Public Release: Distribution Unlimited
Tuning out resonance imperfections
Krishnamoorthy et al., Proceedings of the IEEE, July 2009
20
Approved for Public Release: Distribution Unlimited
25µm ring modulator w/ integrated heater
P. Dong et al., IEEE Summer Topical Meeting on Optics in Data Centers, July 2010
Metal Contact
Ti heater
p contact
Ti
heater
oxide
Si
(a)
p++
p
n
n++
Bus waveguide
• Ring radius = 25 µm, FSR = 3.9 nm
• Total working wavelength range > 150 nm
n contact
to RF pads
to heater
pads
• Heating power: 11.5 mW/nm, or 45 mW per FSR tuning
• 12.5 Gbps is achieved with a Vpp = 3 V, RE>6dB, limited by BERT
• Modulation energy/power of 200 fJ/bit or 2.5 mW
• No performance degradation over one FSR tuning
Heating power = 56 mW
12.5 Gbps, λ : 1556 nm
Heating power = 0 mW
12.5 Gbps, λ: 1551.56 nm
21
Approved for Public Release: Distribution Unlimited
5µm ring modulator w/ integrated heater
P. Dong et al., Optics Express, May 2010
Metal Contact
Ti heater
Metal Contact
n contact
p contact
Ti heater
oxide
Si p++
(a)
p
n
n++
to RF pads
• Ring radius = 5 µm, FSR = 19 nm
• Heating efficiency: 2.4 mW/nm, or 46 mW per
FSR tuning
bus waveguide
to heater pads
• 12.5 Gbps is achieved with a Vpp = 3 V, 6dB ER
• Modulation energy of 40 fJ/bit or 0.5 mW
• no detrimental effects found ver ~93 ºC range
Heating power = 0 mW
12.5 Gbps, l: 1550 nm
Heating power = 18 mW
12.5 Gbps, λ: 1558 nm
22
Approved for Public Release: Distribution Unlimited
Efficient, tunable CMOS rings
• Add/drop tunable filters
• 0.13 µ m SOI 6 metal CMOS process
• Dual heater stages with integrated
waveguide heating
• Integrated back-side etch pit
0 mW
0.7 mW
2.7 mW
Schematic cross-section.
3.9 mW per 2π phase shift
J. Cunningham et al., IEEE Summer Top. Meet. on Optics in Data Centers, July 2010
23
Approved for Public Release: Distribution Unlimited
690fJ/bit all-CMOS Rx (circuits + device)
Performance Summary:
• CMOS integrated germanium photodetector
• 5Gbps, digitally clocked TIA-based receiver
• -18.9 dBm sensitivity at 10-12 BER
with 0.7A/W responsivity, >10GHz BW, and
<20fF detector capacitance
• BER measured below 10-14
• 3.45mW or 690fJ/bit
X. Zheng et al., Optics Express, January 2010
24
Approved for Public Release: Distribution Unlimited
High-responsivity photodetectors
Kotura Ge photodetectors
• Butt-coupling between SOI and Ge
waveguides enables short device lengths
(~ 10 µm)
• => Capacitance a few fF, device not RCtime limited)
• Horizontal p-i-n junction design enables
compatibility with larger Si waveguides
• Narrow Ge WG width (0.65 µm) minimizes
transit-time limitation (speed > 40 GHz)
Performance Summary:
• Responsivity of 1.1 A/W @ 1550 nm
• Dark current of 0.24 µ A @ -0.5 V, 1.3 µ A @ -1 V
• Bandwidth > 32 GHz
0
Response(dB)
►
SEM Cross-section
-0.0V: 3dB BW 17.5GHz
-1.0V: 3dB BW 32.6GHz
-3.0V: 3dB BW 36.8GHz
-3
-6
-9
8
10
9
10
10
10
D. Feng et al., Applied Physics Lett, December 2009
25
Approved for Public Release: Distribution Unlimited
“Macrochip” logical & physical views
Krishnamoorthy et al., Proceedings of the IEEE, July 2009
R. Ho et al., IEEE Communications Mag., July/August 2010
26
Approved for Public Release: Distribution Unlimited
Loss (dB)
Interlayer link components
AR coated
AR coated
0
-12.7dB, -13.8dB, -13.7dB, -12.8dB
-5
-10
-15
-20
-25
-30
-35
-40
-45
-50
-55
1530 1532 1534 1536 1538 1540 1542 1544 1546 1548 1550
D. Lee et al., IEEE Summer Topical Meeting on Optics in Data Centers, July 2010
Wavelength (nm)
z
54.7º mirror
Mode Transformer
y
y
x
x
~12µm
z
~2mm
~3µm
Echelle Grating (< 0.2 mm2)
Etched Waveguide Facet
Mirror Coupler
12 x 2.5 mm2
27
Approved for Public Release: Distribution Unlimited
Rematable power, ground, & alignment
Sacrificial layer etch, spring lift-off
and Au-plating
Micro-spring interconnects
+
Co-integrate both technologies
Package to test rematability
I. Shubin et al., IEEE ECTC, May 2009
28
Approved for Public Release: Distribution Unlimited
Optical interconnect energy roadmap
100
Industry Trend
Link Energy (pJ/bit)
DARPA UNIC Goals
10
1
Krishnamoorthy et al., Proceedings of the IEEE, July 2009
0.1
2006
2008
2010
2012
2014
Year
29
Approved for Public Release: Distribution Unlimited
REFERENCES:
X. Zheng et al., Optics Express, October 2008
Krishnamoorthy et al., IEEE JQE, April 2009
I. Shubin et al., Proc. ECTC, May 2009
Krishnamoorthy et al., Proc. of the IEEE, July 2009
R. Ho et al., IEEE ASSCC, November 2009
D. Feng et al., Applied Physics Lett., December 2009
X. Zheng et al., Optics Express, January 2010
X. Zheng et al., Optics Express, February 2010
J. Cunningham et al., IEEE Photon. Summer Top. OND, TuD3.4, July 2010
P. Dong et al., IEEE Photonics Summer Top. OND, MD2.3, July 2010
D. Lee et al., IEEE Photonics Summer Top. OND, TuD3.3, July 2010
R. Ho et al., IEEE Communications Mag., July/August 2010
30
Approved for Public Release: Distribution Unlimited
UNIC technology highlights to date
• Demonstration of passively-aligned multi-chip, multi-channel
optical proximity communication
• Integration of ball-in-pit alignment with CMOS
• Record low-power silicon photonic link components
>
>
>
>
>
320fJ/bit photonic Tx @ 5Gbps(w/ Kotura ring & custom driver)
690fJ/bit photonic Rx @ 5Gbps (w/ Luxtera Ge PD & custom receiver)
3.9mW FSR tunable mux/demux (w/ Luxtera ring & backside etch pit)
1.1A responsivity, 0.24µ A dark current large-core Ge detector (Kotura)
Areal density of ~730Gbps/sq. mm based on WDM link components
• Record efficiency SOI passive components
> Thin silicon routing waveguide with 0.27dB/cm loss (Kotura)
> 1x2 splitter with 0.1dB excess loss (Kotura)
• Demonstration of rematable power/gnd & chip alignment
Approved for Public Release: Distribution Unlimited
31