“High Performance Collaboration –
The Jump to Light Speed"
Talk to
A Visiting Team from Intel
Calit2@UCSD
June 25, 2006
Dr. Larry Smarr
Director, California Institute for Telecommunications and
Information Technology;
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
From “Supercomputer–Centric”
to “Supernetwork-Centric” Cyberinfrastructure
Terabit/s
1.E+06
32x10Gb “Lambdas”
Bandwidth (Mbps)
1.E+04
Bandwidth of NYSERNet
Research Network Backbones
Gigabit/s
1.E+03
60 TFLOP Altix
1.E+02
1 GFLOP Cray2
1.E+01
1.E+00
T1
1985
Optical WAN Research Bandwidth
Has Grown Much Faster Than
Supercomputer Speed!
Computing Speed (GFLOPS)
1.E+05
Megabit/s
1990
1995
2000
Network Data Source: Timothy Lance, President, NYSERNet
2005
National Lambda Rail (NLR) and TeraGrid Provides
Cyberinfrastructure Backbone for U.S. Researchers
NSF’s TeraGrid Has 4 x 10Gb
Lambda Backbone
Seattle
International
Collaborators
Portland
Boise
Ogden/
Salt Lake City
UC-TeraGrid
UIC/NW-Starlight
Cleveland
Chicago
New York City
Denver
San Francisco
Pittsburgh
Washington, DC
Kansas City
Los Angeles
Albuquerque
Raleigh
Tulsa
Atlanta
San Diego
Phoenix
Dallas
Links Two Dozen
State and
Regional Optical
Networks
Baton Rouge
Las Cruces /
El Paso
Jacksonville
Pensacola
San Antonio
Houston
NLR 4 x 10Gb Lambdas Initially
Capable of 40 x 10Gb wavelengths at Buildout
DOE, NSF,
& NASA
Using NLR
The OptIPuter Project – Creating High Resolution Portals
Over Dedicated Optical Channels to Global Science Data
• NSF Large Information Technology Research Proposal
– Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI
– Partnering Campuses: SDSC, USC, SDSU, NCSA, NW, TA&M, UvA,
SARA, NASA Goddard, KISTI, AIST, CRC(Canada), CICESE (Mexico)
• Industrial Partners
– IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
• $13.5 Million Over Five Years—Now In the Fourth Year
NIH Biomedical Informatics
Research Network
NSF EarthScope and ORION
OptIPuter Software Architecture--a Service-Oriented
Architecture Integrating Lambdas Into the Grid
Source: Andrew Chien, UCSD
Distributed Applications/ Web Services
Visualization
Telescience
SAGE
Data Services
JuxtaView
Vol-a-Tile
LambdaRAM
Distributed Virtual Computer (DVC) API
DVC Runtime Library
DVC Configuration
DVC Services
DVC
Communication
DVC Job
Scheduling
DVC Core Services
Resource
Namespace
Identify/Acquire
Management
Security
Management
High Speed
Communication
Storage
Services
GSI
XIO
RobuStore
Globus
PIN/PDC
GRAM
Discovery
and Control
Lambdas
IP
GTP
CEP
XCP
LambdaStream
UDT
RBUDP
OptIPuter Scalable Adaptive Graphics Environment
(SAGE) Allows Integration of HD Streams
OptIPortal–
Termination
Device
for the
OptIPuter
Global
Backplane
Photo: David Lee,
NCMIR, UCSD
OptIPortal–
Termination Device for the OptIPuter Global Backplane
•
•
•
20 Dual CPU Nodes, 20 24” Monitors, ~$50,000
1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC!
Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC
Source: Phil Papadopoulos SDSC, Calit2
The New Optical Core of the UCSD Campus-Scale Testbed:
Evaluating Packet Routing versus Lambda Switching
Goals by 2007:
>= 50 endpoints at 10 GigE
>= 32 Packet switched
>= 32 Switched wavelengths
>= 300 Connected endpoints
Approximately 0.5 TBit/s
Arrive at the “Optical” Center
of Campus
Switching will be a Hybrid
Combination of:
Packet, Lambda, Circuit -OOO and Packet Switches
Already in Place
Funded by
NSF MRI
Grant
Lucent
Glimmerglass
Force10
Calit2/SDSC Proposal to Create a UC Cyberinfrastructure
of OptIPuter “On-Ramps” to TeraGrid Resources
OptIPuter + CalREN-XD + TeraGrid =
“OptiGrid”
UC Davis
UC San Francisco
UC Berkeley
UC Merced
UC Santa Cruz
UC Los Angeles
UC Santa Barbara
UC Riverside
UC Irvine
UC San Diego
Creating a Critical Mass of End Users
on a Secure LambdaGrid
Source: Fran Berman, SDSC , Larry Smarr, Calit2
Creating a North American Superhighway
for High Performance Collaboration
Next Step: Adding Mexico to
Canada’s CANARIE and the U.S. National Lambda Rail
Countries are Aggressively Creating Gigabit Services:
Interactive Access to CAMERA Data System
Visualization courtesy of
Bob Patterson, NCSA.
www.glif.is
Created in Reykjavik,
Iceland 2003
First Remote Interactive High Definition Video
Exploration of Deep Sea Vents
Canadian-U.S. Collaboration
Source John Delaney & Deborah Kelley, UWash
PI Larry Smarr
Marine Genome Sequencing Project
Measuring the Genetic Diversity of Ocean Microbes
CAMERA will include
All Sorcerer II Metagenomic Data
Dedicated
Compute Farm
(1000 CPUs)
CAMERA
Complex
TeraGrid Backplane
(10000s of CPUs)
DataBase
Farm
10 GigE
Fabric
Flat File
Server
Farm
W E B PORTAL
+ Web Services
Calit2’s Direct Access Core Architecture
Will Create Next Generation Metagenomics Server
Direct
Access
Lambda
Cnxns
User
Environment
Web
Local
Cluster
Source: Phil Papadopoulos,
SDSC, Calit2
Analysis Data Sets, Data Services,
Tools, and Workflows
•
Assemblies of Metagenomic Data
– e.g, GOS, JGI CSP
•
Annotations
– Genomic and Metagenomic Data
•
“All-against-all” Alignments of ORFs
– Updated Periodically
•
Gene Clusters and Associated Data
– Profiles, Multiple-Sequence Alignments,
– HMMs, Phylogenies, Peptide Sequences
•
Data Services
– ‘Raw’ and Specialized Analysis Data
– Rich Query Facilities
•
Tools and Workflows
– Navigate and Sift Raw and Analysis Data
– Publish Workflows and Develop New Ones
– Prioritize Features via Dialogue with Community
Source: Saul Kravitz
Director of Software Engineering
J. Craig Venter Institute
Calit2 and the Venter Institute Will Combine
Telepresence with Remote Interactive Analysis
Live Demonstration
of 21st Century
National-Scale
Team Science
25 Miles
Venter
Institute
OptIPuter
Visualized
Data
HDTV
Over
Lambda
Calit2 Works with CENIC to Provide
the California Optical Core for CineGrid
Partnering with
SFSU’s Institute for
Next Generation
Internet
Discussions
with CITRIS
SFSU
UCB
Calit2’s CineGrid Team
is Working with
Cinema Industry
in LA and SF
Digital Archive
of Films
In addition, 1Gb and 10Gb Connections to:
Prototype of
CineGrid
• Seattle, Asia, Australia, New Zealand
• Chicago, Europe, Russia, China
• Tijuana, Rosarita Beach, Ensenada
Extending SoCal
OptIPuter to USC
School of CinemaTelevision
USC
Calit2
UCI
Calit2
UCSD
First Trans-Pacific Super High Definition Telepresence
Meeting in New Calit2 Digital Cinema Auditorium
Lays
Technical
Basis for
Global
Digital
Keio University
President Anzai Cinema
UCSD
Chancellor Fox
Sony
NTT
SGI