More Wheels of Reincarnation
Or A New PC+, www+ Era?
Infinite processing, memory,
and bandwidth
@ zero cost
Gordon Bell
Bay Area Research Center
Microsoft Corporation
Copyright Gordon Bell & Jim Gray
PC+
The Highly Probable Future c2025
83 items from J. Coates, Futurist, Vol. 84, 1994


8.4 B, english speaking, personally tagged & identified,
prosthetic assisted and/or mutant, tense people who
have access & control of their medical records
Everything will be smart, responsive to environment.
–
–
–
–
–

Sensing of everything… challenge for science & engineering!
Fast broadband network
Smart appliances & AI
Tele-all: shop, vote, meet, work, etc.
Robots do everything, but there may be conflict with labor…
A “managed”, physical and man-made world
–
–
Reliable weather reports
“Many natural disasters e.g. floods, earthquakes, will be
mitigated, controlled or prevented”
 Nobel prize to “economist” for “value of information”
Copyright
No surprises.
can see 10 years, but not 20!
Gordon Bell & We
Jim Gray
PC+
IP On Everything
Copyright Gordon Bell & Jim Gray
PC+
poochi
Copyright Gordon Bell & Jim Gray
PC+
TECHNOLOGY DEVELOPMENT
BIO INTELLIGENCE AGE
CONSUMER ACCEPTANCE
2000 BC
0
1500
1800
1900
2000 AD
TIME (year)
R. Satava 29 July 99
PC At An Inflection Point
PCs
Non-PC
devices and Internet
Copyright Gordon Bell & Jim Gray
PC+
TV/AV
Mobile
Companions
Consumer
PCs
The Dawn Of The PC-Plus Era,
Not The Post-PC Era…
devices aggregate via PCs!!!
Communications
Automation
& Security
Household
Management
PCTV a.k.a. MilliBillg
Using PCs to drive large screens
e.g. tv sets, Plasma Panels
Copyright Gordon Bell & Jim Gray
Copyright 1999 Microsoft Corporation
PC+
Another big bang?
Internet to TV and audio:
The Net, PC meet the TV
“milliBill”
Home Analog/digital cable distribution
CATV
Ethernet Home network
Video
capture
PC broadcasts
are mixed into
home CATV in
analog and/or
MPEG digital
Basic ideas:
1. PC records or plays
thru video cable
channels.
2. PC “broadcasts” art
images, webcams,
presentations,
videos, DVDs, etc.
3. Ethernet not cable?
Settop
box
Images from:
http://www.nextmonet.com
A gallery that sells art on line
The Next Convergence
POTS connects to the Web
a.k.a. Phone-Web Gateways
Web Server
PSTN
Voice to WEB
Bridge
Copyright Gordon Bell & Jim Gray
The
Web
DataBase
PC+
PC will prevail for the next decade
as the dominant platform…
its COTS or COTS’ AND www!



Moore’s Law increases performance; and
alternatively reduces prices
PC server clusters with low cost OS beat
proprietary switches, smPs, and DSMs
Home entertainment & control …
–
–

Very large disks (1TB by 2005)
to “store everything” personal
Screens to enhance use
Lack of last mile bandwidth to move
pictures, data, and interact favors home
mainframes aka PCs
C = Commercial; C’ = Consumer
SNAP
… c1995
Scalable Network And Platforms
A View of Computing in 2000+
We all missed the impact of WWW!
Platform
Gordon Bell
Copyright Gordon Bell & Jim Gray
Network
Jim Gray
PC+
How Will Future Computers Be Built?
Thesis: SNAP: Scalable Networks and Platforms
• Upsize from desktop to world-scale computer
• based on a few standard components
Platform
Network
Because:
• Moore’s law:
exponential progress
• Standardization & Commoditization
• Stratification and competition
When: Sooner than you think!
• Massive standardization gives massive use
• Economic forces are enormous
p
e
r
f
o
r
m
a
n
c
e
Volume drives simple,
cost to standard
price for
platforms Stand-alone
Desk tops
high speed
interconnect
Distributed
workstations
Clustered
Computers
PCs
1-4 processor mP
MPPs
1-20 processor mP
price
Copyright Gordon Bell & Jim Gray
PC+
The economics of
operating systems and databases
Copyright Gordon Bell & Jim Gray
PC+
The Virtuous Economic Cycle
drives the PC industry… & Beowulf
Standards
Copyright Gordon Bell & Jim Gray
PC+
The UNIX Trap:
creating the myth of “open systems”
“Standard” has meant different!
VendorIX platforms have created the “downsizing”
market that provides an apparent, cost reduction
Hardware platform vendors lock-in users with servers of
proprietary UNIX dialects and unique chips
to maintain margins for chip and UNIX development
VendorIX R & D costs $1.4 - $2 billion
Implied selling price $10 - 14 billion for $1.4 billion,
or a sales tax of 1 million UNIX units of $10,000
Users hostage with client-server, database, and apps
An implicit or unconscious cartel has formed that
maintains the industry status quo
Copyright Gordon Bell & Jim Gray
PC+
SNAP Architecture----------
Copyright Gordon Bell & Jim Gray
PC+
Computing
SNAP
built entirely
from PCs
Portables
Wide-area
global
network
Mobile
Nets
Wide & Local
Area Networks
for: terminal,
PC, workstation,
& servers
Person
Person
servers
servers
(PCs)
(PCs)
Legacy
mainframes &
Legacy
minicomputers
mainframe
& terms
servers &
minicomputer
servers & terminals
???
scalable computers
built from PCs
Centralized
&Centralized
departmental
uni& mP servers
&
departmental
(UNIX
& NT)
servers
buit
from
PCs
TC=TV+PC
home ...
(CATV or ATM
or satellite)
A space, time
(bandwidth), &
generation scalable
environment
Copyright Gordon Bell & Jim Gray
PC+
GB with NT, Compaq, & HP cluster
Copyright Gordon Bell & Jim Gray
PC+
In a decade we can/will have:

more powerful personal computers
– processing 10-100x; multiprocessors-on-a-chip
–
–
–

adequate networking? PCs now operate at 1 Gbps
–
–



4x resolution (2K x 2K) displays to impact paper
Large, wall-sized and watch-sized displays
low cost, storage of one terabyte for personal use
ubiquitous access = today’s fast LANs
Competitive wireless networking
One chip, networked platforms e.g. light bulbs,
cameras everywhere, & managed by PCs!
Some well-defined platforms that compete with the
PC for mind (time) and market share
watch, pocket, body implant, home
Inevitable, continued cyberization… the
challenge… interfacing platforms and people.
High Performance Computing
A 60+ year view
Copyright Gordon Bell & Jim Gray
PC+
Copyright Gordon Bell & Jim Gray
PC+
Star Bridge
Copyright Gordon Bell & Jim Gray
PC+
Linux super howls
Copyright Gordon Bell & Jim Gray
PC+
Dead Supercomputer Society
Copyright Gordon Bell & Jim Gray
PC+
Dead Supercomputer Society





















ACRI
Alliant
American Supercomputer
Ametek
Applied Dynamics
Astronautics
BBN
CDC
Convex
Cray Computer
Cray Research
Culler-Harris
Culler Scientific
Cydrome
Dana/Ardent/Stellar/Stardent
Denelcor
Elexsi
ETA Systems
Evans and Sutherland
Computer
Floating Point Systems
Galaxy YH-1






















Goodyear Aerospace MPP
Gould NPL
Guiltech
Intel Scientific Computers
International Parallel Machines
Kendall Square Research
Key Computer Laboratories
MasPar
Meiko
Multiflow
Myrias
Numerix
Prisma
Tera
Thinking Machines
Saxpy
Scientific Computer Systems (SCS)
Soviet Supercomputers
Supertek
Supercomputer Systems
Suprenum
Vitesse Electronics
Steve Squires & Cray
Copyright Gordon Bell & Jim Gray
PC+
1000
100
10
Bell Prize and
Future Peak
Tflops (t)
*IBM
Petaflops
study
target
1
NEC
0.1
CM2
0.01
0.001
XMP
NCube
0.0001
1985
1990
1995
Copyright Gordon Bell & Jim Gray
2000
2005
2010
PC+
Top 10 tpc-c
Top two Compaq systems are:
1.1 & 1.5X faster than IBM SPs;
1/3 price of IBM
1/5 price of SUN
Copyright Gordon Bell & Jim Gray
PC+
Courtesy of Dr. Thomas Sterling, Caltech
Courtesy of Dr. Thomas Sterling, Caltech
Contributions of Beowulf









An experiment in parallel computing systems
Established vision low cost high end computing
Demonstrated effectiveness of PC clusters for
some (not all) classes of applications
Provided networking software
Provided cluster management tools
Conveyed findings to broad community
Tutorials and the book
GB: Provided design standard to rally community!
Standards beget: books, trained people, software
… virtuous cycle
Courtesy of Dr. Thomas Sterling, Caltech
High performance
architecture/program timeline
1950 .
1960 .
1970 .
Vtubes
Trans.
MSI(mini)
1980 .
1990 .
Micro RISC
2000
nMicr
Sequential programming---->-----------------------------(single execution stream)
<SIMD Vector--//--------------Parallelization---
Parallel programs aka Cluster Computing
multicomputers
ultracomputers 10X in size & price!
“in situ” resources 100x in //sm
geographically dispersed
Copyright Gordon Bell & Jim Gray
<--------------<--MPP era-----10x MPP
NOW
VLSCC
Grid
PC+
Computer types
-------- Connectivity-------WAN/LAN
Netwrked
Supers…
SAN
VPPuni
DSM
SM
NEC super
NEC mP
Cray X…T
(all mPv)
Clusters
GRID
Legion
T3E
SGI DSM
Mainframes
Condor
SP2(mP)
clusters &
Multis
BeowulfNOW
SGI DSM WSs PCs
NT clusters
Copyright Gordon Bell & Jim Gray
PC+
Technical computer types
WAN/LAN
Netwrked
Supers…
New
SAN
DSM
SM
NEC mP NEC super
Old
Cray X…T
T series World
(all mPv)
world:
VPPuni
Clustered
GRID
( one
Computing
Legion
SGI DSM program
Mainframes
(multiple program
SP2(mP)
Condor
clusters &
Multis
NOW
stream)
streams)
Beowulf
SGI DSM WSs PCs
Technical computer types
WAN/LAN
SAN
DSM
SM
Netwrked
Supers…
NEC mP NEC super
Vectorize
Cray X…T
Linda,
PVM,
VPPuni
Parallellelize
T series (all mPv)
MPI,
GRID
Cactus, ???
distributed function
Legion
SGI DSM Mainframes
SP2(mP)
Condor
clusters Parallellelize
&
Multis
Computing
NOW
Beowulf
SGI DSM
WSs PCs
Gaussian Parallelism
Copyright Gordon Bell & Jim Gray
PC+
Beyond Moore’s Law …>10 yrs


Just FCB (faster, cheaper, better)…
COTS will soon mean consumer off the shelf
Moore’s Law and technology progress likely to
continue for another decade for:
processing & memory,
storage, LANs, & WANs are really evolving
System-on-a chip of interesting sizes will
emerge to create 0 cost systems
 No DNA, molecular, or quantum computers, or
new stores
 Any displacement technology is unlikely
… Carver Mead’s Law c1980
A technology takes 11 years to get established
 On the
other
hand,
we are on Internet time! PC+
Copyright
Gordon
Bell & Jim
Gray

High Performance Computing

Supers we knew are Japanese…
we have to stay the course. We actually may win!
PC will continue to erode capacity need
Scalability & COTS are in… but you have to roll
your own else pay VendorIX taxes
Beowulf is $14K/TB ( 6 x 4 x 40 GB)
IBM 4000R 1 rack: 2x42 500Mhz processors, 84
GB, 84 disks (3TB @36GB/disk)
$420K … still cheaper than the “big buys”
$10-20K/node for special purpose vs
$2K for a MAC
EMC, IBM at $1 million/TB; vs $14K

We should back radical experiments!






We get more of everything
Copyright Gordon Bell & Jim Gray
PC+
Computer ops/sec x word length / $
1.E+09
doubles every 1.0
1.E+06
.=1.565^(t-1959.4)
1.E+03
y = 1E-248e0.2918x
1.E+00
1.E-03
doubles every 2.3
doubles every 7.5
1.E-06
Copyright
Gordon Bell
& Jim Gray
1880
1900
1920
1940
1960
1980
PC+
2000
Performance in Mflop/s
Growth of microprocessor
performance
10000
1000
100
Cray 2
Cray Y-MP Cray C90
Alpha
RS6000/590
Alpha
RS6000/540
Cray X-MP
Cray 1S
10
Cray T90
Supers
Micros
i860
R2000
1
0.1
0.01
8087
80387
6881
80287
Copyright Gordon Bell & Jim Gray
PC+
Albert Yu predictions ‘96
When
Clock (MHz)
MTransistors
Mops
Die (sq. in.)
2000
900
40
2400
1.1
Copyright Gordon Bell & Jim Gray
2006
4000
350
20,000
1.4
4.4x
8.75x
8.3x
1.3x
PC+
Processor Limit: DRAM Gap
“Moore’s Law”
100
10
1
µProc
60%/yr.
.
Processor-Memory
Performance Gap:
(grows 50% / year)
DRAM
DRAM
7%/yr..
CPU
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
Performance
1000
• Alpha 21264 full cache miss / instructions executed:
180 ns/1.7 ns =108 clks x 4 or 432 instructions
• Caches in Pentium Pro: 64% area, 88% transistors
Copyright Gordon Bell & Jim Gray
*Taken from Patterson-Keeton Talk to SigMod
PC+
Sony Playstation export
limiits
Copyright Gordon Bell & Jim Gray
PC+
Things get cheaper
Copyright Gordon Bell & Jim Gray
PC+
Exponential change of 10X per
decade causes real turmoil!
100000
10000
8 MB
1 MB
Timeshared
systems
1000 256 KB
100
$K
10
64 KB
16 KB
1
0.1
0.01
1960
Single-user
systems
1970
Copyright Gordon Bell & Jim Gray
1980
1990
2000
PC+
VAX Planning Model 1975:
I didn’t believe it

The model was very good
–

1978 timeshared $250K VAXen
cost about $8K in 1997!
Costs declined > 20%
–
users got lots more memory than I predicted
Single user systems didn’t come down
as fast, unless you consider PDAs
 VAX ran out of address bits!
Copyright Gordon Bell & Jim Gray
PC+

System-on-a-chip alternatives
FPGA
Sea of un-committed
gate arrays
Compile Unique processor for
a system every app
Systolic | Many pipelined or
array
parallel processors
DSP |
Special purpose
VLIW
processors
Pc & Mp. Gen. Purpose cores.
Specialized by I/O, etc.
ASICS
Universal Multiprocessor array,
Micro
programmable I/o
Xylinx,
Altera
Tensillica
TI
Intel,
Lucent, IBM
Cradle
Cradle: Universal Microsystem
trading Verilog & hardware for C/C++
UMS : VLSI = microprocessor : special systems
Software : Hardware
 Single part for all apps
 Programming @ run time via FPGA & ROM
 5 quad mPs at 3 Gflops/quad = 15 Glops
 Single shared memory space, caches
 Programmable periphery including:
1 GB/s; 2.5 Gips
PCI, 100 baseT, firewire
 $4 per flops; 150 mW/Gflops
UMS Architecture
DRAM
CONTROL
CLOCKS,
DEBUG

MEMORY
MEMORY
M M M M
S S S S
P P P P
M M M M
S S S S
P P P P
PROG I/O
PROG I/O
PROG I/O
MEMORY
PROG I/O
MEMORY
PROG I/O
PROG I/O
PROG I/O

PROG I/O
PROG I/O

PROG I/O
M M M M
S S S S
P P P P
NVMEM

PROG I/O
PROG I/O
M M M M
S S S S
P P P P
DRAM
Memory bandwidth scales with processing
Scalable processing, software, I/O
Each app runs on its own pool of processors
Enables durable, portable intellectual property
Free 32 bit processor core
Copyright Gordon Bell & Jim Gray
PC+
Linus’s Law:
Linux everywhere







Software is or should be free
All source code is “open”
Everyone is a tester
Everything proceeds a lot faster when
everyone works on one code
Anyone can support and market the
code for any price
Zero cost software attracts users!
All the developers write lots of code
ISTORE Hardware Vision


System-on-a-chip enables computer, memory,
without significantly increasing size of disk
5-7 year target:
MicroDrive:1.7” x 1.4” x 0.2”
2006: ?
1999: 340 MB, 5400 RPM,
5 MB/s, 15 ms seek
2006: 9 GB, 50 MB/s ? (1.6X/yr capacity,
1.4X/yr BW)
Integrated IRAM processor
2x height
Connected via crossbar switch
growing like Moore’s law
16 Mbytes; ; 1.6 Gflops; 6.4 Gops
10,000+ nodes in one rack! 100/board = 1
TB; 0.16 Tf
Copyright Gordon Bell & Jim Gray
PC+
The Disk Farm? or
a System On a Card?
14"
The 500GB disc card
An array of discs
Can be used as
100 discs
1 striped disc
50 FT discs
....etc
LOTS of accesses/second
of bandwidth
A few disks are replaced by 10s of Gbytes
of
RAM
and
a
processor
to
run
Apps!!
Copyright Gordon Bell & Jim Gray
PC+
Nanochip.com
Copyright Gordon Bell & Jim Gray
PC+
Disk vs Tape
At 10K$/TB disks
are competitive
with nearline tape.

Disk

Tape
–
40 GB
– 40 GB
20 MBps
– 10 MBps
5 ms seek time
– 10 sec pick time
3 ms rotate latency – 30-120 second seek time
7$/GB for drive
– 2$/GB for media
3$/GB for ctlrs/cabinet8$/GB for drive+library
Guestimates
4 TB/rack
– 10 TB/rack
Cern: 200 TB
–
1 hour scan
–
–
–
–
–
–
1 week scan
The price advantage of tape is narrowing, and
the performance advantage of disk is growing
3480 tapes
2 col = 50GB
Rack = 1 TB
=20 drives
1988 Federal Plan for Internet
The virtuous cycle of bandwidth
supply and demand
Increased
Demand
Increase Capacity
(circuits & bw)
Standards
Create new
service
Telnet & FTP
EMAIL
Lower
response time
WWW
Audio
Voice!
Video
744Mbps over 5000 km to transmit 14 GB
~
4e15 bit meters per second
4 Peta Bmps (“peta bumps”)
Single Stream tcp/ip throughput
Information Sciences Institute
Microsoft
QWest
University of Washington
Pacific Northwest Gigapop
HSCC (high speed connectivity
consortium)
DARPA
Copyright Gordon Bell & Jim Gray
PC+
Redmond/Seattle,
Map
of GrayWABell Prize results
single-thread single-stream tcp/ip
New York
via 7 hops
desktop-to-desktop …Win 2K
out of the box performance*
Arlington, VA
San Francisco,
CA
5626 km
10 hops
Copyright Gordon Bell & Jim Gray
PC+
Ubiquitous 10 GBps SANs
in 5 years

1Gbps Ethernet are reality now.
–

Also FiberChannel ,MyriNet, GigaNet,
ServerNet,, ATM,…
1 GBps
10 Gbps x4 WDM deployed now
(OC192)
–

3 Tbps WDM working in lab
In 5 years, expect 10x,
wow!!
120 MBps
(1Gbps)
80 MBps
40 MBps
Copyright Gordon Bell & Jim 20
Gray
MBps
5 MBps
PC+
The Promise of SAN/VIA:10x in 2 years
http://www.ViArch.org/

Yesterday:
–
–
–

250
10 MBps (100 Mbps Ethernet)
~20 MBps tcp/ip saturates
2 cpus
round-trip latency ~250 µs
Now
–
Time µs to
Send 1KB
200
150
Transmit
receivercpu
sender cpu
100
Wires are 10x faster
Myrinet, Gbps Ethernet, ServerNet,…
–
50
Fast user-level
communication
-
tcp/ip ~ 100 MBps 10% cpu
round-trip latency is 15 us
Copyright
1.6 Gbps
demoed
Gordon
Bell & Jim on
Graya
WAN
0
100Mbps
Gbps
SAN
PC+
Modern scalable switches …
also hide a supercomputer
Scale from <1 to 120 Tbps
 1 Gbps ethernet switches scale to
10s of Gbps, scaling upward
 SP2 scales from 1.2

Copyright Gordon Bell & Jim Gray
PC+
Where are the challenges?



Continued development based on clusters
… Scalar processors need to compete with
vectors. The U.S. has cast its lot with COTS!
Explore radical alternatives.
WWW is here. Now exploit it in every respect.
– Exploit OSS… though it may not be new!
– Telepresence & interactive communities!!!
– Grid as a prelude to:
– Application Service Providers
Prototype biologist and chemist workbenches
- Labscape @ Cell laboratory, U. of WA
- Sloan sky survey
-
Copyright Gordon Bell & Jim Gray
PC+
1st, 2nd, 3rd, or New Paradigm for science?
Labscape
Copyright Gordon Bell & Jim Gray
PC+
Labscape
Copyright Gordon Bell & Jim Gray
PC+
Labscape
Copyright Gordon Bell & Jim Gray
PC+
Labscape sensors

Location tracking of people/samples
–
–



multiple resolutions
passive and active tags
Manual tasks (e.g., use of reagents,
tools)
Audio/video records, vision and
indexing
Networked instruments (e.g.,
pipettes, refrigerators, etc.)
Copyright Gordon Bell & Jim Gray
PC+
What am I willing to predict?

Processing & data can be anywhere…
–
–
–
–



Maui… in winter. BW is the limiter!
Japan… if supers are so super else use PCs
In the disks
Application Service Providers: can we separate
our data from ourselves and businesses
(ying-yang of personal versus central services)
The GRID e.g. biologist & chemist
workbenches iff the IP doesn’t get in way
Collaboration ala astrophysics (high
energy physics, math, earth sci. and any
pure science if pure science continues!)
OSS is the big bang for supercomputing??
Copyright Gordon Bell & Jim Gray
PC+
The End
Copyright Gordon Bell & Jim Gray
PC+
TECHNOLOGY DEVELOPMENT
BIO INTELLIGENCE AGE
CONSUMER ACCEPTANCE
2000 BC
0
1500
1800
1900
2000 AD
TIME (year)
R. Satava 29 July 99