Sustaining the Exponential Growth of
Embedded DSP Capability†
High Performance Embedded Computing
Workshop
28 September 2004
Dr. Gary Shaw
MIT Lincoln Laboratory
†
Dr. Mark Richards
Georgia Institute of Technology
This work was sponsored by the Department of the Air Force under Contract F19628-00-C-0002. Opinions, interpretations, conclusions, and
recommendations are those of the author and are not necessarily endorsed by the United States Government.
MIT Lincoln Laboratory
Elements Contributing to Embedded
Processor Performance
Signal Processor
Hardware
IC Devices
Computer
Architecture
Software
Algorithms
Functionality
MIT Lincoln Laboratory
040928-2
HPEC GAS
Outline
• Historical perspective – fulfillment of Moore’s Law
• Impediments to continued IC density growth
• Algorithms – the softer side of exponential growth
• Implications regarding sustaining exponential growth
• Summary and Conclusions
MIT Lincoln Laboratory
040928-3
HPEC GAS
Moore’s Law: Prediction and Realization
John von Neumann: “Truth is much too complicated
to allow anything but approximations.”
2x every ~1.9 years
Original Prediction
(Source: Electronics 1965)
Gordon Moore: “If Al Gore invented the
Internet, I invented the exponential”
2x every year
Source: Intel
MIT Lincoln Laboratory
040928-4
HPEC GAS
Top 500 Computer Growth
10000.00
1000.00
100.00
10.00
N=1
N=100
N=500
1.00
2x every 1.1 year
Jun-04
Jun-03
Jun-02
Jun-01
Jun-00
Jun-99
Jun-98
Jun-97
Jun-96
Jun-95
Jun-94
0.10
Jun-93
Peak Performance (GFLOPS)
100000.00
MIT Lincoln Laboratory
040928-5
HPEC GAS
Outline
• Historical perspective - fulfillment of Moore’s Law
•
Impediments to continued IC density growth
– Heat dissipation
– Quantum effects
– Production technology
• Algorithms – the softer side of exponential growth
• Implications regarding sustaining exponential growth
• Summary and Conclusions
MIT Lincoln Laboratory
040928-6
HPEC GAS
Performance Implications of
Shrinking Feature Size
100000
D  feature size
10000
Geometrical
Dependency
1000
Clock Frequency
1/D
100
Transistor Power
D
Transistor Density
1/D2
Total Device Power
1/D
Power Density
1/D
Energy/Instruction
D2
nJ/Instruction
Performance
Metric
Energy per Instruction
Frantz
Smailagic
Intel
trend
10
1
-2x every 1.8 years
0.1
0.01
0.001
1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Year
MIT Lincoln Laboratory
040928-7
HPEC GAS
Moore’s Law Growth in Power Density
Power Density (W/cm2)
1000
nuclear reactor fuel cell
100
Pentium 4
space shuttle tile
2x in 3.3 years
Pentium II
Pentium MMX
10
i486
Pentium
hot plate
i386
1
1985
1990
1995
2000
2005
2010
Year
MIT Lincoln Laboratory
040928-8
HPEC GAS
Moore’s Law is Dead, Long Live Moore’s Law!
Theory & Practice: Feature Size for MOSFET Devices
It's tough to make predictions, especially about the future. - Yogi Berra
Gate Length (mm)
Year
Elect.
Per Bit
Now
~100
+10
~10
+20
~1
Year
Sources:
Combined graph and original concept: Lance Glasser, former Director, DARPA/ETO
Theory: Provided by Prof. David Ferry, Arizona State University
Practice: The National Technology Roadmap for Semiconductors (SIA Publication, 1994)
MIT Lincoln Laboratory
040928-9
HPEC GAS
Capitalization Cost Impediments
Wafer-Fab Capital Cost ($B)
100
Extrapolation
10
ROI Risk Limited?
2x every 3 years
12” fab $3-4B
1
0.1
Intel withdraws from DRAM market due
to estimated ~ $400M capitalization cost
0.01
1970
1980
1990
Year
2000
2010
2020
Adapted from:
MIT Lincoln Laboratory
040928-10
HPEC GAS
Fulfillment and Impact of Moore’s Prediction
Biological
Computing?
• Silicon CMOS IC fabrication technology
Naïve, low-order
implementations
Transistors
Year
Scaling
Rules
Process
Innovations
Design Tool
Innovations
102
109
1965
2005
Paradigm
Shift?
2015?
Fundamental
Limits
Quantum
Computing?
• Examples of far-reaching impact
Altair 8800, 1975
Itanium, 2005
Exponential Improvements
In Computing at a Fixed
Price Point
Embedded Processors
For Real-time Digital
Signal Processing
Low-power Wireless
Applications
Loosely-Coupled
Hardware & Software
Design Methodologies
MIT Lincoln Laboratory
040928-11
HPEC GAS
Outline
• Historical perspective - fulfillment of Moore’s Law
• Impediments to continued IC density growth
•
Algorithms – the softer side of exponential growth
• Implications regarding sustaining exponential growth
• Summary and Conclusions
MIT Lincoln Laboratory
040928-12
HPEC GAS
Elements Contributing to Embedded
Processor Performance
Signal Processor
Hardware
IC Devices
Computer
Architecture
Software
Algorithms
Functionality
MIT Lincoln Laboratory
040928-13
HPEC GAS
Different Character of Hardware (IC) Vs.
Algorithm Improvements
Improvement Metrics
Hardware
Algorithms
Regularity
Predictable
Unpredictable
Dependent variable
Time
Order complexity
Impact on applications
Incremental
Leap-ahead
Useful lifetime
3 years or less
10 years or more
R&D Cost growth
2x in 3 years
1.11x in 3 years
MIT Lincoln Laboratory
040928-14
HPEC GAS
Computational Complexity Reduction Afforded
by the FFT Over a Sum-of-Products DFT
FFT Improvement Over DFT
Complexity Reduction Factor
10000
1000
100
10
1
4
8
16
32
64
128
256
512
1024 2048 4096 8192 16384
FFT Length
MIT Lincoln Laboratory
040928-15
HPEC GAS
Moore’s-Law Equivalent Years Required
to Match FFT Computational Speedup
Equivalent Years of Hardware Improvement
Years of Hardware Improvement
Equivalent Moore's Law Improvements
Required for Equal Computational speedup
Years
25
20
15
10
5
0
4
8
16
32
64
128
256
512
1024
2048
4096
8192 16384
FFT FFT
Length
Radix-2
Length
MIT Lincoln Laboratory
040928-16
HPEC GAS
Exponential Improvement in Modem Rates
Downstream Data Rate (bps)
100000
10000
2x every 1.8 years
1000
100
1984
1986
1988
1990
1992
1994
1996
1998
2000
Year
MIT Lincoln Laboratory
040928-17
HPEC GAS
Application Maturation Cycle
MIT Lincoln Laboratory
040928-18
HPEC GAS
Pulse-Doppler Radar Example
• Algorithmically naïve implementation
LPF
RF
BPF
A/D
I
Time-Domain
Matched Filter
cos(wot)
LPF
LO
Matrix
Transpose
Doppler
DFT
A/D
Q
sin(wot)
• Reduced-order implementation with digital I/Q
RF
LPF
Chirp
A/D
LPF
I + jQ
Range
FFT
Matrix
Transpose
Doppler
FFT
jn
MIT Lincoln Laboratory
040928-19
HPEC GAS
Pulse-Doppler Radar Algorithm Improvements
100
10
GOPS
Pulse Compression
Doppler Filtering
Digital I/Q
1
0.1
Moore’s Law
Reference slope
Year
040928-20
HPEC GAS
19
85
19
83
19
81
19
79
19
77
19
75
19
73
19
71
19
69
19
67
19
65
19
63
19
61
19
59
19
57
19
55
0.01
MIT Lincoln Laboratory
Outline
• Historical perspective - fulfillment of Moore’s Law
• Impediments to continued IC density growth
• Algorithms – the softer side of exponential growth
•
Implications regarding sustaining exponential growth
• Summary and Conclusions
MIT Lincoln Laboratory
040928-21
HPEC GAS
IC Vs. Algorithm Development
(A Contrived but Useful Analogy)
Biological
Computing?
• ICs
Naïve, low-order
implementations
Scaling
Rules
Process
Innovations
Design Tool
Innovations
Paradigm
Shift?
Silicon lithographic fabrication technology
Quantum
Computing?
Nonlinear
Algorithms?
• Algorithms
Naïve, high-order
implementations
Fundamental
Limits
Order
Reductions
Architecture
Innovations
H/W & S/W
Codesign Tools
Paradigm
Shift?
Fundamental
Limits
Linear algebraic representations and processing
Quantum Signal
Processing?
MIT Lincoln Laboratory
040928-22
HPEC GAS
Increased Emphasis on
Codesign Methodologies
Signal Processor
Hardware
IC Devices
Computer
Architecture
Software
Algorithms
Functionality
MIT Lincoln Laboratory
040928-23
HPEC GAS
Wafer-Fab Capitalization Cost Compared to
Annual DSP Algorithm R&D Costs
100
Capital cost for state-of-the-art wafer fab facility
Annual R&D support for entire IEEE SP Society membership (18,500 x $150K in 2001)
10
$B
1.11x every 3 years†
1
2x every 3 years
0.1
0.01
1970
†
1980
1990
Year
2000
2010
2020
Salary inflation rate based on US Bureau of Labor and Statistics Median Engineering Salaries 1983-2003
MIT Lincoln Laboratory
040928-24
HPEC GAS
Summary and Conclusions
•
Fulfilling Moore’s Law
–
–
•
Taking up the slack
–
–
–
•
Over same 40-year time frame as Moore’s Law, algorithm innovation
has yielded exponentially improving performance as well
Algorithm innovation also enabled by diverse R&D, but without as
clear of an industry-wide common vision
Algorithm R&D cost growth significantly lower than fab capital cost
growth (1.1x vs. 2x every 3 years)
Increasing the effectiveness of algorithm R&D
–
–
•
Enabled by diverse, innovative R&D aimed at realizing a common
vision (ITRS semiconductor roadmap)
Continued improvements may be impeded by a combination of
thermal, quantum, and capital cost limits
Develop better methods for quantifying the return on investment for
algorithm R&D
Consider mechanisms for developing a broader industry vision and
commitment to a long-term R&D roadmap
Hardware/software codesign methods increasingly important
MIT Lincoln Laboratory
040928-25
HPEC GAS