advertisement

Sustaining the Exponential Growth of Embedded DSP Capability† High Performance Embedded Computing Workshop 28 September 2004 Dr. Gary Shaw MIT Lincoln Laboratory † Dr. Mark Richards Georgia Institute of Technology This work was sponsored by the Department of the Air Force under Contract F19628-00-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the author and are not necessarily endorsed by the United States Government. MIT Lincoln Laboratory Elements Contributing to Embedded Processor Performance Signal Processor Hardware IC Devices Computer Architecture Software Algorithms Functionality MIT Lincoln Laboratory 040928-2 HPEC GAS Outline • Historical perspective – fulfillment of Moore’s Law • Impediments to continued IC density growth • Algorithms – the softer side of exponential growth • Implications regarding sustaining exponential growth • Summary and Conclusions MIT Lincoln Laboratory 040928-3 HPEC GAS Moore’s Law: Prediction and Realization John von Neumann: “Truth is much too complicated to allow anything but approximations.” 2x every ~1.9 years Original Prediction (Source: Electronics 1965) Gordon Moore: “If Al Gore invented the Internet, I invented the exponential” 2x every year Source: Intel MIT Lincoln Laboratory 040928-4 HPEC GAS Top 500 Computer Growth 10000.00 1000.00 100.00 10.00 N=1 N=100 N=500 1.00 2x every 1.1 year Jun-04 Jun-03 Jun-02 Jun-01 Jun-00 Jun-99 Jun-98 Jun-97 Jun-96 Jun-95 Jun-94 0.10 Jun-93 Peak Performance (GFLOPS) 100000.00 MIT Lincoln Laboratory 040928-5 HPEC GAS Outline • Historical perspective - fulfillment of Moore’s Law • Impediments to continued IC density growth – Heat dissipation – Quantum effects – Production technology • Algorithms – the softer side of exponential growth • Implications regarding sustaining exponential growth • Summary and Conclusions MIT Lincoln Laboratory 040928-6 HPEC GAS Performance Implications of Shrinking Feature Size 100000 D feature size 10000 Geometrical Dependency 1000 Clock Frequency 1/D 100 Transistor Power D Transistor Density 1/D2 Total Device Power 1/D Power Density 1/D Energy/Instruction D2 nJ/Instruction Performance Metric Energy per Instruction Frantz Smailagic Intel trend 10 1 -2x every 1.8 years 0.1 0.01 0.001 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 Year MIT Lincoln Laboratory 040928-7 HPEC GAS Moore’s Law Growth in Power Density Power Density (W/cm2) 1000 nuclear reactor fuel cell 100 Pentium 4 space shuttle tile 2x in 3.3 years Pentium II Pentium MMX 10 i486 Pentium hot plate i386 1 1985 1990 1995 2000 2005 2010 Year MIT Lincoln Laboratory 040928-8 HPEC GAS Moore’s Law is Dead, Long Live Moore’s Law! Theory & Practice: Feature Size for MOSFET Devices It's tough to make predictions, especially about the future. - Yogi Berra Gate Length (mm) Year Elect. Per Bit Now ~100 +10 ~10 +20 ~1 Year Sources: Combined graph and original concept: Lance Glasser, former Director, DARPA/ETO Theory: Provided by Prof. David Ferry, Arizona State University Practice: The National Technology Roadmap for Semiconductors (SIA Publication, 1994) MIT Lincoln Laboratory 040928-9 HPEC GAS Capitalization Cost Impediments Wafer-Fab Capital Cost ($B) 100 Extrapolation 10 ROI Risk Limited? 2x every 3 years 12” fab $3-4B 1 0.1 Intel withdraws from DRAM market due to estimated ~ $400M capitalization cost 0.01 1970 1980 1990 Year 2000 2010 2020 Adapted from: MIT Lincoln Laboratory 040928-10 HPEC GAS Fulfillment and Impact of Moore’s Prediction Biological Computing? • Silicon CMOS IC fabrication technology Naïve, low-order implementations Transistors Year Scaling Rules Process Innovations Design Tool Innovations 102 109 1965 2005 Paradigm Shift? 2015? Fundamental Limits Quantum Computing? • Examples of far-reaching impact Altair 8800, 1975 Itanium, 2005 Exponential Improvements In Computing at a Fixed Price Point Embedded Processors For Real-time Digital Signal Processing Low-power Wireless Applications Loosely-Coupled Hardware & Software Design Methodologies MIT Lincoln Laboratory 040928-11 HPEC GAS Outline • Historical perspective - fulfillment of Moore’s Law • Impediments to continued IC density growth • Algorithms – the softer side of exponential growth • Implications regarding sustaining exponential growth • Summary and Conclusions MIT Lincoln Laboratory 040928-12 HPEC GAS Elements Contributing to Embedded Processor Performance Signal Processor Hardware IC Devices Computer Architecture Software Algorithms Functionality MIT Lincoln Laboratory 040928-13 HPEC GAS Different Character of Hardware (IC) Vs. Algorithm Improvements Improvement Metrics Hardware Algorithms Regularity Predictable Unpredictable Dependent variable Time Order complexity Impact on applications Incremental Leap-ahead Useful lifetime 3 years or less 10 years or more R&D Cost growth 2x in 3 years 1.11x in 3 years MIT Lincoln Laboratory 040928-14 HPEC GAS Computational Complexity Reduction Afforded by the FFT Over a Sum-of-Products DFT FFT Improvement Over DFT Complexity Reduction Factor 10000 1000 100 10 1 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 FFT Length MIT Lincoln Laboratory 040928-15 HPEC GAS Moore’s-Law Equivalent Years Required to Match FFT Computational Speedup Equivalent Years of Hardware Improvement Years of Hardware Improvement Equivalent Moore's Law Improvements Required for Equal Computational speedup Years 25 20 15 10 5 0 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 FFT FFT Length Radix-2 Length MIT Lincoln Laboratory 040928-16 HPEC GAS Exponential Improvement in Modem Rates Downstream Data Rate (bps) 100000 10000 2x every 1.8 years 1000 100 1984 1986 1988 1990 1992 1994 1996 1998 2000 Year MIT Lincoln Laboratory 040928-17 HPEC GAS Application Maturation Cycle MIT Lincoln Laboratory 040928-18 HPEC GAS Pulse-Doppler Radar Example • Algorithmically naïve implementation LPF RF BPF A/D I Time-Domain Matched Filter cos(wot) LPF LO Matrix Transpose Doppler DFT A/D Q sin(wot) • Reduced-order implementation with digital I/Q RF LPF Chirp A/D LPF I + jQ Range FFT Matrix Transpose Doppler FFT jn MIT Lincoln Laboratory 040928-19 HPEC GAS Pulse-Doppler Radar Algorithm Improvements 100 10 GOPS Pulse Compression Doppler Filtering Digital I/Q 1 0.1 Moore’s Law Reference slope Year 040928-20 HPEC GAS 19 85 19 83 19 81 19 79 19 77 19 75 19 73 19 71 19 69 19 67 19 65 19 63 19 61 19 59 19 57 19 55 0.01 MIT Lincoln Laboratory Outline • Historical perspective - fulfillment of Moore’s Law • Impediments to continued IC density growth • Algorithms – the softer side of exponential growth • Implications regarding sustaining exponential growth • Summary and Conclusions MIT Lincoln Laboratory 040928-21 HPEC GAS IC Vs. Algorithm Development (A Contrived but Useful Analogy) Biological Computing? • ICs Naïve, low-order implementations Scaling Rules Process Innovations Design Tool Innovations Paradigm Shift? Silicon lithographic fabrication technology Quantum Computing? Nonlinear Algorithms? • Algorithms Naïve, high-order implementations Fundamental Limits Order Reductions Architecture Innovations H/W & S/W Codesign Tools Paradigm Shift? Fundamental Limits Linear algebraic representations and processing Quantum Signal Processing? MIT Lincoln Laboratory 040928-22 HPEC GAS Increased Emphasis on Codesign Methodologies Signal Processor Hardware IC Devices Computer Architecture Software Algorithms Functionality MIT Lincoln Laboratory 040928-23 HPEC GAS Wafer-Fab Capitalization Cost Compared to Annual DSP Algorithm R&D Costs 100 Capital cost for state-of-the-art wafer fab facility Annual R&D support for entire IEEE SP Society membership (18,500 x $150K in 2001) 10 $B 1.11x every 3 years† 1 2x every 3 years 0.1 0.01 1970 † 1980 1990 Year 2000 2010 2020 Salary inflation rate based on US Bureau of Labor and Statistics Median Engineering Salaries 1983-2003 MIT Lincoln Laboratory 040928-24 HPEC GAS Summary and Conclusions • Fulfilling Moore’s Law – – • Taking up the slack – – – • Over same 40-year time frame as Moore’s Law, algorithm innovation has yielded exponentially improving performance as well Algorithm innovation also enabled by diverse R&D, but without as clear of an industry-wide common vision Algorithm R&D cost growth significantly lower than fab capital cost growth (1.1x vs. 2x every 3 years) Increasing the effectiveness of algorithm R&D – – • Enabled by diverse, innovative R&D aimed at realizing a common vision (ITRS semiconductor roadmap) Continued improvements may be impeded by a combination of thermal, quantum, and capital cost limits Develop better methods for quantifying the return on investment for algorithm R&D Consider mechanisms for developing a broader industry vision and commitment to a long-term R&D roadmap Hardware/software codesign methods increasingly important MIT Lincoln Laboratory 040928-25 HPEC GAS