Memory Power Management via
Dynamic Voltage/Frequency Scaling
Howard David (Intel)
Eugene Gorbatov (Intel)
Ulf R. Hanebutte (Intel)
Chris Fallin (CMU)
Onur Mutlu (CMU)
Memory Power is Significant
Power consumption is a primary concern in modern servers
Many works: CPU, whole-system or cluster-level approach
But memory power is largely unaddressed
Our server system*: memory is 19% of system power (avg)
n
n
n
n
q
Some work notes up to 40% of total system power
Power (W) 400 300 200 100 0 System Power Memory Power lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer Goal: Can we reduce this figure?
n
*Dual 4-core Intel Xeon®, 48GB DDR3 (12 DIMMs), SPEC CPU2006, all cores active.
Measured AC power, analytically modeled memory power.
2
Existing Solution: Memory Sleep States?
n
Most memory energy-efficiency work uses sleep states
q
n
Shut down DRAM devices when no memory requests active
But, even low-memory-bandwidth workloads keep memory
awake
Idle periods between requests diminish in multicore workloads
q  CPU-bound workloads/phases rarely completely cache-resident
Sleep State Residency 8% 6% 4% 2% 0% 3
hmmer povray gamess gromacs namd h264ref perlbench calculix sjeng gobmk bzip2 tonto dealII gcc cactusADM mcf sphinx3 soplex libquantum leslie3d milc GemsFDTD lbm Time Spent in Sleep States q
Memory Bandwidth Varies Widely
n
Workload memory bandwidth requirements vary widely
Bandwidth/channel (GB/s) Memory Bandwidth for SPEC CPU2006 n
8 6 4 2 0 Memory system is provisioned for peak capacity
à often underutilized
4
Memory Power can be Scaled Down
n
DDR can operate at multiple frequencies à reduce power
q
q
q
n
Lower frequency directly reduces switching power
Lower frequency allows for lower voltage
Comparable to CPU DVFS
CPU Voltage/
Freq. System Power Memory Freq. System Power ↓ 15% ↓ 9.9% ↓ 40% ↓ 7.6% Frequency scaling increases latency à reduce performance
q
q
q
Memory storage array is asynchronous
But, bus transfer depends on frequency
When bus bandwidth is bottleneck, performance suffers
5
Observations So Far
n
Memory power is a significant portion of total power
q
n
19% (avg) in our system, up to 40% noted in other works
Sleep state residency is low in many workloads
q
q
Multicore workloads reduce idle periods
CPU-bound applications send requests frequently enough
to keep memory devices awake
n
Memory bandwidth demand is very low in some workloads
n
Memory power is reduced by frequency scaling
q
And voltage scaling can give further reductions
6
DVFS for Memory
n
Key Idea: observe memory bandwidth utilization, then
adjust memory frequency/voltage, to reduce power with
minimal performance loss
à Dynamic Voltage/Frequency Scaling (DVFS)
for memory
n
Goal in this work:
q
q
q
Implement DVFS in the memory system, by:
Developing a simple control algorithm to exploit opportunity
for reduced memory frequency/voltage by observing behavior
Evaluating the proposed algorithm on a real system
7
Outline
n
Motivation
n
Background and Characterization
q
q
q
DRAM Operation
DRAM Power
Frequency and Voltage Scaling
n
Performance Effects of Frequency Scaling
n
Frequency Control Algorithm
n
Evaluation and Conclusions
8
Outline
n
Motivation
n
Background and Characterization
q
q
q
DRAM Operation
DRAM Power
Frequency and Voltage Scaling
n
Performance Effects of Frequency Scaling
n
Frequency Control Algorithm
n
Evaluation and Conclusions
9
DRAM Operation
n
n
Main memory consists of DIMMs of DRAM devices
Each DIMM is attached to a memory bus (channel)
/8
/8
/8
/8
/8
/8
/8
/8
Memory Bus (64 bits)
10
DRAM Operation
n
n
n
Main memory consists of DIMMs of DRAM devices
Each DIMM is attached to a memory bus (channel)
Multiple DIMMs can connect to one channel
to Memory Controller
11
Inside a DRAM Device
12
Row Decoder
Inside a DRAM Device
Banks
Bank 0
•  Independent arrays
•  Asynchronous:
independent of
memory bus speed
Sense Amps
Column Decoder
13
Inside a DRAM Device
Recievers
Drivers
Sense Amps
Column Decoder
Registers
Runs at bus speed
Clock sync/distribution
Bank 0
Bus drivers and receivers
Buffering/queueing
Write FIFO
•
•
•
•
Row Decoder
I/O Circuitry
14
Inside a DRAM Device
Recievers
Registers
Bank 0Termination
On-Die
Write FIFO
Row Decoder
ODT
Column Decoder
Drivers
•  Required by bus electrical characteristics
for reliable operation
•  Resistive element that dissipates power
whenSense
bus is
active
Amps
15
Inside a DRAM Device
Recievers
Drivers
Sense Amps
Column Decoder
Registers
Bank 0
Write FIFO
Row Decoder
ODT
16
Effect of Frequency Scaling on Power
n
n
n
n
n
Reduced memory bus frequency:
Does not affect bank power:
q  Constant energy per operation
q  Depends only on utilized memory bandwidth
Decreases I/O power:
q  Dynamic power in bus interface and clock circuitry
reduces due to less frequent switching
Increases termination power:
q  Same data takes longer to transfer
q  Hence, bus utilization increases
Tradeoff between I/O and termination results in a net
power reduction at lower frequencies
17
Effects of Voltage Scaling on Power
n
Voltage scaling further reduces power because all parts of
memory devices will draw less current (at less voltage)
Voltage reduction is possible because stable operation
requires lower voltage at lower frequency:
Minimum Stable Voltage for 8 DIMMs in a Real System DIMM Voltage (V) n
1.6 1.5 1.4 1.3 1.2 1.1 1 Vdd for Power Model 1333MHz 1066MHz 800MHz 18
Outline
n
Motivation
n
Background and Characterization
q
q
q
DRAM Operation
DRAM Power
Frequency and Voltage Scaling
n
Performance Effects of Frequency Scaling
n
Frequency Control Algorithm
n
Evaluation and Conclusions
19
Bandwidth/channel (GB/s) 7 6 5 4 3 2 1 0 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer How Much Memory Bandwidth is Needed?
Memory Bandwidth for SPEC CPU2006 20
Performance Impact of Static Frequency Scaling
Performance Loss, StaNc Frequency Scaling 80 70 60 50 40 30 20 10 0 1333-­‐>800 1333-­‐>1066 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer n
Performance impact is proportional to bandwidth demand
Many workloads tolerate lower frequency with minimal
performance drop
Performance Drop (%) n
21
Performance Impact of Static Frequency Scaling
8 Performance Loss, StaNc Frequency Scaling ::
::
:: ::
:: :: ::
:: ::
:: :: :: ::
6 4 1333-­‐>800 1333-­‐>1066 2 0 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer n
Performance impact is proportional to bandwidth demand
Many workloads tolerate lower frequency with minimal
performance drop
Performance Drop (%) n
22
Outline
n
Motivation
n
Background and Characterization
q
q
q
DRAM Operation
DRAM Power
Frequency and Voltage Scaling
n
Performance Effects of Frequency Scaling
n
Frequency Control Algorithm
n
Evaluation and Conclusions
23
Memory Latency Under Load
n
At low load, most time is in array access and bus transfer
à small constant offset between bus-frequency latency curves
As load increases, queueing delay begins to dominate
à bus frequency significantly affects latency
Memory Latency as a FuncNon of Bandwidth and Mem Frequency Latency (ns) n
800MHz 180 150 120 90 60 0 2000 1067MHz 4000 1333MHz 6000 8000 UNlized Channel Bandwidth (MB/s) 24
Control Algorithm: Demand-Based Switching
Latency (ns) Memory Latency as a FuncNon of Bandwidth and Mem Frequency 800MHz 180 150 120 90 60 0 1067MHz 2000 T800
4000 T1066
1333MHz 6000 8000 UNlized Channel Bandwidth (MB/s) After each epoch of length Tepoch:
Measure per-channel bandwidth BW
if
BW < T800 : switch to 800MHz
else if BW < T1066 : switch to 1066MHz
else
: switch to 1333MHz
25
Implementing V/F Switching
n
Halt Memory Operations
q
q
q
n
Transition Voltage/Frequency
q
q
q
q
n
Pause requests
Put DRAM in Self-Refresh
Stop the DIMM clock
Begin voltage ramp
Relock memory controller PLL at new frequency
Restart DIMM clock
Wait for DIMM PLLs to relock
Begin Memory Operations
q
q
Take DRAM out of Self-Refresh
Resume requests
26
Implementing V/F Switching
n
Halt Memory Operations
q
q
q
n
Pause requests
Put DRAM in Self-Refresh
Stop the DIMM clock
Transition Voltage/Frequency
Begin voltage ramp
q  Relock memory controller PLL at new frequency
frequency
already adjustable statically
q  Memory
Restart DIMM
clock
q  Wait for DIMM PLLs to relock
q
C
C
n
Voltage regulators for CPU DVFS can work for
Begin Memory Operations
memory DVFS
Take DRAM out of Self-Refresh
transition
q  Full
Resume
requests takes ~20µs
q
C
27
Outline
n
Motivation
n
Background and Characterization
q
q
q
DRAM Operation
DRAM Power
Frequency and Voltage Scaling
n
Performance Effects of Frequency Scaling
n
Frequency Control Algorithm
n
Evaluation and Conclusions
28
Evaluation Methodology
n
Real-system evaluation
q  Dual 4-core Intel Xeon®, 3 memory channels/socket
q
n
Emulating memory frequency for performance
q
q
n
48 GB of DDR3 (12 DIMMs, 4GB dual-rank, 1333MHz)
Altered memory controller timing registers (tRC, tB2BCAS)
Gives performance equivalent to slower memory frequencies
Modeling power reduction
q
q
Measure baseline system (AC power meter, 1s samples)
Compute reductions with an analytical model (see paper)
29
Evaluation Methodology
n
Workloads
q
q
n
SPEC CPU2006: CPU-intensive workloads
All cores run a copy of the benchmark
Parameters
q
q
q
q
Tepoch = 10ms
Two variants of algorithm with different switching thresholds:
BW(0.5, 1): T800 = 0.5GB/s, T1066 = 1GB/s
BW(0.5, 2): T800 = 0.5GB/s, T1066 = 2GB/s
à More aggressive frequency/voltage scaling
30
-­‐1 AVG hmmer povray gamess gromacs 2 namd 3 h264ref perlbench calculix sjeng gobmk bzip2 tonto dealII gcc cactusADM mcf sphinx3 soplex libquantum leslie3d milc GemsFDTD lbm n
Performance DegradaNon (%) Performance Impact of Memory DVFS
Minimal performance degradation: 0.2% (avg), 1.7% (max)
4 BW(0.5,1) BW(0.5,2) 1 0 31
-­‐1 AVG hmmer povray gamess gromacs 2 namd 3 h264ref perlbench calculix sjeng gobmk bzip2 tonto dealII gcc cactusADM mcf sphinx3 soplex libquantum leslie3d milc GemsFDTD n
lbm n
Performance DegradaNon (%) Performance Impact of Memory DVFS
Minimal performance degradation: 0.2% (avg), 1.7% (max)
Experimental error ~1%
4 BW(0.5,1) BW(0.5,2) 1 0 32
n
0% lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer Memory Frequency Distribution
Frequency distribution shifts toward higher memory
frequencies with more memory-intensive benchmarks
100% 80% 60% 40% 1333 20% 1066 800 33
Memory Power ReducNon (%) n
20 15 0 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer AVG Memory Power Reduction
Memory power reduces by 10.4% (avg), 20.5% (max)
25 BW(0.5,1) BW(0.5,2) 10 5 34
System Power ReducNon (%) n
4 3.5 3 2.5 2 1.5 1 0.5 0 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer AVG System Power Reduction
As a result, system power reduces by 1.9% (avg), 3.5% (max)
BW(0.5,1) BW(0.5,2) 35
System Energy ReducNon (%) n
4 3 -­‐1 lbm GemsFDTD milc leslie3d libquantum soplex sphinx3 mcf cactusADM gcc dealII tonto bzip2 gobmk sjeng calculix perlbench h264ref namd gromacs gamess povray hmmer AVG System Energy Reduction
System energy reduces by 2.4% (avg), 5.1% (max)
6 5 BW(0.5,1) BW(0.5,2) 2 1 0 36
Related Work
n
MemScale [Deng11], concurrent work (ASPLOS 2011)
q
q
q
n
n
Also proposes Memory DVFS
Application performance impact model to decide voltage and
frequency: requires specific modeling for a given system; our
bandwidth-based approach avoids this complexity
Simulation-based evaluation; our work is a real-system proof
of concept
Memory Sleep States (Creating opportunity with data placement
[Lebeck00,Pandey06], OS scheduling [Delaluz02], VM subsystem [Huang05];
Making better decisions with better models [Hur08,Fan01])
Power Limiting/Shifting (RAPL [David10] uses memory throttling for
thermal limits; CPU throttling for memory traffic [Lin07,08]; Power shifting
across system [Felter05])
37
Conclusions
n
Memory power is a significant component of system power
q
n
Workloads often keep memory active but underutilized
q
q
n
Channel bandwidth demands are highly variable
Use of memory sleep states is often limited
Scaling memory frequency/voltage can reduce memory
power with minimal system performance impact
q
q
n
19% average in our evaluation system, 40% in other work
10.4% average memory power reduction
Yields 2.4% average system energy reduction
Greater reductions are possible with wider frequency/
voltage range and better control algorithms
38
Memory Power Management via
Dynamic Voltage/Frequency Scaling
Howard David (Intel)
Eugene Gorbatov (Intel)
Ulf R. Hanebutte (Intel)
Chris Fallin (CMU)
Onur Mutlu (CMU)
Why Real-System Evaluation?
n
Advantages:
q
Capture all effects of altered memory performance
n
q
q
n
Able to run full-length benchmarks (SPEC CPU2006) rather
than short instruction traces
No concerns about architectural simulation fidelity
Disadvantages:
q
q
n
System/kernel code, interactions with IO and peripherals, etc
More limited room for novel algorithms and detailed
measurements
Inherent experimental error due to background-task noise,
real power measurements, nondeterministic timing effects
For a proof-of-concept, we chose to run on a real system in
order to have results that capture all potential side-effects
of altering memory frequency
40
CPU-Bound Applications in a DRAM-rich system
n
We evaluate CPU-bound workloads with 12 DIMMs:
what about smaller memory, or IO-bound workloads?
n
12 DIMMs (48GB): are we magnifying the problem?
q
q
n
Large servers can have this much memory, especially for database
or enterprise applications
Memory can be up to 40% of system power [1,2], and reducing its
power in general is an academically interesting problem
CPU-bound workloads: will it matter in real life?
q
q
Many workloads have CPU-bound phases (e.g., database scan or
business logic in server workloads)
Focusing on CPU-bound workloads isolates the problem of varying
memory bandwidth demand while memory cannot enter sleep
states, and our solution applies for any compute phase of a workload
[1] L. A. Barroso and U. Holzle. “The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale
Machines.” Synthesis Lectures on Computer Architecture. Morgan & Claypool, 2009.
[2] C. Lefurgy et al. “Energy Management for Commercial Servers.” IEEE Computer, pp. 39—48, December 2003.
41
Combining Memory & CPU DVFS?
n
Our evaluation did not incorporate CPU DVFS:
q
q
n
Need to understand effect of single knob (memory DVFS) first
Combining with CPU DVFS might produce second-order effects
that would need to be accounted for
Nevertheless, memory DVFS is effective by itself, and
mostly orthogonal to CPU DVFS:
q
q
q
Each knob reduces power in a different component
Our memory DVFS algorithm has neligible performance impact
à negligible impact on CPU DVFS
CPU DVFS will only further reduce bandwidth demands relative
to our evaluations à no negative impact on memory DVFS
42
Why is this Autonomic Computing?
n
Power management in general is autonomic: a system
observes its own needs and adjusts its behavior accordingly
à Lots of previous work comes from architecture community,
but crossover in ideas and approaches could be beneficial
n
n
This work exposes a new knob for control algorithms to
turn, has a simple model for the power/energy effects of
that knob, and observes opportunity to apply it in a simple
way
Exposes future work for:
n
n
n
More advanced control algorithms
Coordinated energy efficiency across rest of system
Coordinated energy efficiency across a cluster/datacenter,
integrated with memory DVFS, CPU DVFS, etc.
43