A Case for
Subarray-Level Parallelism
(SALP) in DRAM
Yoongu Kim, Vivek Seshadri,
Donghyuk Lee, Jamie Liu, Onur Mutlu
Executive Summary
• Problem: Requests to same DRAM bank are
serialized
• Our Goal: Parallelize requests to same DRAM
bank at a low cost
• Observation: A bank consists of subarrays
that occassionally share global structures
• Solution: Increase independence of
subarrays to enable parallel operation
• Result: Significantly higher performance and
energy-efficiency at low cost (+0.15% area)
2
Outline
• Motivation & Key Idea
• Background
• Mechanism
• Related Works
• Results
3
Introduction
DRAM
Bank
Req
Bank
Bank
Req
Bank
Req
Bank
Req
conflict!
Req
Req
Req
4x latency
4
Bank conflicts degrade performance
Three Problems
1. Requests are serialized
2. Serialization is worse after write requests
3. Thrashing in row-buffer
Row
Row
Row
Row
Bank
Row-Buffer
Req
Req
Req
Thrashing:
increases latency
5
Case Study: Timeline
• Case #1. Different Banks
Bank
Wr Rd
Bank
Wr Rd
time
time
in parallel
• CaseServed
#2. Same
Bank Delayed
Bank
Wr 2Wr3WrWr
Rd2 2Rd3 Rd 3 Rd
time
1.
3. Write
Serialization
Thrashing
Row-Buffer
2.
Penalty
6
Our Goal
• Goal: Mitigate the detrimental effects of
bank conflicts in a cost-effective manner
• Naïve solution: Add more banks
– Very expensive
• We propose a cost-effective solution
7
Key Observation #1
A DRAM bank is divided into subarrays
Logical Bank
Row
Row
Row
Row
Row-Buffer
Physical Bank
Subarray
64
Local
Row-Buf
32k rows
Subarray
Local
Row-Buf1
Global Row-Buf
A single row-buffer Many local row-buffers,
cannot drive all rows one at each subarray
8
Key Observation #2
Each subarray is mostly independent…
Bank
Subarray
64
Local
Row-Buf
···
Global Decoder
– except occasionally sharing global structures
Subarray
Local
Row-Buf1
Global Row-Buf
9
Key Idea: Reduce Sharing of Globals
Bank
Local Row-Buf
···
Global Decoder
1. Parallel access to subarrays
Local Row-Buf
Global Row-Buf
2. Utilize multiple local row-buffers
10
Overview of Our Mechanism
Subarray64
···
Local Row-Buf
Subarray1
Local Row-Buf
1. Parallelize
Req
Req
Req
Req
2. Utilize multiple
local
row-buffers
To
same
bank...
but diff. subarrays
Global Row-Buf
11
Outline
• Motivation & Key Idea
• Background
• Mechanism
• Related Works
• Results
12
Organization of DRAM System
Channel
Rank
Rank
Bank
Bus
CPU
DRAM System
Bank
13
Naïve Solutions to Bank Conflicts
1. More channels: expensive
2. More ranks: low performance
3. More banks: expensive
DRAM System
Channel
Channel
Rank
Channel
R R R R
Channel
Bank
Channel
Large load
Bus
Low frequency
Significantly
Many increases
CPU pins
Bus
Bus
DRAM die area
Bus
14
Logical Bank
addr
Decoder
bitlines
data
Precharged
State
Row
Row
Row
Row
Row-Buffer
ACTIVATE
PRECHARGE
VDD
0
0 wordlines
0
0
RD/WR
Activated
State
?
15
Physical Bank
short
Subarray
64
Local
Row-Buf
···
32k rows
very long bitlines:
hard to drive
512 rows
Local bitlines:
Subarray
Local
Row-Buf1
Row-Buffer
Global Row-Buf
16
Bank0
Bank1
Bank2
Bank3
Bank5
Bank6
Bank7
Bank8
Hynix 4Gb DDR3 (23nm)
Subarray
Lim et al., ISSCC’12
Subarray
Decoder
Tile
Magnified
17
Bank: Full Picture
Global
Local bitlines
bitlines
Subarray
64
Local
Row-Buf
Subarray
Local
bitlines
Decoder
···
Latch
Global Decoder
Bank
Subarray
Local
Row-Buf1
Global Row-Buf
18
Outline
• Motivation & Key Idea
• Background
• Mechanism
• Related Works
• Results
19
Problem Statement
···
Local Row-Buf
To different
subarrays
Req
Req
Serialized!
Local Row-Buf
Global Row-Buf
20
Overview: MASA
VDD
Local Row-Buf
···
addr
Global Decoder
MASA (Multitude of Activated Subarrays)
VDD
Local Row-Buf
READ
Global Row-Buf
21
Challenges: Global Structures
1. Global Address Latch
2. Global Bitlines
22
VDD
Local
row-buffer
···
Latch
addr
Global Decoder
Challenge #1. Global Address Latch
VDD
Local
row-buffer
Global
row-buffer
23
VDD
Local
row-buffer
···
Latch
Global Decoder
Solution #1. Subarray Address Latch
VDD
Local
row-buffer
Global
row-buffer
24
Challenges: Global Structures
1. Global Address Latch
• Problem: Only one raised wordline
• Solution: Subarray Address Latch
2. Global Bitlines
25
Challenge #2. Global Bitlines
Global bitlines
Local
row-buffer
Switch
Local
row-buffer
Switch
Global
READ
row-buffer
26
Solution #2. Designated-Bit Latch
Wire
Global bitlines
Local
row-buffer
D
Switch
Local
row-buffer
D
READ
Switch
Global
row-buffer
27
Challenges: Global Structures
1. Global Address Latch
• Problem: Only one raised wordline
• Solution: Subarray Address Latch
2. Global Bitlines
• Problem: Collision during access
• Solution: Designated-Bit Latch
28
MASA: Advantages
• Baseline (Subarray-Oblivious)
1. Serialization
Wr 2 3 Wr 2 3 Rd 3 Rd
2. Write Penalty
• MASA
Wr Rd
Wr Rd
time
3. Thrashing
Saved
time
29
MASA: Overhead
• DRAM Die Size: Only 0.15% increase
– Subarray Address Latches
– Designated-Bit Latches & Wire
• DRAM Static Energy: Small increase
– 0.56mW for each activated subarray
– But saves dynamic energy
• Controller: Small additional storage
– Keep track of subarray status (< 256B)
– Keep track of new timing constraints
30
D
3. Thrashing
2. Wr-Penalty
Latches
1. Serialization
Cheaper Mechanisms
MASA
SALP-2
D
SALP-1
31
Outline
• Motivation & Key Idea
• Background
• Mechanism
• Related Works
• Results
32
Related Works
• Randomized bank index [Rau ISCA’91, Zhang+ MICRO’00, …]
– Use XOR hashing to generate bank index
– Cannot parallelize bank conflicts
• Rank-subsetting [Ware+ ICCD’06, Zheng+ MICRO’08, Ahn+ CAL’09, …]
– Partition rank and data-bus into multiple subsets
– Increases unloaded DRAM latency
• Cached DRAM [Hidaka+ IEEE Micro’90, Hsu+ ISCA’93, …]
– Add SRAM cache inside of DRAM chip
– Increases DRAM die size (+38.8% for 64kB)
• Hierarchical Bank [Yamauchi+ ARVLSI’97]
– Parallelize accesses to subarrays
– Adds complex logic to subarrays
– Does not utilize multiple local row-buffers
33
Outline
• Motivation & Key Idea
• Background
• Mechanism
• Related Works
• Results
34
Methodology
• DRAM Area/Power
– Micron DDR3 SDRAM System-Power Calculator
– DRAM Area/Power Model [Vogelsang, MICRO’10]
– CACTI-D [Thoziyoor+, ISCA’08]
• Simulator
– CPU: Pin-based, in-house x86 simulator
– Memory: Validated cycle-accurate DDR3 DRAM simulator
• Workloads
– 32 Single-core benchmarks
• SPEC CPU2006, TPC, STREAM, random-access
• Representative 100 million instructions
– 16 Multi-core workloads
• Random mix of single-thread benchmarks
35
Configuration
• System Configuration
– CPU: 5.3GHz, 128 ROB, 8 MSHR
– LLC: 512kB per-core slice
• Memory Configuration
– DDR3-1066
– (default) 1 channel, 1 rank, 8 banks, 8 subarrays-per-bank
– (sensitivity) 1-8 chans, 1-8 ranks, 8-64 banks, 1-128 subarrays
• Mapping & Row-Policy
– (default) Line-interleaved & Closed-row
– (sensitivity) Row-interleaved & Open-row
• DRAM Controller Configuration
– 64-/64-entry read/write queues per-channel
– FR-FCFS, batch scheduling for writes
36
80%
70%
60%
50%
40%
30%
20%
10%
0%
MASA
"Ideal"
17%
20%
IPC Improvement
Single-Core: Instruction Throughput
MASA achieves most of the benefit
of having more banks (“Ideal”)
37
Single-Core: Instruction Throughput
IPC Increase
SALP-1
SALP-2
30%
20%
10%
7%
MASA
"Ideal"
20%
17%
13%
0%
DRAM
Die Area
< 0.15%
0.15% 36.3%
SALP-1, SALP-2, MASA improve
performance at low cost
38
IPC Improvement
Single-Core: Sensitivity to Subarrays
30%
25%
20%
15%
10%
5%
0%
MASA
1
2
4
8
16
32
64 128
Subarrays-per-bank
You do not need many subarrays
for high performance
39
Single-Core: Row-Interleaved, Open-Row
MASA
"Ideal"
IPC Increase
20%
15%
10%
12%
15%
5%
0%
MASA’s performance benefit is
robust to mapping and page-policy
40
Single-Core: Row-Interleaved, Open-Row
1.0
0.8
0.6
0.4
0.2
0.0
-19%
Normalized
Dynamic Energy
1.2
Baseline
MASA
100%
80%
60%
40%
+13%
MASA
Row-Buffer Hit-Rate
Baseline
20%
0%
MASA increases energy-efficiency
41
Other Results/Discussion in Paper
• Multi-core results
• Sensitivity to number of channels & ranks
• DRAM die area overhead of:
– Naively adding more banks
– Naively adding SRAM caches
• Survey of alternative DRAM organizations
– Qualitative comparison
42
Conclusion
• Problem: Requests to same DRAM bank are
serialized
• Our Goal: Parallelize requests to same DRAM
bank at a low cost
• Observation: A bank consists of subarrays
that occassionally share global structures
• MASA: Reduces sharing to enable parallel
access and to utilize multiple row-buffers
• Result: Significantly higher performance and
energy-efficiency at low cost (+0.15% area)
43
A Case for
Subarray-Level Parallelism
(SALP) in DRAM
Yoongu Kim, Vivek Seshadri,
Donghyuk Lee, Jamie Liu, Onur Mutlu
Exposing Subarrays to Controller
• Every DIMM has an SPD (Serial Presence Detect)
(Image: JEDEC)
– 256-byte EEPROM
– Contains information about DIMM and DRAM devices
– Read by BIOS during system-boot
• SPD reserves 100+ bytes for manufacturer and user
– Sufficient for subarray-related information
1. Whether SALP-1, SALP-2, MASA are supported
2. Physical address bit positions for subarray index
3. Values of timing constraints: tRA, tWA
45
Multi-Core: Memory Scheduling
WS Increase
Configuration: 8-16 cores, 2 chan, 2 ranks-per-chan
25%
20%
15%
10%
5%
0%
Baseline
FRFCFS
SALP-1
SALP-2
TCM
FRFCFS
8-core system
MASA
TCM
16-core system
Our mechanisms further improve performance when employed
with application-aware schedulers
We believe it can be even greater with subarray-aware schedulers
46
Number of Subarrays-Per-Bank
• As DRAM chips grow in capacity…
– More rows-per-bank  More subarrays-per-bank
• Not all subarrays may be accessed in parallel
– Faulty rows remapped to spare rows
– If remapping occurs between two subarrays…
• They can no longer be accessed in parallel
• Subarray group
– Restrict remapping: only within a group of subarrays
– Each subarray group can accessed in parallel
– We refer to a subarray group as a “subarray”
• We assume 8 subarrays-per-bank
47
Area & Power Overhead
• Latches: Per-Subarray Row-Address, Designated-Bit
– Storage: 41 bits per subarray
– Area: 0.15% in die area (assuming 8 subarrays-per-bank)
– Power: 72.2uW (negligible)
• Multiple Activated Subarrays
– Power: 0.56mW static power for each additional activated subarray
• Small compared to 48mW baseline static power
• SA-SEL Wire/Command
– Area: One extra wire (negligible)
– Power: SA-SEL consumes 49.6% the power of ACT
• Memory Controller: Tracking the status of subarrays
– Storage: Less than 256 bytes
• Activated? Which wordline is raised? Designated?
48