CS 152 Computer Architecture and
Engineering
Lecture 12 - Advanced Out-of-Order
Superscalars
Krste Asanovic
Electrical Engineering and Computer Sciences
University of California at Berkeley
http://www.eecs.berkeley.edu/~krste
http://inst.eecs.berkeley.edu/~cs152
March 1, 2012
CS152, Spring 2012
Last time in Lecture 11
• Register renaming removes WAR, WAW hazards
• In-order fetch/decode, out-of-order execute, in-order
commit gives high performance and precise exceptions
• Need to rapidly recover on branch mispredictions
March 1, 2012
CS152, Spring 2012
2
“Data-in-ROB” Design
(HP PA8000, Pentium Pro, Core2Duo, Nehalem)
Register File
holds only
committed state
Ins# use exec
op
p1
src1
p2
src2
pd dest
data
Reorder
buffer
Load
Unit
FU
FU
FU
Store
Unit
t1
t2
.
.
tn
Commit
< t, result >
• On dispatch into ROB, ready sources can be in regfile or in ROB
dest (copied into src1/src2 if ready before dispatch)
• On completion, write to dest field and broadcast to src fields.
• On issue, read from ROB src fields
March 1, 2012
CS152, Spring 2012
3
Data Movement in Data-in-ROB Design
Read operands
during decode
Architectural Register Read results,
File
write to ARF at
commit
Write sources
after decode
Src Operands
ROB
Result
Data
Read
operands at
issue
Bypass newer
values at
decode
Write results
at completion
Functional Units
March 1, 2012
CS152, Spring 2012
4
Unified Physical Register File
(MIPS R10K, Alpha 21264, Intel Pentium 4 & Sandy Bridge)
• Rename all architectural registers into a single physical
register file during decode, no register values read
• Functional units read and write from single unified register
file holding committed and temporary registers in execute
• Commit only updates mapping of architectural register to
physical register, no data movement
Decode Stage
Register
Mapping
Committed
Register
Mapping
Unified Physical
Register File
Read operands at issue
Write results at completion
Functional Units
March 1, 2012
CS152, Spring 2012
5
Pipeline Design with Physical Regfile
Branch
Resolution
kill
Branch
Prediction
PC
Fetch
kill
kill
Decode &
Rename
kill
Out-of-Order
Reorder Buffer
In-Order
Commit
In-Order
Physical Reg. File
Branch
ALU MEM
Unit
Store
Buffer
D$
Execute
March 1, 2012
CS152, Spring 2012
6
Lifetime of Physical Registers
• Physical regfile holds committed and speculative values
• Physical registers decoupled from ROB entries (no data in ROB)
ld x1, (x3)
addi x3, x1, #4
sub x6, x7, x9
add x3, x3, x6
ld x6, (x1)
add x6, x6, x3
sd x6, (x1)
ld x6, (x11)
Rename
ld P1, (Px)
addi P2, P1, #4
sub P3, Py, Pz
add P4, P2, P3
ld P5, (P1)
add P6, P5, P4
sd P6, (P1)
ld P7, (Pw)
When can we reuse a physical register?
When next write of same architectural register commits
March 1, 2012
CS152, Spring 2012
7
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8
P7
P5
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<x1>
p
p
p
p
p2
Rd
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex op
March 1, 2012
p1
PR1
PR2
LPRd
CS152, Spring 2012
PRd
(LPRd requires
third read port
on Rename
Table for each
instruction)
8
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7
P5
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<x1>
p
p
p
p
p2
Rd
x1
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex op
x
ld
March 1, 2012
p1
p
PR1
P7
PR2
LPRd
P8
CS152, Spring 2012
PRd
P0
9
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1
P5
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<R1>
p
p
p
p
p2
Rd
x1
x3
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex op p1
x
ld
p
x
addi
March 1, 2012
PR1
P7
P0
PR2
LPRd
P8
P7
CS152, Spring 2012
PRd
P0
P1
10
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1
P5 P3
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<R1>
p
p
p
p
p2
PR2
p
P5
Rd
x1
x3
x6
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex op p1
x
ld
p
x
addi
x
sub p
March 1, 2012
PR1
P7
P0
P6
LPRd
P8
P7
P5
CS152, Spring 2012
PRd
P0
P1
P3
11
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1 P2
P5 P3
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<x1>
p
p
p
p
p2
PR2
Rd
p
P5
P3
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex
x
x
x
x
op
p1
ld
p
addi
sub p
add
March 1, 2012
PR1
P7
P0
P6
P1
x1
x3
x6
x3
LPRd
P8
P7
P5
P1
CS152, Spring 2012
PRd
P0
P1
P3
P2
12
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1 P2
P5 P3 P4
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x6>
<x7>
<x3>
<x1>
p
p
p
p
p2
PR2
p
P5
P3
Rd
x1
x3
x6
x3
x6
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
Pn
ROB
use ex op p1
x
ld
p
x
addi
x
sub p
x
add
x
ld
March 1, 2012
PR1
P7
P0
P6
P1
P0
LPRd
P8
P7
P5
P1
P3
CS152, Spring 2012
PRd
P0
P1
P3
P2
P4
13
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1 P2
P5 P3 P4
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
<x1>
p
<x6>
<x7>
<x3>
<x1>
p
p
p
p
p2
PR2
Rd
p
P5
P3
Free List
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
P8
Pn
ROB
use ex
x
x
x
x
x
x
op
p1
ld
p
addi p
sub p
add
ld
p
March 1, 2012
PR1
P7
P0
P6
P1
P0
x1
x3
x6
x3
x6
LPRd
P8
P7
P5
P1
P3
CS152, Spring 2012
PRd
P0
P1
P3
P2
P4
Execute &
Commit
14
Physical Register Management
x0
x1
x2
x3
x4
x5
x6
x7
Rename
Table
P8 P0
P7 P1 P2
P5 P3 P4
P6
Physical Regs
P0
P1
P2
P3
P4
P5
P6
P7
P8
Free List
<x1>
<x3>
p
p
<x6>
<x7>
<x3>
p
p
p
p2
PR2
Rd
p
P5
P3
P0
P1
P3
P2
P4
ld x1, 0(x3)
addi x3, x1, #4
sub x6, x7, x6
add x3, x3, x6
ld x6, 0(x1)
P8
P7
Pn
ROB
use ex
x
x
x
x
x
x
x
op
p1
ld p
addi p
sub p
add p
ld
p
March 1, 2012
PR1
P7
P0
P6
P1
P0
x1
x3
x6
x3
x6
LPRd
P8
P7
P5
P1
P3
CS152, Spring 2012
PRd
P0
P1
P3
P2
P4
Execute &
Commit
15
Separate Pending Instruction
Window from ROB
The instruction window holds
instructions that have been
decoded and renamed but not
issued into execution. Has
register tags and presence
bits, and pointer to ROB entry.
use ex
op
Ptr2
next to commit Done?
p1 PR1 p2
Rd
PR2
LPRd
PC
PRd ROB#
Except?
Reorder buffer used to hold
exception information for
commit.
Ptr1
next available
ROB is usually several times larger than instruction
window – why?
March 1, 2012
CS152, Spring 2012
16
Reorder Buffer Holds Active Instructions
(Decoded but not Committed)
… (Older instructions)
ld x1, (x3)
add x3, x1, x2
sub x6, x7, x9
add x3, x3, x6
ld x6, (x1)
add x6, x6, x3
sd x6, (x1)
ld x6, (x1)
… (Newer instructions)
Commit
Execute
Fetch
Cycle t
March 1, 2012
…
ld x1, (x3)
add x3, x1,
sub x6, x7,
add x3, x3,
ld x6, (x1)
add x6, x6,
sd x6, (x1)
ld x6, (x1)
…
x2
x9
x6
x3
Cycle t + 1
CS152, Spring 2012
17
Superscalar Register Renaming
• During decode, instructions allocated new physical destination register
• Source operands renamed to physical register with newest value
• Execution unit only sees physical register numbers
Update
Mapping
Op Dest Src1 Src2
Write
Ports
Inst 1
Op Dest Src1 Src2 Inst 2
Read Addresses
Register
Free List
Rename Table
Read Data
Op PDest PSrc1 PSrc2
Op PDest PSrc1 PSrc2
Does this work?
March 1, 2012
CS152, Spring 2012
18
Superscalar Register Renaming
Update
Mapping
Write
Ports
Inst 1 Op Dest Src1 Src2
Read Addresses
=?
Rename Table
Read Data
Must check for
RAW hazards
between
instructions
issuing in same
cycle. Can be
done in parallel
with rename
Op PDest PSrc1 PSrc2
lookup.
Inst 2
Op Dest Src1 Src2
=?
Register
Free List
Op PDest PSrc1 PSrc2
MIPS R10K renames 4 serially-RAW-dependent insts/cycle
March 1, 2012
CS152, Spring 2012
19
CS152 Administrivia
• Quiz 2, Tuesday March 6
– Covers lectures 6-9, PS 2, Lab 2, readings
• Complete Undergrad EECS town hall survey at:
– https://www.surveymonkey.com/s/HWGPSRT
– Town hall overlaps class on March 15 (1:30-3:30) 
March 1, 2012
CS152, Spring 2012
20
Memory Dependencies
sd x1, (x2)
ld x3, (x4)
When can we execute the load?
March 1, 2012
CS152, Spring 2012
21
In-Order Memory Queue
• Execute all loads and stores in program order
=> Load and store cannot leave ROB for execution until
all previous loads and stores have completed
execution
• Can still execute loads and stores speculatively, and
out-of-order with respect to other instructions
• Need a structure to handle memory ordering…
March 1, 2012
CS152, Spring 2012
22
Conservative O-o-O Load Execution
sd x1, (x2)
ld x3, (x4)
• Split execution of store instruction into two phases: address
calculation and data write
• Can execute load before store, if addresses known and x4 != x2
• Each load address compared with addresses of all previous
uncommitted stores
– can use partial conservative check i.e., bottom 12 bits of address, to
save hardware
• Don’t execute load if any previous store address not known
(MIPS R10K, 16-entry address queue)
March 1, 2012
CS152, Spring 2012
23
Address Speculation
sd x1, (x2)
ld x3, (x4)
• Guess that x4 != x2
• Execute load before store address known
• Need to hold all completed but uncommitted
load/store addresses in program order
• If subsequently find x4==x2, squash load and all
following instructions
=> Large penalty for inaccurate address speculation
March 1, 2012
CS152, Spring 2012
24
Memory Dependence Prediction
(Alpha 21264)
sd x1, (x2)
ld x3, (x4)
• Guess that x4 != x2 and execute load before store
• If later find x4==x2, squash load and all following
instructions, but mark load instruction as store-wait
• Subsequent executions of the same load instruction
will wait for all previous stores to complete
• Periodically clear store-wait bits
March 1, 2012
CS152, Spring 2012
25
Speculative Loads / Stores
Just like register updates, stores should not modify
the memory until after the instruction is committed
- A speculative store buffer is a structure introduced to hold
speculative store data.
March 1, 2012
CS152, Spring 2012
26
Speculative Store Buffer
Speculative
Store
Buffer
V
V
V
V
V
V
S
S
S
S
S
S
Load Address
Tag
Tag
Tag
Tag
Tag
Tag
Data
Data
Data
Data
Data
Data
L1 Data
Cache
Tags
Data
Store Commit Path
Load Data
• On store execute:
– mark entry valid and speculative, and save data and tag of instruction.
• On store commit:
– clear speculative bit and eventually move data to cache
• On store abort:
– clear valid bit
March 1, 2012
CS152, Spring 2012
27
Speculative Store Buffer
Speculative
Store
Buffer
V
V
V
V
V
V
S
S
S
S
S
S
Tag
Tag
Tag
Tag
Tag
Tag
Load Address
Data
Data
Data
Data
Data
Data
L1 Data
Cache
Tags
Store Commit Path
Data
Load Data
• If data in both store buffer and cache, which should we use?
Speculative store buffer
• If same address in store buffer twice, which should we use?
Youngest store older than load
March 1, 2012
CS152, Spring 2012
28
Datapath: Branch Prediction
and Speculative Execution
Branch
Prediction
kill
kill
Branch
Resolution
kill
PC
Fetch
Decode &
Rename
kill
Reorder Buffer
Commit
Reg. File
March 1, 2012
Branch
ALU MEM
Unit
Execute
CS152, Spring 2012
Store
Buffer
D$
29
Instruction Flow in Unified Physical
Register File Pipeline
• Fetch
– Get instruction bits from current guess at PC, place in fetch buffer
– Update PC using sequential address or branch predictor (BTB)
• Decode/Rename
– Take instruction from fetch buffer
– Allocate resources to execute instruction:
» Destination physical register, if instruction writes a register
» Entry in reorder buffer to provide in-order commit
» Entry in issue window to wait for execution
» Entry in memory buffer, if load or store
– Decode will stall if resources not available
– Rename source and destination registers
– Check source registers for readiness
– Insert instruction into issue window+reorder buffer+memory buffer
March 1, 2012
CS152, Spring 2012
30
Memory Instructions
• Split store instruction into two pieces during decode:
– Address calculation, store-address
– Data movement, store-data
• Allocate space in program order in memory buffers during
decode
• Store instructions:
–
–
–
–
Store-address calculates address and places in store buffer
Store-data copies store value into store buffer
Store-address and store-data execute independently out of issue window
Stores only commit to data cache at commit point
• Load instructions:
– Load address calculation executes from window
– Load with completed effective address searches memory buffer
– Load instruction may have to wait in memory buffer for earlier store ops
to resolve
March 1, 2012
CS152, Spring 2012
31
Issue Stage
• Writebacks from completion phase “wakeup” some
instructions by causing their source operands to
become ready in issue window
– In more speculative machines, might wake up waiting loads in
memory buffer
• Need to “select” some instructions for issue
– Arbiter picks a subset of ready instructions for execution
– Example policies: random, lower-first, oldest-first, critical-first
• Instructions read out from issue window and sent to
execution
March 1, 2012
CS152, Spring 2012
32
Execute Stage
• Read operands from physical register file and/or
bypass network from other functional units
• Execute on functional unit
• Write result value to physical register file (or store
buffer if store)
• Produce exception status, write to reorder buffer
• Free slot in instruction window
March 1, 2012
CS152, Spring 2012
33
Commit Stage
• Read completed instructions in-order from
reorder buffer
– (may need to wait for next oldest instruction to complete)
• If exception raised
– flush pipeline, jump to exception handler
• Otherwise, release resources:
– Free physical register used by last writer to same architectural
register
– Free reorder buffer slot
– Free memory reorder buffer slot
March 1, 2012
CS152, Spring 2012
34
Acknowledgements
• These slides contain material developed and
copyright by:
–
–
–
–
–
–
Arvind (MIT)
Krste Asanovic (MIT/UCB)
Joel Emer (Intel/MIT)
James Hoe (CMU)
John Kubiatowicz (UCB)
David Patterson (UCB)
• MIT material derived from course 6.823
• UCB material derived from course CS252
March 1, 2012
CS152, Spring 2012
35