CS 152 Computer Architecture and
Engineering
Lecture 4 - Pipelining
Krste Asanovic
Electrical Engineering and Computer Sciences
University of California at Berkeley
http://www.eecs.berkeley.edu/~krste
http://inst.eecs.berkeley.edu/~cs152
Last time in Lecture 3
• Microcoding became less attractive as gap between
RAM and ROM speeds reduced
• Complex instruction sets difficult to pipeline, so
difficult to increase performance as gate count grew
• Iron-law explains architecture design space
– Trade instructions/program, cycles/instruction, and time/cycle
• Load-Store RISC ISAs designed for efficient
pipelined implementations
– Very similar to vertical microcode
– Inspired by earlier Cray machines
2/3/2009
CS152-Spring’09
2
“Iron Law” of Processor Performance
Time = Instructions
Cycles
Time
Program
Program * Instruction * Cycle
– Instructions per program depends on source code,
compiler technology, and ISA
– Cycles per instructions (CPI) depends upon the
ISA and the microarchitecture
– Time per cycle depends upon the
microarchitecture and the base technology
This lecture
2/3/2009
Microarchitecture
Microcoded
Single-cycle unpipelined
Pipelined
CS152-Spring’09
CPI
>1
1
1
cycle time
short
long
short
3
An Ideal Pipeline
stage
1
stage
2
stage
3
stage
4
• All objects go through the same stages
• No sharing of resources between any two stages
• Propagation delay through all pipeline stages is equal
• The scheduling of an object entering the pipeline
is not affected by the objects in other stages
These conditions generally hold for industrial
assembly lines.
But can an instruction pipeline satisfy the last
condition?
2/3/2009
CS152-Spring’09
4
Pipelined MIPS
To pipeline MIPS:
• First build MIPS without pipelining with
CPI=1
• Next, add pipeline registers to reduce
cycle time while maintaining CPI=1
2/3/2009
CS152-Spring’09
5
Pipelined Datapath
0x4
Add
PC
addr
rdata
Inst.
Memory
IR
we
rs1
rs2
rd1
ws
wd rd2
GPRs
Imm
Ext
ALU
we
addr
rdata
Data
Memory
wdata
write
fetch
decode & Reg-fetch
execute
memory
-back
phase
phase
phase
phase
phase
Clock period can be reduced by dividing the execution of an
instruction into multiple cycles
tC > max {tIM, tRF, tALU, tDM, tRW} ( = tDM probably)
However, CPI will increase unless instructions are pipelined
2/3/2009
CS152-Spring’09
6
Technology Assumptions
• A small amount of very fast memory (caches)
backed up by a large, slower memory
• Fast ALU (at least for integers)
• Multiported Register files (slower!)
Thus, the following timing assumption is reasonable
tIM tRF tALU tDM  tRW
A 5-stage pipelined Harvard architecture will be
the focus of our detailed design
2/3/2009
CS152-Spring’09
7
5-Stage Pipelined Execution
0x4
Add
PC
addr
rdata
we
rs1
rs2
rd1
ws
wd rd2
GPRs
IR
Inst.
Memory
I-Fetch
(IF)
Data
Memory
Imm
Ext
wdata
Write
Decode, Reg. Fetch Execute
Memory
(ID)
(EX)
(MA)
Back
t0 t1 t2 t3 t4 t5 t6 t7 . . . . (WB)
time
instruction1
instruction2
instruction3
instruction4
instruction5
2/3/2009
ALU
we
addr
rdata
IF1
ID1 EX1 MA1 WB1
IF2 ID2 EX2 MA2 WB2
IF3 ID3 EX3 MA3 WB3
IF4 ID4 EX4 MA4 WB4
IF5 ID5 EX5 MA5 WB5
CS152-Spring’09
8
5-Stage Pipelined Execution
Resource Usage Diagram
0x4
Add
PC
addr
rdata
we
rs1
rs2
rd1
ws
wd rd2
GPRs
IR
Inst.
Memory
Resources
I-Fetch
(IF)
2/3/2009
we
addr
rdata
ALU
Data
Memory
Imm
Ext
wdata
t5
Write
Memory
(MA)
Back
t6 t7 . . . . (WB)
I5
I4
I3
I2
I5
I4
I3
Decode, Reg. Fetch Execute
(ID)
(EX)
time
IF
ID
EX
MA
WB
t0
I1
t1
I2
I1
t2
I3
I2
I1
t3
I4
I3
I2
I1
t4
I5
I4
I3
I2
I1
CS152-Spring’09
I5
I4
I5
9
Pipelined Execution:
ALU Instructions
0x4
IR
Add
PC
addr
IR
IR
31
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
A
ALU
GPRs
Y
B
we
addr
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
MD2
Not quite correct!
We need an Instruction Reg (IR) for each stage
2/3/2009
CS152-Spring’09
10
Pipelined MIPS Datapath
without jumps
F
D
E
M
IR
W
IR
IR
31
0x4
Add
RegDst
RegWrite
PC
addr
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
OpSel
MemWrite
A
ALU
GPRs
Y
we
addr
rdata
B
Data
Memory
wdata
Imm
Ext
R
wdata
MD1
ExtSel
WBSrc
MD2
BSrc
Control Points Need to
Be Connected
2/3/2009
CS152-Spring’09
11
How Instructions can Interact with
each other in a pipeline
• An instruction in the pipeline may need a
resource being used by another instruction
in the pipeline  structural hazard
• An instruction may depend on something
produced by an earlier instruction
– Dependence may be for a data value
 data hazard
– Dependence may be for the next instruction’s
address
 control hazard (branches, exceptions)
2/3/2009
CS152-Spring’09
12
Data Hazards
r1 …
r4  r1 …
0x4
IR
Add
PC
addr
31
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
A
ALU
GPRs
Y
B
we
addr
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
...
r1 r0 + 10
r4 r1 + 17
...
2/3/2009
IR
IR
MD2
r1 is stale. Oops!
CS152-Spring’09
13
CS152 Administrivia
• PS 1 due Tuesday Feb 10 in class
• Section covering PS 1 on Wednesday Feb 11
– Room/time TBD
• First Quiz on Thursday Feb 12
– In class, closed-book, no computers or calculators
– Covers lectures 1-5 (this week’s lectures)
• Lecture 7, Tuesday Feb 17 in 320 Soda
• Lecture 8, Thursday Feb 19 back in 306 Soda
• See website for full schedule
2/3/2009
CS152-Spring’09
14
Resolving Data Hazards (1)
Strategy 1:
Wait for the result to be available by freezing
earlier pipeline stages  interlocks
2/3/2009
CS152-Spring’09
15
Feedback to Resolve Hazards
FB1
FB2
stage
1
•
FB3
stage
2
FB4
stage
3
stage
4
Later stages provide dependence information to earlier stages
which can stall (or kill) instructions
• Controlling a pipeline in this manner works provided
the instruction at stage i+1 can complete without
any interference from instructions in stages 1 to i
(otherwise deadlocks may occur)
2/3/2009
CS152-Spring’09
16
Interlocks to resolve Data Hazards
Stall Condition
0x4
nop
Add
PC
addr
IR
IR
31
inst IR
Inst
Memory
...
r1 r0 + 10
r4 r1 + 17
...
2/3/2009
IR
we
rs1
rs2
rd1
ws
wd rd2
GPRs
A
ALU
Y
B
we
addr
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
CS152-Spring’09
MD2
17
Stalled Stages and Pipeline Bubbles
time
t0 t1 t2 t3 t4 t5
(I1) r1 (r0) + 10 IF1 ID1 EX1 MA1 WB1
(I2) r4 (r1) + 17
IF2 ID2 ID2 ID2 ID2
(I3)
IF3 IF3 IF3 IF3
(I4)
stalled stages
(I5)
Resource
Usage
IF
ID
EX
MA
WB
time
t0 t1
I1
I2
I1
t2
I3
I2
I1
t3
I3
I2
nop
I1
t4
I3
I2
nop
nop
I1
t5
I3
I2
nop
nop
nop
t6
CS152-Spring’09
....
EX2 MA2 WB2
ID3 EX3 MA3 WB3
IF4 ID4 EX4 MA4 WB4
IF5 ID5 EX5 MA5 WB5
t6
I4
I3
I2
nop
nop
nop 
2/3/2009
t7
t7
I5
I4
I3
I2
nop
....
I5
I4
I3
I2
I5
I4
I3
I5
I4
pipeline bubble
18
I5
Interlock Control Logic
stall
ws
Cstall
rs
rt
?
0x4
nop
Add
PC
addr
IR
IR
IR
31
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
GPRs
A
ALU
Y
B
we
addr
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
MD2
Compare the source registers of the instruction in the decode
stage with the destination register of the uncommitted
instructions.
2/3/2009
CS152-Spring’09
19
Interlock Control Logic
ignoring jumps & branches
stall
Cstall
rs
rt
ws
we
?
re1
re2
nop
Add
PC
addr
Cdest
Cdest
Cre
0x4
ws
ws we
we
IR
IR
IR
31
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
GPRs
Cdest
A
ALU
Y
B
we
addr
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
MD2
Should we always stall if the rs field matches some rd?
not every instruction writes a register we
not every instruction reads a register re
2/3/2009
CS152-Spring’09
20
Source & Destination Registers
R-type:
op
rs
rt
I-type:
op
rs
rt
J-type:
op
ALU
ALUi
LW
SW
BZ
J
JAL
JR
JALR
2/3/2009
rd
func
immediate16
immediate26
rd  (rs) func (rt)
rt  (rs) op imm
rt M [(rs) + imm]
M [(rs) + imm]  (rt)
cond (rs)
true: PC  (PC) + imm
false: PC  (PC) + 4
PC  (PC) + imm
r31  (PC), PC  (PC) + imm
PC  (rs)
r31  (PC), PC  (rs)
CS152-Spring’09
source(s) destination
rs, rt
rd
rs
rt
rs
rt
rs, rt
rs
rs
rs
rs
31
31
21
Deriving the Stall Signal
Cdest
ws = Case opcode
ALU
rd
ALUi, LW
rt
JAL, JALR
R31
Cre
we = Case opcode
ALU, ALUi, LW (ws  0)
JAL, JALR
on
...
off
Cstall
2/3/2009
re1 = Case opcode
ALU, ALUi,
LW, SW, BZ,
JR, JALR
on
J, JAL
off
re2 = Case opcode
on
ALU, SW
off
...
stall = ((rsD =wsE).weE +
(rsD =wsM).weM +
(rsD =wsW).weW) . re1D
((rtD =wsE).weE +
(rtD =wsM).weM +
(rtD =wsW).weW) . re2D
CS152-Spring’09
+
22
Hazards due to Loads & Stores
Stall Condition
What if
(r1)+7 = (r3)+5 ?
0x4
nop
Add
PC
addr
IR
31
inst IR
Inst
Memory
we
rs1
rs2
rd1
ws
wd rd2
GPRs
A
ALU
Y
B
...
M[(r1)+7]  (r2)
r4  M[(r3)+5]
...
we
addr
rdata
Data
Memory
Imm
Ext
2/3/2009
IR
IR
R
wdata
wdata
MD1
MD2
Is there any possible data hazard
in this instruction sequence?
CS152-Spring’09
23
Load & Store Hazards
...
M[(r1)+7]  (r2)
r4  M[(r3)+5]
...
(r1)+7 = (r3)+5  data hazard
However, the hazard is avoided because our
memory system completes writes in one cycle !
Load/Store hazards are sometimes resolved in
the pipeline and sometimes in the memory
system itself.
More on this later in the course.
2/3/2009
CS152-Spring’09
24
Resolving Data Hazards (2)
Strategy 2:
Route data as soon as possible after it is
calculated to the earlier pipeline stage  bypass
2/3/2009
CS152-Spring’09
25
Bypassing
time
(I1) r1 r0 + 10
(I2) r4 r1 + 17
(I3)
(I4)
(I5)
t0
IF1
t1 t2 t3 t4 t5
ID1 EX1 MA1 WB1
IF2 ID2 ID2 ID2 ID2
IF3 IF3 IF3 IF3
stalled stages
t6
t7
....
EX2 MA2 WB2
ID3 EX3 MA3
IF4 ID4 EX4
IF5 ID5
Each stall or kill introduces a bubble in the pipeline
CPI > 1
A new datapath, i.e., a bypass, can get the data from
the output of the ALU to its input
time
(I1) r1 r0 + 10
(I2) r4  r1 + 17
(I3)
(I4)
(I5)
2/3/2009
t0
t1
IF1
t2 t3
ID1 EX1
IF2 ID2
IF3
CS152-Spring’09
t4
MA1
EX2
ID3
IF4
t5
WB1
MA2
EX3
ID4
IF5
t6
t7
....
WB2
MA3 WB3
EX4 MA4 WB4
ID5 EX5 MA5 WB5
26
Adding a Bypass
stall
r4  r1...
0x4
r1 ...
nop
Add
IR
E
IR
M
IR
31
ASrc
PC
addr
inst IR
Inst
Memory
D
we
rs1
rs2
rd1
ws
wd rd2
GPRs
A
ALU
Y
B
rdata
Data
Memory
Imm
Ext
R
wdata
wdata
MD1
(I1)
(I2)
we
addr
MD2
When does this bypass help?
...
r1 r0 + 10
r1 M[r0 + 10]
JAL 500
r4 r1 + 17
r4 r1 + 17
r4 r31 + 17
yes
no
no
2/3/2009
CS152-Spring’09
27
W
The Bypass Signal
Deriving it from the Stall Signal
stall = ( ((rsD =wsE).weE + (rsD =wsM).weM + (rsD =wsW).weW).re1D
+((rtD =wsE).weE + (rtD =wsM).weM + (rtD =wsW).weW).re2D )
ws = Case opcode
ALU
rd
ALUi, LW
rt
JAL, JALR R31
we = Case opcode
ALU, ALUi, LW (ws  0)
JAL, JALR
on
...
off
ASrc = (rsD=wsE).weE.re1D
Is this correct?
No because only ALU and ALUi instructions can benefit
from this bypass
Split weE into two components: we-bypass, we-stall
2/3/2009
CS152-Spring’09
28
Bypass and Stall Signals
Split weE into two components: we-bypass, we-stall
we-bypassE = Case opcodeE
ALU, ALUi  (ws  0)
...
off
we-stallE = Case opcodeE
LW
 (ws  0)
JAL, JALR on
...
off
ASrc
= (rsD =wsE).we-bypassE . re1D
stall
= ((rsD =wsE).we-stallE +
(rsD=wsM).weM + (rsD=wsW).weW). re1D
+((rtD = wsE).weE + (rtD = wsM).weM + (rtD = wsW).weW). re2D
2/3/2009
CS152-Spring’09
29
Fully Bypassed Datapath
PC for JAL, ...
stall
0x4
nop
Add
PC
addr
ASrc
inst IR
Inst
Memory
D
Is there still
a need for the
stall signal ?
2/3/2009
IR
M
IR
31
we
rs1
rs2
rd1
ws
wd rd2
A
ALU
GPRs
Imm
Ext
IR
E
Y
B
BSrc
we
addr
rdata
Data
Memory
R
wdata
wdata
MD1
MD2
stall = (rsD=wsE). (opcodeE=LWE).(wsE0 ).re1D
+ (rtD=wsE). (opcodeE=LWE).(wsE0 ).re2D
CS152-Spring’09
30
W
Resolving Data Hazards (3)
Strategy 3:
Speculate on the dependence. Two cases:
Guessed correctly  do nothing
Guessed incorrectly  kill and restart
2/3/2009
CS152-Spring’09
31
Next Time: Control Hazards
• Branches/Jumps
• Exceptions/Interrupts
2/3/2009
CS152-Spring’09
32
Acknowledgements
• These slides contain material developed and
copyright by:
–
–
–
–
–
–
Arvind (MIT)
Krste Asanovic (MIT/UCB)
Joel Emer (Intel/MIT)
James Hoe (CMU)
John Kubiatowicz (UCB)
David Patterson (UCB)
• MIT material derived from course 6.823
• UCB material derived from course CS252
2/3/2009
CS152-Spring’09
33