Computer Systems Organization:
Lecture 7
Ankur Srivastava
University of Maryland, College Park
Based on Slides from Mary Jane Irwin ( www.cse.psu.edu/~mji )
www.cse.psu.edu/~cg431
[Adapted from Computer Organization and Design,
Patterson & Hennessy, © 2005, UCB]
ENEE350
Review: MIPS Pipeline Data and Control Paths
PCSrc
ID/EX
EX/MEM
Control
IF/ID
Add
RegWrite
4
PC
Instruction
Memory
Read
Address
Shift
left 2
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
ALUSrc
ALU
Read
Data 2
16
Rt
Rd
ENEE350
Address
Read
Data
Write Data
ALU
cntrl
Sign
Extend
MEM/WB
Branch
32
ALUOp
RegDst
MemWrite MemRead
MemtoReg
Review: One Way to “Fix” a Data Hazard
IM
Reg
DM
Reg
IM
Reg
ALU
O
r
d
e
r
add $1,
ALU
I
n
s
t
r.
Fix data hazard
by waiting –
stall – but
impacts CPI
IM
Reg
stall
stall
sub $4,$1,$5
HOW IS A STALL PHYSICALLY IMPLEMENTED?
ENEE350
ALU
and $6,$7,$1
DM
Reg
DM
Reg
Review: Another Way to “Fix” a Data Hazard
and $6,$7,$1
$8,$1,$1
sw
$4,4($1)
ENEE350
Reg
IM
Reg
IM
Reg
IM
Reg
DM
Reg
DM
DM
Reg
DM
ALU
or
IM
ALU
sub $4,$1,$5
Reg
ALU
IM
ALU
O
r
d
e
r
add $1,
ALU
I
n
s
t
r.
Fix data hazards
by forwarding
results as soon as
they are available
to where they are
Reg
needed
Reg
DM
Reg
Data Forwarding (aka Bypassing)

Take the result from the earliest point that it exists in any of
the pipeline state registers and forward it to the functional
units (e.g., the ALU) that need it that cycle

For ALU functional unit: the inputs can come from any
pipeline register rather than just from ID/EX by

adding multiplexors to the inputs of the ALU

connecting the Rd write data in EX/MEM or MEM/WB to either (or
both) of the EX’s stage Rs and Rt ALU mux inputs

adding the proper control hardware to control the new muxes

Other functional units may need similar forwarding logic
(e.g., the DM)

With forwarding can achieve a CPI of 1 even in the
presence of data dependencies
ENEE350
Forwarding Illustration
IM
sub $4,$1,$5
IM
Reg
IM
Reg
DM
EX/MEM hazard
forwarding
ENEE350
Reg
DM
ALU
and $6,$7,$1
Reg
ALU
O
r
d
e
r
add $1,
ALU
I
n
s
t
r.
Reg
DM
Reg
MEM/WB hazard
forwarding
Datapath with Forwarding Hardware
PCSrc
ID/EX
EX/MEM
Control
IF/ID
Add
Shift
left 2
4
PC
Instruction
Memory
Read
Address
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
Read
Data 2
32
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit
ENEE350
MEM/WB
Branch
Data Forwarding Control Conditions
1.
EX/MEM hazard:
if (EX/MEM.RegWrite
and (EX/MEM.RegisterRd
and (EX/MEM.RegisterRd
ForwardA = 10
if (EX/MEM.RegWrite
and (EX/MEM.RegisterRd
and (EX/MEM.RegisterRd
ForwardB = 10
2.
Forwards the
!= 0) ($zero is not destination)
result from the
= ID/EX.RegisterRs))
previous instr.
to either input
of the ALU
!= 0)
= ID/EX.RegisterRt))
MEM/WB hazard:
if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd
and (MEM/WB.RegisterRd
ForwardA = 01
if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd
and (MEM/WB.RegisterRd
ForwardB = 01
ENEE350
!= 0)
= ID/EX.RegisterRs))
!= 0)
= ID/EX.RegisterRt))
Forwards the
result from the
second
previous instr.
to either input
of the ALU
Yet Another Complication!
I
n
s
t
r.
add $1,$1,$2
add $1,$1,$4
ENEE350
Reg
IM
Reg
IM
Reg
DM
Reg
DM
ALU
O
r
d
e
r
add $1,$1,$3
IM
ALU
Another potential data hazard can occur when there is a
conflict between the result of the WB stage instruction
and the MEM stage instruction – which should be
forwarded?
ALU

Reg
DM
Reg
Yet Another Complication!
I
n
s
t
r.
add $1,$1,$2
add $1,$1,$4
ENEE350
Reg
IM
Reg
IM
Reg
DM
Reg
DM
ALU
O
r
d
e
r
add $1,$1,$3
IM
ALU
Another potential data hazard can occur when there is a
conflict between the result of the WB stage instruction
and the MEM stage instruction – which should be
forwarded?
ALU

Reg
DM
Reg
Corrected Data Forwarding Control Conditions
2.
MEM/WB hazard:
if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd != 0)
and (EX/MEM.RegisterRd != ID/EX.RegisterRs)
and (MEM/WB.RegisterRd = ID/EX.RegisterRs))
ForwardA = 01
if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd != 0)
and (EX/MEM.RegisterRd != ID/EX.RegisterRt)
and (MEM/WB.RegisterRd = ID/EX.RegisterRt))
ForwardB = 01
ENEE350
Datapath with Forwarding Hardware
PCSrc
ID/EX
EX/MEM
Control
IF/ID
Add
Shift
left 2
4
PC
Instruction
Memory
Read
Address
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
Read
Data 2
32
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit
ENEE350
MEM/WB
Branch
Datapath with Forwarding Hardware
PCSrc
ID/EX
EX/MEM
Control
IF/ID
Add
Shift
left 2
4
PC
Instruction
Memory
Read
Address
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
MEM/WB
Branch
ALU
Read
Data 2
Address
Read
Data
Write Data
ALU
cntrl
32
EX/MEM.RegisterRd
ID/EX.RegisterRt
ID/EX.RegisterRs
ENEE350
Forward
Unit
MEM/WB.RegisterRd
Memory-to-Memory Copies

For loads immediately followed by stores (memory-tomemory copies) can avoid a stall by adding forwarding
hardware from the MEM/WB register to the data memory
input.

sw $1,4($3)
ENEE350
IM
Reg
IM
Reg
DM
ALU
O
r
d
e
r
lw $1,4($2)
ALU
I
n
s
t
r.
Would need to add a Forward Unit and a mux to the memory
access stage
Reg
DM
Reg
Forwarding with Load-use Data Hazards
or
$8,$1,$9
xor $4,$1,$5
IM
Reg
IM
Reg
IM
Reg
IM
Reg
DM
Reg
DM
Reg
DM
Reg
DM
Reg
DM
ALU
ENEE350
Reg
ALU
and $6,$1,$7
IM
ALU
sub $4,$1,$5
Reg
ALU
$1,4($2)IM
ALU
O
r
d
e
r
lw
ALU
I
n
s
t
r.
Reg
DM
Forwarding with Load-use Data Hazards
or $6,$1,$7
$8,$1,$9
and
or
xor $8,$1,$9
$4,$1,$5
ENEE350
IM
Reg
IM
Reg
IM
Reg
IM
Reg
DM
Reg
DM
Reg
DM
Reg
DM
Reg
DM
ALU
xor $4,$1,$5
Reg
ALU
and $4,$1,$5
$6,$1,$7
sub
IM
ALU
stall $4,$1,$5
sub
Reg
ALU
$1,4($2)IM
ALU
O
r
d
e
r
lw
ALU
I
n
s
t
r.
Reg
DM
Load-use Hazard Detection Unit

Need a Hazard detection Unit in the ID stage that inserts
a stall between the load and its use
ID Hazard Detection
if (ID/EX.MemRead
and ((ID/EX.RegisterRt = IF/ID.RegisterRs)
or (ID/EX.RegisterRt = IF/ID.RegisterRt)))
stall the pipeline
2.

The first line tests to see if the instruction now in the EX
stage is a lw; the next two lines check to see if the
destination register of the lw matches either source
register of the instruction in the ID stage (the load-use
instruction)

After this one cycle stall, the forwarding logic can handle
the remaining data hazards
ENEE350
Stall Hardware

Along with the Hazard Unit, we have to implement the stall

Prevent the instructions in the IF and ID stages from
progressing down the pipeline – done by preventing the
PC register and the IF/ID pipeline register from changing


Insert a “bubble” between the lw instruction (in the EX
stage) and the load-use instruction (in the ID stage) (i.e.,
insert a noop in the execution stream)


Hazard detection Unit controls the writing of the PC (PC.write)
and IF/ID (IF/ID.write) registers
Set the control bits in the EX, MEM, and WB control fields of the
ID/EX pipeline register to 0 (noop). The Hazard Unit controls the
mux that chooses between the real control values and the 0’s.
Let the lw instruction and the instructions after it in the
pipeline (before it in the code) proceed normally down the
pipeline
ENEE350
Adding the Hazard Hardware
PCSrc
Hazard
Unit
ID/EX
EX/MEM
0
1
IF/ID
Control
Add
Shift
left 2
4
PC
Instruction
Memory
Read
Address
Add
Read Addr 1
Data
Memory
Register Read
Read Addr 2Data 1
File
Write Addr
Write Data
16
Sign
Extend
Read
Data 2
32
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit
ENEE350
MEM/WB
Branch
Adding the Hazard Hardware
PCSrc
Hazard
Unit
ID/EX
ID/EX.MemRead
EX/MEM
0
1
IF/ID
Control 0
Add
Shift
left 2
4
PC
Instruction
Memory
Read
Address
Read Addr 1
Read Addr 2Data 1
File
Write Addr
16
Sign
Extend
Read
Data 2
32
ID/EX.RegisterRt
MEM/WB
Branch
Data
Memory
Register Read
Write Data
ENEE350
Add
ALU
Address
Read
Data
Write Data
ALU
cntrl
Forward
Unit