Lecture 18: Pipelining I
Pipelining I (1)
Fall 2005
Laundry Pipeling Example
° Ann, Brian, Cathy, Dave
each have one load of
clothes to wash, dry,
fold, and put away
A B C D
° Washer takes 30 minutes
° Dryer takes 30 minutes
° “Folder” takes 30 minutes
° “Stasher” takes 30 minutes
to put clothes into drawers
Pipelining I (2)
Fall 2005
Sequential Laundry
6 PM 7
T
a
s
k
O
r
d
e
r
Pipelining I (3)
A
8
9
10
11
12
1
2 AM
30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
Time
B
C
D
• Sequential laundry takes
8 hours for 4 loads
Fall 2005
Pipelined Laundry
6 PM 7
T
a
s
k
8
9
3030 30 30 30 30 30
10
11
12
1
2 AM
Time
A
B
C
O
D
r
d
e • Pipelined
r 3.5 hours
Pipelining I (4)
laundry takes
for 4 loads!
Fall 2005
General Definitions
• Latency: time to completely execute a
certain task
• for example, time to read a sector from
disk is disk access time or disk latency
• Throughput: amount of work that can
be done over a period of time
Pipelining I (5)
Fall 2005
Pipelining Lessons (1/2)
6 PM
T
a
s
k
Pipelining I (6)
8
9
Time
30 30 30 30 30 30 30
A
B
O
r
d
e
r
7
C
D
• Pipelining doesn’t help
latency of single task, it
helps throughput of
entire workload
• Multiple tasks
operating
simultaneously using
different resources
• Potential speedup =
Number pipe stages
• Time to “fill” pipeline
and time to “drain” it
reduces speedup:
2.3X v. 4X in this
example
Fall 2005
Pipelining Lessons (2/2)
• Suppose new
Washer takes 20
6 PM
7
8
9
minutes, new
Time
T
Stasher takes 20
a
30 30 30 30 30 30 30
minutes. How
s A
much faster is
k
pipeline?
B
O
r
d
e
r
Pipelining I (7)
C
D
• Pipeline rate
limited by slowest
pipeline stage
• Unbalanced
lengths of pipe
stages also
reduces speedup
Fall 2005
Steps in Executing MIPS
1) IFetch: Fetch Instruction, Increment PC
2) Decode Instruction, Read Registers
3) Execute:
Mem-ref: Calculate Address
Arith-log: Perform Operation
4) Memory:
Load:
Read Data from Memory
Store:
Write Data to Memory
5) Write Back: Write Data to Register
Pipelining I (8)
Fall 2005
Pipelined Execution Representation
Time
IFtch Dcd Exec Mem WB
IFtch Dcd Exec Mem WB
IFtch Dcd Exec Mem WB
IFtch Dcd Exec Mem WB
IFtch Dcd Exec Mem WB
IFtch Dcd Exec Mem WB
• Every instruction must take same number
of steps, also called pipeline “stages”, so
some will go idle sometimes
Pipelining I (9)
Fall 2005
+4
1. Instruction
Fetch
ALU
Data
memory
rd
rs
rt
registers
PC
instruction
memory
Review: Datapath for MIPS
imm
5. Write
2. Decode/
3. Execute 4. Memory
Back
Register Read
• Use datapath figure to represent pipeline
IFtch Dcd Exec Mem WB
Pipelining I (10)
Reg
ALU
I$
D$
Reg
Fall 2005
Graphical Pipeline Representation
(In Reg, right half highlight read, left half write)
Time (clock cycles)
Reg
Reg
D$
Reg
I$
Reg
D$
Reg
I$
Reg
ALU
D$
Reg
I$
Reg
ALU
I$
D$
ALU
Reg
ALU
Pipelining I (11)
I$
ALU
I
n
s Load
t Add
r.
Store
O
Sub
r
d Or
e
r
D$
Reg
Fall 2005
Example
• Suppose 2 ns for memory access, 2 ns
for ALU operation, and 1 ns for register
file read or write; compute instr rate
• Nonpipelined Execution:
• lw : IF + Read Reg + ALU + Memory + Write
Reg = 2 + 1 + 2 + 2 + 1 = 8 ns
• add: IF + Read Reg + ALU + Write Reg
= 2 + 1 + 2 + 1 = 6 ns
• Pipelined Execution:
• Max(IF,Read Reg,ALU,Memory,Write Reg) =
2 ns
Pipelining I (12)
Fall 2005
Pipeline Hazard: Matching socks in later load
6 PM 7
T
a
s
k
8
9
3030 30 30 30 30 30
A
10
11
12
1
2 AM
Time
bubble
B
C
O
D
r
d E
e
r F
A depends on D; stall since folder tied up
Pipelining I (13)
Fall 2005
Limits to pipelining
• Hazards prevent next instruction from
executing during its designated clock
cycle
• Structural hazards: HW cannot support
this combination of instructions (single
person to fold and put clothes away)
• Control hazards: Pipelining of branches &
other instructions stall the pipeline until
the hazard; “bubbles” in the pipeline
• Data hazards: Instruction depends on
result of prior instruction still in the
pipeline (missing sock)
Pipelining I (14)
Fall 2005
Structural Hazard #1: Single Memory (1/2)
Time (clock cycles)
ALU
I
n
I$
D$
Reg
Reg
s Load
I$
D$
Reg
Reg
t Instr 1
r.
I$
D$
Reg
Reg
Instr 2
O
I$
D$
Reg
Reg
Instr 3
r
I$
D$
Reg
Reg
d Instr 4
e
r
Read same memory twice in same clock cycle
ALU
ALU
ALU
ALU
Pipelining I (15)
Fall 2005
Structural Hazard #1: Single Memory (2/2)
• Solution:
• infeasible and inefficient to create
second memory
• so handle this by having two Level 1
Caches (a temporary smaller [of usually
most recently used] copy of memory)
• have both an L1 Instruction Cache and
an L1 Data Cache
• need more complex hardware to control
when both caches miss
Pipelining I (16)
Fall 2005
Structural Hazard #2: Registers (1/2)
Reg
Reg
D$
Reg
I$
Reg
D$
Reg
I$
Reg
D$
Reg
I$
Reg
ALU
I$
D$
ALU
Reg
ALU
I$
ALU
O Instr 2
r
Instr 3
d
e Instr 4
r
Time (clock cycles)
ALU
I
n
s
t sw
r. Instr 1
D$
Reg
Can’t read and write to registers simultaneously
Pipelining I (17)
Fall 2005
Structural Hazard #2: Registers (2/2)
• Fact: Register access is VERY fast:
takes less than half the time of ALU
stage
• Solution: introduce convention
• always Write to Registers during first half
of each clock cycle
• always Read from Registers during
second half of each clock cycle
• Result: can perform Read and Write
during same clock cycle
Pipelining I (18)
Fall 2005
Things to Remember
• Optimal Pipeline
• Each stage is executing part of an
instruction each clock cycle.
• One instruction finishes during each clock
cycle.
• On average, execute far more quickly.
• What makes this work?
• Similarities between instructions allow us
to use same stages for all instructions
(generally).
• Each stage takes about the same amount
of time as all others: little wasted time.
Pipelining I (19)
Fall 2005