Trace Caches
Michele Co
CS 451
Page 1
Motivation

High performance superscalar processors
 High
instruction throughput
 Exploit ILP
–Wider dispatch and issue paths
 Execution units designed for high parallelism
–Many functional units
–Large issue buffers
–Many physical registers

Fetch bandwidth becomes performance bottleneck
Page 2
Fetch Performance Limiters

Cache hit rate

Branch prediction accuracy

Branch throughput
 Need
to predict more than one branch per cycle

Non-contiguous instruction alignment

Fetch unit latency
Page 3
Problems with Traditional Instruction Cache

Contain instructions in compiled order
 Works
well for sequential code with little branching, or code
with large basic blocks
Page 4
Suggested Solutions

Multiple branch target
address prediction

Branch address cache
(1993, Yeh, Marr, Patt)
– Provides quick access to
multiple target addresses
– Disadvantages
• Complex alignment
network, additional
latency
Page 5
Suggested Solutions (cont’d)

Collapsing buffer

Multiple accesses to btb
(1995, Conte, Mills,
Menezes, Patel)
– Allows fetching nonadjacent cache lines
– Disadvantages
• Bank conflicts
• Poor scalability for
interblock branches
• Significant logic added
before and after
instruction cache

Fill unit

Caches RISC-like
instructions derived from
CISC instruction stream
 (1988, Melvin, Shebanow,
Patt)
Page 6
Problems with Prior Approaches

Need to generate pointers for all noncontiguous
instruction blocks BEFORE fetching can begin
 Extra
stages, additional latency
 Complex alignment network necessary

Multiple simultaneous access to instruction cache
 Multiporting

is expensive
Sequencing
 Additional
stages, additional latency
Page 7
Potential Solution – Trace Cache

Rotenberg, Bennett, Smith (1996)

Advantages
 Caches
dynamic instruction sequences
–Fetches past multiple branches
 No additional fetch unit latency

Disadvantages
 Redundant
instruction storage
–Between trace cache and instruction cache
–Within trace cache
Page 8
Trace Cache Details

Trace
 Sequence
of instructions potentially containing branches and
their targets
 Terminate on branches with indeterminate number of targets
–Returns, indirect jumps, traps

Trace identifier
 Start

address + branch outcomes
Trace cache line
 Valid
bit
 Tag
 Branch
flags
 Branch mask
 Trace fall-through address
 Trace target address
Page 9
Page 10
Next Trace Prediction (NTP)


History register
Correlating table


Secondary Table


Complex history indexing
Indexed by most recently
committed trace ID
Index generating function
Page 11
NTP Index Generation
Page 12
Return History Stack
Page 13
Trace Cache vs. Existing Techniques
Page 14
Trace Cache Optimizations

Performance
 Partial
matching [Friendly, Patel, Patt (1997)]
 Inactive issue [Friendly, Patel, Patt (1997)]
 Trace preconstruction [Jacobson, Smith (2000)]

Power
 Sequential
access trace cache [Hu, et al., (2002)]
 Dynamic direction prediction based trace cache [Hu, et al.,
(2003)]
 Micro-operation cache [Solomon, et al., 2003]
Page 15
Trace Processors

Trace Processor Architecture

Processing elements (PE)
– Trace-sized instruction buffer
– Multiple dedicated functional units
– Local register file
– Copy of global register file
 Use hierarchy to distribute execution resources

Addresses superscalar processor issues

Complexity
– Simplified multiple branch prediction (next trace prediction)
– Elimination of local dependence checking (local register file)
– Decentralized instruction issue and result bypass logic
 Architectural limitations
– Reduced bandwidth pressure on global register file (local register
files)
Page 16
Trace Processor
Page 17
Trace Cache Variations

Block-based trace cache (BBTC)
 Black,
Rychlik, Shen (1999)
 Less storage capacity needed
Page 18
Trace Table:
BBTC Trace Prediction
Page 19
Block Cache
Page 20
Rename Table
Page 21
BBTC Optimization

Completion time multiple branch prediction (Rakvic, et al.,
2000)
 Improvement
over trace table predictions
Page 22
Tree-based Multiple Branch Prediction
Page 23
Tree-PHT
Page 24
Tree-PHT Update
Page 25
Trace Cache Variations (cont’d)

Software trace cache
 Ramirez,
Larriba-Pey, Navarro, Torrellas (1999)
 Profile-directed code reordering to maximize sequentiality
–Convert taken branches to not-taken
–Move unused basic blocks out of execution path
–Inline frequent basic blocks
–Map most popular traces to reserved area of i-cache
Page 26