Routing Algorithms
© Sudhakar Yalamanchili, Georgia Institute of Technology (except as indicated)
Overview
• Routing Algorithm fixed by
 Routing function
 Selection function
A single function for deterministic protocols
• Determinant of key properties




Connectivity
Deadlock and routing freedom
Fault tolerance
adaptivity
ECE 8813a (2)
Taxonomy
Routing Algorithms
# Destinations
Routing Decisions
Implementation
Adaptivity
multicast
Unicast
Centralized
Source
Distributed
Multi-phase
Table Lookup
Finite State Machine
Deterministic
Adaptive
Progressiveness
Progressive
Backtracking
Minimality
Profitable
Misrouting
Complete
Partial
Number of Paths
ECE 8813a (3)
Generic Router Architecture
ECE 8813a (4)
Elaboration
• Source Routing
 Static paths defined at the source
 Extensions to street-sign routing
• Multiphase routing
 Used for reliable routing
 Load balancing
• Irregular networks typically need table look-up
or source routing
• Interval routing
 Making table-based routing more efficient
ECE 8813a (5)
Interval Routing
0
1
•
5
8
9
12
13
2
6
10
14
3
7
11
15
Channel
Node
Interval
X+
6
8-15
X-
6
0-3
Y+
6
4-5
Y-
6
7
Each output link corresponds to an interval of nodes


•
4
Union of intervals at a node is the set of all destination nodes
Must be able to distinguish invalid intervals (at the edges of a
mesh)
Use overlapping intervals for fault tolerance
ECE 8813a (6)
Some Finer Points
• Oblivious vs. deterministic protocols
 Typically referring to use of network state
 Deterministic protocols are oblivious
o
Converse may not be true
• Fully, maximal, and true fully adaptive
 Fully: maximize alternative physical paths
 Maximal: maximize all routing options
 True: no constraints on VC usage
o
Deadlock recovery-based methods
ECE 8813a (7)
Classes of Routing Algorithms
• Deterministic Routing
• Partially Adaptive
• Fully Adaptive
• Maximally Adaptive
• Non-minimal Routing
• Routing in MINs
ECE 8813a (8)
Deterministic Routing Algorithms
Mesh
Binary Hypercube
• Strictly increasing or decreasing order of
dimension
• Acyclic channel dependencies
Deterministic routing
ECE 8813a (9)
Tori
c0
n0
n1
c1
c3
n3
n2
c2
c10
c10
n0
c13
c03
c11
c01
c03
n1
c00
• Dimension order routing
• Create an acyclic channel
dependency graph with the
following routing function c0i
when j<i, c1i when j >i
c01
c11
c02
c12
n3
n2
c02
Deterministic routing
c12
ECE 8813a (10)
Deterministic Routing Algorithms:
Implementation Issues
• Relatively, the most inexpensive to implement
 Absolute addressing
 Relative Addressing
• Header update
 Necessary to maintain uniformity of implementation
• Source routing
 Street sign addressing: only encode the turns
Deterministic routing
ECE 8813a (11)
Partially Adaptive Routing Algorithms
• Trade-off between hardware resources and
adaptivity
 Maximize adaptivity for given resources
 Minimize resources required for a given level of
adaptivity
• Typically exploit regular topologies
Partially Adaptive Routing
ECE 8813a (12)
Planar Adaptive Routing
A0
A0
A1
Fully adaptive
Planar adaptive
A1
A2
Planar adaptive
• Packets are routed adaptively in a series of two
dimensional planes
 Order of planes (dimensions) is arbitrary
• Routing in two dimension uses two virtual
networks
 Increasing and decreasing networks
Partially Adaptive Routing
ECE 8813a (13)
Adaptive Routing in Two Dimensions
0
1
2
3
0
1
2
3
4
5
6
7
4
5
6
7
8
9
10
11
8
9
10
11
12
13
14
15
12
13
14
15
Increasing Network
Decreasing Network
Di+1
Di
Partially Adaptive Routing
ECE 8813a (14)
PAR in Multidimensional Networks
Dimension i+2
A0
Dimension i
Dimension i+1
•
•
Routing is fully adaptive
in a plane
When can you skip a
plane?
Partially Adaptive Routing
A1
ECE 8813a (15)
PAR Properties
• Each plane is comprised of the following
channels
Ai = di,2 + di+1,0 + di+1,1
• Three virtual channels/link in meshes and six
virtual channels/link in Tori
Partially Adaptive Routing
ECE 8813a (16)
The Turn Model
abstract cycles
XY routing
west-first routing
• What is a turn?
 From one dimension to another : 90 degree turn
 To another virtual channel in the same direction: 0
degree turn
 To the reverse direction: 180 degree turn
• Turns combine to form cycles
• Goal: prohibit the least number of turns to
break all possible cycles
Partially Adaptive Routing
ECE 8813a (17)
Turn Constraints
• Choice of prohibited turns is not arbitrary
equivalent
• Alternative designs
 Three combinations unique (within symmetry)
o
Three algorithms: west-first, north-last, negative-first
Partially Adaptive Routing
ECE 8813a (18)
West First Routing
deterministic region
fully adaptive region
• Fully adaptive to the east, deterministic to the
west (for non-minimal routing)
• Non-minimal is partially adaptive to the west
Partially Adaptive Routing
ECE 8813a (19)
Generalization of the Turn Model
Application to Binary hypercubes
•Base is e-cube routing
•Identify up channels and down
channels
•Adaptively route though one class
and then the other
•Turns prohibited from one class
to the other
• Identify channel classes and prohibit turns
between them
• Cycles are infeasible within a channel class
• Transitions between channel classes are acyclic
• Partially adaptive: number of shortest paths
are reduced
Partially Adaptive Routing
ECE 8813a (20)
Fully Adaptive Routing
• Using resources in the form of buffers,
channels and networks
• Migration from initial proposals to more
efficient implementations
Fully Adaptive Routing
ECE 8813a (21)
Structured Buffer Pools
buffer D-1
buffer 1
buffer 0
• Positive hop algorithm




D+1 buffers at each node (D = diameter)
Nodes request/use buffers in strictly increasing order
Minimal path algorithm, valid for any topology
Large buffer requirements: O(Diameter)
Fully Adaptive Routing
ECE 8813a (22)
Structured Buffer Pools: Extension
set 0
set 1
subset 0
subset 1
subset S-1
• Negative hop algorithm
 Partition nodes into non-adjacent subsets
 Order subsets
 Down transitions request higher numbered buffer,
else same numbered buffer
 Number of buffers required in each node is given by
 D( S  1)  + 1
 S



Fully Adaptive Routing
ECE 8813a (23)
Extensions to Wormhole Switching
Acyclic central buffer
dependencies are
transformed into
acyclic virtual channel
dependencies
•
•
replace central buffers with
equivalent number of
virtual channels across
each physical channel
Basic version produces unbalanced used of virtual channels

distance rarely equal to diameter


Number is equal to unused hops: diameter - #req
Use this number to increase the number of choices of virtual
channels at any node
Not the same as adaptivity
Extension: bonus cards.

Fully Adaptive Routing
ECE 8813a (24)
Virtual Networks
• Establish multiple virtual networks
 Each network works for a specific destination set
 Routing functions are acyclic, but typically not
connected
• Establish constraints between virtual networks
Fully Adaptive Routing
ECE 8813a (25)
Virtual Networks in a Mesh
Y
0
1
2
3
0
1
2
3
4
5
6
7
4
5
6
7
8
9
10
11
8
9
10
11
12
13
14
15
12
13
14
15
X
X+Y+
•
Each virtual network is constructed to have acyclic
channel dependencies

•
•
•
X-Y+
Routing function in a virtual network is not connected
Packets are injected into the appropriate virtual network
Fully adaptive, no transitions between networks
2n virtual networks with (n.2n) virtual channels/node
Fully Adaptive Routing
ECE 8813a (26)
Optimize
0
1
2
3
0
1
2
3
4
5
6
7
4
5
6
7
8
9
10
11
8
9
10
11
12
13
14
15
12
13
14
15
Virtual network 0
•
•
Virtual network 1
Reduce the number of networks by 50% with one
additional channel/network
Extensions to k-ary n-cubes by introducing levels in each
virtual network



One level for each wrap-around channel
(n+1).(n+1).2(n-1) virtual channels/node for dimensions >0
See Figure 4.20
Fully Adaptive Routing
ECE 8813a (27)
Extensions to Tori
• Addition of layers
(virtual networks)
for each wraparound
connection
• Number of layers
increases by
#dimensions
Fully Adaptive Routing
ECE 8813a (28)
Deadlock Avoidance via Message
Dependencies
• Two virtual channel classes across each
physical link
 Adaptive channels & deterministic channels
• Fully adaptive use of adaptive channels
 Keep track of #dimension reversals for each message
o
Moving from a dimension p to a lower dimension q
 Label each channel with the DR# of the message
 Messages cannot block on a channel with lower DR#
o
If no channel available, permanent transition to the
deterministic channel
 Dependencies between messages are acyclic
Fully Adaptive Routing
ECE 8813a (29)
Summary of Design Techniques
• Ordered use of topological features
 Dimensions
 Packaged components
 Paths
• Ordered use of resources
 Buffers
 Channels
• Order the message population
 Each message is uniquely identified by some
attribute, e.g., number of wrap-around channel
crossings
 Order blocking based on message population
membership
ECE 8813a (30)
Design Methodology
•
Start with a network, set of channels C1, and routing
function R1

•
R1 is connected and deadlock free and may be
deterministic/adaptive, minimal/non-minimal
Split each physical channel into a set of additional virtual
channels and define the new routing function
R( x, y )  R1( x, y )  (Cxy  (C  C1))


•
Set of channels includes escape channels and adaptive
channels
Selection function can be defined in many ways
For wormhole switching, verify that the extended channel
dependency graph is acyclic: likely if R is restricted to
minimal paths
ECE 8813a (31)
Example Binary Hypercube
• Start with
dimension order ecube algorithm
• Add additional
channels for
adaptive routing
2
3
6
4
0
7
5
1
ECE 8813a (32)
Maximally Adaptive Routing
• Establish a relationship between routing
freedom and resources
 Maximize adaptivity for fix resources
 Minimize resources for target adaptivity
• Relationship between adaptivity and
performance
 Not obvious
 Unbalanced use of physical or virtual channel
resources
Maximally Adaptive Routing
ECE 8813a (33)
In 2D Meshes: Double Y
0
1
2
3
Y2
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Y1
Y2
4
0
6
Y1
increasing network
Y1+ X+
X-
Y1-
Maximally Adaptive Routing
decreasing network
X-
Y1+
Y1-
X+
Y2+ X+
X-
Y2-
X-
Y2+
Y2-
X+
ECE 8813a (34)
In 2D Meshes: Mad-Y
permitted
X-
Y1+ X+
Y1+
Y2+ X+
X-
Y2+
Y2
6
X-
Y1-
Y1-
X+
X-
Y2-
Y2-
X+
Y1
No coupling
between Y2 and Y1
• Permit turns
 From the Y1 channels to the X+ channels
 From X- channels to Y2 channels
• Remove unnecessary turn restrictions
• Still overly restrictive!
Maximally Adaptive Routing
ECE 8813a (35)
In 2D Meshes: Opt Y
Y1+ X+
X-
Y1-
X- Y1+
Y1-
X+
Y2+ X+
X-
Y2-
X-
Y2+
Y2+ Y1+
Y2-
X+
Y1+ Y2+
Y1Y2-
Y2Y1-
• Further reduce the number of restrictions
 Only restrict turns from Y1 to X Turns from X- to Y1 and 0-degree turns in Y only
when X offset is 0 or positive
• Extensions to multidimensional meshes
• Basic idea: fully adaptive routing in one set of
channels, and dimension order in the other set
until specified lower dimension traversals are
complete
Maximally Adaptive Routing
ECE 8813a (36)
Routing with Minimum Buffer Requirements
• Key Idea:
 Organize packet traffic into disjoint groups that use
separate buffers in each node
 Place acyclic routing restrictions in buffer usage
• Based on node orderings
Maximally Adaptive Routing
ECE 8813a (37)
Node Labeling for 2D Torus
0
1
2
3
15
14
13
12
4
5
6
7
11
10
9
8
8
9
10
11
7
6
5
4
12
13
14
15
3
2
1
0
right increasing node ordering
left increasing node ordering
0
1
3
2
15
14
12
13
4
5
7
6
11
10
8
9
12
13
15
14
3
2
0
1
8
9
11
10
7
6
4
5
inside increasing node ordering
Maximally Adaptive Routing
outside increasing node ordering
ECE 8813a (38)
Algorithm 1
• Algorithm
 Packet moves from the injection queue to the A
queue
 Stay in the A queues as long as we can move to right
along at least one dimension along a minimal path
 Transition to the B queues under same rule for left
traversals
 Transition to the C queue and remain there until
packet is delivered
• Note the de-coupling of node labeling from
buffer labeling
Maximally Adaptive Routing
ECE 8813a (39)
Application
• Orderings analogous
to virtual planes
0
 Note the orderings are
acyclic
1
2
3
• Extensions to edge
buffers
4
5
 Check Algorithm 2
6
7
0
1
2
3
4
Maximally Adaptive Routing
5
6
7
ECE 8813a (40)
True Fully Adaptive Routing
• Adaptivity extends across physical and virtual
channels
• Deadlock recovery vs. deadlock avoidance
Maximally Adaptive Routing
ECE 8813a (41)
Non-minimal Routing
• Non-minimal routing
 Wormhole degrades performance while VCT has less
secondary effects
 Fault tolerance is the main motivator
• Classes
 Backtracking
 Randomized routing and the Chaos router
Non-Minimal Routing
ECE 8813a (42)
Backtracking Protocols
• Backtracking search + resource reservation
• Constrain the search
 Minimal paths vs. #misroutes
Non-Minimal Routing
ECE 8813a (43)
Optimization
•
Sensitive to choice of switching technique


•
•
Deadlock is avoided by not blocking on busy channels
Livelock is avoided by maintaining and using search
history


•
Naturally suited to circuit switching and pipelined circuit
switching
Overhead is large with SAF
In the header: large headers
In the routers: local state, headers comparable to e-cube
Protocol variations





Multi-links
k-family
exhaustive: profitable and misrouting
limited misrouting
multi-phase
Non-Minimal Routing
ECE 8813a (44)
Routing in MINs
0
0
1
1
2
2
3
3
4
4
5
5
6
6
7
7
•
•
•
“Square” vs. non-square MINs vs. dilated MINs
Permutation routing and centralized control
Message routing, distributed control, and contention
•
Basic elements of a logkN network and topological
equivalence


Single source destination path
Logarithmic delay
Routing in MINs
ECE 8813a (45)
The Blocking Condition
sn-1,sn-2,…,s2,s1,s0,dn-1,dn-2,…,d2,d1,d0
rn-1,rn-2,…,r2,r1,r0,tn-1,tn-2,…,t2,t1,t0
• Addresses of ports at intermediate switches are
computed from the source-destination
addresses
 Blocking condition: two paths collide iff they compete
for the same link at any stage
Routing in MINs
ECE 8813a (46)
Self Routed MINs
Routing in MINs
ECE 8813a (47)
Self Routed MINs (cont.)
• Self routing property
 Routing decision a function of only the destination
• Computation of destination routing tag
tn-1,tn-2,…,t2,t1,t0
 ti = di+1 0 <= i <= n-2, and tn-2 = d0 for butterfly
MINs
 ti = dn-i-1 0 <= i <= n-1 for Omega and Cube
networks
Routing in MINs
ECE 8813a (48)
Self Routed MINs: example
Routing in MINs
ECE 8813a (49)
Routing in Bidirectional MINs
• Routing uses the function FirstDifference(S,D)
 Identifies the nearest stage to “turnaround”
• Multiple choices of switches at last stage
 Can randomize forward path selection
Routing in MINs
ECE 8813a (50)
Relevance of Fat Trees
• Popular interconnect
for commodity
supercomputing
• Active research
problems
Routing in MINs
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
 Efficient, low latency
routing
 Fault tolerant routing
 Scalable protocols
(coherence)
ECE 8813a (51)
Segment Based Routing
• Topology dependent vs. topology agnostic
routing
 Reliability
 Increasing important on-chip
• Restriction-based approach
 Multiple restriction options
o
Select restrictions based on performance goals
 Source based routing
o
Routing table generation
Topology Agnostic Routing
ECE 8813a (52)
Key Idea: Segments & Subnets
• Partition topology into subnets and then
segments in a subnet
• Goal: islands of regularity
Topology Agnostic Routing
ECE 8813a (53)
Requirements
• Avoid deadlock in a segment
• Avoid deadlock when traversing multiple
segments
• Ensure routing connectivity when physical
connectivity exists
• Avoiding congestion in path construction
Topology Agnostic Routing
ECE 8813a (54)
Construction of Segments
Starting node
Failed links
Terminal node
Unitary segment
bridge segment
• Search for initial segment + “regular”
segments
 Unitary segments
• Add one bidirectional restriction in each
segment
• Source routing
Topology Agnostic Routing
ECE 8813a (55)
Deadlock Freedom
• One routing restriction per segment
 No cycles in a segment
• Every cycle contains a segment
 Hence cannot be “closed” to create deadlock
• No cycle from the start node back to itself
 Cannot create cycles across subnets
• Think of a subnet as a union of 1-D segments
Topology Agnostic Routing
ECE 8813a (56)
Performance
From A. Mejia , J. Flich, J. Duato, Sven-Arne Reinomo and Tor Skeie, “Segment Based Routing: An Efficient Fault-Tolerant Routing
Algorithm for Meshes and Tori,” Proceedings of the International Parallel and Distributed Processing Symposium, April 2006.
Topology Agnostic Routing
ECE 8813a (57)
Region-Based Routing
• Problem: reduce the size of routing tables for
on-chip routers
• Recognize that finite state machine routers
implicitly check for region membership
 Think meshes
• Generalize the idea of regions
 Can naturally be adapted for fault tolerant routing
Topology Agnostic Routing
ECE 8813a (58)
Example of Regions
{node set}
{node set}
• What are the characteristics of these regions?
 Note #regions = f(routing options)
From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable
Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007
Topology Agnostic Routing
ECE 8813a (59)
Approach
• Observe that table-based routing is really
region based
 Each entry identifies a region
• Merge entries into compact region
specifications at each switch
• Region construction is based on the paths
 Any set of paths  fault tolerant routing
 No virtual channels
Topology Agnostic Routing
ECE 8813a (60)
Region Construction
@each node
@each node
•
•
•
•
Start with routing restrictions
Compute deadlock free-paths
Compute regions
Coalesce regions (pack if necessary to cap
#regions)
• Construct routing table
From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable
Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007
Topology Agnostic Routing
ECE 8813a (61)
Creating Regions
• Coalesce routing options based on inputs and
outputs
• Represents a compact routing table
From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable
Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007
Topology Agnostic Routing
ECE 8813a (62)
Implementation
• Initialization of region registers and parallel
evaluatiom of all regions
From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable
Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007
Topology Agnostic Routing
ECE 8813a (63)
Hardware Overheads
• Each region requires
 Four registers that define the region
 Mask registers to define input and output ports
 Logic to determine routing options
• Hardware cost grows as the number of regions
 Growth as f(network_size) is much slower
From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable
Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007
Topology Agnostic Routing
ECE 8813a (64)
Microarchitecture Issues
• Routing algorithm performance is sensitive to
resource allocation schemes in the router
• Key resource management functions include
 Routing function
 Selection function
 Arbitration/scheduling
• Mismatch can lead to poor performance
ECE 8813a (65)
Resource Allocation: Selection Functions
VC status/control
Routing function
VC buffer
Selection function
Input VC
Output VCs
•
Selection function may be oblivious or informed

•
Examples:

•
Common to favor minimal paths and lightly loaded links
Meshes: minimum congestion, maximum flexibility, straight
lines
Unlike routing functions, selection function must be
serialized

Result updates the channel status – a centralized resource
ECE 8813a (66)
Selection Functions
• Favor adaptive channels
 Improve probability of escape channel availability
 Time dependent selection functions: give adaptivity a
chance
• Selection functions for real time traffic
 Separate best effort and guaranteed packets via VCs
or virtual networks
• Note the impact on bisection utilization
• Selection functions for cache coherent
systems?
ECE 8813a (67)
Resource Allocation: Arbitration
arbitration
Message size
Flow control
• Tradeoffs: channel bandwidth vs. message
sizes and types
 Mix of buffering strategies across message types
• All three strategies must be co-designed for a
tuned system
ECE 8813a (68)
Routing, Selection & Arbitration
• Input driven vs. output driven scheduling
 Output driven scheduling requires replication of
routers amongst inputs
• Lessons from the microprocessor world
 Impact of complexity, workloads, and concurrency
• Impact of….
 Symmetry of the topology
 Locality of traffic
 Packet size
Locality,
uniformity
deterministic
routing
Irregular,
hot spot
adaptive
routing
ECE 8813a (69)
Summary
• Best routing algorithm driven by multiple
considerations
Hot spots
Deterministic vs. adaptive
Packet sizes
Compatible micro-architecture
Locality of traffic
Uniform vs. non-uniform traffic
Power envelope
On-chip vs. off-chip
Symmetric vs. asymmetric topology
ECE 8813a (70)