1. Introduction
1.1.
Environment
1.1.1. Description
A test-bench environment that instantiates a NxN Router
network, where every Router connects to 4 other Routers at 4
of its ports (ports 0-3) and connects to a Module via every port
hat is not connected to a Router.
Each Module has an Injector which autonomously injects
random packets into the Router network. The packets travel
through the network and terminate at the Collector component
of one of the Modules in the Environment.
Both the Injector and the Collector components log all packet
activities which then can be analyzed (offline) using the
Network Analyzer set of scripts that calculate the network's
performance throughout the test.
1.1.2. Components:
1.1.2.1.Router
1.1.2.2.Module
1.1.2.3.Network Analyzer
1.1.3. Architecture:
R(0,0)
R(0,1)
M(0,0)
R(1,0)
R(1,1)
M(1,0)
1.1.4. Features:
1.1.4.1.Network Load – Determines the frequency at
which packets are injected for each Injector
in the Environment. Each Injector Network
Load is set separately by the Environment.
1.1.4.2.Packet Size – Determines the average packet
size for all the Injectors throughout the test.
1.2.
Module
1.2.1. Description
A Module is a stub unit that replaces a system on a chip and
functions as a start point and end point for all the packets in the
network.
Each Module has a unique ModueID which consists of 2
coordinates. The coordinates of the Module are the coordinates
of the Router to which the Module is connected.
1.2.2. Components:
1.2.2.1. Injector
1.2.2.2. Collector
1.2.3. Features:
1.2.3.1.Local Clock Domain – Operates at its own
clock domain. The Environment generates a
base clock for all the Modules, and a divider
value per Module. Each Module creates its
own clock by dividing the Base clock by its
own divider value.
1.3.
Injector
1.3.1. Description
The Injector generates and injects new packets via the port
through which it is connected to the Router.
It generates all of the packet's parameters (some parameters
may be globally controlled by the Environment) including the
Source Routing of the packet and then sends the packet to the
router.
In order to uniquely identify each packet in the Environment,
the data of each of the packets is unique. The Injector sends
using the packet's data, the packet's unique PacketID which
consists of the sending ModuleID concatenated with a unique
ascending counter number.
1.3.2. Features:
1.3.2.1.Multiple SL Injection – Injects packets of
different SLs at random. The Environment
sets a global setting of how many SLs are in
use.
1.3.2.2.Multiple VC Injection – Injects packets at
different VSs at random. The Environment
sets a global setting of how many VCs are in
use.
1.3.2.3.Multiple Packet Size Injection – Injects
packets of random sizes. (uniform value)
1.3.2.4.Routing – The Module selects a random
destination Module, and generates a valid
source routing vector for the packet to
follow (horizontal movement followed by
vertical movement).
1.3.2.5.Network Load – The frequency at which
packets are injected is controlled by the
Environment. Each Injector Network Load is
set separately by the Environment.
1.3.2.6.Logging – Logs all the sent packets into a
log file.
1.4.
Collector
1.4.1. Description
The Collector receives flits from the Router, collects
them into buffers, until it receives the last flit of the
packet, and then, adds a report to its log.
1.4.2. Features:
1.4.2.1.Packet Reception – Receives packets from
the Router.
1.4.2.2.Has several buffers (SL number * VC
Number), which enables to collect
simultaneously more than one packet.
1.4.2.3.Logging – Logs all the received packets into
a log file.
1.5.
Network Analyzing Script
1.5.1. Description
An offline script that analyzes the log file and calculates the
network performance.
1.5.2. Features:
1.5.2.1.Latency Calculation – Calculates the average
Latency of the network after a single
Environment run. Besides total latency
calculation, it also performs latency
calculation per SL.
1.5.2.2.Throughput Calculation – Calculates the
average Throughput of the network after a
single Environment run. Besides total
throughput calculation, it also performs
throughput calculation per SL.
2. Implementation
2.1.
Environment
2.1.1. Environment Parameter
2.1.1.1.Network Size – An Integer parameter that
defines the size of the Router network to the
size of NxN. This parameter is set in the
Environment package file.
2.1.1.2.Network Load Matrix – A matrix that sets
the average Network Load of each of the
Modules in the Environment. The parameter
size is set between 0 and 10 (Defines tens
percents of workload). This parameter is set
in the Environment settings file.
2.1.1.3.SLs in Use – The maximum number of SLs
(starting from SL0) that the Modules are
allowed to use for randomization. This
parameter is set in the Environment settings
file.
2.1.1.4.VCs in Use – The maximum number of VCs
that the Modules are allowed to use for
randomization. This parameter is set in the
Environment settings file.
2.1.2. Hierarchy Instance Generation
2.1.2.1.Figure :
2.1.2.2.Router Generation - The Environment
generates NxN instances of the Router. Each
Router instance is designated with a unique
RouterID, which consists of 2 coordinates
that are determined by the Router's location
in the network.
2.1.2.3.Module Generation – The Environment
generates NxN instances of the Module, to
be connected to it's matching Router's Port
no. 4. Each Module instance is designated
with a unique ModuleID, which consists of 2
coordinates. The coordinates are determined
by the RouterID which the Module is
connected to.
2.1.2.4.Interface Generation – For each of the 0-3
ports of every Router, each output port
connects to the neighboring Router's input
port, unless the Router is at the edge of the
network and the port is pointing to that edge.
In that case, the port not connected. Each
one of the no. 4 ports in each of the Routers
is connected to the appropriate Module in
the network.
2.2.
Module
2.2.1. Hierarchy
Module
Interface
Logging
Interface
Packet
Generator
Logging
2.2.2. Local Clock Generation
The Module generates its own clock for the use of the Injector
and the Collector. The Environment generates a base clock for
all the Modules, and a divider value per Module. Each Module
creates its own clock by dividing the Base clock by its own
divider value.
2.2.3. Counters
2.2.3.1.Clock Counter – The Module counts the
amount of base clock cycles that has passed
since the start of the Module's operation.
2.3.
Injector
2.3.1. Hierarchy
2.3.1.1. Packet Generator – generates new packets.
2.3.1.2. Interface – sends the generated packets to
the Router.
2.3.1.3. Logging – writes to a log information about
the packets which are being sent.
2.3.2. Counters
2.3.2.1.Packets Counter – The Injector counts all
transmitted packets in order to create their
unique PacketID.
2.3.3. Packet Generation
The Packet Generator generates all the packet
parameters that the Injector Interface needs in order
to send a packet to the Router. The Packet Generator
receives an indication from the Injector Interface to
randomize a delay period, after which it generates
new packet parameters.
2.3.3.1.Injector Packet Load – The Injector
randomizes the delay between the
transmissions of any 2 consecutive packets.
The delay is dependant on the time it took to
send the last packet and the work load
parameter that the Environment has set for
that Module. The randomization is an
uniform random variable. The workload is
defined by percents (10%, 20%,… 100%).
The formula of the delay calculation is:
Twait = U[0,2*Tprev*((10/Workload)-1)]
2.3.3.2.Service Level and Virtual Channel – The
packet's SL, and VC are randomized, with
dependence on the limits that are provided
by the Environment.
2.3.3.3.Packet Size – The packet size is a uniformly
random variable, which is set in the limits
provided by the Environment.
Each SL has different packet size limits.
SL 3 – can have 3-15 flits.
SL2 and SL1 – can have 16-127 flits.
SL0 - can have 128-1023 flits.
2.3.3.4.Source Routing – The packet's route is pre
determined by the Injector. The Injector
randomizes the destination and generates a
route that starts in the X direction, continues
in the Y direction and ends with port4. The
route must end with port4 in order to make
sure that none of the packets finish their path
at a Router.
2.3.3.5.Packet Data – The Injector sets each packet
data to be the PacketID of the packet. The
PacketID is a concatenation of the ModuleID
and the Packet Counter padded with zeroes.
2.3.4. Packet Logging
The Logger logs every packet that the Injector sends
into a log. It does so when it is activated by the Injector
Interface. The Logger writes the following information
into its log:
2.3.4.1.Time Stamp – The base clock counter.
2.3.4.2.ModuleID – The Module ID of the Injector.
2.3.4.3.PacketID – The PacketID of the sent Packet.
2.3.4.4.Packet SL – The Service Level of the sent
packet.
2.3.4.5.Sent Indication – An indication that the
packet has been sent (as apposed to
"received")
2.3.4.6.Packet Size – The size of the sent packet (in
flits).
2.3.5. Injector/Router Interface
The Injector Interface receives the Packet
parameters that the Packet Generator generates,
creates flits that correspond to these parameters and
sends them to the Router.
2.3.5.1.The Injector is connected to the Router's
input port. Once all the parameters of the
packet have been decided, the Injector
divides the packet into Router flits, and
sends them one by one using the Router
req/ack interface.
2.3.5.2. As soon as the first header flit is sent, an
indication is sent to the Logger for it to
record the new Packet in its log.
2.3.5.3.When the Interface finishes sending the
Packet, it sends an indication to the Packet
Generator, so it can generate the next packet.
2.4.
Collector
2.4.1. Hierarchy
2.4.1.1.Interface – receives packets from the Router.
2.4.1.2. Logging – writes to log information about
the packets which are being received.
2.4.2. Collector/Router Interface
2.4.2.1.The Collector is connected to the Router's
output port. It receives incoming flits from
the Router, and constructs them into whole
packets using the Router req/ack interface.
2.4.2.2.The Collector Interface has buffers that
allow it to receive packets that have different
SLs and VCs whose flits arrive to the
Collector interleaved.
2.4.2.3.Once the Collector Interface finishes
receiving a packet (receives the tail flit) it
sends an indication to the Logger to record
the packet in its log.
2.4.3. Packet Logging
The Logger records all the packets that the Collector
receives. It is activated by the Collector Interface, and at
each activation the Logger writes the following packet
information:
2.4.3.1.Time Stamp – The base clock counter.
2.4.3.2.Target ModuleID – The Module ID of the
Collector.
2.4.3.3.Source ModuleID – The Module ID of the
Injector that sent the packet.
2.4.3.4.PacketID – The PacketID of the sent Packet.
2.4.3.5.Packet SL – The Service Level of the sent
packet.
2.4.3.6.Received Indication – An indication that the
packet has been received (as apposed to
"sent")
2.4.3.7.Packet Size – The size of the sent packet (in
flits).
2.5.
Network Analyzing Script
The analyzing Script is written in Perl. It analyzes the log file
that the Loggers generate, and generates its own (comma
separated) output file that adds data to the original log file.
2.5.1. Script Features
2.5.1.1.Sender/Receiver Matching – The script
searches for pair of sender and receiver log
entries, and combines them into a single
entry.
2.5.1.2.Packet Latency Calculation – The Script
calculates the Latency (reception time minus
the transmission time) of each packet and
adds it to the packet's entry.
2.5.1.3.Packet Throughput Calculation – The Script
calculates the throughput of the packet
(flit_number * flit_size / latency) and adds it
to the packet's entry.
2.5.1.4.Average per SL Latency Calculation – The
Script calculates the algebraic average of the
Latency for each SL.
2.5.1.5.Average per SL Throughput Calculation –
The Script calculates the algebraic average
of the Throughput for each SL.
2.5.1.6.Average Latency Calculation – The Script
calculates the algebraic average of the
Latency for all the packets.
2.5.1.7.Average Throughput Calculation – The
Script calculates the algebraic average of the
Throughput for all the packets.