Correlation Module Specification
Table of Contents
Purpose of Module ....................................................................................................................................... 2
Module Architecture ..................................................................................................................................... 3
Correlation Engine .................................................................................................................................... 3
Correlation Strategies ............................................................................................................................... 3
Correlation Message Acceptor.................................................................................................................. 3
Association Store ...................................................................................................................................... 3
Upstream Components ............................................................................................................................. 3
Purpose of Module
The purpose of the Correlation Module is to identify a relationship between two pieces of input data, in
this case messages received or sent within the network being monitored. The Correlation Module finds
these relationships in order to establish a link between two points in the network: one where the
message was sent from, and one where it was received. The travel time across the link can then be
calculated by taking the difference of the receipt and send times.
In order to achieve this purpose, the module must:
1.
2.
3.
Provide a way to add messages to the pool being correlated
Provide a way to specify what constitutes a relationship between messages
Store the relationships and latencies discovered
The implementation strategy for these functionalities is discussed in the following sections of this
document.
Module Architecture
Correlation Engine
The central piece of the Correlation Module, the correlation engine maintains the pool of messages,
executes strategies to find relationships between messages, and stores the results. The message pool
consists of a set of messages from each source (a point on the network graph), and edges (also from the
network graph) which link two sets. An edge specifies that the engine should search for relationships
between the two sets it links, and furthermore that there is a directionality between the two sets (so
messages from the end set can only correlate to messages from the start set which have an earlier
timestamp).
Correlation Strategies
The correlation engine contains a strategy for each of the edges relating message sets in its pool. A
strategy determines what constitutes a relationship between a message at the start set of the edge and
the end set of the edge. Usually this involves a mapping from the fields of one message to the fields of
the other. A more complex strategy can be created through a composition of simpler strategies.
Correlation Message Acceptor
A correlation message acceptor is created for each source (point on the network graph) which will add
messages to the module. The acceptor receives messages from the source, and then adds them to the
corresponding message pool set within the correlation engine.
Association Store
When the correlation engine finds a relationship between two messages, it sends the relationship to the
association store in order to save that result. Different types of association stores can be implemented
depending on how the results of correlation are to be used.
Upstream Components
Upstream components are additional message acceptors which wrap a correlation message acceptor, or
from a different perspective provide an adapter between it and a message source. Each added acceptor
performs some transformation on the messages which pass through it. For example, a message adapter
can be implemented to split messages from a single source into two separate sources (as in the case of
splitting input and output messages from a single log file which contains both).