Deliverable D3.6
Internal progress report on software evaluation and
testing
1. STATUS OF THE DEVELOPMENT AND INTEGRATION
SANTA-G (Task 3.3)
Typically with Grid Information Systems an information provider inserts data to the system at
regular intervals. The user can then retrieve this data using the information system client (e.g.
API, GUI). A difficulty arises when dealing with information sources that generate a large
amount of data at a very fast rate in a form unsuitable for direct insertion to the information
system, for example instrument monitors (e.g. logic analysers). SANTA-G is a framework
that supports monitoring with this type of information source in the Grid environment by
allowing access to data through the Grid Information System. The monitoring data, rather
than being inserted into the information system, is left in its raw form at the location where it
was created.
Information
Provider
Information
Source
Data
Grid
Information
System
Request /Data
Client
Request /Data
SANTA-G
Figure 1-1 SANTA-G Framework
The demonstrator of this, developed within the EU CrossGrid project, is the SANTA-G
NetTracer, a network monitor that allows users to access log files stored in libpcap (a network
packet capture library) format through the EU DataGrid’s (EDG) Relational Grid Monitoring
Architecture (R-GMA) monitoring and information system. Examples of tools that generate
logs in this format are Tcpdump, and SNORT (a network intrusion detection system). It is
aimed at system administrators for network traffic analysis across multiple sites within a Grid,
and also for performance analysis. It is also intended to use the SNORT functionality of the
NetTracer to construct a Grid-wide intrusion detection system.
The latest version of the NetTracer is v1.4.1.
RPMS can be downloaded from:
http://gridportal.fzk.de/distribution/crossgrid/autobuilt/i386-rh7.3-gcc3.2.2/wp3/RPMS/
The source code can be browsed and downloaded from the CrossGrid CVS:
http://savannah.fzk.de/cgi-bin/viewcvs.cgi/crossgrid/crossgrid/wp3/wp3_3-moninfr/wp3_3_2santag/
User, installation and developer guides for the NetTracer can be obtained from the following
URL:
http://savannah.fzk.de/autobuild/i386-rh7.3-gcc3.2.2/wp3_3_2-santag/userdoc/
CG3.0-D3.6-v0.7-PSNC010-Plans&State.doc
Internal
1 /4
Deliverable D3.6
Internal progress report on software evaluation and
testing
1.1.1
Current State of Implementation
In the design document [Deliverable D3.2] it was stated that the SANTA-G NetTracer, was
composed of two main modules, the publishing module, and the viewer module. The
publishing module is itself composed of a further two components, the QueryEngine, and the
Sensor (which is installed on the node(s) to be monitored).
QueryEngine
Tcpdump
Sensor
Viewer
R-GMA
CanonicalProducer API
R-GMA
Consumer
API
R-GMA
LatestProducer API
Log Files
Figure 1-2 SANTA-G NetTracer, Network traffic monitoring
The purpose of these two components is to allow for monitoring data stored in libpcap format
log files to be accessed through the R-GMA. This includes log files generated by Tcpdump
(an open source packet capture application) (Figure 1-2), and the SNORT network intrusion
detection system (Figure 1-3).
QueryEngine
Sensor
SNORT
Viewer
R-GMA
CanonicalProducer API
R-GMA
Alerts
Consumer
API
R-GMA
LatestProducer API
Packet
log file
Figure 1-3 SANTA-G NetTracer, Network intrusion detection
The Sensor invokes Tcpdump, in the case of network traffic monitoring, and then monitors
the log files created. The Sensor notifies the QueryEngine when new log files are detected.
The QueryEngine records these events. The QueryEngine also implements the interface to the
R-GMA by using the CanonicalProducer API. The Viewer module provides a Java Swing
GUI that presents a graphical interface to users. It allows users to collect data from the RGMA, by using the Consumer API, and to graphically view packets stored in the log files.
The full implementation and functionality of these components is described in detail in the
SANTA-G Developer guide.
Since the description of the prototype contained in Deliverable 3.5 significant changes and
improvements have been made to the NetTracer, although the basic functionality has
remained relatively unchanged. Bugs found during the test and validation process have been
CG3.0-D3.6-v0.7-PSNC010-Plans&State.doc
Internal
2 /4
Deliverable D3.6
Internal progress report on software evaluation and
testing
corrected and the source code has been commented and refactored in order to comply with the
quality assurance recommendations.
Major changes have included the addition of a recovery mechanism to the QueryEngine
component. Should the QueryEngine be restarted it will recover the state of any currently
running sensors. Another important change was the removal of the use of NFS between the
QueryEngine and the sensor. In previous versions the QueryEngine accessed the remote log
files using NFS. This was considered too insecure. This is now done using remote file reads
over an SSL socket connection, making use of the EDG Java Security packages.
1.1.2
Current State of Integration
The SANTA-G NetTracer is currently installed at all sites, on both the development and
production testbeds.
The NetTracer has been integrated with the JIMS monitoring system (Task 3.3.2). A plugin
has been implemented for the QueryEngine that allows the NetTracer to obtain monitoring
data from JIMS, and to publish it to the R-GMA information system. This allows for
monitoring data from both monitoring tools to be accessed through the R-GMA (see Figure
1-4).
Figure 1-4 SANTA-G - JIMS Integration
The plugin periodically collects data from the JIMS system, and publishes it to the R-GMA.
JIMS information published from multiple sites can then be collected at the Grid Operations
Center using an R-GMA Archiver. This provides for historical analysis of the JIMS
monitoring data, as well as providing a common interface to the data collected by the two
monitoring systems.
The SNORT functionality of the SANTA-G NetTracer is also to be used as the basis of a
Grid-wide intrusion detection system. This system is to be deployed by the Grid-Ireland
CG3.0-D3.6-v0.7-PSNC010-Plans&State.doc
Internal
3 /4
Deliverable D3.6
Internal progress report on software evaluation and
testing
project (http://www.grid-ireland.org/) in January 2005. It is envisaged that each site within
Grid-Ireland will host the NetTracer along with a set of SNORT sensors publishing alerts to
the R-GMA (Figure 1-5). By using other R-GMA components, such as an Archiver, alerts
published from multiple sites will be aggregated to form a Grid-wide intrusion log. A high
level incident detection, tracking and response platform will then be created by using custom
coded Consumers to filter and analyse this log in order to detect patterns that would signify an
attempted distributed attack on the Grid infrastructure. If such a pattern is found the
Consumer would trigger a pattern-specific alert, and thereby generate Grid-wide intrusion
alerts.
Figure 1-5 Grid-wide Intrusion Detection
CG3.0-D3.6-v0.7-PSNC010-Plans&State.doc
Internal
4 /4