Log Correlation Engine 4.2
Administration and User Guide
May 14, 2014
(Revision 2)
Table of Contents
Introduction ......................................................................................................................................... 4
Standards and Conventions ....................................................................................................................... 4
Components of the Log Correlation Engine .............................................................................................. 4
IDS Collection and Correlation ................................................................................................................... 5
IDS Collection Only .................................................................................................................................... 5
Prerequisites ................................................................................................................................................ 6
Supported Operating Systems/Platforms ................................................................................................... 6
Licenses .................................................................................................................................................... 6
SecurityCenter ........................................................................................................................................... 6
Secure Shell Public Keys ........................................................................................................................... 6
Secure the Log Correlation Engine Server System .................................................................................... 6
LCE Server Installation ....................................................................................................................... 7
Getting Started ............................................................................................................................................. 7
Installation Location .................................................................................................................................... 7
Installing the Package ................................................................................................................................. 7
Files and Layout ........................................................................................................................................ 13
Base Directories ...................................................................................................................................... 13
The admin Directory................................................................................................................................. 13
The daemons Directory............................................................................................................................ 13
The db Directory ...................................................................................................................................... 13
Installing the License ................................................................................................................................ 13
Hostname Determination ......................................................................................................................... 13
Key Installation ........................................................................................................................................ 14
Upgrading the License ............................................................................................................................. 14
Configuration .................................................................................................................................... 15
The lce.conf File......................................................................................................................................... 15
LCE Configuration ................................................................................................................................... 15
Pointing to a New “Plugins” Directory ....................................................................................................... 23
Adding Log Correlation Engine Client Information .................................................................................... 23
Debug Mode ............................................................................................................................................ 24
Correlation Statistics ................................................................................................................................ 25
Utilizing Tiered LCEs ................................................................................................................................. 26
Configuring the Primary LCE Server ........................................................................................................ 26
Configuring the Auxiliary LCE Server ....................................................................................................... 26
Syslog Considerations .............................................................................................................................. 26
Sending Syslog Messages to Other Hosts ............................................................................................... 26
syslog Compliant Messages .................................................................................................................... 27
Content of Forwarded syslog Messages .................................................................................................. 27
Storing All Logs with “save-all” ............................................................................................................... 27
Different File System................................................................................................................................ 28
Multiple Plugin Matches per Log File “multiple-matches”...................................................................... 28
Quick Example......................................................................................................................................... 28
The rules.conf File ..................................................................................................................................... 28
Email Syntax ............................................................................................................................................ 29
Syslog Syntax .......................................................................................................................................... 29
Custom Command Syntax ....................................................................................................................... 29
2
LCE Rule Filters....................................................................................................................................... 29
LCE Shell Command Options .................................................................................................................. 31
Email/Alerting/Execution........................................................................................................................... 31
LCE Operations ................................................................................................................................. 32
Starting LCE ............................................................................................................................................... 32
Halting LCE ................................................................................................................................................ 33
Restarting LCE ........................................................................................................................................... 33
Determine LCE Status ............................................................................................................................... 33
Operating the stats Daemon ..................................................................................................................... 33
Updating Plugins (PRM Files) and TASL Scripts ..................................................................................... 33
Automatic Plugin (PRM Files) and TASL Updates ................................................................................... 33
Updating Individual PRM Files ................................................................................................................. 34
Excluding PRM Files ................................................................................................................................ 35
Excluding TASL Files ............................................................................................................................... 35
Managing Client Configuration Files ........................................................................................................ 35
Upgrading LCE .................................................................................................................................. 35
Additional Features .......................................................................................................................... 36
Importing LCE Data Manually ................................................................................................................... 36
User Tracking............................................................................................................................................. 37
Working with SecurityCenter ........................................................................................................... 38
Adding the LCE to SecurityCenter ........................................................................................................... 38
Configuring Organizations ........................................................................................................................ 40
Analyzing Security Events ........................................................................................................................ 41
Identifying Vulnerabilities ......................................................................................................................... 41
TASL Scripts ............................................................................................................................................ 41
Full Text Searches ..................................................................................................................................... 42
For More Information ........................................................................................................................ 43
Appendix 1: Sample lce.conf File .................................................................................................... 44
Appendix 2: Sample LCE rules.conf File ........................................................................................ 55
Appendix 3: Troubleshooting .......................................................................................................... 59
Appendix 4: Manual SC4/LCE Key Exchange................................................................................. 60
Appendix 5: Non-Tenable License Declarations ............................................................................ 62
About Tenable Network Security ..................................................................................................... 63
3
Introduction
This document describes the installation, configuration, and administration of Tenable Network Security’s Log
Correlation Engine 4.2 for both standalone and SecurityCenter deployments. Please email any comments and
suggestions to support@tenable.com.
If the LCE is to be managed by Tenable’s SecurityCenter, knowledge of SecurityCenter operation and architecture is
assumed. Familiarity with system log formats from various operating systems, network devices, and applications and a
basic understanding of Linux and Unix command line syntax is also assumed.
Standards and Conventions
Throughout the documentation, filenames, daemons, and executables are indicated with a courier bold font such as
gunzip, httpd, and /etc/passwd.
Command line options and keywords are also indicated with the courier bold font. Command line examples may or
may not include the command line prompt and output text from the results of the command. Command line examples will
display the command being run in courier bold to indicate what the user typed while the sample output generated by
the system will be indicated in courier (not bold). Following is an example running of the Unix pwd command:
# pwd
/opt/local/lce
#
Important notes and considerations are highlighted with this symbol and grey text boxes.
Tips, examples, and best practices are highlighted with this symbol and white on blue text.
Components of the Log Correlation Engine
Note that some features described in this document are only meaningful if LCE is deployed with
SecurityCenter.
The Log Correlation Engine (LCE) has three basic components: the client, the daemon/server component (lced) and the
user interface (SecurityCenter GUI).
Throughout this document, the LCE daemon/server component will be referred to as the LCE server, while the
GUI component will be referred to as SecurityCenter for simplicity.
The LCE client is installed on hosts to monitor and collect events that are forwarded on to the LCE server daemon. When
received by the LCE server, events are both stored as raw logs and normalized and correlated with vulnerabilities (if
applicable). The SecurityCenter UI makes both the raw and normalized event data available to the user for event analysis
and mitigation.
LCE users work with log data from a wide variety of sources. Each organization can make queries to one or more LCE
servers that contain events from a wide variety of devices including firewalls, servers, routers, honeypots, applications,
and many other sources. The LCE supports many types of agents including:
Windows Event Logs (collected locally or remotely via a WMIC client)
4
Windows, Linux, and Unix system and application logs
Check Point OPSEC events
Cisco RDEP events
Cisco SDEE events
NetFlow
Splunk
Sniffed TCP and UDP network traffic (Tenable Network Monitor)
Sniffed syslog messages in motion
File monitoring (Linux, Unix, and Windows)
LCE has many signature processing libraries to parse logs and can normalize and correlate most network IDS devices, as
well as messages from SecurityCenter. The LCE supports the following IDS sources:
IDS Collection and Correlation
Bro
Cisco IDS
Enterasys Dragon
HP TippingPoint
IBM Proventia (SNMP)
Juniper NetScreen IDP
McAfee IntruShield
Fortinet IDS events
Tenable’s Passive Vulnerability Scanner (PVS)
Snort (and Snort-based products)
TippingPoint’s syslog event format must be modified to use a comma delimiter rather than a tab delimiter
before it can be processed by the LCE.
IDS Collection Only
AirMagnet
Check Point (Network Flight Recorder)
Portaledge
Toplayer IPS
There are thousands of normalization rules that support most operating systems, firewalls, network routers, intrusion
detection systems, honeypots, and other network devices. The list of officially supported log sources is frequently updated
on the Tenable web site.
5
Prerequisites
It is important to ensure that the prerequisite requirements for LCE are met before beginning installation. These
requirements include:
A CentOS/RHEL OS platform with all unnecessary services disabled
LCE license
LCE management installation (SecurityCenter)
LCE clients (if applicable)
Secure Shell (SSH) key generation
Supported Operating Systems/Platforms
The LCE server component is available for the Red Hat Enterprise Linux (RHEL) and CentOS 5.x and 6.x operating
systems for 32-bit and 64-bit platforms. One or more LCE servers can be installed to operate with a single SecurityCenter
or LCE Manager.
The LCE server can be installed on the SecurityCenter’s host system, but this configuration is not recommended for
performance reasons. If you are using the LCE Manager to manage the LCE server, they can both be installed on the
same system without as much of a performance impact since it does not have any of the vulnerability data that
SecurityCenter manages.
Licenses
LCE servers are licensed to the specific hostname of the system it is to be installed on. There is no licensed limit to the
number of events or IPs that the LCE can be configured to monitor.
There are different licenses available for the LCE based on the total amount of storage used by the LCE. The licenses are
based on 1 TB, 5 TB, and 10 TB storage sizes. The maximum number of silos available to each license size is 103, 512,
and 1024, respectively. There is no difference in the LCE software that is installed, just the maximum storage size that
can be used by the LCE. Data silos are always limited to a maximum size of 10 GB per silo.
SecurityCenter
If the LCE is to be managed by SecurityCenter, you must download and install SecurityCenter, which is available from the
Tenable Support Portal. Please refer to the SecurityCenter documentation for more information on installation and
configuration.
The current versions of both products are required to ensure complete feature integration between LCE
Server and SecurityCenter.
Secure Shell Public Keys
LCE analysis is provided to SecurityCenter through the use of command execution across a Secure Shell (SSH) network
session. When SecurityCenter queries a LCE server, it invokes a SSH session to the configured LCE server. All execution
and analysis of LCE data occurs on the LCE server.
SSH public keys are configured such that SecurityCenter can invoke commands on the LCE server. Non systemadministrator accounts are used to perform these queries. The trust relationship is only needed from SecurityCenter to the
LCE server.
Secure the Log Correlation Engine Server System
It is recommended that the server operating system be locked down before installation to ensure that no unnecessary
services are running. The only service that is required to support remote users is SSH. While the LCE daemon is
operational, it will listen by default on UDP port 514 for syslog messages, UDP port 162 for SNMP, TCP port 601 for
reliable syslog service messages over TCP, TCP port 31300 for the LCE API (needed if LCE clients are
operational), and TCP port 31302 for load balanced LCE servers.
6
The system running the LCE can operate a syslog daemon, but the syslog daemon must not be listening
on the same port(s) that the LCE server is listening on.
LCE Server Installation
If the LCE server is to be managed by SecurityCenter, you must upgrade SecurityCenter to version 4.6.2.2 or
greater to use the new functionality in LCE 4.2. Please contact Tenable Support at support@tenable.com if
you have any questions regarding upgrading SecurityCenter.
Getting Started
Before beginning the LCE installation, it is important to understand the high-level steps required to facilitate a successful
installation. These steps are typically performed in the following order:
1. Download the LCE server RPM and confirm the integrity of the installation package by comparing the downloaded
MD5 checksum with the one listed in the product release notes.
2. Install the LCE server RPM.
3. Copy the LCE license key (lce.key) to /opt/lce/daemons
4. Execute the post install script to configure LCE (/opt/lce/tools/lce-post-install.sh)
5. Start the LCE daemon (service lce start)
6. If the LCE server is to be managed by SecurityCenter, use the SecurityCenter’s web interface to add it as a
resource.
Installation Location
The installation file may be placed anywhere on the installed system. The installation steps described below assume
execution from the same directory where the installation package is located.
Installing the Package
To ensure consistency of audit record time stamps between the LCE and SecurityCenter, make sure that the
underlying OS makes use of the Network Time Protocol (NTP) as described in:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sectDate_and_Time_Configuration-Command_Line_Configuration-Network_Time_Protocol.html
If you are upgrading from a previous version of LCE, please skip this section and see the section titled “Upgrading the Log
Correlation Engine” below. Please follow the instructions in this section for new installations.
As the root user, install the LCE RPM using the following command:
# rpm -ivh lce-4.2.x-es6.x86_64.rpm
An example is shown below:
# rpm -ivh lce-4.2.0-RC1-el6.x86_64.rpm
7
Preparing...
########################################### [100%]
1:lce
########################################### [100%]
The installation process is complete.
Please refer to /var/log/lce_upgrade.log to review installation messages.
This is a new installation. To configure LCE, please run:
/opt/lce/tools/lce-post-install.sh
# /opt/lce/tools/lce-post-install.sh
Beginning in LCE 4.2 you will need to run the post install script “/opt/lce/tools/lce-post-install.sh” as shown
above. This script will prompt for additional information to complete the initial configuration of the LCE server. The script
will attempt to locate a valid license key as /opt/lce/daemons/lce.key and will prompt for the location of the key if it
does not exist in that location. The script will then determine if specific ports are open and available for use. Next, the
script will prompt for networks to include and exclude for statistical analysis. The database directory will be checked for
validity and the approximate free disk space will be reported. If necessary (for example, if there is insufficient disk space),
a different directory may be entered. The IP address and names of syslog sensors are also entered at this time. See the
configuration output below for an example:
------------------------------------------------------------------------------LCE CONFIGURATION
------------------------------------------------------------------------------TENABLE NETWORK SECURITY
http://www.tenable.com
support@tenable.com
Copyright 2003-2013
Welcome to the LCE configuration!
This will assist you in configuring your newly installed LCE.
It should take about a minute to complete.
Press ENTER to continue
The configuration script has detected that LCE is currently running.
It is being shut down so that the configuration can be completed...
Stopping LCE Indexer
Stopping LCE Report Proxy
[
[
OK
OK
]
]
-------------------------------------------------------------------------LCE CONFIGURATION : Key File
-------------------------------------------------------------------------We will now check /opt/lce/daemons/lce.key for validity...
LCE is unable to find a valid key/license in: /opt/lce/daemons/lce.key
It is possible that:
8
* the file doesn't exist or is not a license file,
* the license has expired, or
* the license does not belong to this server.
Enter the full path to your license file, or,
just press ENTER when you have copied it to: /opt/lce/daemons/lce.key
>>
The key in /opt/lce/daemons/lce.key is valid!
------------------------------------------------------------------------------LCE CONFIGURATION : Activation
------------------------------------------------------------------------------In order to receive plugin updates, the Log Correlation Engine
must be activated.
Please enter your activation code:
-------------------------------------------------------------------------LCE CONFIGURATION : Interface Ports
-------------------------------------------------------------------------LCE listens for data on a number of ports.
LCE will check now to be sure that none of those ports are in use.
If any of the ports are in use, you may reconfigure LCE to use a different port,
or stop the service using the port.
The LCE ports will now be checked for validity...
Press ENTER to continue
...checking the syslog port (udp port 514)...
...checking the reliable syslog port (tcp port 601)...
...checking the LCE client-server communications port (tcp port 31300)...
...checking the snmp port (udp port 162)...
...checking the LCE load balancing port (tcp port 31302)...
-------------------------------------------------------------------------LCE CONFIGURATION : Networks
-------------------------------------------------------------------------LCE contains a statistics engine for anomaly detection, and a correlation
engine for advanced alerting.
For best performance, these engines need to know what internal network ranges
to track in your log data, as well as what network ranges NOT to track.
9
First, please configure the networks you wish to INCLUDE in analysis.
Press ENTER to continue
-------------------------------------------------------------------------LCE CONFIGURATION : Networks to include
-------------------------------------------------------------------------Current include-networks:
--------------------------------------Network ranges may be specified in two ways:
1. IP/Netmask - for example, 192.168.0.0/255.255.0.0
2. IP/CIDR
- for example, 192.168.0.0/16
Please enter a network range to include (or press ENTER to quit).
>>
-------------------------------------------------------------------------LCE CONFIGURATION : Networks
-------------------------------------------------------------------------Next, please configure the networks you wish to EXCLUDE in analysis.
Press ENTER to continue
-------------------------------------------------------------------------LCE CONFIGURATION : Networks to exclude
-------------------------------------------------------------------------Current exclude-networks:
--------------------------------------Network ranges may be specified in two ways:
1. IP/Netmask - for example, 192.168.0.0/255.255.0.0
2. IP/CIDR
- for example, 192.168.0.0/16
Please enter a network range to exclude (or press ENTER to quit).
>>
------------------------------------------------------------------------------LCE CONFIGURATION : Vulnerability Reporting
------------------------------------------------------------------------------LCE will provide reports to SecurityCenter containing vulnerability
information. This is done over a secure connection requiring a
10
username and password. Default ones are provided, but it is recommended
that you choose your own now if you have not done so already.
The username and password will be the same as the ones you enter into
SecurityCenter when you add this LCE as a passive scanner for
vulnerability information.
The current USERNAME is "username"
Press ENTER to use this name, or enter a new one now.
>>
The current PASSWORD is "passwd"
Press ENTER to use this password, or enter a new one now.
>>
-------------------------------------------------------------------------LCE CONFIGURATION : Database Directory
-------------------------------------------------------------------------Depending on your license, each LCE may store anywhere from 250 GB to
10 TB of data in the event database.
The database directory will now be checked for validity...
Press ENTER to continue
The current database directory (/opt/lce/db/) has 4 GB free.
If you would like to change the database directory, you may enter it now,
or simply press enter to continue using the current selection.
>>
-------------------------------------------------------------------------LCE CONFIGURATION : Syslog Sensors
-------------------------------------------------------------------------All events analyzed and stored by LCE have an associated sensor name.
For events without a sensor name in the data itself, LCE may still assign
a sensor name you designate based on the IP address of the sender.
If you wish to name IP addresses as particular sensor names, you may do
so now.
This can also be updated later by modifying /opt/lce/admin/syslog_sensors.txt.
The current configured Sensors are:
---------------------------------------
11
--------------------------------------IP Address = ""
Sensor Name = ""
Please enter the IP address of the next Syslog Sensor,
or press ENTER to quit entering Syslog Sensors:
>>
Done entering Syslog Sensors.
-------------------------------------------------------------------------LCE CONFIGURATION : Complete
-------------------------------------------------------------------------Congratulations!
LCE configuration has been completed.
To begin collecting and analyzing data from your network in just minutes,
please refer to the LCE Quick Start Guide. The LCE Administration and User
Guide provides a detailed discussion of advance configuration items in
/opt/lce/daemons/lce.conf, including:
-
Database archiving
Syslog forwarding
Automatic plugin updates
Load balancing across multiple LCE servers
NAT setup for LCE clients
IDS sensors
Processing of usernames and hostnames
Statistical anomaly parameters
Note: If you are running this script directly instead of through
the rpm install, you will need to manually restart the LCE
services yourself by running:
"service lce restart"
After you have accumulated a week of data or more, you should
run the statistical anomaly engine by running:
"service stats restart"
Press ENTER to complete the configuration
The installation process is complete.
Please refer to /var/log/lce_upgrade.log to review installation messages.
#
Once this data is entered, the initial configuration is complete. More detailed configuration may be performed by editing
the /opt/lce/daemons/lce.conf file to include syslog forwarding, load balancing across multiple LCE servers, NAT
setup for LCE clients, and other advanced settings.
12
For more information on large scale deployments, please refer to the Log Correlation Engine 4.2 High
Availability Large Scale Deployment Guide.
The installation process will create a user and group named “lce” and install the LCE to the /opt/lce directory. All files
will be installed with the user and group of “lce” except for the actual lced daemon, which is set-user-id root. This must
be started as the “lce” user, and once the daemon has bound to the appropriate port(s) it will drop privileges. If the lced
daemon terminates abnormally for any reason, the system will automatically restart the daemon and add a warning to the
LCE logs.
Files and Layout
Base Directories
Within the /opt/lce directory are some main tools and various sub-directories including, of particular interest: admin,
daemons, and db.
The admin Directory
This directory contains all of the LCE’s log files. There is a subdirectory named log that contains various log files. System
log file names are based on the format of year and month date such as 2012May.log. Log files in the main log directory
are general LCE log system files. Within subdirectories of the log directory are logs for specific aspects of the LCE such
as clientmanager, indexing, stats, queries, and imports.
The daemons Directory
This directory holds the actual lced binary, its configuration files (lce.conf and rules.conf), and several helper
functions to update the LCE plugins. The LCE Client Manager binary and support files are also located within this
directory and its subdirectories. The license key must also be placed in this directory and renamed to lce.key.
Within this directory is the plugins directory that contains all of the libraries used by the LCE to parse events. When the
LCE loads, it will load all libraries in this directory unless they are disabled.
Within the lce.conf file, the plugins-directory keyword is used to specify the location of the active
directory containing the LCE plugins. Information about LCE plugins and writing them is included in a separate
document entitled “Log Correlation Engine Log Normalization Guide”.
The db Directory
As the LCE is operating, it keeps all of the event data in the db directory. Each silo will be labeled with a
lce(number).ndb and its log_store and db_index_c directories.
When a silo is “rolled” and is no longer the active silo, there will also be a cache and an index field. For example, silo #5
would have lce5-cache and lce5-index directories. These will contain the results from previous cached queries and
indexing.
Installing the License
Hostname Determination
The LCE uses the hostname (not the domain name) of the system it is being installed on. To determine the hostname
needed for a LCE license, simply run the hostname tool and report what is returned. For example:
# hostname
badger
#
13
Key Installation
By default, the demo or commercial license key file for the LCE must be placed on the system running the LCE and
moved to the /opt/lce/daemons directory. The installation script will prompt for the name and location of the key file if
it is not already in place and properly named during installation. It is recommended that you put the license key in place
after the rpm package has been run, but prior to running the post install script.
If a key must be installed manually, it must be named lce.key. It must be readable by user and group “lce”. The lced
process will generate an error at run time if the license is not valid. From the directory where the key was copied to initially, in
this example named tenable-lce255-16-33.key, run the following commands with the appropriate key name:
# cp tenable-lce255-16-33.key /opt/lce/daemons/lce.key
# cd /opt/lce/daemons
# chown lce:lce lce.key
# chmod 640 lce.key
# ls -l lce.key
-rw-r----- 1 lce lce 1285 May 3 15:14 lce.key
# /sbin/service lce start
#
Use of a file transfer program that utilizes “secure FTP” (SFTP) or “secure copy” (SCP) via SSH to transfer the
ASCII key file to the correct location (/opt/lce/daemons/) is recommended if the key originates on a
remote system.
Upgrading the License
It is possible to upgrade from your silo license to one with a higher capacity (e.g., 1 TB to 10 TB). A replacement license
key file will be required. To upgrade your license:
1. Stop the LCE service.
2. Replace the existing /opt/lce/daemons/lce.key with the new, upgraded license key.
3. Edit the file /opt/lce/daemons/lce.conf to change the number of silos to an appropriate number for your
configuration.
4. Start the LCE service.
To verify what key is currently installed, use the grep command to examine the LCE’s log file in /opt/lce/admin/log.
The line with the most recent date will indicate the maximum number of silos permitted with the license:
# grep “number of silos” /opt/lce/admin/log/2013Apr.log
Apr 26, 13 07:41 (LCE Daemon) lced - number of silos is 103 Apr 26, 13 07:41 (LCE
Daemon) lced - number of silos is 512
Apr 26, 13 07:41 (LCE Daemon) lced - number of silos is 1024
The number of silos can indicate the type of license in use. For example, 103 silos indicates a 1 TB license,
512 silos indicates a 5 TB license, and 1024 silos indicates a 10 TB license, when the maximum silos for a
license are used.
14
Configuration
The lce.conf File
The /opt/lce/daemons/lce.conf file is used to specify all LCE configuration parameters. Any changes to this file
must be performed using a text editor. For changes to go into effect, either the lced process needs to be restarted or only
reload the lce.conf file by using the script and commands below. This will reload the LCE configuration file and restart
the ancillary LCE services without halting LCE completely.
#
#
#
#
#
/opt/lce/tools/lce-reload-conf.sh
service lce_indexer restart
service lce_query restart
service lce_report_proxy restart
service stats restart
LCE Configuration
Unless noted otherwise, the following options are specified in the /opt/lce/daemons/lce.conf file:
Option
Description
LCE Basic Configuration Options
key-file
Path to the LCE’s demo or commercial key file.
database-directory
Specifies the location of the LCE database directory. This can be reconfigured to make
use of directories on another filesystem that may have more disk space. Ensure that the
“lce” user has permissions to write and read from any directories outside of /opt/lce.
silo-size
Specifies the maximum amount of data from matched log events that will be stored in
one indexed file (silo). Uses an “M” extension to specify megabytes. For example,
10240M specifies the maximum silo size of 10 Gigabytes. Extensions of “G” specify
gigabytes. For example, 1G specifies 1 gigabyte. By default, this is set to 10G.
Note that the filesystem must support the file size selected within this
setting.
number-silos
Specifies the number of silos that lced will create. The maximum number of silos that
can be created is 1024 for a 10 TB license, 512 for a 5 TB license, and 103 for a 1 TB
license. When configuring this setting, consider the silo-size setting and maximum disk
space available for storage. Example: 1 TB is available for storage and silos
configured for 10 GB would allow for a maximum of 102 silos before disk exhaustion.
store-unnormalizedlogs
Enabling this option will store non-matching logs in the event database. When stored
in this manner, these logs are available to be searched from SecurityCenter text
search feature as an “unnormalized” event type.
server-address
This option allows you to specify the network interfaces that lced will listen on. More
than one interface may be specified on separate lines as in:
server-address 127.0.0.1
server-address 172.0.0.2
By default, this option is commented out and lced will listen on all available network
15
addresses.
public-server-address
If the server is run from behind a device performing network address translation (NAT),
and the LCE clients that it manages are on the public side of the device, the publicserver-address field must be populated with the NAT address so that the managed
clients can connect to it. The LCE Client Manager will use, in order of preference: the
public-server-address setting, the server-address setting, or the first IP
that it finds LCE using that is not 127.0.0.1.
When this setting is used, all managed clients on either side of the NAT
device must use this defined address to connect.
server-port
This option specifies the port number that lced listens on. By default, it is set to
31300, but may be reset to another value.
client-assignment-rule
If a LCE server is multi-homed or the environment has multiple LCE servers, this
setting may be set to configure connecting clients from a particular network to the
appropriate LCE interface or server.
syslog-listen-port
LCE listens for UDP syslog traffic on the standard port of 514 by default. If the
environment requires the LCE to listen on a different port, this setting may be changed.
tcp-syslog-port
This setting determines the port to listen on for reliable syslog messages via the TCP
protocol.
syslog-sensors-file
For logs received via syslog, a sensor name can be assigned to each IP address
sending data to LCE. This sensor name will be associated with all logs from the
designated source, regardless of whether or not another sensor name is extracted
from the log text.
IDS
LCE has the ability to receive IDS events from multiple sources. In addition to being
normalized and stored in the log database, each event will be checked against any
SecurityCenter vulnerability databases. If a host is vulnerable to attack, the event is
marked as such, allowing rules to trigger on this scenario so that the information can
be distributed to the affected administrators.
For each IDS sensor, a sensor name and type must be defined as in the example
below. The supported types are Snort, Bro, RealSecure, Dragon, IntruShield, Juniper,
NetScreen, NFR, Fortinet, PVS, LCE, Cisco, TippingPoint-Sensor, and TippingPointSMS.
sensor-name
Name to be used within the SecurityCenter logs.
sensor-type
IDS sensor type.
Advanced Configuration Options
log-processors
This option leverages multicore processors and determines how many threads will be
dedicated to log processing.
It is recommended that this setting be no higher than the number of CPU cores in the
LCE host system. This is an upper-limit, and should not be changed unless you have
greater than 8 total cores (e.g., a dual quad-core CPU system).
16
For systems with hyper-threading technology, the value may be scaled accordingly.
additional-querymemory
The lce_queryd process is responsible for high-performance processing of all query
requests. By default, up to about 1 gigabyte of memory is used. For systems with large
amounts of available memory, the additional-query-memory option can be used
to allocate additional memory for the query daemon. This will increase the number of
query results that can be cached, improving response time during event analysis in
SecurityCenter. The option can be specified in kilobytes, megabytes, or gigabytes by
appending a “K”, “M”, or “G” to the value.
Do not set his value above 1500M on 32-bit systems.
disk-alert-percentage
When enabled, the disk-alert-percentage option generates an alert when the
LCE is in danger of running out of disk space to store logs. When disk usage reaches
the specified percentage of capacity, a daily TASL alert will be generated.
Event Rules
Specifies actions to execute in response to particular log events or other conditions.
Note: This option is configured in the /opt/lce/daemons/rules.conf file.
discover-hosts
This option enables or disables host discovery. When set to yes, new hosts on the
network will be discovered and reported based on log data.
hosts-learning-period
This option determines how many days a host has not been seen before an alert will be
generated. A setting of at least 1 or 2 days is recommended. After that, any host that
was not discovered during the period will be alerted on as new. Without this setting, LCE
would “discover” all of your hosts that are currently running and are not really “new”.
report-frequency
The frequency, in minutes, in which the report file will be generated and updated on
disk. The default is 60 minutes.
report-lifetime
The lifetime of a report in days. The report will be cleared after this amount of time.
The default is 7 days.
reporter-user
The username that SecurityCenter will use to log in and collect reports from the
reporter module. This username was created when the post install script was run, but
can be updated in the configuration file if necessary.
reporter-password
The password that SecurityCenter will use to log in and collect reports from the
reporter module. This password was created when the post install script was run, but
can be updated in the configuration file if necessary.
reporter-port
The port on which the reporter module will listen for requests to generate and send
report files. The default port is 1243.
ssl
This section lists the filenames for the server key certificate authority, and server
certificate.
ssl {
key_file /opt/lce/reporter/ssl/serverkey.pem
ca_file
/opt/lce/reporter/ssl/cacert.pem
cert_file /opt/lce/reporter/ssl/servercert.pem
}
save-database
When the maximum number of silos has been reached and an older silo must be
overwritten for the next silo roll, the silo to be overwritten can first be saved for future
use.
17
This option specifies whether or not to save the normalized database file.
If there is insufficient disk space on the silo archive device, LCE will no
longer attempt to save a silo before overwriting. If this occurs, log
messages will be generated warning of the event. The event alerting
functionality of LCE can be leveraged to automatically notify concerned
individuals (e.g., email alert) when this sort of event occurs. Please
reference the section of this document titled “The rules.conf File” for
more information.
save-index
This option specifies if the index files are to be saved. The save-database option
must be enabled for this option to be useful.
save-raw
This option specifies if the lce#.raw files are to be saved. These files contain the
original matched log messages before normalization.
location
This option specifies the location to save the data for the previous three options. The
location setting allows the silo to be saved to any mounted volume or other physical
location.
auto-update-prms
auto-update-tasls
The auto-update-prms keyword allows PRM files to be updated automatically by
LCE. The auto-update-tasls keyword does the same for TASL scripts. If the value
is set to “yes”, files will be updated on each silo roll, or every three days, whichever
comes first. The /opt/lce/daemons/lce_update_plugins.pl script allows both
PRM and TASL scripts to be updated from the command line.
proxy-address
proxy-user
proxy-password
Proxy address and credentials used for both manual and automatic updates of LCE
plugins.
accept-letters
accept-numbers
additional-validcharacters
max-usernamecharacters
LCE keeps track of network users on the basis of their usernames. These options set
restrictions on which usernames are considered valid. Any usernames failing to match
the specified criteria are disregarded and “invalid” is reported as the user for the
associated log entries.
accept-letters: This is a yes/no option that specifies whether alpha characters [azA-Z] are allowed.
accept-numbers: This is a yes/no option that specifies whether numbers [0-9]are
allowed.
max-username-characters: Specifies the maximum number of characters allowed
in a username.
additional-valid-characters: Specifies which special characters are
considered valid for usernames. By default, the following characters are considered
valid:
The “dash” character, as in “-”
The “underscore” character, as in “_”
The “dot” character, as in “.”
The “at sign” character, as in “@”
For example, the following address would be considered valid under the default criteria:
18
b.j-smith@a_b.com
Only the special characters that are specified with the additional-validcharacters setting are considered to be valid.
The semicolon character, “;” is not permitted in this context.
excluded-users-file
This option allows you to specify a text file that contains a list of users whose activity
you do not want tracked. This is useful for application user accounts that are not used
to log into the system.
trusted-plugins-file
This keyword allows you to specify a text file that lists the plugins for which user
tracking will be enabled.
max-memory-usage
cache-results-days
pre-cache-file
When a log message is defined in a plugin, LCE provides the option of specifying a
hostname instead of an IP address for the srcip and dstip fields. In this case, LCE
automatically attempts to resolve the provided hostname to an IP address using DNS.
Since the same hostname is typically encountered multiple times, caching the results
of lookups can greatly increase performance. These options configure DNS caching in
LCE. The max-memory-usage option can go up to 360K domain names. The
cache-results-days option specifies the number of days to cache a hostname-toIP mapping before updating the result with a new lookup.
Hosts listed in the pre-cache-file are looked up and stored when LCE starts. The
format for the file is one hostname per line, a line of hostnames separated by a space,
or a combination of the two formats.
For example:
tenablesecurity.com
tenable.com blog.tenable.com www.tenable.com
nessus.org
excluded-domain-name
excluded-domains-file
A particular hostname or all domain names with a certain extension can be excluded
using the exclude-domain-name option. In this case, the matching hosts are looked
up at every occurrence. The excluded-domains-file option can be used to
maintain a more extensive list of domains to exclude by directing LCE to read the list
of domains from a file. The file is read when LCE starts up; therefore, changes to the
list will not be read until LCE is restarted.
lce-load-balancestatus-port
When the LCE server is configured to offload log data to auxiliary servers, TCP port
31302 is the default port used. Uncomment and change the setting here to change the
port on which the LCE server communicates.
primary-lce-address
When used as an auxiliary LCE server, this setting designates the IP address and port of
the primary LCE server in the format of ipaddress:port on which it listens for status
data. The port setting is optional when the primary is using the default of port 31302.
lce-load-balancelocal-addr
When there is more than one network interface available to receive data from the
primary LCE, enter the IP address of the interface to use. Otherwise, the default
interface’s IP address will be used. This can be used to balance bandwidth between
multiple interfaces.
19
mirror-mode
Optionally, instead of receiving a subset of logs, this LCE may register itself as a mirror
and receive ALL logs processed by the primary LCE, effectively creating a live backup
of the primary database. To enable this mode, uncomment this line.
virtual-ip-address
This is the IP address used by devices such as syslog sensors and clients to send
data to LCE.
virtual-ip-interface
When specifying a virtual-ip-address, also specify an existing network adapter
on which the LCE will bind the virtual IP defaults to eth0.
virtual-router-id
If you have a VRRP solution deployed or plan on adding one in the future to the same
network your LCE is deployed on, use this option to specify a router ID for the LCE
cluster, that differs from your other VRRP setup.
load-balance-key
When load balancing between primary and auxiliary LCE servers, all messages are
encrypted. To enhance security, a user-specified key may be added. Uncomment the
option and enter a 32 character encryption phrase.
Allowed characters are alphanumeric and the following characters:
[].^$()|*+?{}/#_-~!@%=`'<>:|&\",
forward-syslog
This option allows you to specify one or more syslog servers to which events are
forwarded.
The format of the option is “forward-syslog IP:port,exclude_header”. If no
port is included, UDP 514 is assumed. The exclude_header option determines if the
LCE header is included on the syslog record that is forwarded. When set to “1”, only
the original log is forwarded without any information that it was forwarded from the
LCE server. The default is to include the LCE header when forwarding logs.
If you have more than one syslog server, specify each on as separate line as follows:
forward-syslog 192.168.10.10
forward-syslog 192.168.10.50:1234,0
forward-syslog 192.168.20.30:514,1
This option is turned off by default.
checksum-forward
Each time the active silo changes, an MD5 checksum is generated for the previous
silo. This allows the integrity of the .raw and .db data files to be checked at a later
time. This option allows the MD5 checksums to be forwarded to one or more off-site
locations. You may specify multiple addresses to forward checksum data on separate
lines as shown:
checksum-forward 192.168.10.10
checksum-forward 192.168.20.30
A new event is logged whenever a set of MD5 silo checksums is calculated. The event
has type “lce” and name “silo-MD5-checksums”. The same message is also logged to
the save-all file.
max-tasl-queue-memory
To maximize performance on multi-processor and multi-core systems, correlated TASL
events are processed in parallel to receive regular incoming events. Since some TASL
scripts can run for an extended period of time, the primary event processor can
potentially receive many TASL-triggering events while a TASL script is still being
executed. In this case, the TASL job is stored in a queue for later processing. This
20
option defines the maximum size of this queue. On systems with extremely large
volumes of data, setting the maximum queue size higher results in increased
performance. If a TASL script that can be sampled is triggered while the queue is full,
its callback functions will not be executed.
sampleable-tasls-file
This option allows you to specify a file listing the filenames of TASL scripts that can be
ignored when the system is busy and the processing queue has reached the specified
maximum amount of memory.
plugins-directory
Specifies the directory where active PRM files are stored.
correlation-scriptsdirectory
Specifies the directory where TASL scripts are stored.
log-directory
Specifies the location of where the ISO compliant LCE log files will be created.
pid-file
When the lced process starts, it will store its process ID file in the file specified here.
die-file
While the lced process is running, it will periodically check for the existence of this
file. If it is found, a graceful exit will be performed.
report-directory
This setting specifies the full path to the directory that stores reports.
add-delimiter
The setting is to allow non-standard frame delimiters for logs received via reliable
syslog. By default, only the linefeed character is recognized. Others may be specified
with the proper ASCII decimal value (0-255). To use multiple delimiters, enter each on
a separate line.
port-restricted-idscorrelation
IDS events are correlated with vulnerabilities based on matching the event’s IP
address, port, protocol, and ID. Ports 0, 22, and 445 are common for client-side
applications. LCE does not require those three ports to match for correlation. Enabling
this option will cause the specified ports to be considered for correlation.
override-sensor-name
The sensor name can be set by the source of the log, the configured sensor name of
the client or syslog source, or the plugin that normalizes the log. If this option is
enabled, the sensor name will always be that of the configured client or syslog source.
auto-authorizeclients-time
LCE Clients version 4 and greater must be authorized by the LCE administrator to
send data after the client attempts to connect to the LCE server. Enable this option to
automate authorization for a specified number of minutes after LCE server startup or
reconfiguration. This automatically authorizes clients that have never previously tried
to connect to the LCE server for 10 minutes after startup.
Debugging Options
save-nonmatched
If this keyword is enabled, LCE will create a file named notmatched.txt in the
database directory and fill it with log events that have not matched any LCE plugin.
This is an excellent way to analyze events that may be inadvertently ignored. There is
a hardcoded limit of 2 GB for this option in addition to the number of events specified.
save-all
Specifics a log file where all events (not just the ones matched with a LCE plugin) are
stored. This log file does not rotate and must be managed by the logrotate process.
Note that this will require significantly more disk space than just keeping the events
that match plugin criteria. This option is most useful when used in conjunction with
logrotate and an external storage device.
21
With the store-unnormalized-logs option introduced in LCE 4.2,
the save-all option is only useful if a text version of all incoming logs is
desired.
debug block
Uncomment this section for additional logging generated by the lced process.
WARNING - the debug logic is extremely verbose and several Tenable
customers have created log files greater than 4 GB. These logs are
purely for diagnostic purposes. It is recommended to only use this debug
mode when directed by Tenable support.
client block
This block of keywords specifies credential information for a LCE client. Clients may be
configured as a single IP address, a CIDR range, ranges in the third and fourth octet of
an IP address, or a range specified by the start and end IP addresses. This option is
not exposed in the configuration file by default.
This option is only for clients prior to version 4.0 connecting to LCE Server
4.0 and newer. Newer clients are managed using the LCE Client Manager.
For more information about the configuration, see the section of the
document titled “Adding Log Correlation Engine Client Information”.
Statistics Daemon Configuration Options (stats_options)
event-directory
Directory containing the LCE events.txt file.
log-directory
Log files are named according to date and stored in the specified directory.
syslog-alert
The stats daemon will generate syslog messages to one or more recipients. By
default, a local address of 127.0.0.1 is used to send messages to the lced services.
However, more than one syslog server can be configured on separate lines in the
following format:
syslog-alert 127.0.0.1
syslog-alert 192.168.10.10
stddev-threshold
This keyword specifies the minimum standard deviation that must occur for an event
before an alert will be generated for it. The higher this number, the more statistically
significant a sequence of events needs to be before an alert is raised.
stddev-unit-threshold
If an event occurs more or less than 5.0 standard deviation units, an alert will be
generated. Setting this value higher will cut down on any sequence of events that
occur close to the standard deviation.
iteration-threshold
This keyword specifies the number of iterations (days) per-event are required before
alerts will be generated. If a large amount of LCE data is already present, set this
number to a low value or even to zero. The stats daemon can be started to read in all
or just part of the existing LCE data. If you have NO LCE data, leave this value around 7
so the stats daemon will not alert on anything until it has 7 days of event data.
22
nhits-frequencythreshold
If an event occurs less than 10% of the time then an alert will be generated. Even if an
event may be statistically significant, that sequence of events may also occur
periodically. For example, 50% of the time you are within a standard deviation, however,
occasionally (the other 50%) you have outliers 2 and 3 standard deviations away. Those
outliers may be the cause of 90% of the alerts generated in this case. Setting this value
to 10, 20, or other values would only alert for hours that were both out of the allotted
standard deviation, and also are event counts that have not occurred before.
Network Range
include-networks
Make sure this range matches IP addresses that are considered
“internal” from an event perspective. Starting with LCE 3.6.1, this range is
used by a number of TASL scripts and the Stats daemon to define
inbound/outbound/internal specifications for LCE events. Prior to 3.6.1,
these ranges were solely used by the Stats daemon.
This is different from the “Directions” filter on the SecurityCenter 4 events
page, which uses the logged-in user’s managed ranges to determine
event direction.
The following sections define your internal network range. All networks specified in the
first section are included, while the exclude-networks block is used to make exceptions.
exclude-networks
Provides exceptions to the “include-networks” directive ranges specified above.
Pointing to a New “Plugins” Directory
To reconfigure the LCE to use only a sub-set of the plugins it is shipped with, create a new directory in the
/opt/lce/daemons directory. For example, we can create an active-plugins directory. The plugins-directory
keyword in the lce.conf file will need to point to this new directory as shown below:
plugins-directory
/opt/lce/active-plugins/
The desired plugins must be manually copied into this new directory and then the lced process restarted.
Adding Log Correlation Engine Client Information
This option is only valid for LCE clients prior to version 4.0 connecting to LCE Server 4.0 and newer. Newer
clients are managed using the LCE Client Manager. See the LCE 4 Client Guide for details, available from the
Tenable Support portal.
To configure a LCE client, add the IP address it will be connecting from and a secret key it will be using to connect with.
The shared secret key may contain any ASCII character except a semicolon (;) or space ( ).
The LCE only supports one type of remote authentication that is based on the shared secret.
All client designations are specified inside the lce_options section of the lce.conf file. The following example
shows an LCE client connecting in from different individual IP addresses with different shared keys:
23
client 192.168.3.7 {
client-auth auth-secret-key s3kret
}
client 192.168.2.1 {
client-auth auth-secret-key myp455word
}
Multiple clients can be added using a variety of methods. The client directive supports a single IP address, an IP
address range (e.g., 10.0.0.1-10.0.0.255 or 10.0.0.1-255) and the use of CIDR notation (e.g., 172.20.1.1/26). The
following examples show an IP range and a CIDR address block used to define the client addresses. Each example also
shows how to define multiple secret keys, any of which may be used when configuring the clients for connecting to the
LCE server from the included IP address range.
client 192.168.16.1-192.168.16.5 {
client-auth auth-secret-key s3kret
client-auth auth-secret-key s3kr3tpa$$
}
client 172.20.101.190/26 {
client-auth auth-secret-key s3kret
client-auth auth-secret-key t0ps3kr3t
}
The client directive can be configured with an optional sensor-name field. This field defines a default sensor name to be
reported for logs from particular clients that do not provide a sensor name to their associated plugin rule. An example of
this designation is shown below:
client 172.20.1.1 {
client-auth auth-secret-key tenable
sensor-name TNS2012
}
To view a sample of the version 3.x LCE Clients configuration, see the version 3.x Clients Configuration
section in the lce.conf.v3_clients file located in the /opt/lce/admin directory of your LCE server.
Debug Mode
The lce.conf file can also include a variety of debug information. Information about plugins loaded, LCE client status,
and operation are all written to the current log file. To place the LCE into debug mode, uncomment the following section of
the lce.conf file:
debug {
lce-api {
client-auth
packets-event
packets-all
}
matches {
24
match-success
match-failure
match-applied
}
silo {
silo-switch
silo-addition
}
match-tree {
tree-construction
tree-matches
}
load-balancing {
status-msg
data-msg
connections
high-availability
}
}
The lce-api keyword logs all remote client authentication attempts. Enabling this can be helpful when diagnosing
remote agent problems. The matches and match-tree keywords detail how a message is analyzed by the existing
plugins. This is a good way to diagnose if a particular plugin is not matching. Lastly, the silo debug keyword logs when a
silo is rotated and indexed.
Enabling these debug messages is a great way to learn how the LCE operates, however, they generate a lot
of information and can create multi-gigabyte log files.
If the lced daemon terminates abnormally for any reason, the system will automatically restart the daemon and add a
warning to the LCE logs.
Correlation Statistics
Runtime statistics pertaining to logging and correlation are collected including:
Logs/bytes per second
Number/percentage of logs matched/unmatched
Number of events correlating with vulnerabilities
Number/percentage of logs from clients, syslog, and IDS
Number of TASL alerts generated
This information is logged once per hour and is written both to the application log and to the normalized database under
the event name “LCE-Server_Statistics” (type “lce”).
Example Correlation Statistics Output:
An average of 50 logs are being received each second.
A total of 5,778 logs (521,046 bytes) have been received.
25
2,232 logs have been matched by plugins (38.63%). 3,546 logs did not match (61.37%).
Log source breakdown: 5,774 from clients (99.93%), 2 via syslog (0.07%), 0 from IDS
devices (0.00%).
No log events have correlated with vulnerabilities.
2 TASL alerts have been generated.
Utilizing Tiered LCEs
Beginning with LCE 4.0, multiple LCEs may be configured in a tiered system. This allows for one LCE to be designated as
the primary LCE, which can send incoming log messages to one or more auxiliary LCE servers (depending on load
requirements, which are automatically calculated on a regular interval). This distributes the storage and processing of the
log messages among up to 256 different LCE servers. Taking advantage of this configuration allows for all the LCE clients
and log sources to be configured for a single LCE server, and that primary LCE server load balances the incoming
requests between itself and its auxiliary servers. Additionally, clients may be configured to send their logs directly to an
auxiliary server, bypassing the primary LCE if there is a need to do so. One example would be if you want all firewall logs
to go to a specific LCE for storage, then they would have their logs point to that specific LCE, bypassing the primary LCE.
Load balancing messages and logs sent between the primary and auxiliary LCEs are encrypted. To provide additional
encryption, the load-balance-key option may be configured. This option can use a phrase between 1-32 characters.
When set, all of the connected LCEs must be configured with the same passphrase in their lce.conf files.
When using tiered LCE servers, each one must be configured in SecurityCenter in order to be queried. If SecurityCenter
user only has access to three out of four LCE servers in a group, that user will receive incomplete results based only on
the data stored in the three LCE servers to which the user has access.
Configuring the Primary LCE Server
The primary LCE server listens on TCP port 31302 (by default) for status data from auxiliary LCE servers. The listening
port of the primary LCE server may be changed by modifying the lce-load-balance-status-port option in the
lce.conf file. There may only be one primary LCE server configured in a group, and servers may not play a dual role of
primary and auxiliary. Unless the server is specifically configured to be an auxiliary LCE server, it considers itself a
primary LCE server and listens on port 31302 (by default).
Configuring the Auxiliary LCE Server
When configured as an auxiliary LCE, the server will accept log files sent to it by the primary. To enable the auxiliary
mode, configure the primary-lce-address setting in the lce.conf file with the IP address and port number of the
primary LCE. If the primary LCE is running on the default of 31302, adding the port number is not required. Append the IP
address with a colon (:) and the port number, such as 192.168.1.20:65444 if the primary LCE server is listening on
the alternate port of 65444.
Note that when utilizing tiered LCE servers, processing of log-related options such as syslog forwarding,
storing not-matched logs, and similar are performed on the server processing the logs. Such options must be
configured identically on all the LCE servers for consistent results.
Syslog Considerations
Sending Syslog Messages to Other Hosts
The LCE can be the focal point of your entire log aggregation strategy. If a Storage Area Network, syslog server, or
some other type of log aggregation solution is deployed in your network, the LCE can be configured to send a copy of any
received message to one or more syslog servers. These messages include any message received from any client.
To configure the LCE to forward these messages, simply enter a line for each syslog server in the lce.conf file with the
forward-syslog keyword. The actual syslog service is not used to forward the messages. All packet generation is
handled by the lced process.
The format of the forward-syslog option is IP:port,exclude-header. The IP is the address of the syslog server to
which the messages are sent. The port indicates the UDP port in which the receiving syslog server is listening. If this is
26
not declared, the default UDP port of 514 is used. The exclude-header option determines if the LCE appends a custom
header to indicate if the messages are sent from the LCE server or not. When omitted or set to “0”, the header is
appended. When set to “1”, the header is not added and only the original log message is sent without indication that it was
forwarded from the LCE server.
The following is an example section of the lce.conf that forwards messages to multiple syslog servers. The first line
forwards to UDP port 514 and appends a LCE server header to each entry. The second forwards to UDP port 1234, and
appends a LCE server header to each entry. The third forwards to UDP port 514 and strips the LCE server header,
sending the original logs to the syslog server.
forward-syslog 192.168.10.10
forward-syslog 192.168.10.50:1234,0
forward-syslog 192.168.20.30:514,1
syslog Compliant Messages
Logs forwarded by the LCE will retain the original syslog alert level and facility, if one was present. If one was not
present, the LCE assigns a log level of “auth.warning”.
Typically, LCE clients do not send syslog compliant messages. If a LCE client were configured to monitor a log file that
retained an original message’s syslog alert level and facility, then this would be retained if forwarded by the LCE.
This allows for a remote syslog server that is receiving events from the LCE to process the received messages and
place them in specific files. Depending on the type of syslog server, it may be possible to place logs from a router into
one file, operating system logs into another and so on.
Content of Forwarded syslog Messages
When the LCE forwards a message, it also adds any matched information to the log file as shown below if configured to
do so:
Jun 30 17:45:36 lce: [not-matched] 0.0.0.0:0 -> 172.20.1.1:0 ::
<37>sshd(pam_unix)[15322]: authentication failure; logname= uid=0 euid=0
tty=NODEVssh ruser= rhost=172.20.1.1
The “::” characters are used to separate LCE’s heading from the original message. In this case, the message would also
have been sent with a syslog facility of <37> since that was the facility of the original message.
Additionally, notice that the LCE tagged the example event above with a not-matched keyword. This means that the LCE
did not possess a .prm file to process the log. If it did, the matched event name would be present in the same location.
If configured to strip the LCE headers from the forwarded syslog messages, only the original log message is sent to the
remote syslog server.
Storing All Logs with “save-all”
Many organizations have regulatory requirements to save all of their log data for 30 days or some other length of time. It
may also be part of that requirement that the data not be manipulated, normalized, or otherwise processed in case it must
be used in a legal proceeding. Any exculpatory evidence in the original logs must not be missing as well.
The LCE’s method of storing data in silos for high-speed normalization and analysis by many different administrators is
not the best place to keep one central log file. The LCE has means to save every message, even ones that do not match
a certain plugin to a central log file.
This log file is saved with the save-all keyword in the lce.conf configuration file. The save-all keyword merely
specifies the path and output directory of a given log file. It defaults to the location of /opt/lce/db/lce.log which is in
the same directory as the silos.
27
As the LCE daemon receives events through the API or from syslog, it will save the message into the file specified by
the save-all keyword. This log file will grow very large. Maintain rotation and compression of these logs with the
logrotate program that is already installed on all Linux systems supported by the LCE.
Different File System
Since the lce.log file will grow to extremely large sizes, it is highly recommended to place this file on a different physical
file system. If the LCE server is placed on a system with two hard drives, consider creating physically separate partitions
for both the LCE silo data and the lce.log files.
If your network has use of a Storage Area Network (SAN), consider using this to store the lce.log file. Many times,
these storage devices can be mounted through a network file system (NFS) or Windows file share (SMB) resource. Make
sure that write permissions from the LCE server are available and there is sufficient network bandwidth to send the data, if
you use a SAN.
Multiple Plugin Matches per Log File “multiple-matches”
By default, the LCE daemon will stop processing a log file as soon as one match has been made. This behavior may be
overridden by adding the multiple-matches keyword in the lce.conf file. With this feature enabled, the LCE daemon
will attempt to exercise the entire plugin set across every log message. This behavior is useful for extracting multiple
forms of information out of a log file. For example, there may be a plugin that looks for a generic user login failure and
another that looks for a login failure for user “root”. Without the multiple-matches keyword, only one of the plugins
will match, even though both are valid.
Even more so than with normal LCE operation, be sure to remove unneeded libraries with the multiplematches keyword enabled, otherwise, the LCE’s performance can be diminished.
Quick Example
Tenable implemented this feature when a customer was encountered who had a firewall log with NAT addresses. For each
transaction, the firewall logged the external Internet address, the customer’s Internet address and their internal RFC1918
address. What they wanted was the ability to type in any of the IP addresses in question to produce a report of the history.
For example, a student may receive 192.168.20.10 via DHCP inside a high school. The school’s public IP address at the
firewall may be 64.64.64.64 and the student may have been attacking a web site at 99.99.99.99.
These “public” addresses were chosen at random and are in no way intended to be example organizations or
potential targets. We did not want to use RFC1918 addresses as example external addresses.
For any network browsing, a firewall log may have all three IP addresses. Without the multiple-match keyword, there
is only one pair of IP addresses that can be matched on. However, with it enabled, two rules can be used to process the
same log file and extract the specific IP addresses.
In the customer’s case, they decided to log “external to public IP” and “public IP to internal IP” firewall logs. They generated
two LCE events for each firewall log event. However, when they added in the DHCP logs, they were able to use the IP
address of a potential attacked target to get the actual internal IP address and MAC address. When someone outside of
their network contacted them and complained of a spammer, worm, or malicious activity, they were able to type in the IP
address of the target, see which public IP was in use at the time, and then see which internal IP addresses were related.
The rules.conf File
The rules.conf file, like lce.conf, is located in the /opt/lce/daemons directory and is used to configure active
response operations used by the LCE daemon. LCE rules are configured to analyze LCE event content and fire if preset
conditions are met. Active responses include the ability to send automatic emails (sendemail), syslog alerts
(sendsyslog), or run custom commands on the LCE system.
28
Email Syntax
sendemail -subject "$Name" -body "$Log" -to "rgula@example.com,joe@example.com"
Syslog Syntax
sendsyslog 172.20.1.1 -message "$Log"
Custom Command Syntax
my_custom_firewall_reconfig_command -block $sip
LCE Rule Filters
The following fields are optional filters. A plus sign signifies that events matching the specified values will receive rule
application, while a minus sign signifies that matching events will not. If no “+” filter is used, all events are matched by
default for the field, unless excluded specifically with the “-” filter. Multiple values can be specified for any filter.
Do not use spaces to precede LCE rules. If there is a space at the beginning of an option, that option will be
ignored.
Option
Description
IPS
This filter allows for the search of IP addresses that are or are not present as either
source or destination. The following five formats are supported for both +IPS and -IPS:
SrcIPS
This filter will search for source IP addresses that are or are not present. The following
five formats are supported for both +SrcIPS and –SrcIPS:
DstIPS
172.16.1.1/255.255.255.0
172.16.1.1/32
172.16.1.1-255
172.16.1.1-172.16.1.255
172.16.1.1
172.16.1.1/255.255.255.0
172.16.1.1/32
172.16.1.1-255
172.16.1.1-172.16.1.255
172.16.1.1
This filter will search for destination IP addresses that are or are not present. The
following five formats are supported for both +DstIPS and –DstIPS:
172.16.1.1/255.255.255.0
172.16.1.1/32
172.16.1.1-255
172.16.1.1-172.16.1.255
172.16.1.1
Events
Considers both the primary and secondary event names. The “Events” field allows
spaces in event names (because Nessus IDS signatures contain spaces), and thus
events must only be separated by commas and not spaces. Spaces, commas or both
may be used to separate entries in the other fields.
Sensors
Sensor that detected the LCE event
29
Types
LCE event type
Ports
Source or destination port within the LCE event
Protocols
Specified by TCP, UDP, ICMP or a number
Users
Username associated with the event
Text
Filter on any text token in the log that is or is not present (tokens can include spaces
and punctuation but not commas) by using +Text or –Text.
IText
This is the same filter as above but the token can be case insensitive, and +IText or –
IText must be used.
Vulnerable
“yes” or “no”
Ignore
Single keyword causes all events matching the rule’s filters to be ignored by LCE. If an
event is ignored in this manner, there will be no LCE database entry written for it, no
other matching rules will fire and no TASLs filtering on the event will be executed.
Ignored events are only evidenced through the save-all file.
RateLimit
A string indicating the maximum number of event responses per time period that will
be allowed. When the quantity of incoming matching logs exceeds this constraint, the
remaining logs will be queued or ignored. This string follows the format:
(integer) per [second, minute, hour, day, week, month, year]
Command
Runs the given command at the command line as user “lce” (i.e., echo "log
matched" >> /opt/lce/my_log_file.log).
See the /opt/lce/tools/directory for a few tools supplied with LCE for emailing
logs, forwarding them via syslog, etc.
When using “Command:” to run a command, you may insert some or portions of the log
into your command using the following replacement macros. The following example
sends the original log text and the src IP:port dst IP:port via syslog to another
machine for network or connection type logs:
Name: Example command
+Types: network,connection
Command: /opt/lce/tools/send_syslog 192.168.1.1 -message "LOG
MATCHED RULE ($sip:$sport -> $dip:$dport) [$log]"
For more Command examples please review Appendix 2: Sample LCE
rules.conf File.
MaxQueue
The maximum number of matching events to queue; those coming in while the queue
is full will be ignored.
30
Threshold
A string indicating the minimum number of matching events that must occur in a given
time period before event responses are generated. This string follows the format:
(integer) in a [second, minute, hour, day, week, month, year]
LCE Shell Command Options
The following case sensitive variables may be included in the shell command string:
Option
Description
$sip
Source IP of event
$dip
Destination IP of event
$sport
Source port of event
$dport
Destination port of event
$proto
Protocol of event, displayed as N/A, TCP, UDP, ICMP or a number for other protocols
$vuln
“no” if the event was not correlated with a vulnerability, “yes” otherwise
$sensor
Name of sensor generating the event
$event1
Primary event name
$event2
Secondary event name
$type
Type name of event
$time
Time event was recorded at LCE (format: Mon MM, YYYY H:M:S)
$user
Username associated with the event
$log
Raw text of log
$queued_logs
All logs currently in the event rules queue. Use of this variable has the effect of
emptying the rule’s queue
An example of the LCE rules.conf file can be found in Appendix 2: Sample LCE rules.conf File.
Email/Alerting/Execution
LCE can be configured with the ability to interpret received log events based on log content and use configurable rules to
generate active responses from the LCE server. These rules are configured on the LCE server in the
/opt/lce/daemons/rules.conf file and can perform three primary responses:
email alerting
31
syslog alerting
command execution
The LCE server will generate email alerts using the settings found msmtp.conf file, which can be located in
the /opt/lce/tools/ directory on the LCE server. This file will need to include your email server
information for alerting to function correctly.
Examples of practical applications include configuring rules to rate limit certain types of log events, email administrators
immediately when an attack is detected, and send customized commands to a firewall when an inbound attack is detected
and firewall reconfiguration needs to take place.
Various fields within the received log alert are automatically placed in variables that may be used as parameters within the
active response. For example, consider the following rules.conf configuration:
Name: DMZ Login
Matching-IPS: 192.168.20.15,192.168.20.100,192.168.20.110-112
Event: SC4-Login
Command: sendemail -subject "$Name" -body "$Log" -to
"rgula@example.com,joe@example.com"
RateLimit: 5m
This rule takes LCE events labeled “SC4-Login” to the specified IP addresses and automatically generates an email alert
to the specified administrator email addresses. In addition, a rate limit is applied such that only one email would be sent
every five minutes to prevent the LCE server from overwhelming the email server system. Configuration possibilities are
limited only by the imagination of the LCE server administrator.
LCE Operations
At any time, the version of the lced binary can be determined by running it with the -v option, as follows:
# /opt/lce/daemons/lced -v
Log Correlation Engine version 4.2
#
Use the following command to see how the LCE is configured during Linux startup and shutdown (installation defaults are
shown):
# chkconfig --list lce
lce
0:off
#
1:off
2:on
3:on
4:on
5:on
6:off
To change how the LCE will behave during Linux startup and shutdown use the following command:
# chkconfig [--level <levels>] lce <on/off/reset>)
Please refer to your own Red Hat Linux documentation on how to use chkconfig in conjunction with Linux run levels to
configure the LCE startup and shutdown to your requirements.
Starting LCE
The RPM installation places a LCE start-up (/etc/rc.d) script in /etc/rc.d/init.d.
32
Use the following command to start the LCE:
# service lce start
If the lced daemon terminates abnormally for any reason, the system will automatically restart the daemon and add a
warning to the LCE logs.
Halting LCE
Similarly, the /etc/rc.d script can be used to halt the LCE and gracefully exit any log analysis or log writing it is
performing. Use the following command to stop the LCE server:
# service lce stop
Restarting LCE
The /etc/rc.d script can be used to restart the LCE, gracefully exiting any log analysis or log writing it is performing
and starting the LCE again. Use the following command to restart the LCE server:
# service lce restart
Determine LCE Status
The /etc/rc.d script can be used to determine the status of the LCE components and their PIDs. Use the following
command to acquire the status of the LCE server processes:
# service lce status
Operating the stats Daemon
Although this document does not cover all aspects of the stats daemon, a separate RC script is included in the LCE
RPM for starting and stopping the daemon. Use the following commands to stop and start the stats daemon:
# service stats stop
# service stats start
# service stats restart
# service stats status
Updating Plugins (PRM Files) and TASL Scripts
This section describes methods for updating LCE plugins (files with a .prm extension) and TASL scripts.
Automatic Plugin (PRM Files) and TASL Updates
Plugin updates occur automatically after every silo roll unless explicitly disabled in the LCE configuration file. Tenable
Network Security strongly recommends that PRM files and TASL scripts be updated automatically as part of the LCE’s
maintenance processes or updated directly with the lce_update_plugins.pl script located in the
/opt/lce/daemons directory.
33
The script will automatically look at each PRM and TASL that is present in the plugins directory and compare it to the one
online at Tenable’s web site. If there is a difference, the script will download and install any new scripts and optionally
restart the LCE.
Usage of the lce_update_plugins.pl script can be found on the command line by running the script without any
options or using the “–h” (help) option.
Updating Individual PRM Files
Individual PRM files can be updated using lce_update_plugins.pl located in the /opt/lce/daemons directory.
This script offers several options for updating PRM files and TASL scripts, which are displayed using the “-h” (help)
option as shown below:
# /opt/lce/daemons/lce_update_plugins.pl -h
LCE Plugin Updated Help Screen
Usage: ./lce_update_plugins.pl -OPTIONS
Available options:
-a (download & update all latest plugins from Tenable)
-p (download & update latest PRM plugins from Tenable)
-t (download & update latest TASL scripts from Tenable)
-c (download & update latest client packages from Tenable)
-y (download & update latest client policies from Tenable)
-x (download & update latest threat list data from Tenable)
-o (download & update only plugins/TASL scripts available in corresponding
directories)
-i (Only inspect if new plugins are available. This switch will report plugins
that changed, but will not update them.)
-s (silent mode)
-v (verbose mode)
-D (default mode - equivalent to -aov)
-f (force LCE to reload plugins, TASLs, policies, etc regardless of whether
underlying content has been modified)
Examples:
./lce_update_plugins.pl -a
(Update all plugins/TASL scripts)
./lce_update_plugins.pl -as (Update all plugins and remain silent)
./lce_update_plugins.pl -aov (Update plugins/TASL scripts available in corresponding
directories and report all actions)
./lce_update_plugins.pl -aiv (Check for updated versions of all installed plugins, but
do not update them, and report all actions)
./lce_update_plugins.pl -tiv (Check for updated versions of all installed TASL
scripts, but do not update them, and report all actions)
The directories containing the PRM files and TASL scripts are specified in the lce.conf file with the “pluginsdirectory” and “correlation-scripts-directory” keywords. When the lce_update_plugins.pl script is
manually invoked, the files contained in these plugins and correlation scripts (TASL) directories will be archived to the
/opt/lce/daemons/plugins_archive directory. The backups of the files in the TASL directory will appear in the
plugins_archive directory as a time stamped file such as tasls_2008091222457616.tar.gz, and the backups of
the files in the plugins directory will appear in the plugins_archive directory as a time stamped file such as
plugins_2008091222457421.tar.gz.
For example, if the script is invoked with the -p option, only the PRM files will be updated, without affecting the TASL
scripts. In this case, all files in the plugins directory will first be archived to the plugins_archive directory before
34
downloading and updating the latest plugins. Likewise, invoking the script with the -t option will only update the TASL
scripts without affecting the PRM files. In this case, all files in the correlation scripts (TASL) directory will first be archived
to the plugins_archive directory before downloading and updating the latest TASL scripts. Invoking the script with the
-a option will first archive all files in both the plugins and correlation scripts (TASL) directories before downloading and
updating all the latest PRM files and TASL scripts.
Excluding PRM Files
In some cases, a user may wish to allow the global updates of PRM files, but specifically exclude some from being run.
This can be facilitated by using the /opt/lce/admin/disabled-prms.txt file. The PRM files to be processed but not
loaded can be specified in this file, without a pathname, separated by any amount or type of whitespace.
If there is a need to customize a plugin or plugins, rename the original file before making modifications. Once done,
include the name of the original plugin in the disabled-prms.txt file. If an existing PRM file is modified and not
renamed, it will be overwritten on the next PRM update. If the original is not disabled, and the multiple-matches
option is not enabled, only one of the two PRM files will match.
Excluding TASL Files
TASLs may be disabled selectively by adding the TASL script file name (e.g., program_accounting.tasl) to the
/opt/lce/admin/disabled-tasls.txt file and then restarting the LCE daemon. This is useful for cases where a
particular TASL script is not needed by an organization or where the TASL might be causing performance issues and
needs to be disabled either temporarily or permanently.
Any disabled TASLs can be re-enabled by removing them from the disabled-tasls.txt file and restarting the LCE
daemon.
Managing Client Configuration Files
Starting with version 4.2 of the LCE clients, the client configuration files are managed centrally from the SecurityCenter
4.6 or LCE server using the /opt/lce/daemons/lce_client_manager command line utility. This allows a central
server to manage the configuration files of all the deployed LCE clients that are configured for the server.
For more information on this option, see the LCE Clients Guide available from http://support.tenable.com.
Upgrading LCE
The LCE is upgraded simply by using the “rpm” command with the “-U” switch to force an upgrade. The LCE stops and
starts the service during the upgrade process, which makes a manual stop/start unnecessary. After the upgrade is
completed it is recommended that you run the /opt/lce/daemons/activate.sh script. The script will require your
LCE activation code. The activation code for LCE can be located by logging into the Tenable Support Portal and selecting
“Activation Codes” from the menu on the right followed by “Log Correlation Engine”. After you have entered the activation
code, the script will complete once the plugin update is finished.
# rpm -Uvh lce-4.2.x-el6.x86_64.rpm
Preparing...
########################################### [100%]
1:lce
########################################### [100%]
New configuration options have been added to /opt/lce/daemons/lce.conf.
Please review the release notes and user guide for details.
The installation process is complete.
Please refer to /var/log/lce_upgrade.log to review installation messages.
The Log Correlation Engine must be activated in order for plugin updates to be
retrieved.
To activate, please run /opt/lce/daemons/activate.sh. #
35
# /opt/lce/daemons/activate.sh
Enter activation code: XXXX-XXXX-XXXX-XXXX-XXXX
Registering activation code XXXX-XXXX-XXXX-XXXX-XXXX..
Your plugins are now being updated...
[LCE-Updater]: Disabling LCE Client update - there are no new changes.
[LCE-Updater]: The following files changed:
You will also need to edit the lce.conf file located in /opt/lce/daemons directory to include the username and
password you will be utilizing for the Event Vulnerability Data feature now available with LCE 4.2. The section of the
lce.conf file is shown below:
# The username and password that SecurityCenter will use to login
# and collect reports from the reporter module.
reporter-user username
reporter-password password
The same username and password will need to be added to your LCE configuration in SecurityCenter 4.6.2.2 as
explained in the “Adding the LCE to SecurityCenter” section of this guide.
Additional Features
Importing LCE Data Manually
LCE data can be collected both via real-time logging and manually in batch mode using the “import_logs” tool. These
events will show up in the normalized event view along with events collected in real-time. This command-line tool allows
data to be imported into the LCE that may not be available in real-time, but is still important for correlation of vulnerability
data and for analysis of security posture and events.
Usage:
# /opt/lce/tools/import_logs <list of log files and directories to import> [-d, -disable-rules] [-a, --approximate-timestamps] [-c, --current-time] [-o, -output-prefix <prefix>]
Each item in the <list of log files and directories to import> is a file name or directory name. A directory name may or not
end with a slash. For example:
# /opt/lce/tools/import_logs /directory1 file1 file2 /directory2/
Directory imports are non-recursive.
The following table describes the options available for import_logs:
Option
Description
-d –disable-rules
Do not apply LCE event rules to imported logs.
-a, --approximatetimestamps
If no timestamp can be determined for an event, assign the most recent known
timestamp.
36
-c, --current-time
Use the current system time for all imported logs rather than the timestamps contained
within the event text.
-o, --output-prefix
<prefix>
Use the specified prefix when naming newly generated silos. For example, the “-o
Snort” option will generate silos with names like SnortJun142009Aug242009.db.gz. The default prefix is “lce”. This option can aid in the process of
searching for logs created by a particular import instance.
The log importer tool logs its actions to /opt/lce/admin/log/importer and archives within this directory can be
checked in the event that an import does not execute as expected.
The log import tool only supports importing logs into an archived silo.
User Tracking
The LCE server has a feature that is designed to track users. User tracking can be applied to any event coming into the
LCE server, regardless of the source of the event. Events correlated from Windows, Linux, Unix, or other network devices
can be monitored.
When LCE encounters a log that has no username field, it will assign the username of the user most recently associated
with the source IP of the incoming log, or associated with the destination IP of the log if a destination IP (dstip) is provided
but a source IP (srcip) is not. If no user was previously tracked at either of the IPs, or if no IP is provided, an “(unknown)”
entry is assigned.
37
When a user changes IP addresses (i.e., a LCE receives a log where the user’s srcip differs from the srcip in the previous
log tagged with the username), the new IP address is also associated with the user. The last three IP addresses per user
are stored for the user, allowing for cases where a single user logs into multiple systems at the same time. For example,
the following event shows a user becoming active at a new IP address:
Network user IP address change: user someguy94 became active at 169.254.96.232 with
event login (169.254.96.232:0)
The data used to track usernames is stored in the files usernames.txt, ip_user.dat, and user_ip.dat in the LCE
database directory. The .dat files are written when the LCE service is shut down gracefully. In case of a server crash,
the data is automatically backed up every 10 minutes.
A maximum of 65,534 unique usernames can be stored. If the maximum is reached, incoming logs with new users will
have the user fields marked with the “(unknown)” entry.
User tracking in LCE will function if the following conditions are met:
The LCE server has plugins that can match the events and pull usernames from the events. For example, plugin
3209 in os_win2k_sec.prm has the following line:
log=event:Windows-Account_Used_For_Login sensor:$1 dstip:$2 type:login user:$4
event2:WindowsEvent-680
The “user:$4” directive tells the plugin to add the username to the available event searchable fields. As a result,
searches that query this event based on the username will return results.
The plugin IDs have been added to the /opt/lce/admin/trusted_plugins.txt configuration file (one
plugin ID per line).
A list of the plugins provided by Tenable that include user information is found at the end of
/opt/lce/daemons/plugins/prm_map.prm.
The user tracking settings have been properly configured in the lce.conf file. Please refer to the Configuration
section of this document for a description of the following applicable keywords:
-
accept-letters
-
accept-numbers
-
additional-valid-characters
-
max-username-characters
If these conditions are not met, usernames may still be stored in normalized events; however, they cannot be searched
using the event filter “username” parameter. Another way to search for usernames in logs is through the raw log search
feature of SecurityCenter described below.
Working with SecurityCenter
Adding the LCE to SecurityCenter
To add your LCE server to SecurityCenter, log into SecurityCenter as the admin user and click on “Resources” and then
“Log Correlation Engines”. A screen similar to the one below is displayed with the currently available LCE servers.
38
The “Add” button displays a dialog box with the following fields:
Option
Description
Name
The unique name that this LCE server will be known as.
Description
Descriptive text for the LCE server.
Host
The IP address of the LCE server.
When the SecurityCenter resides on the same host as the LCE server, it
is recommended to use the localhost IP address of 127.0.0.1.
Organizations
Select the customer that this LCE is assigned to from the drop down menu.
Event Vulnerability Data
Import Vulnerabilities
Selecting this box will allow you to configure your LCE use Event data to detect
vulnerabilities.
Repositories
This will allow you to select which repository you would like to keep the vulnerability
data collected from LCE events.
Event Vulnerability Host
Host
This is the IP address of your LCE server.
Port
This allows you to configure the port used for communication between SecurityCenter
and LCE. The default port is 1243.
Username
This is the username that was set when running the post install script.
Password
This is the password set during the post install script.
An example of this screen is shown below:
39
After clicking on “Submit”, the LCE admin credentials (“root” user or equivalent) are requested to establish an
authenticated session between SecurityCenter and the LCE. After the LCE server is successfully added, highlight the new
LCE server to display options pertinent to that server.
If company policy prohibits remote root login, a manual key exchange process can be used. The key file can be
generated manually by the “tns” user on the SecurityCenter using ssh-keygen or with the SecurityCenter
administrator web interface (“System” -> “Keys”). The key would be copied between systems by an authorized
user and then installed as described in the “Appendix 4: Manual SC4/LCE Key Exchange” section of this
document.
If you are using DNS in your environment, make sure it is configured for reverse DNS resolution to facilitate
query speeds. If you are not using DNS, modify the /etc/hosts file to include your SecurityCenter IP
address and hostname. For example:
192.168.1.22 SecurityCenter4.example.com SecurityCenter4
More information about SecurityCenter configuration options is available through the “SecurityCenter Administration
Guide” available on the Tenable Support Portal.
Configuring Organizations
As a SecurityCenter administrator, LCE servers can be associated with various organizations. Through the web interface,
SecurityCenter can be configured such that users of specific organizations can make queries to each LCE server. This is
documented in the SecurityCenter documentation.
40
Analyzing Security Events
A wide variety of LCE analysis and reporting tools are available to SecurityCenter users. These users can make use of
any LCE event that intersects with their range of managed IP addresses. All analysis and reporting options are described
in the “SecurityCenter 4 User Guide”.
Identifying Vulnerabilities
A new feature in LCE 4.2.0 leverages log data to find vulnerabilities. The Tenable plugins that report this information will
have the plugin ID range of 800,000 - 899,999. A sample screen capture of data that can be found is shown below:
You can filter for the vulnerabilities identified by LCE in SecurityCenter by using the ID Filters and selecting Plugin ID,
then selecting “>=” and then entering “800000”.The filter setting is pictured below:
TASL Scripts
Sending IDS events to the LCE allows the LCE to run the PRM and .prmx rules to normalize the events and TASL scripts to
provide further event processing. TASL scripts are used for many types of detection events such as thresholds, successful
attack detection, and alerting. By default, all TASL scripts are included on the LCE server; however they can be disabled
manually using the /opt/lce/admin/disabled-tasls.txt file described in detail earlier in this document.
41
For more information regarding TASL scripts review the LCE 4.2 TASL Reference Guide.
Full Text Searches
Full text searches may be performed on the data stored within the attached LCE servers. When viewing the events page
the Search field will accept text strings as valid search criteria. Additionally, when using LCE server 4.0.1 and newer,
search terms are case insensitive and Boolean searches may be utilized to further enhance search results. This enables
searching the raw logs for details contained in the events.
42
For More Information
Tenable has produced a variety of additional documents detailing the LCE’s deployment, configuration, user operation,
and overall testing. These documents are listed here:
Log Correlation Engine Architecture Guide – provides a high-level view of LCE architecture and supported
platforms/environments.
Log Correlation Engine Quick Start Guide – provides basic instructions to quickly install and configure an LCE
server. A more detailed description of configuration and management of an LCE server is provided in the “LCE
Administration and User Guide” document.
Log Correlation Engine Client Guide – how to configure, operate, and manage the various Linux, Unix, Windows,
NetFlow, OPSEC, and other clients.
LCE High Availability Large Scale Deployment Guide – details various configuration methods, architecture
examples, and hardware specifications for performance and high availability of large scale deployments of
Tenable's Log Correlation Engine (LCE).
LCE Best Practices – Learn how to best leverage the Log Correlation Engine in your enterprise.
Tenable Event Correlation – outlines various methods of event correlation provided by Tenable products and
describes the type of information leveraged by the correlation, and how this can be used to monitor security and
compliance on enterprise networks.
Tenable Products Plugin Families – provides a description and summary of the plugin families for Nessus, Log
Correlation Engine, and the Passive Vulnerability Scanner.
Log Correlation Engine Log Normalization Guide – explanation of the LCE’s log parsing syntax with extensive
examples of log parsing and manipulating the LCE’s .prm libraries.
TASL Reference Guide – explanation of the Tenable Application Scripting Language with extensive examples of a
variety of correlation rules.
Log Correlation Engine Statistics Daemon Guide – configuration, operation, and theory of the LCE’s statistic
daemon used to discover behavioral anomalies.
Log Correlation Engine Large Disk Array Install Guide – configuration, operation, and theory for using the LCE in
large disk array environments.
Example Custom LCE Log Parsing - Minecraft Server Logs – describes how to create a custom log parser using
Minecraft as an example.
Documentation is also available for Nessus, the Passive Vulnerability Scanner, and SecurityCenter through the Tenable
Support Portal located at https://support.tenable.com/.
There are also some relevant postings at Tenable’s blog located at http://www.tenable.com/blog and at the Tenable
Discussion Forums located at https://discussions.nessus.org/community/lce.
For further information, please contact Tenable at support@tenable.com, sales@tenable.com, or visit our web site at
http://www.tenable.com/.
43
Appendix 1: Sample lce.conf File
lce_options {
# Do not modify the "Quick-install" area prior to installation!
# Quick-install customizations
# End of quick-install customizations
# <------------------- Basic Configuration Options ------------------->
# <------ License Key Location ------>
key-file /opt/lce/daemons/lce.key
# <------ Active Data Storage ------>
# The database-directory partition should contain enough free disk
# space to store the specified amount of active data.
database-directory
/opt/lce/db/
# The silo-size variable specifies the maximum amount of
# data from matched log events that will be stored in one
# indexed file. This may be set to at most 10GB. The option
# must be specified in kilobytes, megabytes, or gigabytes
# by appending a 'K', 'M', or 'G' to the value. For example,
# 512M indicates a 512 megabyte maximum silo size.
# The 'number-silos' variable specifies the number of silos.
# With a silo-size of 1G and number-silos set to 3, the
# system would use a maximum of 3 gigabytes of total storage.
silo-size
10G
# The 10 terabyte, 5 terabyte, and 1 terabyte LCE licenses support
# a maximum of 1024, 512, and 103 silos, respectively. The 250GB
# license supports a maximum of 25 silos.
number-silos
1024
# The next setting provides the option of storing non-matching logs in
# the event database. When enabled, these events will appear in
# SecurityCenter under the "unnormalized" event type. Opting to store
# the logs in this manner will make them available for text searches,
# but will require additional disk space.
store-unnormalized-logs yes
# <------ Network Setup ------>
#
#
#
#
#
#
#
#
The next section determines the interfaces on
which LCE will listen for both syslog messages and
connection requests from clients. For each address
specified, LCE will utilize the corresponding interface.
If this option is used, all clients must be configured
to connect to one of the addresses below. If no addresses
are specified, LCE will listen on all available interfaces.
server-address 127.0.0.1
# If the server is run from behind a device performing
# network address translation as it will be seen from
# the clients, then the public-server-address field
44
# must be populated with the address to which the clients
# should attempt to connect.
# public-server-address 127.0.0.1
#
#
#
#
#
The server-port option allows specification of a port
number on which LCE will listen for connection requests
from clients. The clients must be configured to connect
to the port indicated below.
server-port
31300
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
If your LCE server is multi-homed or you wish to
have more control over the destination IP that LCE 4.x
clients will be assigned when they first connect to this
server, then you may specify rules here to override
the other server assignment logic.
When new clients first connect, the LCE server will search
this list, and assign the designated LCE server IP or hostname
to the client automatically.
Client ranges may be specified as IP/cidr or IP/netmask.
Server assignments may be IP:port or hostname:port, where
the port is optional and defaults to the port specified in
server-port above.
For example, this assigns all 203.0.113.x clients to a server named
My_LCE_Server, on port 5000:
client-assignment-rule 203.0.113.0/24 My_LCE_Server:5000
# <------ Syslog Configuration ------>
#
#
#
#
#
#
The syslog-listen-port option allows the user to
specify a non-default UDP syslog port. Normally this
is port 514, and if it is not specified, LCE will
default to listen on port 514 for each server-address
interface specified above.
syslog-listen-port
514
# The following option allows specification of the port
# number on which to accept reliable syslog messages.
# tcp-syslog-port 601
# For logs received via syslog, a sensor name can be assigned to each
# IP address sending data to LCE. This sensor name will be associated
# with all logs from the designated source, regardless of whether or
# not another sensor name is extracted from the log text.
syslog-sensors-file /opt/lce/admin/syslog_sensors.txt
# <------ IDS Configuration ------>
#
#
#
#
#
#
#
The Log Correlation Engine has the ability to receive IDS events from
multiple sources. In addition to being normalized and stored in the
log database, each event will be checked against any Security Center
vulnerability databases. If a host is vulnerable to attack, the event
is marked as such, allowing rules to trigger on this scenario so that
the information can be distributed to the affected administrators.
For each IDS sensor, a sensor name and type must be defined as in the
45
#
#
#
#
#
#
#
example below. The supported types are Snort, Bro, RealSecure,
Dragon, IntruVert, IntruShield, Juniper, NetScreen, NFR, Fortinet,
PVS, LCE, Cisco, TippingPoint-Sensor, and TippingPoint-SMS.
IDS 192.168.1.2 {
sensor-name Corporate_Snort
sensor-type Snort
}
# <----------------- Advanced Configuration Options ----------------->
# <------ CPU, Memory, and Disk Utilization ------>
# The Log Correlation Engine is capable of leveraging multiple
# processor cores in order to process a number of incoming logs
# in parallel, yielding higher throughput. The following setting
# limits how many threads will be dedicated to log processing.
# For maximum perfomance, this should generally be set to or above
# the total number of CPU cores available on your system. For systems
# utilizing hyper-threading technology, the value can be scaled
# accordingly.
log-processors 8
#
#
#
#
#
#
#
#
#
#
#
The lce_queryd process is responsible for high-performance processing
of all query requests. By default, up to about 1 gigabyte of memory
is used. For systems with large amounts of available memory, the
following option can be used to allocate additional memory for the
query daemon. This will increase the number of query results that
can be cached, improving response time during event analysis in
SecurityCenter. The option can be specified in kilobytes, megabytes,
or gigabytes by appending a 'K', 'M', or 'G' to the value.
Note that this value should never be set above 1500M on 32-bit
systems.
additional-query-memory 1G
# The following option allows an alert to be generated when LCE becomes
# in danger of running out of disk space to store logs. When disk usage
# reaches the specified percentage of capacity, the LCE-High_Disk_Usage
# event will be added.
disk-alert-percentage 75
# <------ Event Rules ------>
# To specify actions to execute in response to particular log events
# or other conditions, edit /opt/lce/daemons/rules.conf.
# <------ Discovery Options ------>
# The following option enables or disables host discovery. When set to
# yes, new hosts on your network will be discovered and reported based
# on log data.
discover-hosts yes
#
#
#
#
Alert on new hosts not seen after running for 'N' days.
A setting of at least 1 or 2 days is recommended. After
that, any host that was not discovered during the period
will be alerted on as new. Without this setting, LCE would
46
# 'discover' all of your hosts that are currently running and
# not really 'new'.
hosts-learning-period 2
# The frequency, in minutes, in which the report file should be
# generated and updated on disk.
report-frequency 60
# The lifetime of a report in days. The report will be cleared
# after this amount of time.
report-lifetime 7
# The username and password that SecurityCenter will use to login
# and collect reports from the reporter module.
reporter-user username
reporter-password passwd
# The port on which the reporter module will listen for requests
# to generate and send report files.
reporter-port 1243
# This section lists the filenames for the server key, cert authority,
# and server cert.
ssl {
key_file /opt/lce/reporter/ssl/serverkey.pem
ca_file
/opt/lce/reporter/ssl/cacert.pem
cert_file /opt/lce/reporter/ssl/servercert.pem
}
# <------ Data Archiving ------>
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
When the maximum number of silos has been reached and an older silo
must be overwritten for the next silo roll, the silo to be
overwritten can first be saved for future use. The following options
determine whether the database, index and original log text are
saved. Valid options for each setting are "yes" and "no". The
location setting allows the silo to be saved to any mounted volume
or other physical location.
Saving the database alone is sufficient to allow most queries on the
archived data. Retaining the original log text in addition will
permit the use of the Raw Syslog tool. Saving the index will allow
queries on archived data to execute much faster, but consume
additional storage space.
save-database yes
save-index
yes
save-raw
yes
location
/opt/lce/silo_archive/
# <------ Automatic Updates & Proxy Settings ------>
#
#
#
#
The 'auto-update-prms' keyword allows PRM files to be automatically
updated. When enabled, LCE will update the installed PRM files with
the latest versions from Tenable, as well as download any newly
published plugins. Both this and the TASL update are launched
47
# whenever a silo rolls, or every 3 days, whichever occurs first.
auto-update-prms yes
# The 'auto-update-tasls' keyword allows TASL scripts to be
# automatically updated. When enabled, any TASL scripts currently
# installed will be updated with the latest versions from Tenable. To
# ensure that only desired TASL scripts are used, newly available
# scripts will not be installed. These can be obtained manually, or
# with the /opt/lce/daemons/lce_update_plugins.pl script.
auto-update-tasls yes
#
#
#
#
#
#
The following options configure both the automatic and manual update
processes to use a web proxy server when downloading files from
Tenable. When these values are commented out, proxy use is disabled.
proxy-address 192.168.10.10
proxy-user username
proxy-password password
# <------ Username Management ------>
# The Log Correlation Engine keeps track of network users on the basis
# of their usernames. The following section sets restrictions on which
# usernames are considered valid. Any usernames failing to match the
# below criteria are disregarded, and "invalid" is reported as the user
# for their associated logs.
accept-letters yes
accept-numbers yes
additional-valid-characters -_.@
max-username-characters 20
# The following setting defines a list of usernames whose IP addresses
# are not tracked. These usernames will appear with their associated
# logs, but LCE will not generate alerts when the users with a matching
# name switch to a new IP address.
excluded-users-file /opt/lce/admin/untracked_usernames.txt
# In order to apply user tracking only to appropriate logs, a list of
# trusted plugins is provided in the following file. If a plugin is not
# on this list, any specified username will appear with the associated
# log if it matches the above criteria, but otherwise will not be
# tracked. Each plugin is specified in the file by its plugin ID.
trusted-plugins-file /opt/lce/admin/trusted_plugins.txt
# <------ DNS Caching ------>
#
#
#
#
#
#
#
#
#
#
When a log message is defined in a plugin, LCE provides the option of
specifying a hostname instead of an IP address for the srcip and
dstip fields. In this case, LCE automatically attempts to resolve
the provided hostname to an IP address using DNS. Since the same
hostname is typically encountered multiple times, caching the results
of lookups can greatly increase performance. The following section
configures DNS caching in LCE. The cache-results-days option
specifies the number of days for which to cache a hostname-to-IP
mapping before updating the result with a new lookup. Hosts listed
in the pre-cache-file are looked up and stored when LCE starts. A
48
# particular hostname or all domain names with a certain extension can
# be excluded using the exclude-domain-name option. In this case, the
# matching hosts are looked up at every occurrence.
max-memory-usage 100M
cache-results-days 10
pre-cache-file /opt/lce/admin/hostlist.txt
excluded-domains-file /opt/lce/admin/excluded_domains.txt
# excluded-domain-name .edu
# excluded-domain-name tenablesecurity.com
# <------ Load Balancing ------>
#
#
#
#
#
#
#
#
#
#
#
#
#
The following section allows users to offload log data that this LCE
receives to other LCE servers. This can be done manually, but using
this mechanism ensures the best steady-state load balancing and
simplifies configuration because all clients may be specified in the
primary LCE. Auxiliary LCEs only need to know how to connect to the
primary LCE. To set this LCE as an auxiliary LCE, read further. LCE
servers may only be primary or auxiliary, but not both.
It is acceptable to leave this commented if this is the primary LCE;
the default lce-load-balance-status-port is 31302.
#
#
#
#
#
#
#
#
To specify that this LCE is an auxiliary LCE to which logs should be
forwarded, uncomment this line. This specifies the primary LCE IP
or hostname, followed by a colon and the port on which it listens for
status data. If the port is unspecified, the status port defaults to
31302 (same as the example). If you specified
lce-load-balance-status-port 31302 in your primary LCE, and your
primary LCE is on 192.168.1.20, then you will specify here:
primary-lce-address 192.168.1.20:31302
#
#
#
#
#
To specify a local interface for receiving data from the primary LCE,
enter an IP address below. Otherwise, the default will be selected.
This can be used to balance bandwidth between separate interfaces,
if desired.
lce-load-balance-local-addr 192.168.1.19
#
#
#
#
#
Optionally, instead of receiving a subset of logs, this LCE may register itself
as a mirror and receive ALL logs processed by the primary LCE, effectively
creating a live backup of the primary database.
To enable this mode, uncomment the following line.
mirror-mode
#
#
#
#
#
#
#
#
#
When using mirror-mode in an LCE cluster, whether this is the primary or
mirror LCE, specifying a virtual-ip-address allows one LCE to
take over all responsibility if the other LCE machine experiences a
hard failure.
This IP address should be the IP address used by syslog sensors,
clients, etc to send data to LCE.
For example, if LCE 1 is at 1.1.1.1/24 and LCE 2 is at 1.1.1.2/24,
choose a third unused IP address such as 1.1.1.3, and direct all
sources to 1.1.1.3 (note that all primary and mirror LCEs must be
This port is the local listen port for status data from auxiliary
LCEs.
lce-load-balance-status-port 31302
49
#
#
#
#
#
#
#
#
#
on the same subnet).
When specifying a virtual-ip-address, also specify an existing network
adapter on which the LCE should bind the virtual IP (default: eth0).
Optionally, if you have a VRRP solution deployed on the same network,
you may specify a router ID for the LCE cluster that differs from your
other VRRP setup with virtual-router-id.
virtual-ip-address 203.0.113.1
virtual-ip-interface eth0
virtual-router-id 1
#
#
#
#
#
#
#
#
#
#
Load balancing messages between Primary / Aux LCEs are encrypted.
To provide additional encryption, a user-specified key may be
added to the built-in encryption. The key should be between 1 and 32
ASCII characters, and must be the same for all Primary / Aux LCEs
in the server pool. To use the additional key (strongly recommended),
uncomment the following line and replace p@ssw0rd_eXampl3 with your
desired 1-32 character encryption phrase.
Allowed characters are alphanumerics and the following characters:
[].^$()|*+?{}/#_-~!@%=`'<>:|&\",
load-balance-key p@ssw0rd_eXampl3
# <------ Data Forwarding ------>
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
One IP address/port may be specified per forward-syslog keyword
which will forward any log event regardless of method
(OPSEC, log file, syslog, windows events, etc.) to a syslog
server. LCE forwards each event to multiple syslog servers
when multiple lines with the forward-syslog keyword are
included.
#
#
#
#
#
#
#
Each time the active silo changes, an MD5 checksum is generated for
the previous silo. This allows the intgerity of the .ndb data files
to be checked at a later time. The following section allows the MD5
checksums to be forwarded to one or more off-site locations. The IP
address list may include any syslog servers.
checksum-forward
192.168.10.10
checksum-forward
192.168.13.63
The forward-syslog format should be:
forward-syslog IP:port,exclude_header
Exclude_header is optional and defaults to 0. If changed to a 1, the
LCE header will NOT be prepended to logs that are forwarded - only
the original log is forwarded and the receiver will not be able to
identify that it came from LCE.
forward-syslog 192.168.10.16
# port defaults to 514
forward-syslog 192.168.20.12:27691,1 # custom port without header
# <------ TASL Performance ------>
#
#
#
#
#
In order to maximize performance on multi-processor and multi-core
systems, correlated TASL events are processed in parallel to the
receiving of regular incoming events. Since some TASL scripts can run
for an extended period of time, the primary event processor can
potentially receive many TASL-triggering events while a tasl script
50
# is still being executed. In this case, the TASL job is stored in a
# queue for later processing. The following option defines the maximum
# size of this queue. On systems with extremely large volumes of data,
# setting the maximum queue size higher results in increased
# performance. If a TASL script designated as being sampleable is
# triggered while the queue is full, its callback functions will not
# be executed.
max-tasl-queue-memory 25M
# The following file contains the filenames of TASL scripts that are
# designated as being sampleable, which results in the behavior
# described above.
sampleable-tasls-file /opt/lce/admin/sampleable_tasls.txt
# <------ File & Directory Settings ------>
plugins-directory
/opt/lce/daemons/plugins/
correlation-scripts-directory /opt/lce/daemons/plugins/
log-directory
/opt/lce/admin/log/
pid-file
/opt/lce/daemons/lced.pid
die-file
/opt/lce/daemons/lced.die
report-directory
/opt/lce/reports/
# <------ Miscellaneous Options ------>
#
#
#
#
#
#
#
#
The below keyword can be used to specify non-standard frame
delimiters for logs received via TCP syslog. This
option can be used with certain products such as NetScreen
that utilize a special delimiter. By default, only the
industry-standard linefeed character is recognized. Other
delimiters must be specified with the corresponding ASCII
decimal value (0-255).
add-delimiter 0
#
#
#
#
#
#
#
#
#
In general, IDS events are correlated with vulnerabilities based on a
match between the event's IP address, port, protocol, and ID. The
vulnerability port is usually 0, 22, or 445 for client-side
applications and credentialed audits. Since these may refer to
client side, web server, or other vulnerabilities, by default, LCE
does not require those three ports to match for correlation. When
the following option is enabled, the vulnerability port will always
be considered.
port-restricted-ids-correlation
#
#
#
#
#
#
Sensor names can be set by the source of the log, the configured
sensor name of the client or syslog source, or the plugin
which normalizes the log. If this option is enabled,
then the sensor name will always be that of the configured
client or syslog source.
override-sensor-name
#
#
#
#
#
LCE Clients version 4+ must be authorized to send data by the LCE
administrator after the client attempts to connect to the LCE server.
To automate authorization for N minutes after LCE server startup
or reconfiguration, enable the auto-authorize-clients-time option.
This would automatically authorize clients that have never previously
51
# tried to connect to the LCE server for 10 minutes after startup:
# auto-authorize-clients-time 10
# <----------------- Debugging Options ----------------->
#
#
#
#
#
#
#
#
#
#
#
#
As events are sent to LCE, if a signature does not exist
for the log event, the data from the log will be stored in
a file named 'notmatched.txt' in the configured database
directory. The maximum number of log entries that will be
stored in the file is specified by the 'save-nonmatched'
variable below. In addition, there is a hard file size
limit of 2GB. If either the specified entry limit or the
2GB file size limit is exceeded, notmatched.txt will be
deleted and opened again to store subsequent new events.
The save-nonmatched option is useful for seeing what kinds
of events are not being matched by the LCE plugins.
save-nonmatched
20000
#
#
#
#
#
#
#
#
#
#
#
The 'save-all' keyword allows all log events to be stored in a
specific file. When used with the 'logrotate' facility and storing
this file on a different physical file system, or possibly a
network file mount, extremely large amounts of data can be saved
by LCE. The 'save-all' keyword specifies the location of this
file. Any event sent to LCE or collected by an agent will be
stored in the file specified. Please consult the LCE documentation
for advice on configuring the 'logrotate' facility to store data for
a specific length of time, specific size of data, or to simply create
rotating log files.
save-all
/opt/lce/db/lce.log
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
The following section should be completely uncommented for additional
logging generated by the lced process. WARNING - the debug logic
is extremely verbose and several Tenable customers have created log
files greater than 4 GB. These logs are purely for diagnostics.
In lieu of specifying the subsections like "client-auth" you
you may specify "debug-all" to turn on all subsections in each topic.
debug {
lce-api {
client-auth
packets-event
packets-all
}
matches {
match-success
match-failure
match-applied
}
silo {
silo-switch
silo-addition
}
match-tree {
52
#
#
#
#
#
#
#
#
#
#
# }
tree-construction
tree-matches
}
load-balancing {
status-msg
data-msg
connections
high-availability
}
}
stats_options {
# Quick-install stats customizations
# End of quick-install stats customizations
# Directory containing the LCE events.txt file
event-directory
/opt/lce/db/
# Log files are named according to date and stored in
# the following directory
log-directory
/opt/lce/admin/log/stats/
# Syslog alerting
# The stats daemon will generate SYSLOG messages to one or more
# recipients. By default, a local address of 127.0.0.1 is used to
# send messages to the lced services. However, more than one
# SYSLOG server can be configured.
syslog-alert 127.0.0.1
# The minimum standard deviation that must occur for an event
# before we'll alert on it. The higher this number, the more
# statistically significant a sequence of events needs to be
# before an alert is raised.
stddev-threshold
1.0
# If an event occurs more/less than 5.0 standard deviation units
# then we'll alert. Setting this value higher, will cut down on
# any sequence of events which occur close to the standard deviation.
stddev-unit-threshold
5.0
# The number of iterations (days) per-event required before
# we'll start alerting for an event. If a large amount of LCE
# data is already present, this number should be set low or even
# to zero. The stats daemon can be started to read in all or just
# part of the existing LCE data. If you have NO LCE data,
# you are encouraged to leave this value around 7 which means the
# stats daemon won't alert on anything until it has 7 days of
# event data. Tenable recommends not running the stats daemon until
# at least 7 days of traffic have been collected.
iteration-threshold
7
# If an event occurs less than 10% of the time then we'll alert.
# Even if an event may be statistically significant, that sequence
# of events may also occur periodically. For example, 50% of the
53
# time you are within a standard deviation, however, occasionally,
# (the other 50%) you have outliers 2 and 3 standard deviations
# away. Those outliers may be the cause of 90% of the alerts thrown
# in this case. Setting this value to 10, 20 or other values would
# only alert for hours which were both out of the allotted standard
# deviation, and also have been event counts which have not occurred
# before.
nhits-frequency-threshold
10.0
}
# The following sections define your network range. All networks specified
# in the first section are included, while the exclude-networks block is
# used to make exceptions. This information is used in determining statistical
# anomalies and event correlations.
include-networks {
# Quick-install customization of include-networks
192.168.0.0/16;
# End of quick-install customization of include-networks
}
exclude-networks {
# Quick-install customization of exclude-networks
# End of quick-install customization of exclude-networks
}
54
Appendix 2: Sample LCE rules.conf File
LCE ships with the following example rules.conf file:
#
#
#
#
#
#
#
#
#
#
#
Log Correlation Engine rules allow special for special commands to be run
or logs to be ignored based on a set of filters for each log event.
Each filter is preceeded by + or - to designate it as a positive or
negative filter (+Types: lce filter matches only logs with Type=lce,
where -Types: lce filter matches logs with Type!=lce).
Each filter type may have multiple tokens separated by commas; positive
filters will match any log containing at least one of those tokens, and
negative filters will match any log containing none of those tokens.
(i.e. +Types: lce,login,intrusion matches lce OR login OR intrusion logs)
(i.e. -Types: lce,login,intrusion will match logs that are NOT (lce or login or
intrusion),
# which is the same as saying NOT lce AND NOT login AND NOT intrusion)
#
# If multiple filter types are used, each filter type must match, for example,
# this rule will only match events to/from IP 192.168.1.1, AND the type is either lce
OR login,
# AND the src or dst port is 22 - it will ignore logs that match:
#
# Name: Example filter rule for ignoring logs
# +IPS: 192.168.1.1
# +Types: lce,login
# +Ports: 22
# ignore
#
# Here are the value filter types:
#
#
IPS:
Filter on src or dst IP (IP, CIDR, or range allowed, i.e.
192.168.0.0/16)
#
SrcIPS:
Filter strictly on src IP (IP, CIDR, ranges allowed)
#
DstIPS:
Filter strictly on dst IP (IP, CIDR, ranges allowed)
#
Events:
Filter on LCE normalized event name (i.e. CiscoIDS_Command_Execution)
#
Sensors:
Filter on sensor name
#
Types:
Filter on LCE event type (i.e. login, lce, intrusion, scanning,
system, etc)
#
Ports:
Filter on the src or dst port
#
Protocols:
Filter on the protocol of the event (i.e. 1 for ICMP, 2 for IGMP, 6
for TCP, 17 for UDP)
#
Users:
Filter on the username in a log
#
Text:
Filter on any text token in the log (tokens can include spaces and
punctuation but not commas)
#
IText:
Same as above, but case insensitive
#
# This is a special filter type that does not have a negation availble because the
value is either "yes" or "no":
#
#
Vulnerable:
"yes" or "no" - yes if you want to only match logs that correlate
to vulnerable hosts
#
# When a filter matches a log, you may specify two things to do:
#
#
ignore
Drops the log instead of storing it to the LCE db.
55
#
Command:
Runs the given command at the command line as user lce (i.e. echo
"log matched" >> /opt/lce/my_log_file.log)
#
# See the /opt/lce/tools/ directory for a few tools supplied with LCE for emailing
logs, forwarding them via syslog, etc.
#
# When using "Command:" to run a command, you may insert some or portions of the log
into your command using the following
# replacement macros. The following example sends the original log text and the src
IP:port dst IP:port via syslog to
# another machine for network or connection type logs:
#
# Name: Example command
# +Types: network,connection
# Command: /opt/lce/tools/send_syslog 192.168.1.1 -message "LOG MATCHED RULE
($sip:$sport -> $dip:$dport) [$log]"
#
# Here is the full list of replacement macros available:
#
#
$sip
Replaced by the src IP
#
$dip
Replaced by the dst IP
#
$sport
Replaced by the src port
#
$dport
Replaced by the dst port
#
$proto
Replaced by the protocol
#
$vuln
Replaced by whether or not the log correlated to a vulnerable host
#
$sensor
Replaced by the sensor name
#
$event1
Replaced by the LCE normalized event name
#
$event2
Replaced by the secondary LCE normalized event name (usually applies
to Windows logs)
#
$type
Replaced by the LCE event type
#
$time
Replaced by the log timestamp
#
$user
Replaced by the username
#
$log
Replaced by the entire original log text
#
# Further Examples:
#
# Below is an example Log Correlation Engine rule. It filters on a specified set of
# IP addresses, applying the rule to only those logs with attributes matching
constraints
# on event, sensor, IP address, and user. The threshold setting requires that a
minimum of
# 5 matching events occur within a five-minute window before the rule goes into
effect.
# After this threshold is reached, the LCE syslog utility is invoked to transmit a log
at
# most once per minute. If the quantity of matching incoming logs exceeds the rate
limit,
# up to 100 will be queued and transmitted at the specified maximum rate of 1 per
minute.
#
# Name: Potential SSH account username/password guessing
# +Events: SSH-Invalid_User, SSH-Failed_Password
# +IPs: 127.0.0.1, 10.0.0.0/8, 192.168.20.0/128.0.0.0
# -IPs: 192.168.20.0-192.168.20.5
# -IPs: 10.0.0.1, 10.0.0.7-15
# +Sensors: DMZ-1, DMZ-2
56
# -Users: (unknown)
# Command: /opt/lce/tools/send_syslog 192.168.2.1 -message "Possible password guessing
evidence: $log" -priority 97
# Threshold: 5 in a minute
# RateLimit: 1 per minute
# MaxQueue: 100
#
# The following example rule will cause all login events from the local server to be
ignored.
# Ignored events will only be evidenced in the LCE save-all file or log archive.
#
# Name: Ignore local logins
# +Types: login
# +IPs: 127.0.0.1
# ignore
#
# The last example demonstrates a rule to e-mail all vulnerability correlations to an
administrator.
#
# Name: E-mail vulnerability correlations
# Vulnerable: yes
# Command: echo "IDS event correlated with vulnerability ($sip -> $dip) at $time:
$log" | /opt/lce/tools/msmtp -C /opt/lce/tools/msmtp.conf admin@yourdomain.com
[root@LCE_Test_2 daemons]#
[root@LCE_Test_2 daemons]#
-bash: rules.conf: command
[root@LCE_Test_2 daemons]#
GNU nano 2.0.9
clear
rules.conf
not found
nano rules.conf
File: rules.conf
# After this threshold is reached, the LCE syslog utility is invoked to transmit a log
at
# most once per minute. If the quantity of matching incoming logs exceeds the rate
limit,
# up to 100 will be queued and transmitted at the specified maximum rate of 1 per
minute.
#
# Name: Potential SSH account username/password guessing
# +Events: SSH-Invalid_User, SSH-Failed_Password
# +IPs: 127.0.0.1, 10.0.0.0/8, 192.168.20.0/128.0.0.0
# -IPs: 192.168.20.0-192.168.20.5
# -IPs: 10.0.0.1, 10.0.0.7-15
# +Sensors: DMZ-1, DMZ-2
# -Users: (unknown)
# Command: /opt/lce/tools/send_syslog 192.168.2.1 -message "Possible password guessing
evidence: $log" -priority 97
# Threshold: 5 in a minute
# RateLimit: 1 per minute
# MaxQueue: 100
#
# The following example rule will cause all login events from the local server to be
ignored.
# Ignored events will only be evidenced in the LCE save-all file or log archive.
#
# Name: Ignore local logins
# +Types: login
57
# +IPs: 127.0.0.1
# ignore
#
# The last example demonstrates a rule to e-mail all vulnerability correlations to an
administrator.
#
# Name: E-mail vulnerability correlations
# Vulnerable: yes
# Command: echo "IDS event correlated with vulnerability ($sip -> $dip) at $time:
$log" | /opt/lce/tools/msmtp -C /opt/lce/tools/msmtp.conf admin@yourdomain.com
58
Appendix 3: Troubleshooting
The following are troubleshooting steps for determining LCE client/server functionality:
1. Install and configure the LCE and clients by following the instructions in the documentation.
2. Verify the clients are connecting by viewing the file /opt/lce/admin/log/client.status.
a. If the clients never connect, review configuration.
b. If the configuration is correct, then there is a network issue. Check for proxies, firewalls or ACLs that may
be blocking traffic.
c.
If the clients connect but do not stay connected, continue to test.
3. The LCE client will not remain connected with the LCE server unless the client has some data to send. To “force”
a client to forward data to the LCE server, an observed log on the LCE client machine can be appended with
entries that are known to cause alerts within SC4. This gives the LCE client some data to send to the server. It is
advised to put “TEST OF FUNCTIONALITY” in the beginning of the log entries to ensure that these tests do not
interfere with actual alerts. Check your client logs to ensure communication is taking place.
a. Yes? Communication is taking place. Continue to Step 4.
b. No? Contact Tenable Support for an LCE Client Issue.
4. Once the logs are appended, check the client.status file. Has it changed?
a. Yes? Functionality is working.
b. No? Continue with next step.
5. Check SC4 for the IP address in question and the time of the test. Were there entries found?
a. Yes? Your LCE is functioning properly. However, there may be an issue with the client.status
heartbeat. Notify Tenable Support of the issue.
b. No? Continue to the next step.
6. Grep the logs in the LCE’s notmatched.txt file for the IP address in question and the time of test. Were there
entries found?
a. Yes? Your LCE is functioning and logs are being updated properly. However there may be an issue with
the client.status heartbeat. Notify Tenable Support of the issue.
b. No? Continue to the next step.
7. Perform a TCPDump on the LCE and capture traffic from the IP address of the client in question. Repeat step 3 to
force communications. Did you receive traffic?
a. Yes? Notify Tenable Support of the issue for further assistance.
b. No? You may have a network issue. Please work with your network support to troubleshoot the issue.
59
Appendix 4: Manual SC4/LCE Key Exchange
A manual key exchange between SecurityCenter and the LCE is normally not required; however, in some cases where
remote root login is prohibited or key exchange debugging is required, you will need to manually exchange the keys.
For the remote LCE to recognize SecurityCenter, you need to copy the SSH public key of SecurityCenter and append it to
the “/opt/lce/.ssh/authorized_keys” file on the LCE server. The “/opt/lce/daemons/lce-install-key.sh”
script performs this function. The following steps describe how to complete this process:
The LCE server must have a valid license key installed and the LCE daemon must be running before
performing the steps below.
1. Download the SSH public key for SecurityCenter by logging in as the SecurityCenter administrator user and
navigating to the “Keys” section (“System” -> “Keys”).
2. Click on “Download Key”, choose the desired key format (both DSA or RSA work for this process) and then click
on “submit”.
3. Save the key file (SSHKey.pub) to your local workstation. Do not edit the file or save it to any specific file type.
4. From the workstation where you downloaded the key file, use a secure copy program, such as “scp” or “WinSCP” to
copy the SSHKey.pub file to the LCE system. You will need to have the credentials of an authorized user on the
LCE server to perform this step. For example, if you have a user “bob” configured on the LCE server (hostname
“lceserver”) whose home directory is /home/bob, the command on a Linux or Unix system would be as follows:
# scp SSHKey.pub bob@lceserver:/home/bob
5. After the file is copied to the LCE server move the file to /opt/lce/daemons by doing the following:
# mv /home/bob/SSHKey.pub /opt/lce/daemons
6. On the LCE server, as the root user, change the ownership of the SSH key file to ‘lce’ as follows:
# chown lce /opt/lce/daemons/SSHKey.pub
7. Then append the SSH public key to the “/opt/lce/.ssh/authorized_keys” file with the following steps:
# su lce
# /opt/lce/daemons/lce-install-key.sh /home/bob/SSHKey.pub
8. To test the communication, as the user “tns” on the SecurityCenter system, attempt to run the ‘id’ command:
# su tns
# ssh -C -o PreferredAuthentications=publickey lce@<LCE-IP> id
If a connection has not been previously established, you will see a warning similar to the following:
The authenticity of host '192.168.15.82 (192.168.15.82)' can't be established.
RSA key fingerprint is 86:63:b6:c3:b4:3b:ba:96:5c:b6:d4:42:b5:45:37:7f.
Are you sure you want to continue connecting (yes/no)?
60
Answer “yes” to this prompt.
If the key exchange worked correctly, a message similar to the following will be displayed:
# uid=251(lce) gid=251(lce) groups=251(lce)
9. The IP address of SecurityCenter can be added to the LCE system’s /etc/hosts file. This prevents the SSH
daemon from performing a DNS lookup that can add seconds to your query times.
10. The LCE can now be added to SecurityCenter via the normal administrator “LCE add” process documented in the
SecurityCenter Administration Guide.
61
Appendix 5: Non-Tenable License Declarations
Below you will find the command that will list all the third-party software packages that Tenable provides for use with the
Log Correlation Engine.
# /opt/lce/daemons/lced -l
62
About Tenable Network Security
Tenable Network Security is relied upon by more than 20,000 organizations, including the entire U.S. Department of
Defense and many of the world’s largest companies and governments, to stay ahead of emerging vulnerabilities, threats
and compliance-related risks. Its Nessus and SecurityCenter solutions continue to set the standard to identify
vulnerabilities, prevent attacks and comply with a multitude of regulatory requirements. For more information, please visit
www.tenable.com.
GLOBAL HEADQUARTERS
Tenable Network Security
7021 Columbia Gateway Drive
Suite 500
Columbia, MD 21046
410.872.0555
www.tenable.com
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
63
