ALERT LOGIC LOG MANAGER
DEPLOYMENT ARCHITECTURE
Log Manager can support log collection from on-premise, hosted, Virtual Private and Public Cloud environments with four primary
methods for collecting log data; agent-only, via a Remote Collector, a Virtual Appliance, and through a dedicated Physical Appliance.
This document provides an overview of the Log Manager architecture, core solution components, and deployment scenarios.
Contents
Overview of Log Manager Architecture ..................................................................................................................................................... 2
Overview of Log Manager Architectural Components............................................................................................................................... 3
Log Manager Deployment Options ............................................................................................................................................................ 4
Specifications & Performance Profiles ....................................................................................................................................................... 5
Log Manager Deployment Scenarios ......................................................................................................................................................... 6
Agent Only Collection ........................................................................................................................................................................ 6
Public Cloud & Elastic Environments ................................................................................................................................................. 7
Remote Collectors .............................................................................................................................................................................. 8
Virtual Appliances .............................................................................................................................................................................. 9
Dedicated Physical Appliance .......................................................................................................................................................... 10
Agent-less Syslog Collection..................................................................................................................................................................... 11
Load Balancing Log Traffic ....................................................................................................................................................................... 12
Firewall Ports for Data Collection ............................................................................................................................................................ 12
Frequently Asked Questions .................................................................................................................................................................... 13
Additional Documentation ....................................................................................................................................................................... 16
Alert Logic, Inc.
1776 Yorktown, 7th Floor, Houston, TX 77056 | 877.484.8383 (toll free) | 713.484.8383 (main) | 713.660.7988 (fax) | www.alertlogic.com
Alert Logic and the Alert Logic logo are trademarks, registered trademarks, or service marks of Alert Logic Inc. All other trademarks listed in this document are the property of their respective
owners. Revised August 2011.
© 2012 Alert Logic, Inc. All rights reserved.
LOG MANAGER DEPLOYMENT ARCHITECTURE
Overview of Log Manager Architecture
Log Manager provides a dramatically simplified, yet powerful log collection solution for customers concerned with IT security and
compliance regulations. At a high level, logs are simply collected from the customer’s environment, encrypted, compressed and sent
back to Alert Logic’s cloud for processing and storage. All data is secured and replicated for disaster recovery as part of standard
service delivery. Customer access is available through a web-based UI where they can search, alert, correlate, and report on log data.
An optional Log Review Service, delivered by Alert Logic security analysts, is also available for daily log review compliance mandates.
There are four primary methods of deploying Log Manager within a customer environment; agent-only, via a Remote Collector, a
Virtual Appliance, through a dedicated Physical Appliance. Selection of which method to use is dependent on the IT environment
which contains the host devices, the types of devices, and the preferred collection method. For the majority of customers, an agentonly or virtual appliance is the preferred method. However, a dedicated physical log collection appliance is available when collection
agents are not desired.
Log Manager Collection Components
Page 2
LOG MANAGER DEPLOYMENT ARCHITECTURE
Overview of Log Manager Architectural Components
Log Manager provides four methods for collecting log data within a customer environment; agent-only, via a Remote Collector, a
Virtual Appliance, and a dedicated physical appliance.
Agents: Log Manager Agents support Windows, Syslog, and Flat File log collection. The agent collects, compresses, and encrypts the
log data for transmittal directly back to the Alert Logic cloud via HTTPS; no appliance required. Agents also collect host metadata for
presentation in the log manager UI and are able to report on collection status, statistics, and history. The agents run at a low priority,
are light on resources, and can be upgraded remotely if desired. If a single point of network egress is desired, a NAT is the
recommended method. Log Manager Agents also support log collection from auto-scaling instances; often found in Public Cloud
environments.
Remote Collector: Log Manager Remote Collectors collect and forward Syslog data to the Alert Logic cloud. They can be installed on
either a Windows or Linux machine, are remotely upgradeable, and do not require a virtual VMware instance (as does the Virtual
Appliance). Syslog data is compressed and encrypted prior to sending. Ideal for collecting Syslog data from network
devices/firewalls, which cannot run an Agent - and VMware is not available for the Virtual Appliance. Windows log collection is
accomplished via the Windows Agent.
Virtual Appliance: The Log Manager Virtual Appliance collects and forwards Syslog data to the Alert Logic Cloud. Although similar in
function to the Remote Collector (Syslog collection from devices which cannot run an Agent), the Virtual Appliance must be installed
within a VMware virtual machine. Virtual Appliances can also provide a single point of network egress and are remotely
upgradeable. Windows log collection is accomplished via the Windows Agent. (Note: Public Clouds utilize agent-only collection and
cannot use virtual appliances).
Physical Appliance: Log Manager also offers a dedicated physical 1U log collection appliance. The physical appliance is able to collect
Windows, Syslog, and Flat File without collection agents and provides dedicated compute resources for log collection. Best used for
environments where collection agents are not desired and/or high log volumes.
Remote Collectors, Physical, and Virtual Appliances support agent-less collection of syslog data. This is the primary method of
collecting log messages from Unix servers (AIX, Solaris, etc.) and network equipment (firewalls, routers, etc.). Log appliances accept
TCP or UDP syslog messages on port 514.
Customer identity for all deployment scenarios is established with a specific License Key which is married with the agent, remote
collector, appliance at installation. The key is provided via the Alert Logic portal as part of the new customer provisioning process.
The License key is used to bootstrap unique a cryptographic identity for each agent or appliance.
Page 3
LOG MANAGER DEPLOYMENT ARCHITECTURE
Log Manager Collection Method Support
Log Manager Deployment Options
Log Manager solution components can be used independently or in a mixed environment. The selection of which method to deploy
is dependent on the IT environment which contains the host devices, the device types and whether log collection agents will be (or
can be) used.
Page 4
LOG MANAGER DEPLOYMENT ARCHITECTURE
Specifications & Performance Profiles
Windows Agent
CPU Utilization
RAM
DISC
Internet connection
Supported OS
Log collection support
Encryption
Log collection frequency
Host permissions
Syslog Agent
CPU Utilization
RAM
DISC
Internet connection
Supported OS
Log collection support
Encryption
Collection frequency
Host permissions
Remote Collector
Types
CPU
RAM
DISC
Internet connection
Log collection support
Encryption
Virtual Appliance
CPU
RAM
DISC
Internet connection
Supported virtual environment
Log collection support
Encryption
Note
Physical Appliance
CPU
RAM
DISC
Chassis
Power
Internet connection
Log collection support
Encryption
Specifications
1-10% depending on log volume
15 MB minimum
30 MB minimum
Port 443 - required for log transport and agent maintenance updates
Windows Server (2012, 2008, 2003) Windows (8, 7,Vista, XP) Platform: 32-bit / 64-bit
Agent-only deployments & with virtual and physical appliances, VPC and Public Clouds
TLS Standard (SSL): 2048bit key encryption, 256bit AES bulk encryption
Every 5 minutes (logs collected & sent back to Alert Logic Cloud)
LocalSystem account has all the requisite permissions by default
1-10% depending on log volume
10 MB minimum
500 MB minimum
Port 443 - required for log transport and agent maintenance updates
Debian (Squeeze, Lenny), Ubuntu 7.x-12.x, CentOS 5.x -6.x RedHat 5.x -6.x), Platform: 32-bit / 64-bit
Agent-only deployments & with virtual and physical appliances, VPC and Public Clouds
TLS Standard (SSL): 2048bit key encryption, 256bit AES bulk encryption
Every 5 minutes (logs collected & sent back to Alert Logic Cloud)
No special permissions are required
Windows Remote Collector & Linux Remote Collector
2 cores (recommended)
2GB(recommended)
1GB – 50GB(recommended)
Port 443 - required for log transport and appliance maintenance updates
Syslog without an agent
TLS Standard (SSL): 2048bit key encryption, 256bit AES bulk encryption
2 cores
2GB
1GB – 50GB
Port 443 - required for log transport and appliance maintenance updates
VMware only
Syslog without an agent
TLS Standard (SSL): 2048bit key encryption, 256bit AES bulk encryption
Not designed to run in a public cloud environment. Use agent-only deployments instead.
Intel Xeon
4GB DDR3
500GB
1U rack mounted
250W
Port 443 - required for log transport and appliance maintenance updates
Both agent-based & agent-less Windows, Syslog, Flat File log collection
TLS Standard (SSL): 2048bit key encryption, 256bit AES bulk encryption
Page 5
LOG MANAGER DEPLOYMENT ARCHITECTURE
Log Manager Deployment Scenarios
Agent Only Collection
Since Log Manager Agents can collect Windows, Syslog, and Flat File logs, enable fast easy provisioning, provide better status
reporting, and are remotely upgradeable, an agent-based collection is recommended by Alert Logic and should be one of the first
deployment considerations.
Log Manager provides two types of agents, Host Agents and Role Agents, where Role Agents are exclusively used for auto-scaling
instances in a Public Cloud. For a dedicated instance within a Public Cloud or agent-based collection in other non-auto-scaling
environments, Host Agents are used.
Host Agents
Host Agents are used in non-auto-scaling environments. The latest versions and installation instructions are available directly from
the Log Manager UI. There are two methods for deployment; Windows GUI and a Command Line Interface (CLI). If mass
deployment is preferred, standard Microsoft Windows remote installation tools such as SMS and GPO are supported; for Syslog, use
remote shell or PDSH. Utilizing a script for deployment has the added benefit of enabling host customization during the agent
installation (ex. Setting a custom host name and asset tags); alternatively, customization can be accomplished as a second step
directly within the Log Management UI after agent installation.
Installing Agents on a Domain controller is a good practice for capturing log data for improved Windows insight. Once installed, each
agent independently connects back to the Alert Logic Cloud for log transport. If a single point of egress from the network is desired,
a NAT is recommended as a best practice.
Page 6
LOG MANAGER DEPLOYMENT ARCHITECTURE
Public Cloud & Elastic Environments
For Public Cloud deployments, there are typically two methods for creating a machine instance which includes a Log Manager
collection agent; Bake-it-In method, and Script-it Method. Both are suitable for either Windows or Linux instances.
Bake-it-In Method
If standard practice is to make images for machines, bake-in the latest agent before it is provisioned. This is advantageous if utilizing
the agent’s remote upgrade feature. If the instance is to be used in an auto-scaling environment, such as Amazon EC2, the
recommendation is to utilize the Bake-it-in method with an already provisioned log collection Role Agent.
Auto-scaling is supported within the Log Manager Management interface by the creation of “host roles” where multiple instances
can be included within one logical group and managed as a single identity. For example, an auto-scaling web server environment
may increase/decrease the number of instances through a day, month, or year depending on the demand for those resources. These
devices can be configured as one logical role, assigned policies, and tracked through log message data. Log Manager captures and
preserves historical metadata on each of the individual instances for improved insight and instance tracking within these dynamic
environments.
Script-It Method
If the standard practice is to start with a generic image, and then install custom software and configurations, you can easily include a
command to add the latest Role Agent during this set up process.
This method should be used for creating individual instances only and it not suitable for auto-scaling instances.
Page 7
LOG MANAGER DEPLOYMENT ARCHITECTURE
Remote Collectors
A Remote Collector is an application which can be installed on a customer-owned Windows or Linux device to forward Syslog data to
Alert Logic. Although similar in function to a Virtual Appliance, they do not require a dedicated virtual instance (VMware) in which to
operate. Remote Collectors only forward Syslog data. An Agent must be used for Windows Event Logs (assumes no other log
appliance is available for remote Windows collection) The Remote Collector will encrypt and compress the Syslog data prior to
sending to the AL Cloud via Port 443.
Common use case: “I must collect syslog from network devices or firewalls (which cannot run agents), but do not want a physical
appliance – and do not have VMware in my environment for the Virtual Appliance”
Remote Collector resource utilization testing was conducted on a 2 core machine with 2 GB RAM and 10GB disk space. Results may
vary depending on environments and logging requirements.
Remote Collectors are available for download via the Support page within the Log Manager user interface. After installation, syslog
streams are pointed to the Remote Collector to forward to Alert Logic. Status, statistics, and error states are reported within the UI
on the Collectors page.
Page 8
LOG MANAGER DEPLOYMENT ARCHITECTURE
Virtual Appliances
Log Manager Virtual Appliances collect and send Syslog data to Alert Logic. While this function is the same as a Remote Collector,
they differ in two ways: Virtual Appliances must be installed within a VMware instance, and they can be used as a single point of
egress for all log data exiting the network. Enabling Windows or Flat File log collection requires the use of a collection agent. Syslog
can utilize either agent-based or agent-less log collection. Public Clouds utilize agent-only collection and cannot use virtual
appliances.
Common use case: “I must collect syslog from network devices or firewalls (which cannot run agents), but do not want a physical
appliance – and I do have VMware in my environment for the Virtual Appliance”
Virtual Appliances are available for download via the Support page within the Log Manager user interface. After installation, syslog
streams are pointed to the Virtual Appliance to forward to Alert Logic. Status, statistics, and error states are reported within the UI
on the Collectors page.
If Windows Event Log collection is desired, the Windows agent must be installed on each target Windows machine. During agent
installation, two options are available for sending Windows logs to Alert Logic: use the default destination to send Windows events
directly back to Alert Logic (vaporator.alertlogic.com), or direct the Windows log data to the Virtual Appliance which then forwards
to Alert Logic (single point of network egress).
Page 9
LOG MANAGER DEPLOYMENT ARCHITECTURE
Dedicated Physical Appliance
The dedicated physical appliance provides the broadest support for both agent-based and agent-less log collection and can be
utilized for: mixed collection environments, environments where log collection agents are not desired, or for devices which cannot
run an agent. The primary advantage of a dedicated physical appliance is the ability to support log collection without agents.
Placing the physical Log Appliance in the DMZ provides visibility to both internal IT assets & the Alert Logic Cloud. Log traffic is
passed back the Alert Logic Cloud via 443; management of the appliance is via port 22.
Multiple dedicated appliances can be used to support high log volumes or multiple locations
Page 10
LOG MANAGER DEPLOYMENT ARCHITECTURE
Agent-less Syslog Collection with Appliances
For Syslog collection without a log agent, there are two alternatives when utilizing either a Virtual or Physical Appliance:
Existing Central Syslog Server
If you already have a central Syslog server, it is easy to forward Syslog traffic to either a virtual or physical a Log Manager appliance.
New Syslog Collection
If you do not have a central Syslog server already established or want a separate collection path, you can install either the virtual or
physical appliance and send Syslog traffic directly.
Page 11
LOG MANAGER DEPLOYMENT ARCHITECTURE
Load Balancing Log Traffic
For high log volumes, you can use a load balancer between the log source and either a virtual or physical appliance.
Firewall Ports for Data Collection
Reference chart for firewall settings to enable log collection via agent-based and agent-less configurations:
Page 12
LOG MANAGER DEPLOYMENT ARCHITECTURE
Frequently Asked Questions
Where can I find more information on Log Manager?
Please refer to the links in the Additional Documentation section of this paper or go to: http://docs.alertlogic.com/ for more
information.
What methods of Syslog collection are supported?
For Syslog, we support TCP and UDP.
What methods of Flat File log collection are supported?
To collect flat files, Log Manager’s Flat File log collector uses a CIFS file share connection from the Alert Logic Log Manager appliance
to the log file data. In cases where a CIFS file share cannot be enabled on a specific log host computer, it might be necessary to copy
the affected log file(s) to a separate computer where a CIFS file share can be accessed by the Log Manager appliance.
Alert Logic Flat File Collector supports several methods of log rotation so that the collector will start collecting the oldest log
messages and move to the most recent. It will get to a "watermark" point so that it will collect the newest messages and not collect
the previously found messages.
Flat File log rotation support is as follows:
YYYYMMDD (IIS Native Method)
YYYYMMDD (append method)
MM-DD-YYYY
MM-DD-YY
MM.DD.YYYY
MM.DD.YY
MM_DD_YYYY
MM_DD_YY
Epoch Timestamp
Incrementing Integer Method (logrotate)
For any of the above rotation schemes, Log Manager supports gzip, bzip2, and zip compressed logs. For a video tutorial on setting up
flat file collection go to Log Manager/Support/Tutorials or visit: http://www.youtube.com/playlist?list=PLs4m-8xwg9MLvUh41kTdtEhHY68swwuUO
Does Log Manager support DNS?
If an agent is installed on a log source and that device knows its host name (by DNS or static configuration), we will report that name
and IP. We do not support DNS for agent-less collection, that is connecting to a log source by name and not IP address.
What types of logs should I collect for security and compliance regulations?
For recommendations on best practices for log management, please refer to the whitepapers linked at the end of this document.
Does Alert Logic offer any APIs for system integration?
Yes, Alert Logic offers a number of APIs for data access and system integration. Contact your Account Manager to learn more.
I use Snare to capture Windows logs, which changes them to Syslog. Can Log Manager collect and parse these Windows/Syslog logs?
Yes, we can collect these logs, however, they will be presented in the Log Manager UI as un-parsed text logs. Recommendation is to
replace Snare with either agent-based or agent-less Windows log collection.
How do remote upgrades work for the agents and the appliances? Can I opt-out of automatic updates?
By default, automatic updates are turned off. If you desire auto-updates, you can opt-in through the Log Manager UI. Additionally,
you will have the capability to select “upgrade agent now” within the UI at the time desired.
Page 13
LOG MANAGER DEPLOYMENT ARCHITECTURE
Can I set a collection “black-out” period to prohibit log transmittal back to the Alert Logic Cloud. What happens to this log data?
Yes, setting a collection black-out policy within the Log Manager UI is easy to accomplish. This is often used in retail storefronts to
limit log traffic during business hours; with logs transmitting only after hours. During the black-out period, logs will be cached on the
local device and sent in their entirety at the scheduled time. With Windows Event Log, you must ensure that the local log retention
policy has sufficient storage settings to prevent logs from being overwritten.
What happens to log data if the connection back to the Alert Logic Cloud is interrupted?
For hosts utilizing a collection agent, the logs will be cached on the host until the connection is restored. For Virtual Appliances, logs
will be cached on the host on which the virtual appliance is running . In each deployment model, once the connection is reestablished, the system will “catch up “on cached data and continue collection as normal; without the loss of data.
Log Collection Agents
What is the network impact to using a Log Collection Agent?
Utilizing an agent for log collection will be lighter on the network as the agent compresses log data prior to sending over the network
and back to Alert Logic’s cloud.
What are the benefits of using Log Collection Agents?
1. Automatic Discovery of log sources: Customer will be able to install the Agent software with no further manual
configuration on a per host basis to enable host discovery and log collection.
2. Collect Logs from sources behind firewalls: Agent-based collection eliminates the need for opening special firewall ports.
3. Collect Logs without log source credentials: An agent-based solution enables the host to push logs proactively without the
need for releasing credentials or having to provide an administrative account. This is especially relevant with Windows log
sources.
4. Collects host metadata: Agents collect metadata from the host on which it resides including hostname, local IPV4/6 IP,
public hostname, creation date/user, host and source IDs.
5. Compresses log data: Agents compress log data (aprox 14:1) prior to sending back to Alert Logic which means lower
network utilization for log traffic
What are the methods for installing the agents on a large number of hosts?
For Windows, the MSI package can be installed with standard Microsoft Windows remote installation tools such as SMS or by using
GPO. For Linux agents, the .deb and .rpm packages can use remote shell or PDSH for installation.
What scripting capability is available for the agents?
You can use scripting for initial agent deployment or configuring already installed agents.
Is there a way to configure the agent to use a proxy server to access the Internet?
Yes. While the agent does not use a proxy server by default, it is possible to enable it via the command-line interface of the installed
agent. When enabled, the agent uses the system proxy settings.
What rule(s) do I need to configure on my Proxy Server to allow communication to Alert Logic for log transport?
You need to configure an allow rule for all HTTPS traffic to vaporator.alertlogic.com:443
Is there any throttling available when using the agent?
Page 14
LOG MANAGER DEPLOYMENT ARCHITECTURE
The agent will try to transfer log data as quickly as possible given current network conditions. Agents provide collection blackout
periods to configure transfer of log data only at desired times.
Can I install agents for multiple log collection appliances and have them intelligently route logs to the more available log appliance?
If you are worried about scalability and availability of the log collection appliances, you can eliminate the need for an appliance by
configuring your routers and the agents in a way that agent traffic can easily reach https://vaporator.alertlogic.com and we'll take
care of both scalability and availability for you.
How often are logs collected and sent back to the Alert Logic Cloud?
Logs are collected from the host every 5 minutes and immediately sent back to the Alert Logic Cloud.
How would the agent communicate back to Alert Logic if the server does not have direct Internet access?
The recommended way is to configure the network routers to pass agents' traffic to vaporator.alertlogic.com:443. Alternatively,
agents can be configured to send log traffic to an Alert Logic appliance and the appliance will serve as the router forwarding traffic to
vaporator.alertlogic.com:443.
How do I move a log agent to a new appliance?
Installed agents have a command-line function for doing this: al-log-agent configure --host <new host> [ --port <new port> ]
Can I move an agent to another LM appliance without reinstalling?
Yes.
How do I change an agent configuration?
Through the Log Manager User Interface.
Log Manager Portal
What browsers are supported for Log Manager portal access?
Alert Logic supports the latest stable releases of Chrome, Safari, Firefox, Opera, and Internet Explorer.
Is there a limit to the number of users allowed to access the Log Manager UI?
No, there is no limitation on the number of users who can access and use Log Manager.
What types of user access control is available?
Users may be granted rights to view groups of log sources. Alert Logic enables access control management for all user objects.
Permissions include management of Users (Create, Delete, Lock and Modify), and Log Management (Modify Policy, Modify
Correlation
Does Log Manager support filtering and aggregation of log data?
Yes, log data can be filtered and aggregated across multiple data values and time ranges; both within the UI and via reports.
Does Log Manager support cross-device correlation?
Logs can be correlated using the user interface to build correlation policies. Policies can be built off of a number of particular log
messages occurring in a time frame, a part of a log message occurring, or any combination of time, content, or frequency.
Page 15
LOG MANAGER DEPLOYMENT ARCHITECTURE
Additional Documentation
For additional information on component specifications, performance profiles, and solution architecture, please refer to the
following:
Whitepaper
Log Management Best Practices
Configuring Log Sources for Best Practice Reports
Description
Best practices for log security and compliance log collection
Best practices for report types and how to configure log collection to enable
Page 16
