2/11/2019
99: Centralized Monitoring
99: Centralized Monitoring
Objective
Manually upload log data to a central Splunk server.
Set up servers so to get syslog data from a remote system into Splunk.
Scenario
It is a challenge for organizations to analyze the massive amounts of network information and metadata associated
with networking, especially when understaffed. Splunk is one solution that uses big data from websites, applications,
servers, networks, sensors, and mobile devices for more efficient analysis. It is particularly useful for event correlation
and supports a wide variety of environments. You will follow the steps in the lab to upload data to a central log server
(Splunk) and configure a remote server to forward syslog events.
Virtual Machines
1. LAN_WIN7_syshard
2. LAN_KALI_GOLD
Manually Upload Data to Splunk
1. Win 7 Log In
Log in to the Windows 7 virtual machine as student with the password P@ssw0rd.
2. Open Splunk
Open Splunk by clicking start, and typing "splunk". Then, click "Splunk Enterprise"
Logon as admin and the password changeme.
Change the password to something rememberable.
Don't forget your password, as you'll need this later.
3. Import Data Sources
First you are going to import data sources to Splunk using the “Add Data” link. (Add or forward data to
Splunk Enterprise.). Under Explore Splunk Enterprise, click on “Add Data”.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
1/11
2/11/2019
99: Centralized Monitoring
4. Uploading Previous Log Files
Next you will upload previously saved log files to analyze using Splunk. Click on “Upload” to upload the log
files. Click “Select File” then navigate to c:\labfiles\logfiles. Open “mailsecure.log”.
5. Set Sourcetype
Click “Next”. You should now be at Set Sourcetype. Here you can preview data before it is indexed by Splunk.
We will keep the defaults and select “Next”.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
2/11
2/11/2019
99: Centralized Monitoring
6. Sourcetype: Mailserver
The Save Sourcetype dialog box will appear. Enter “mailserver” in the Name field and “mailserver” in the
description. Click “Save”.
7. Input Settings
You should now be at Input Settings. Change the Host field value to “mailserver” then click “Review”.
8. Review Entries - Finalize Upload
Review your entries and then click “Submit”. Your file is now uploaded and indexed in Splunk.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
3/11
2/11/2019
99: Centralized Monitoring
9. Add Additional Files
Click on Add More Data. Repeat the steps you just performed to upload the remaining log files located in
c:\labfiles\logfiles folder. Be careful to set the source type name to corresponding server (www1, www2 or
www3). After uploading the log files, click on “splunk” in the upper left corner.
10. Search and Reporting
Back at the Splunk home screen, Click on “Search & Reporting” to begin analyzing your log files. Notice that
data is displayed as indexed under What to Search.
11. Search Specific Host
You identified that the potentially malicious IP address 182.236.164.11 has accessed your servers. Enter
“182.236.164.11”, then search to query this IP in the logfiles.
From here you could identify and correlate other servers that have communicated with the same IP
address.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
4/11
2/11/2019
99: Centralized Monitoring
12. Leave Splunk Running in Background
Leave Splunk Running.
Congratulations, you have completed the manual, splunk data upload lab.
Configure Splunk and Syslog
1. Authorize TCP/UDP Input
First you will need to authorize your TCP/UDP input into your project. To begin this process click on Add
Data.
In these next steps you will learn how to set up servers so that you can get syslog data from a
remote system into Splunk.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
5/11
2/11/2019
99: Centralized Monitoring
2. Configure Splunk to Monitor
Instead of uploading data like you did previously, click on Monitor then select TCP/UDP so you can configure
Splunk to listen on a network port. Enter 20000 in the port field. Select Next.
In an operational network, you choose any port that is not being used by other services on your
network. We will use TCP since this protocol is more reliable.
3. Input Settings
Change the Source type to syslog and Method to IP.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
6/11
2/11/2019
99: Centralized Monitoring
You should now be at the Input Settings screen.
4. Review Settings
Select Review. Review your settings then select Submit.
5. Kali LAN - Log In
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
7/11
2/11/2019
99: Centralized Monitoring
Switch to the Kali LAN virtual machine. Log in to the Kali LAN virtual machine as root and password toor.
Now you will need to configure a client to forward syslog data to your Splunk server.
6. Kali LAN - Open rsyslog.conf
Open the Kali LAN rsyslog configuration file by first double clicking on the Computer icon located on the
desktop, click File System in the tree to the left, double click the “etc” folder, then double click the file
“rsyslog.conf”.
7. Kali LAN - Edit rsyslog.conf File
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
8/11
2/11/2019
99: Centralized Monitoring
Edit your /etc/rsyslog.conf file by adding the text below.
Add the input to the bottom of the file
$ModLoad imfile
$InputFileName /var/log/nginx/error.log
$InputFileTag nginx:
$InputFileStateFile stat-nginx-error
$InputFileSeverity error
$InputRunFileMonitor
$InputFilePollingInterval 10
*.* @@192.168.0.30:20000
Remember to scroll down to view all of the instructions listed in this step.
This configuration will make rsyslog send all of your logs to Splunk. The InputFileTag line tells
rsyslog what to add as the tag in the log records. The InputFileStateFile is the file that will keep
track of how much of that file you have already sent in and should be unique for each file that you
are using.
8. Save and Reboot Kali LAN
Save your file by clicking on File then select save. Now reboot your Kali LAN machine by clicking on "root"
located in the upper right of your Kali Desktop screen. Select Shut Down and then Restart.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
9/11
2/11/2019
99: Centralized Monitoring
9. Kali LAN - Forwarding Test
Log back into the Kali LAN virtual machine using root and toor. Open a Terminal on the Kali LAN by clicking
on the shortcut located on the top. Type the following at the command prompt:
logger -t test "this is a test of splunk forwarding"
When the Kali LAN virtual machine has finished restarting, you should be able to test sending
events to Splunk by using the command line tool logger.
10. Win7 - Search for Events in Splunk
Switch back to your Windows 7 virtual machine. Click on Start Searching. Then log into the Splunk search
interface and search for your events, starting with "this is a test ".
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
10/11
2/11/2019
99: Centralized Monitoring
11. Lab Completion
This completes all the steps required for this lab. We've set up splunk, uploaded log files for analysis and
finally learned how to forward logs across a network to splunk interface.
Congratulations, you have completed the Splunk configuration lab.
https://labondemand.com/LabProfile/Manual/40700?showScreenshots=1
11/11