Event Logs
Management
Anthony LAI, CISSP, CISA
Founder & Editor
InfoSec Hong Kong
1
Why do I come up an
idea to present this?
First of all, I am not the event log expert
but the one suffered from event log
burden. 
Think of ideas to improve it in a bank.
No budget and complicated approval
process there , forcing me to seek for
free software.
2
3
InfoSec Hong Kong
www.infosechk.org
Goal

Promote security awareness to public and education sectors.
Our advantages

We act as a bridge between security concepts and commercial
solutions
Members


Over 370 registered members
Enjoy discounted products from our sponsors
Current Status


Over 64,000 visit since Aug 2004
News, seminar, training, advertisement, consultancy service for
SMEs.
Future



Reflect the voice to government from public and focus on more
security education and provide solutions to public.
It is a long term commitment
Invitation for more commercial sponsorship
4
Author Profile
Anthony Lai a speaker for the International Information Systems
Security Certification Consortium, Inc. [(ISC)2®], the non-profit
international leader in educating, qualifying and certifying information
security professionals worldwide. (ISC)2 issues the Certified
Information Systems Security Professional (CISSP) and Systems
Security Certified Practitioner (SSCP) credentials and related
concentrations to those meeting necessary competency requirements.
Anthony is also a founder and editor of the “InfoSec Hong Kong”
Website with over 300 registered public members. He currently is a
chapter leader of OWASP (The Open Web Application Security ProjectHK Chapter) and serves as a program committee member in PISA
(Professional Information Security Association) and ISSA (HK
Chapter).
His major interest is in web application security, various penetration
tests and forensics. He is a column writer on information security in
Hong Kong local newspaper – “Apple Daily”. He has certified as
CISSP and CISA.
5
What will I cover?
Event Logs Monitoring Problems
Solutions (You may most concern about
it)
Demonstration (You may most be
interested in it )
Experience sharing among all of you.
6
Real-Case: Event Log
Monitoring Problem in a
Bank
It have 50 systems, and 12 windows servers and
other legacy systems.
In the past, the one taking the information security
officer just use his “X-Ray” Eyes to scan over all the
Logs print-outs (over 400~600 papers) to detect
errors every working day (Remark: I believe he is
readily a superman, otherwise, he will be blind!)
No filtering, No selection for critical events.
Use half day to review the logs.
No standard document about log review process.
No solutions from other branches in other countries.
7
Real-Case: Event Log
Monitoring Problem in a
Bank
Their current solutions:
Print out the event logs with heaps of
papers, and glance over it and then
chop, chop chop with their signature.
Package them up and put into the
inventory
Auditor has no comment on this kind of
log review process.
8
The Challenges
Bureaucratic approval process ( I can’t count how
many tiers I need to pass even I just want to get a
few thousand dollars)
Free and cost saving
Efficient to solve the problem
Shorten the log review time
Automate the log download process
Patch the log file with appropriate headers and
format.
Save our world, save papers!
In addition, increase the opportunity to detect any
critical events and suspicious activity.
No one has knowledge in this area.
9
What did I suggest to
bank?
Long Term

Security Operation Center (SOC)


(but I think it is too early because I cannot find
they have put some efforts in Incidence
Response Planning and Implementation)
Logs Filtering and Processing Server
Short & Mid Term

Commercial event log review software
10
Roadmap to building log
processing standard
Log Nature and Element


Define the elements needs for a log records
What are the critical events?
Operation

Log patching, download, process and retention as
well as purging.
Review


What are we looking at?
The procedure to report any suspicious events.
11
More challenges from
my observation
Various systems with various standard
in logging -> Hard to integrate.
Cultural problem and political problem
No one thinks about the system
infrastructure.
12
Log Wiping Risk and Attack
13
Event Logs Monitoring Tools
Microsoft Log Parser 2.2
http://www.microsoft.com/technet/scriptcenter
/tools/logparser/default.mspx
Kiwi Syslog Products:
http://www.kiwisyslog.com/Syslogs
Remstats http://remstats.sourceforge.net/release/logserver.html
Set up a Linux log server
http://www.linuxsecurity.com/content/view/117
514/49/
14
Log Parser
Search for Data - Search for the logons of a specific user
among the events in the Windows Event Log:
C:\>LogParser "SELECT TimeGenerated, SourceName,
EventCategoryName, Message INTO report.txt FROM Security WHERE
EventID = 528 AND SID LIKE '%TESTUSER%'" -resolveSIDs:ON
15
Log Parser (2)
Create Reports - Create customformatted HTML reports.
16
Log Parser (3)
Calculate Statistics -Calculate
the distribution of the HTTP
response status codes from
your IIS log files:
C:\>LogParser "SELECT sc-status, COUNT(*)
AS Times INTO Chart.gif
FROM <1> GROUP BY sc-status ORDER BY Times
DESC" –
chartType:PieExploded3D chartTitle:"Status Codes"
And produce a chart formatted as desired:
17
Log Parser - Syntax
Examples:
LogParser "SELECT date, REVERSEDNS(c-ip) AS Client, COUNT(*)
FROM file.log
WHERE sc-status<>200 GROUP BY date, Client" -e:10
LogParser
file:myQuery.sql?myInput=C:\temp\ex*.log+myOutput=results.c
sv
LogParser -c -i:BIN -o:W3C file1.log file2.log "ComputerName
IS NOT NULL"
Help:
-h GRAMMAR
: SQL Language Grammar
-h FUNCTIONS [ <function> ] : Functions Syntax
-h EXAMPLES
: Example queries and commands
-h -i:<input_format>
: Help on <input_format>
-h -o:<output_format>
: Help on <output_format>
-h -c
: Conversion help
18
Log Parser – Sample
Output
Server
EventID Total
------------ ----HKGKABS1 528 420
HKGKABS1 529 1
HKGKABS1 538 419
HKGKABS1 539 1
HKGKABS1 576 420
HKGKABS1 578 2
HKGUATS1 528 73
HKGUATS1 538 71
HKGUATS1 576 73
HKGUATS1 578 11
………….
Statistics:
----------Elements processed: 1130
Elements output: 10
Execution time: 0.19 seconds
19
From logs download to
output reports (1a)
Automatic Download
strTarget = ""“P:\SecurityLogs\log\t_Server1.txt"""
Set objShell = CreateObject("WScript.Shell")
Set objExec = objShell.Exec("cmd.exe /C dumpel -f " &
strTarget & " -s Server1 -d 1 -l security -c -format
dtIucs")
strPingResults = LCase(objExec.StdOut.ReadAll)
‘dumpel is an executable to download logs
20
DUMPEL Usage:
dumpel -f file [-s \\server] [-l log [-m source]] [-e n1 n2 n3..] [-r] [-t] [-d
x]
-d <days>
Filters for event last days (number larger than zero)
-e nn
Filters for event id nn (up to 10 may be specified)
-f <filename>
-l <name>
-b
Output filename (default stdout)
Dumps the specified log (system, application, security)
Dumps a backup file (use -l to specify file name)
-m <name>
-r
Filters for events logged by name
Filters out events logged by name (must use -m too)
-s <servername> Remote to servername
-t
Use tab to separate strings (default is space)
-c
Use comma to separate fields
-ns
Do not output strings
-format <fmt>
Specify output format. Default format is
dtTCISucs
where
t - time
From logs
download
to output
reports
(1b)
d - date
T - event type
C - event category
I - event ID
S - event source
u - user
c - computer
s - strings
21
From logs download to output
reports (2a)
strTarget1 = ""“Q:\log\window\script\Daily\header.txt"""
strTarget2 = ""“P:\SecurityLogs\log\" & genmonth & genday &
"t_Server1.txt"""
‘Copy log files to target directory with appended header
Set objShell = CreateObject("WScript.Shell")
Set objExec = objShell.Exec("cmd.exe /C copy " & strTarget1 &
"+" & strTarget & " " & strTarget2)
strPingResults = LCase(objExec.StdOut.ReadAll)
`Invoke and run Log Parser, it outputs chart.
Set objShell = CreateObject("WScript.Shell")
Set objExec = objShell.Exec("cmd.exe /C
P:\SecurityLogs\Logparser.exe
file:P:\SecurityLogs\EventIDDistrib_Overview.sql?sourcefile
=P:\SecurityLogs\2005log\" & genmonth & genday &
"t_Server1.txt+destfile=P:\SecurityLogs\BarChart\" &
genmonth & genday & “Overview_Server1.gif -i:csv -o:CHART charttype:BarClustered -categories:ON -values:ON charttitle:Event_Logs_Overview_Report_" & genmonth & genday
& “Server1 -legend:ON -groupsize:1024x700")
strPingResults = LCase(objExec.StdOut.ReadAll)
22
From logs download to
output reports (2b)
‘Output CSV file with logs breakdown
Set objShell = CreateObject("WScript.Shell")
Set objExec = objShell.Exec("cmd.exe /C
P:\SecurityLogs\Logparser.exe
file:P:\SecurityLogs\EventIDDistrib_with_select
ed_event_ID.sql?sourcefile=P:\SecurityLogs\log\
" & genmonth & genday &
"t_Server1.txt+destfile=P:\SecurityLogs\BarChar
t\" & genmonth & genday & "Selected_Server1.csv
-i:csv -o:csv")
strPingResults = LCase(objExec.StdOut.ReadAll)
23
Event Logs Review
Process (3)
Add the header to the beginning of the file.

Date,Time,EventID,SourceName,Dummy,Server,D
escription . They are used in the SQL query.
SQL Query in the
EventIDDistrib_with_selected event_ID.sql:
SELECT StrCat(TO_STRING(EventID),Description) as
EventID_And_Source,Count(*) as Total
FROM %sourcefile% To %destfile%
WHERE EventID in (529; 530; 531; 532; 535; 537; 539;608; 609;
612; 613; 614; 615; 616; 617; 620; 624; 625;626; 627; 628;
629; 630; 631; 632; 633; 634; 635; 636;637; 638; 639; 640;
641; 642; 643; 644; 645; 646; 647;648; 649; 650; 651; 652;
653; 654; 655; 656; 657; 658;659; 660; 661; 662; 663; 664;
665; 666; 667; 675; 676; 677)
GROUP BY EventID_And_Source
24
Final Log Review Report
Element
Scope and Content

Which servers do we monitor?
Grand Total Figures:

It shows no. of logs for each event for every server
Group the events into different categories:
Appendix with detailed event statistics


Statistics with no breakdown.
Detailed statistics breakdown with log description
Appendix of selected critical events
25
Group the events for the management report
Event Group
Event
Suspicious Finding After
Following Up (Yes/No/No
Occurrence)
Audit Log and Policy
Access/Change
517, 612
No Occurrence
Account Lockout
539,644
No
Failed Account Access
529-535, 537
No
Account Profile Change
(Normal User/Administrator)
608, 609, 624-630, 642-647
No
User Role/Group
Change/Addition
631-639, 641, 648-667
No
Domain Policy & User
Database Change
640,643
No
26
Event
ID
Message
Type
Description
512
User
Windows NT is starting up
513
User
Windows NT is shutting
down
514
User
An authentication package
loaded by Local Security
Authority
515
User
A trusted logon process
registered with Local
Security Authority
516
User
Internal resources allocated
for queuing of audit
messages exhausted
517 *
User
Audit log cleared
518
User
A notification package
loaded by Security A/C Mgr
528
User
Successful Logon
Risk (H/M/L)
Critical
Events
Selection
Different
parties will
have different
risk ranking
on particular
event.
27
Challenge again…
How to justify the event is not
suspicious?
How to filter out those duplicated event
logs further?
We need to keep an eye on any new
patch or/and updates, it may add some
new event we have not captured.
28
Current weakness
No co-relation could be drawn.
No short-cut, all manual effort in
investigation
29
Incidence Response or
Ignorance Response?
Again, log filtering and review is not a goal,
we need to response any irregularities once it
is discovered.
One more mindset from CEO: “We are doing
“XXXX” business, not “information security”!
If the IR is reviewed once a year due to the
coming of HKMA, generally, the staff and
management awareness is lacking, it is hard
to go ahead, may be, people may response
with ignorance.
30
Summary
Logs are here, but not the one you desire.
Logs are obtained but useless if you don’t
know how to review.
Logs are reviewed but useless if there is no
process to response.
Shorten the Fault Detection Time. However, it
needs management support.
As an system architect, they may need to
discuss with other architects and security
officer about log facilities for better
management and integration.
31
Resources
An unofficial Log Parser support site http://www.logparser.com/
Log Parser Toolkit (book)
http://www.syngress.com/catalog/?pid=3110
How to generate web-based report (I have discussed with
others in http://www.logparser.com) , search “How to create a
customized HTML report?” in the forum.
Understanding Windows Logging
 http://www.windowsecurity.com/articles/Understanding_Wind
ows_Logging.html
Tutorial and Tools
 http://www.windowsecurity.com/pages/search.asp?query=Lo
g+Monitoring
32
Any questions?
Please feel free to share with me.
You could reach me at
anthonylai@infosechk.org or
anthonylai@owasp.org
33