Kevin Savoy
Director of Hospital and IT Audits
University of Virginia






SIEM – just another acronym?
What is it
Why
UVA approach
Audit Objectives
Audit Program




Depends on who you talk to but usually defined
as a holistic approach to security management.
Security event management was the world of real
time monitoring (Firewalls, IDS, etc.), detection,
notification, and action.
Security information management was the world
of log retention, analysis, and reporting.
Some have the two terms flipped. But put both
together and you have SIEM!



Used to be “good enough” to put the fire out
then worry as an afterthought about what
went wrong or was prevented.
Controls were always either preventive,
detective, or corrective.
Compliance with laws such as notification of
affected parties, E-discovery and the related
preservation of data, and due diligence
requires a strong SIEM as well.
Preserved Data
Legal/Audit/IT Security
Firewall
Routers
Mgmt. Console
IDS
Correlation Engine
Warnings
Unix App
Database
Syslogs
Personal Devices

Think of every step that should be performed in
managing events:
◦
◦
◦
◦
◦
◦
◦
◦
◦
◦
Determining risks
Blocking or Quarantining
Detecting issue or anomaly or pattern
Event Correlation
Logging
Alerting
Analysis
Data integrity
Correcting or recovery
Reporting and debriefing


If we don’t catch or properly handle attacks
or other issues we may mess up the end
game such as correcting, and notifying
affected parties.
Just as bad is if we have the tools and logs
and other devices but fail to use them. The
laws and courts may say that we failed in our
due diligence to our patients, employees, and
other stakeholders.


Should not be thought of as just appliances
and software, although that is certainly what
makes it easier.
SIEM should be thought of as the total
process. From detection (or prevention) all
the way to providing enough information for
lawyers, courts, stakeholders, law
enforcement, and ourselves (auditors).


Vendors are creating new appliances and
software to do log parsing, correlation and
anomaly analysis, continuous monitoring,
and spot file integrity issues and more.
The systems take input from firewalls, IDS,
routers, operating system, databases,
applications, authentication servers and on
and on to check for issues and collect and
store information for later processing.



Most SIEM systems are a myriad of
components that feed into others. For
instance a dashboard may be presented
through a central SIEM console.
However analyzing may take place within
other servers and the saving of log
information in other servers.
Its like anything in life you can go piece meal
or big bang!



We have automated payroll, AR, EMR,
manufacturing, insurance claims and on and on.
Why not security management. (Don’t worry there
will still be security folks).
One day your console may blink that a risky
Internet Protocol based on known patterns got
through your router, firewall, and a Unix server
directory permissions are now different, and that
all e-backups have been locked down for secure
restoration.



Definition of an APT is a concerted effort with
the resources to attack you often and in force
(manpower and brainpower).
Often foreign governments or criminal
entities.
Banking and defense were initial targets. Now
everyone is fair game including Universities
and Health Care..



Your university and/or hospital may be Fort
Knox, so those performing APTs often are
going after smaller entities.
They know that defenses may have had been
implemented to a lesser degree at smaller
operations. (think smaller higher education,
outpatient clinics and physician offices)
College at Wise story


Think of all the data that is present every
second that can be a potential signal that
something is amiss.
SIEMs attempt to take that overwhelming
information and make sense of it in addition
to preservation.




Cost
Standardization
Myriad of protocols to deal with from
appliances feeding information
What data to keep and where to keep it



To misquote a recently deceased dictator:
the audit of SIEM could be the “mother of all
audits…”
You are in essence doing an audit of an
automated IT audit function…..
May want to attack it piece meal or swallow
it whole. It depends on how much time you
have.

Two main audit objectives:
◦ NUMBER 1 Objective - An audit of the SIEM automation itself and
what it provides and is it useful to the organization









Policies
Procurement of SIEM infrastructure (cost/benefit of control)
Configuration diagram (what’s talking to what and how)
Prevention (the usual, do you stop, allow with warning etc.)
Warnings (how soon, who receives, what technology)
Response
Follow-up
Legal and operational log retention (does it meet legal obligations)
Legal actions

Two main audit objectives:
◦ NUMBER 2 Objective - An audit of the Security of the
SIEM automation
 Access to electronic and paper logs
 Security over audit tripwires (what causes logging or alarm)
 Security over interfaces between component appliances (who
controls, encrypted?)
 Security of the component appliances (physical and logical).
Some of these may be covered in some of your other audits
such as database, operating system, network audits.

Objective 1
◦ Determine the effectiveness of the SIEM automation
Step 1.
Obtain policies and procedures to gain an
understanding of the SIEM strategy of your
organization.
Step 2.
Determine that procurement of appliances are in
line with organization strategy and that costs do not
exceed benefit of the control either singularly or in
combination with other controls. (Use NPV etc.)
Step 3.
Obtain a diagram of all components with their
functions labeled (prevention, warning, logging,
retention). Also diagram should show all data
flows between components. Manual processes
should be noted as well.
Step 4.
Determine that all components and data flows are
used efficiently to attain prevention (if possible),
logging, warning, response, and preservation of data.
Step 5.
Determine that warning tripwires have been planned
ahead of time and escalation procedures are
appropriate. What technology is used and is it
appropriate: email, text message, phone call etc. and
at what level is authority given to stop the
alarm from being reported higher and is that
appropriate based on risk.
Step 6.
Determine that events are acted upon (could be part of
another audit such as incident response). Determine that
responses to events were appropriate.
Step 7.
Determine that legal actions were sufficient for any
incident (again could be part of another audit such as
incident response). Affected parties should have been
notified if your organization was responsible for doing
so.
Step 8.
Determine that any SIEM generated data that must be
preserved by law has been stored appropriately.

Objective 2
◦ Determine the security over the SIEM automation
Step 9.
Determine that access to paper and electronic logs is
well thought out and that procedures exist for the
granting, provisioning and removal of access to logs.
Step 10.
Determine the appropriateness of who determines what
protocols and events are logged and acted upon. Usually
this is a group effort as firewall administrators have their
concerns, a Unix administrator would have concerns, an
application owner would have their concerns etc…
Step 11.
Determine that access to audit tripwire configuration is
controlled by proper access procedures.
Step 12.
Determine what interfaces move data between
components. Be vigilant of data that can be modified
or read in clear text.
Step 13.
Determine that security over warning transmissions is
appropriate and that high risk messages can not be
read or stopped during the escalation process.
Step 14.
Determine that individual appliances are secure at
the operating system, database level, network level,
and application level. This may be done in other
audits.


SIEM may be old thinking in new packaging, but
technology has advanced where audit needs to
take advantage of any components that have
been installed.
In the past we have looked at IDS, Firewalls,
Routers, Servers, Interfaces, but the more SIEM
appliances that coordinate these functions exist
we need to stay on top of the extra control our
organizations may be achieving without our
knowledge. So ask is the first step!!!

savoy@virginia.edu