Application-level IT Risk Assessment
ISACA Denver Chapter Meeting
February 21, 2008
Kerry L. Shackelford
KLS Consulting LLC
Outline
Why this topic?
SEC interpretive guidance
ABC’s implementation approach
Design of the ITRA model
Model walk-through / Q&A
KLS Consulting LLC
Why This Topic?
GRC Spending Skyrockets
Governance
Risk
Compliance
Board and Entity
Management
Enterprise Risk Mgt
(COSO, COCO)
Public Companies
(Sarbanes-Oxley, NYSE,
Nasdaq, Turnbull, etc.)
Corporate Policy and
Procedure Management
Operational Risk Mgt
SOX-Like
(Japan, Canada, EU)
IT Governance
(CobiT, ISO 17799 &
27001-ISM)
IT Risk Mgt
(CobiT, ITIL, etc.)
Specific Areas
(PCI-DSS, AML, etc.)
Internal Audit
Departments
Financial Institution Risk
Mgt (Basel II, etc.)
Personal Information
(FTC, HIPAA, GLBA,
COPPA, EUD, etc.)
KLS Consulting LLC
Why This Topic?
US Congress Responds
PCAOB Created
(07/30/02)
ICFR Opinions
Large Accelerated
Filers
(FYEs 11/15/04+)
PCAOB Proposes
AS2
(10/07/03)
Roundtable
Feedback
(04/13/05)
PCAOB & SEC
Approve AS2
(03/09/04 &
06/17/04)
PCAOB Policy
Statement
(05/16/05)
KLS Consulting LLC
Why This Topic?
Corporate Outcry Begins
“The first-year implementation of new
requirements for public companies’
internal control over financial reporting
(ICFR) proved more burdensome and
costly than expected, resulting in an
outcry from corporate America.”
Journal of Accountancy, Two Years and Counting, June 2007
KLS Consulting LLC
Why This Topic?
Fix: Audit Firms
Per the PCAOB Policy statement issued
5/16/05, the auditors should—
Integrate their audits
Tailor audit plans to their client’s risks
Use a top-down approach
Use the work of others
Communicate directly and timely with clients
KLS Consulting LLC
Why This Topic?
SOX Year Two - 2005
PCAOB SAG Re:
Internal Control
(06/08/05)
Internal Control
Audit Inspections
Report
(05/01/06)
ICFR Opinions
Accelerated Filers
(FYEs 07/15/05+)
Roundtable
Feedback
(05/10/06)
AS2
Implementation
Report
(11/30/05)
And Then?
KLS Consulting LLC
Why This Topic?
Corporate Outcry (Cont)
The average cost of being a public
company with revenue under $1 billion
rose $1.6 million, or 130%, since the
Sarbanes-Oxley era began.
Source: “Second Anniversary: The Impact of Sarbanes-Oxley,” Institutional
Shareholder Services, www.issproxy.com
KLS Consulting LLC
Why This Topic?
Fix: Issuer (& Audit Firms)
PCAOB Announces
AS2 Rewrite
(12/xx/06)
PCAOB Adopts AS5
Replacing AS2
(05/24/07)
Management’s
Report Required
(FYEs 12/15/07+)
PCAOB Proposes
Guidance for
Issuers
(12/20/06)
SEC Interpretive
Guidance
Effective
(06/27/07)
SEC Announces
Small Biz C&B
Study
(02/01/08)
SEC Approves
Interpretive
Guidance
(05/23/07)
AS5 Effective
(FYEs 11/15/07+)
ICFR Opinions NonAccelerated Filers
(FYEs 12/15/09+)
KLS Consulting LLC
SEC Interpretive Guidance
For Issuer Management
Guidance Regarding Management’s
Report on Internal Control Over Financial
Reporting
Effective Date: June 27, 2007
www.sec.gov/rules/interp/2007/33-8810.pdf
ACTION: Interpretation.
KLS Consulting LLC
SEC Interpretive Guidance
Underlying Principles
Management should:
Evaluate whether it has implemented
controls that adequately address the risk that
a material misstatement of the financial
statements would not be prevented or
detected in a timely manner.
Base its assessment of risk on the evaluation
of evidence about the operation of its
controls.
KLS Consulting LLC
SEC Interpretive Guidance
Benefits
KLS Consulting LLC
ITRA
Overview - Approach
 Use risk factors (risk assessment evaluation
criteria) to assess the level of inherent risk and
control risk for each application system.
 Use the resultant risk ratings to determine the
level of overall risk according to the Company's
methodology.
 Use the overall risk assessment rating to guide
the appropriate level of internal control
evaluation procedures to be applied.
KLS Consulting LLC
KLS Consulting LLC
ITRA
Model Walk-Through
KLS Consulting LLC
ITRA
Run Settings
Assignment of point values to risk factors
Break points which define Low, Medium,
and High risk applications
Excluding risk factor categories from
results
Excluding missing / unknown data
KLS Consulting LLC
ITRA
Risk Factors
Information Categories
APPL (Application Systems)
ADOS (Application / Database Server
Operating Systems
DBMS (Data Base Management Systems)
Plus basic APPL information
Bias towards objective vs subjective
evaluation criteria
KLS Consulting LLC
ITRA
APPL Basic Information
Name
SOX-Indicator-IC-Dept
Vendor-Name
Original-ImplementationDate
 Major-ReleaseImplementation-Date
 Software-Version
 Support-Source




 Infrastructure ManagementSource
 App-Server-OS-Vendor,
Product, Version, & SPLevel
 DB-Server-OS-Vendor,
Product, Version, & SPLevel
 DB-DBMS-Vendor, Product,
Version, & SP-Level
KLS Consulting LLC
ITRA
APPL Risk Factors (1 of 2)
 Vendor-Reputation
 Months-Post-OriginalImplementation-Date
 Months-Post-Major-ReleaseDate
 Version-Supported
 Users-Count
 Customization








User-Configurable
Simple-or-Complex-Logic
Interfaces-Total-Count
Interfaces-Manual-Count
Changes-Count-Normal
Changes-Count-Emergency
Failures-Count
Restores-Count
KLS Consulting LLC
ITRA
APPL Risk Factors (2 of 2)
Gaps-Security-Count
Gaps-Changes-Count
Gaps-QAAR-Count
Gaps-SOD-Count
Gaps-Other-Count
Outages-Count-Days
Outages-Hours
Processes-SupportedCount
BP-Risk-AverageInherent
Materiality-I-Count
Materiality-G-Count
Materiality-S-Count
IT Tier
KLS Consulting LLC
ITRA
ADOS Risk Factors
 Outsourcer-SAS 70 Report
Opinion, Testing
Exceptions-Moderate, &
Testing Exceptions-Major
 App Server OS-VendorReputation
 DB Server OS-VendorReputation
 App Server OS-VersionSupported
 DB Server OS-VersionSupported
 Changes-Count
 Failures-Count
 Gaps-Security-Count
 Gaps-Changes-Count
 Gaps-QOSR-Count
 Gaps-Other-Count
 Production-Server-Count
KLS Consulting LLC
ITRA
DBMS Risk Factors
Vendor-Reputation
Version-Supported
Changes-Count
Failures-Count
Gaps-Security-Count
Gaps-Changes-Count
Gaps-QDBR-Count
Gaps-Other-Count
KLS Consulting LLC
ITRA
Model Walk-Through (cont)
KLS Consulting LLC
ITRA
Major Data Sources
 IC Department
APPL Lists
CMS Reports
APPL Narratives
Detailed Assessment
ITGC Documentation
Gap Logs
 Evaluator Judgment
 Internet Research
 IT Department
APPL Lists
Infrastructure Lists
Change Records
Outage Reports
Problem Reports
 Outsourcers
SAS 70 Reports
Change Records
Problem Reports
KLS Consulting LLC
Q&A
Kerry L. Shackelford
720-839-6359
Kerry@KLSConsultingLLC.com