Building the Future | Human Resources
SM
Building a Successful
Analytics Group
IIA Beach Cities Seminar
March 22, 2013
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Objective
• How has the SCE Audit Services
Department (ASD) built a data
analytics group, where are we now,
and where are we going? What
processes do we have and what tools
are we using.
• What do you want covered?
2
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Agenda
• Introduction & what started it for us
• Foundation
• What we are doing and where are we
going
• What tools are we using & what
training are we providing
• References
• Questions
3
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Introduction ─ Company
Edison International
Edison International, through its subsidiaries, is a generator and distributor of
electric power and an investor in infrastructure and energy assets, including
renewable energy. Headquartered in Rosemead, California, Edison International
is the parent company of Southern California Edison—a regulated electric
utility—and Edison Mission Group, a competitive power generation business.
Southern California Edison (SCE) is one of the largest electric utilities in California,
serving more than 14 million people in a 50,000 square-mile area of central, coastal
and Southern California, excluding the City of Los Angeles and certain other cities.
Based in Rosemead, California, the utility has been providing electric service in the
region for more than 120 years. SCE's service territory includes more than 180 cities.
4
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Introduction ─ Department
Audit Services
5
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Introduction ─ ATG
Audit Technology Group
An IT Audit Manager at SCE and head of "Audit Technology
Group." Fifteen years at SCE. Over thirty-six years of
experience, primarily in Information Technology auditing
and consulting in the utility, banking, insurance, and public
accounting industries, including IT management and
system implementation experience. Was a Senior Manager
at Deloitte and Vice President & Audit Manager at First
Interstate Bank of California, where I managed 5
programmers who supported the department in analytics
and continuous auditing and also performed audits.
Larry Hanson
CTO & Past President & Director ISACA LA Chapter.
CPA, CIA, CISA, CRISC, CDP. BS in Business and MSMS
from Graduate School of Business at USC.
6
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Introduction ─ ATG
Audit Technology Group
Corporate Auditor at SCE since 2005. Professional
experience includes specializing in Information Technology
data extraction, data mining, programming and other
related analysis in support of audit projects, fraud
investigations, and financial analytics. Has been the
primary data analyst within Audit Services until John joined
the group. He also participates or leads IT audits, including
what we call “Blended” reviews and SOX reviews.
Victor Alvarado
Victor previously was a Senior Systems Project Specialist
with Automobile Club of Southern California. He is a CISA
and has a Certificate in Project Management from the
University of California at Irvine and a BS in Economics,
California State Polytechnic University, Pomona.
7
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Introduction ─ ATG
Audit Technology Group
Corporate Auditor at SCE since August 2011. Was a data
analyst at Zurich Group/Farmers Insurance and supported
over 200 auditors with Computer Assisted Audit Techniques
(CAATs). Besides also performing IT audits, he conducted
training for Zurich Group Audit staff on ACL/ACL scripting,
their audit management tool (Auditor Assistant) and audit
standards. He also prepared training materials on using
ACL, Excel functions and macros, MS Access and SQL,
and data mining using Weka.
John Lee
AuditWork
Technology
Group Here
Your
Group Name
He is a CPA, CIA, and CISA and also has the following
certifications: ACL Data Analyst, Advanced SAS
Programmer, and Advanced Lotus Notes Programmer.
John has an MS in Computer Science, California State
Polytechnic University, Pomona, an MBA from Boston
University, and a BA in English in China.
8
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
What Started the Change in Focus:
ASD Strategic Plan
• ASD Strategic Plan
– Value, People, &Technology
– Technology:
• IT Tools & Training
• Embed IT Expertise
• Continuous Controls
9
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
10
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
Drivers for Data Analytics
Reliability/Productivity on Top!!!
SM
Reliability &
Productivity
11
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Better Use of
Auditor Resources
Increasingly, the knowledge and skill level of
auditors is rising to keep pace with complex
demands of comprehensive audits.
• Automation will allow the auditor to spend more time on
activities requiring the application of auditor judgment.
• By having the computer handle repetitive task using data
analytics, the use of the auditor resource can be maximized.
• Volumes of data can be sorted, matched, recalculated, and
analyzed to identify suspect data for additional investigation.
• Scripts and/or macros can be created to make analyses and
tests replicable.
Source: Internal Audit – Efficiency Through Automation – David Coderre
AuditWork
Technology
Group Here
Your
Group Name
12
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Benefits of
Data Analytics
•
•
•
•
•
•
•
•
Close control loopholes before fraud escalates
Quantifies the impact of fraud
Cost-effective
Acts as a deterrent
Can be automated for continuous auditing
Provide focus based on risk and probability of fraud
Direct pointers to critical evidence
Support for regulator compliance
Source: ACL Services Ltd.
AuditWork
Technology
Group Here
Your
Group Name
13
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Data Analytics
Processing Benefits
•
•
•
•
Review 100 percent of transactions
No limit of file size
Compare data from different applications & systems
Perform tests that are designed for audit and control
purposes
• Conduct tests proactively
• Ability to automate high-risk areas to catch fraud before it
escalates
• Maintain comprehensive logs of all activities performed
Source: ACL Services Ltd.
AuditWork
Technology
Group Here
Your
Group Name
14
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
ASD Strategic Plan & New Role
• New Role, New Group – Audit
Technology Group (ATG)
Using automation & analytics to help focus audits
on areas of greater risk; allowing us to
continuously detect issues rather than maybe once
a year or over a longer period; allowing us to look
at 100% rather than samples in many cases:
– Data Mining/Data Analytics
– Continuous Control Monitoring/Auditing
– Fraud Detection using automation
Also:
– Training on IT Risks & Controls plus Tools &
Techniques
– Access to data (SAP and more)
– Working with IT on support escalation, AIMS
technical issues, large IT projects, Windows 7…
15
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Foundation
• Set up Audit Technology Group
• Hired John as experienced data analyst—so now
have 2
• Met with each ASD Director and then presented
to their All Hands meetings on new role and how
we could help them
• On a particular procurement review provided
much more analytics than asked for. Presented
results near end of audit to General Auditor.
Results lead to future follow-up reviews.
• Reached out to multiple other compliance related
groups, including Ethics & Compliance
(responsible for investigations), Information
Security, and internal control groups.
• Set up internal training sessions
• Implemented various tools
• Established an Analytics & Monitoring Committee
AuditWork
Technology
Group Here
Your
Group Name
16
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
What we are doing ─ Data Analytic Approach
•
•
•
•
•
Audit Manager & Team engage ATG early on possible analytics
Include ATG in Walk-throughs with client
Agree on budget hours for ATG involvement
Be aware of data challenges, sensitivity & confidentiality
Data Analytics is only the starting point. Data will need to be
analyzed & verified and may require additional work.
• If possible perform 100% population testing. If not, use data
analytics to sample smarter.
• As data is retrieved with clients, ask them what SAP
T-Codes they use or other tools they use for extracting the data
and reports.
• ATG is enhancing our SAP roles and data access capabilities.
We want the capability of performing all our data extracts with
limited IT support.
17
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Possible Approaches vs. Ours
• Hire external data specialist
• Develop a central team of experts
• Develop analytics skills across the
entire audit function
There is no ‘one-size-fits-all’ model to building a team of
analytics experts.
Audit Director’s Roundtable of Corporate Executive Board:
http://cebviews.com/2010/08/02/developing-a-data-analytics-team/
18
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Using Data Analytics in
Recent ASD Engagements
• Trading Operations
– Data Analytics on Excel Spreadsheet Data
• Administration of Benefit Plans – Payroll Data Feeds
– Data Analytics on Accuracy / Completeness on Data
• HR Payroll Systems – Reconciliation
– Data Analytics on identification of “Ghost Employees”
• Inventory Audits
– Data Analytics on suspect inventory data and improve sampling
• Supply Chain Process – Bidding through Contract Admin
– Data Analytics on approval dollar limits – help identify contracts that deviate
from master agreements with vendors
• Fleet Gas Card and Gas Pump Usage
– Data Analytics on data indicating potential fraud
• Cloud Computing Risk Assessment
19
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Generic
Data Analytics Opportunities
Generic data analytic tests are applicable to
numerous audit and fraud detection situations.
• Split Transactions
• Incorrect Totals
• Even Dollar
Transactions
• Transaction Aging
• Fuzzy Name Matches
• Matched Join
• High Standard Deviation
• Transaction Volume
Summary
• Duplicate Transactions
• Suspicious Date Range
20
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Cost Management
Data Analytics Opportunities
Designed to investigate large data sets in order to identify fraud,
errors, risk and inefficiencies within organizations, Cost Management
analytics have helped organizations find money, increase operational
efficiencies and address key areas within their audit plan. They
include analytics to address the following areas:
•
•
•
•
•
•
Vendor Payments
Duplicate Payments
Phantom Vendors
Vendor Master
Vendor Discounts
Purchase Authorization
Limits
• Customer Credit
• Sales Orders to Purchase
Orders
• Payroll
• Corporate Credit Cards
• Travel & Entertainment
Expenses
21
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
General Ledger
Data Analytics Opportunities
Designed to enhance an auditor’s effectiveness in identifying errors and fraud
in financial statements, data analytic scripts are available to examine General
Ledger data based on the Statement on Auditing Standards No. 99:
Consideration of Fraud in a Financial Statement Audit (SAS 99). These
powerful analytics scripts allow auditors to conduct more efficient, consistent
and comprehensive general ledger audits and provide an increased level
assurance of the integrity of an organization’s financial statements. They
include:
•
•
•
•
•
•
•
•
Validation of Trial Balance
Temporary Accounts
Validation of Control Accounts
Analysis of Month End Balances
Profiling of Suspense Accounts
Data Validity
Segregation of Duties
Duplicate Journal Entries
•
•
•
•
•
•
•
•
Suspicious General Ledger Accounts
Suspicious Journal Entry Dates
Invalid Account Classification
Suspicious Journal Entry Amounts
Suspicious Journal Entry Description
Period Close Cut-off
Reversed Journal Entries
Unreconciled Journal Entries
22
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
LIMITS OF DATA ANALYTICS IN
CONTINIOUS CONTROLS
MONITORING, DATA MINING, AND
FRAUD DETECTION:
The only limits on what can be
accomplished are the limits of your
imagination, creativity, and access to
the data.
23
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
ASD End State
• Data Analytics
– All auditors consider data analytics on all audits
• Data Analytics on ERA IT Risk Questionnaire or part of Work
Program
• Audit Managers & Team engage ATG early in review
• All Auditors trained in analytic tools, plus some super-users
• Individual All Hands meeting topic/training & brown-bag meetings
• ASD Analytics & Monitoring Committee
• Continuous Control Monitoring & Auditing
– Due to SAP CCM monitoring, eliminate F&C annual SOX testing of
configuration controls
– Use CCM/A for risk assessments to know which areas to focus on
and not focus on
– Automate scripts to continually monitor for potential fraud
situations
24
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Data Analysis in Procurement Audit
• Including:
–
–
–
–
–
–
–
–
–
Compare Service vs. Material Spent
Reassemble PO line items into PO
Match PO by Vendor and Contract
Summarize Non-PO line items by Vendor
Assemble PO Change History
Identify Duplicate Vendor
Fraud related analyses
Material cost variance analysis
Employee Vendor match by Tel, Tax ID, and Address.
25
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Price/Cost Variance Analysis
• Including:
– Ratio analysis: Max/2nd Max, Max/Min
• Help pinpoint suspect transactions and trends
– Benford law: Leading two-digit analysis
– Completeness and Integrity: Wrong data type or
mandatory field blank
– Cross-Tabulation: Vendor and PO Category
• Organizing data to find trends
– Data Profile: Sum, Count, Mean, Median, Std. Dev.
– Identify the cost difference of the same material,
purchased by different plant.
26
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Benford Analysis
• Help identify anomalous data.
Benford Leading 2 Digits
6000
5000
4000
3000
2000
1000
0
10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97
Actual_Count
Expected_Count
Zstat_Ratio
27
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Vendor/Employee Match
• Identify whether a current employee is also a
vendor/contractor.
– Matching employee to vendor by telephone number.
– Matching employee to vendor by Tax ID.
– Matching employee to vendor by home or mailing
address.
• Identify prior employee as current
vendor/contractor and determine the basis for
award.
28
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Red Flags of Ghost Employees
•
•
•
•
•
•
•
•
•
•
•
•
Use of a common name, e.g., Smith
No physical address
No personal or vacation leave
No deductions for voluntary deductions (i.e., health insurance,
pension, etc.)
High withholding allowance to minimize income tax withholding
Missing employee information
Invalid social security numbers
IRS notice regarding invalid social security numbers
No evidence of work performance
Changes to direct deposit bank account numbers
Payroll checks that are cashed in company accounts
Duplicate bank accounts for direct deposits
29
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
CA and CCM: An Integrated Approach
From ACL Webinar: Continuous Auditing: A Core Competency for Regulatory Compliance
30
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Continuous Auditing/Monitoring
• ASD has gained direct access in production to the
top 20 SAP tables used by auditors (headers and
transactional) via ACL Direct Link.
• ATG is in the process of implementing ACL scripts
using Direct Link to potentially continuously analyze
these key tables for trends and anomalies using
summarization, classification, stratification, aging,
and other audit techniques.
• ATG is at the same time is looking for good CA
opportunities. This is ongoing but also in conjunction
with Analytics & Monitoring Committee.
31
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Continuous Auditing/Monitoring
• Working with the CCI* Compliance & Process
Controls group on implementing Continuous
Controls Monitoring (CCM) on SAP GRC version 10.
• Looking for opportunities to continuously monitor
risks to help either move up or out the start of audits.
* “Center for Continuous Improvement” (CCI) is equivalent to what other companies with SAP
refer to as “Center of Excellence” or COE.
32
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Continuous Monitoring
Queries
• Corporate Credit Cards
• Conflict of Interest (Employees and Vendors)
• Top 100 PO and Non-PO Vendors
33
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Copyright © 2008 ACL Services Ltd. 7
Continuum of Audit Analytics
24
7
365
ad hoc
•
One-off analysis
and testing
repetitive
•
•
Automated analyses
and tests
Managed and
deployed from a
central environment
continuous
•
Continual execution
of automated audit
and monitoring tests
to identify errors,
fraud and anomalies
on a timely basis
From ACL Webinar: ACL Solutions for Continuous Auditing & Monitoring
34
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
5 Levels of Capability Model
From ACL Whitepaper: The ACL Audit Analytic Capability Model
AuditWork
Technology
Group Here
Your
Group Name
35
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Copyright © 2008 ACL Services Ltd. 7
Query & Analysis
•
•
•
•
ACL’s
AuditExchange
Managed Analytics Platform for
Audit
In-depth analysis
Audit-specific commands & scripting
Advanced analytics and predictive
modeling
Centralized logging
Management & Automation
•
•
•
•
•
Audit repository
User access & rights, data security
Centralized tests and processing
Continuous auditing management
Configuration & management
Data Access
•
•
•
Access, extract, transform, load
Specialized format connectors
Audit data repository
Reporting & Presentation
•
•
•
From ACL Webinar: ACL Solutions for Continuous
Auditing & Monitoring
AuditWork
Technology
Group Here
Your
Group Name
Templates, charting
Dashboard integration
Report deployment and maintenance
Analytic Library
•
Packaged analytics, key business
processes
36
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Tools and Training
• Use ACL Network version (3 concurrent users)
• Held ACL 105-Foundations class on-site for superusers
• Implemented ACL DirectLink (for efficient and
effective access to SAP production data)
• Held Visio and Excel Analytics training sessions
• Working on using SAP Business Object
development tools (Crystal Reports, SAS, Xcelsius,
Webi).
37
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Tools and Training
• Implemented Dell Precision T7500 workstation with
SQL Server 2008 Express.
• Implementing IBM’s SPSS Modeler for data and text
mining. Other open-source versions: KNIME & Weka 3
• Implemented “R” which is a free software environment
for statistical computing and graphics.
• Using SCE’s Vovici Enterprise survey tool for
independent assessments.
• Other: Monarch, Acrobat 9 Standard, SnagIt,
Camtasia
38
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Tools ─ “R”
39
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Tools ─ KNIME
KNIME, pronounced [naim], is open-source and a modern data analytics platform that allows you to
perform sophisticated statistics and data mining on your data to analyze trends and predict potential
results. Its visual workbench combines data access, data transformation, initial investigation, powerful
predictive analytics and visualization. KNIME also provides the ability to develop reports based on your
information or automate the application of new insight back into production systems.
40
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
References and Sources
• ACL (www.acl.com)
• Richard Lanza of AuditSoftware.net
• David Coderre’s Fraud Analysis Techniques Using
ACL
• Idea (www.caseware.com/products/idea)
• R language (www.r-project.org)
• Edward R. Tufte’s The Visual Display of
Quantitative Information
• SPSS (www-01.ibm.com/software/analytics/spss)
• KNIME (www.knime.org)
• Weka 3 (www.cs.waikato.ac.nz/ml/weka)
• Dr. Nigrini on Benford’s Law (www.nigrini.com) 41
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
More References
• Audit Director Roundtable of Corporate Executive
Board (www.executiveboard.com/corporateintegrity/audit-director-roundtable/index.html)
Example in 3/21/13 CEB Email:
42
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®
Building the Future | Human Resources
SM
Questions
Audit Technology Group:
– Larry Hanson (Larry.Hanson@sce.com)
– Victor Alvarado (Victor.Alvarado@sce.com)
– John Y Lee (John.Y.Lee@sce.com)
43
AuditWork
Technology
Group Here
Your
Group Name
EDISON CALIFORNIA
INTERNATIONAL
SOUTHERN
EDISON®