Compliance System Validation
- An Audit Based Approach
December 2012
Uday Gulvadi, CPA, CIA, CISA, CAMS
Mahesh Viswanathan, CAMS
Director - Internal Audit, Risk and Compliance
Sr. Vice President
Current Challenges
• Wide range of service providers and skills
• Inconsistent quality of the assessment and deliverables
• Often independent contractors are used resulting in
lost continuity year to year
• Lacking consistent standards of performance
• Findings frequently not tied to risk and potential
impact
• Level of independence is not always clear
2
Terminology
System
Verification
System
Review
Independent
Assessment
System Audit
System
Validation
3
Independent
Review
Need for an Audit Based Approach
• Boards and management are recognizing both
o Need to perform independent validations of systems and
o Lack of consistent high quality “audit based” assessments in
the past
• Critical role of technology in BSA/AML Compliance program
• Increased scrutiny by regulators
• Mitigate the probability and impact of critical risk events
• Avoid severe regulatory penalties and reputational risk
4
Need for Audit Based Approach
• Required by FFIEC BSA Examination Manual:
o “A periodic review of the effectiveness of the suspicious activity
monitoring systems (manual, automated, or a combination) used for
BSA/AML compliance.”
o Evaluate the system’s methodology for establishing and applying
expected activity or filtering criteria
o Evaluate the system’s ability to generate monitoring reports
(Cases/alerts)
o Determines whether the system filtering criteria are risk based &
reasonable.
o Validate the auditor’s reports and work papers to determine whether
the bank’s independent testing is comprehensive, accurate, adequate,
and timely.
5
What is an Audit based approach?
Independent
& Objective
Systematic,
Disciplined
approach
Assess
conformance
to
regulations,
policies &
procedures
Assess the
culture of
compliance
6
Identify
control
weaknesses
and remedial
measures
Follow up on
action taken
Essential Requirements for Audit Based approach
Appropriate,
robust report,
work papers
Knowledge of
regulatory
expectations
Internal or
Third-Party
Credentials
and Experience
Risk Based
approach
Distinguish
regulatory
violations and
best practices.
Audit
Understandin
g of the “red
flags” unique
to the
business
Compliance
7
Technology
Audit based approach phases
Planning
and Scoping
Follow up
Review
Assessment
Validation
Report
8
Independent Validation - Components
• Should be performed by qualified individuals within the FI or by
a qualified third party
• Should be performed annually or should match the frequency of
Risk Assessment
• Should consider the alignment of BSA AML System with Risk
Assessment including
o
o
o
o
Customers
Geographies
Lines of Business
Products and Services
9
Independent Validations - Coverage
Typical Coverage
•Match Level Management
•Sanctions Filtering Rules – Thresholds,
Effectiveness & Efficiency
•Batch, Real Time and Incremental
Filtering
•Business and Functional Requirements
•User Acceptance Testing
•Application Security and administration
•Data Mapping, Interfaces and
Reconciliations
•Risk Model
•Customer Due Diligence and EDD
•Profile configurations
•AML Monitoring rules – Thresholds,
Effectiveness & Efficiency
•Audit Trails
•Case Management
10
Technical Challenges
• Assessing the functionality of rules and that the data supports
rule processing
o Logic is not always transparent
o Flaws in logic processing
o Too many false positives
• Validating all required SWIFT Messages are being scanned
• Inconsistent thresholds on rules/scenarios leading to incorrect
or no alerts
• Absence of data or poor data quality providing incorrect
customer risk classification
11
Organization’s Roles & Responsibilities
Staff and
Management
Implements
BSA/AML
Compliance
Monitors
12
Independent
Audit
Assesses
independently
Keys to an Effective Validation
Identify
high risk
services,
products
and clients
13
Audit based Performance Standards
• Consistent with professional practice standards
• Audit procedures and testing commensurate with
risk
• Quality Assurance reviews
• Build on knowledge of best practices
• Continuous improvements methodology
• Confidentiality and Security protocols
• Specialized analytical tools
14
Deliverables
• Assessment Report
o Key observations
o Associated risks and potential impact
o Recommendations for risk remediation
• Significant Items Management Action Plan
o Living document with significant findings
o Management responses
o Remedial action plan with “Ownership” and due dates
• Test Work Papers and Supporting Documentation
15
How to select a Third Party Vendor?
• Should integrate three essential skillsets:
o Audit expertise
o Compliance & regulatory knowledge
o Strong technology and in-depth product knowledge
• Well defined structured process/framework that is
adaptive
• Completely independent
• Continuity of permanent staff
• Professional Certifications – CPA, CIA, CAMS CCRP etc.
• Good customer references
16
Essential qualifications
Internal Staff or
Third-Party
Credentials and
Experience
Understands
Your Institution
Knowledge of
Regulatory
Requirements
Audit
Establishing
Expectations
Compliance
17
Technology
Questions
18