Managing the Business Risk of Fraud using Sampling and Data Mining Mike Blakley Presented to: Fall 2009 Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC PWC Global Survey – Nov, 2009 “Economic crime in a downturn” Sharp rise in accounting fraud over the past 12 months Accounting fraud had grown to 38 percent of the economic crimes in 2009 Employees face increased pressures to : – – – meet performance targets keep their jobs keep access to funding Managing the business risk of fraud EZ-R Stats, LLC Survey findings Greater risk of fraud due to increased incentives or pressures More opportunities to commit fraud, partially due to reductions in internal finance staff While companies are expecting more fraud, they have not done much People who look for fraud are more likely to find it Managing the business risk of fraud EZ-R Stats, LLC Session objectives Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Managing the business risk of fraud EZ-R Stats, LLC Session agenda - 1 Introduction and the Process for Managing the Business Risk of Fraud Introductions All Around Course Objectives Framework of risk management for fraud Fundamentals of data mining Data mining: The Engine That Drives analysis – Analytics and Regression Sources of Analytics Data Basic and Intermediate ARTs SAS 56 IIA Practice Advisory 2320 The Yellow Book (2007 revision) The Guide – “Managing the Business Risk of Fraud” Managing the business risk of fraud EZ-R Stats, LLC Session Agenda (cont’d) – Sampling refresher Sampling The sampling process Sampling methods RAT-STATS – – – – – – – – Random Numbers Determining Sample Size Case Study Attribute sampling Variable Sampling Case study Stratified Sampling Obtaining and Interpreting the results Other Sampling Approaches DCAA Audit Package Sequential Sampling Overview of the process Attribute Sampling Variable Sampling Managing the business risk of fraud EZ-R Stats, LLC Session Agenda (cont’d) – Linear regression as an audit tool Regression Analysis Overview Terms Statistical basis Charting Regression … Seeing Is Believing Plotting Data – Statistical Intervals – – – Inserting a “Trend line” Confidence Intervals Prediction Intervals Calculation of Statistical “Confidence Bounds” Case Study - Wake County Schools Bus Maintenance Managing the business risk of fraud EZ-R Stats, LLC Session Agenda (cont’d) – Data mining, or How to test 100% Overview Statistical Basis Data Conversion and Extraction Data mining objectives – – – – Classification Trends Identification of extremes Major types of data analysis Numeric Date Text Managing the business risk of fraud EZ-R Stats, LLC Session Agenda (cont’d) – Excel as an Analytics tool Macros Tools – Data Analysis The Macro facility – – Adding a little “class” to your audit VBA – “friend” or “foe” Managing the business risk of fraud EZ-R Stats, LLC Handout (CD) CD with articles and software PowerPoint presentation More info at www.ezrstats.com Managing the business risk of fraud EZ-R Stats, LLC “Cockroach” theory of auditing If you spot one roach…. Managing the business risk of fraud EZ-R Stats, LLC “Cockroach” theory of auditing There are probably 30 more that you don’t see… Managing the business risk of fraud EZ-R Stats, LLC Statistics based “roach” hunting Many frauds coulda/woulda/shoulda been detected with analytics Managing the business risk of fraud EZ-R Stats, LLC Overview Fraud patterns detectable with digital analysis Basis for digital analysis approach Usage examples Continuous monitoring Business analytics Managing the business risk of fraud EZ-R Stats, LLC Objective 1 The Why and How Three brief examples ACFE/IIA/AICPA Guidance Paper Practice Advisory 2320-1 Auditors “Top 10” Process Overview Who, What, Why, When & Where Managing the business risk of fraud EZ-R Stats, LLC Objective 1a Example 1 Wake County Transportation Fraud Supplier Kickback – School Bus parts $5 million Jail sentences Period of years Managing the business risk of fraud EZ-R Stats, LLC Objective 1a Too little too late Understaffed internal audit Software not used Data on multiple platforms Transaction volumes large Managing the business risk of fraud EZ-R Stats, LLC Objective 1a Preventable Need structured, objective approach Let the data “talk to you” Need efficient and effective approach Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Regression Analysis Stepwise to find relationships – – Forwards Backwards Intervals – – Confidence Prediction Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Data outliers Sometimes an “out and out Liar” But how do you detect it? Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Data Outliers Plot transportation costs vs. number of buses “Drill down” on costs – – – Preventive maintenance Fuel Inspection Managing the business risk of fraud EZ-R Stats, LLC Scatter plot with prediction and confidence intervals Managing the business risk of fraud EZ-R Stats, LLC Objective 1a Example 2 Cost of six types of AIDS drugs Total Cost of AIDS Drugs Dollar Amount 200 150 NDC1 NDC2 100 NDC3 50 NDC4 0 NDC5 NDC1 NDC2 NDC3 NDC4 NDC5 NDC6 NDC6 Drug Type Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Medicare HIV Infusion Costs CMS Report for 2005 South Florida - $2.2 Billion Rest of the country combined $.1 Billion Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Pareto Chart Medicare HIV Infusion Costs - 2005 ($Billions) data source: HHS CMS 120.0% Annual Medicare Costs 100.0% 80.0% Pct 60.0% Cum Pct 40.0% 20.0% 15 13 11 9 7 5 3 1 0.0% County Managing the business risk of fraud EZ-R Stats, LLC Objective 1a Example 2 Typical Prescription Patterns AIDS Drugs Prescription Patterns Dollar Value 60.0 NDC1 50.0 NDC2 40.0 NDC3 30.0 NDC4 20.0 NDC5 10.0 NDC6 0.0 Prov 1 Prov 2 Prov 3 Prov 4 Prov 5 Prov 6 Prescriber Managing the business risk of fraud EZ-R Stats, LLC Example 2 Objective 1a Prescriptions by Dr. X Dollar Amount Dr. X compared with Total Population 350 300 250 200 150 100 50 0 Population Dr. X NDC1 NDC2 NDC3 NDC4 NDC5 NDC6 Drug Type Managing the business risk of fraud EZ-R Stats, LLC Example 2 Objective 1a Off-label use Serostim – – – Treat wasting syndrome, side effect of AIDS, OR Used by body builders for recreational purposes One physician prescribed $11.5 million worth (12% of the entire state) Managing the business risk of fraud EZ-R Stats, LLC Example 3 Objective 1a Revenue trends Overall Revenue Trend Annual Billings 1.2 1.15 1.1 Overall 1.05 Linear (Overall) 1 0.95 0.9 2001 2002 2003 Calendar Year Managing the business risk of fraud EZ-R Stats, LLC Example 3 Objective 1a Dental Billings Rapid Increase in Revenues Annual Billings ($millions) 5 4 Billings A 3 Billings B 2 Linear (Billings A) 1 0 2001 2002 2003 Calendar Year Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Guidance Paper A proposed implementation approach “Managing the Business Risk of Fraud: A Practical Guide” http://tinyurl.com/3ldfza Five Principles Fraud Detection Coordinated Investigation Approach Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Managing the Business Risk of Fraud: A Practical Guide ACFE, IIA and AICPA Exposure draft issued 11/2007, final 5/2008 Section 4 – Fraud Detection Managing the business risk of fraud EZ-R Stats, LLC Guidance Paper Five Sections – – – – – Fraud Risk Governance Fraud Risk Assessment Fraud Prevention Fraud Detection Fraud Investigation and corrective action Managing the business risk of fraud EZ-R Stats, LLC Risk Governance Fraud risk management program Written policy – management’s expectations regarding managing fraud risk Managing the business risk of fraud EZ-R Stats, LLC Risk Assessment Periodic review and assessment of potential schemes and events Need to mitigate risk Managing the business risk of fraud EZ-R Stats, LLC Fraud Prevention Establish prevention techniques Mitigate possible impact on the organization Managing the business risk of fraud EZ-R Stats, LLC Fraud Detection Establish detection techniques for fraud “Back stop” where preventive measures fail, or Unmitigated risks are realized Managing the business risk of fraud EZ-R Stats, LLC Fraud Investigation and Corrective Action Reporting process to solicit input on fraud Coordinated approach to investigation Use of corrective action Managing the business risk of fraud EZ-R Stats, LLC “60 Minutes” – “World of Trouble” 2/15/09 – Scott Pelley – – – – – Fraud Risk Governance – “one grand wink-wink, nod-nod “ Fraud Risk Assessment - categorically false Fraud Prevention – “my husband passed away” Fraud Detection - We didn't know? Never saw one. Fraud Investigation and corrective action - Pick-APayment losses $36 billion Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Section 4 – Fraud Detection Detective Controls Process Controls Anonymous Reporting Internal Auditing Proactive Fraud Detection Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Proactive Fraud Detection Data Analysis to identify: – Anomalies – Trends – Risk indicators Managing the business risk of fraud EZ-R Stats, LLC Fraud Detective Controls Operate in the background Not evident in everyday business environment These techniques usually – – – – – Occur in ordinary course of business Corroboration using external information Automatically communicate deficiencies Use results to enhance other controls Managing the business risk of fraud EZ-R Stats, LLC Examples of detective controls Whistleblower hot-lines (DHHS and OSA have them) Process controls (Medicaid audits and edits) Proactive fraud detection procedures – – – Data analysis Continuous monitoring Benford’s Law Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Specific Examples Cited Journal entries – suspicious transactions Identification of relationships Benford’s Law Continuous monitoring Managing the business risk of fraud EZ-R Stats, LLC Objective 1b Data Analysis enhances ability to detect fraud Identify hidden relationships Identify suspicious transactions Assess effectiveness of internal controls Monitor fraud threats Analyze millions of transactions Managing the business risk of fraud EZ-R Stats, LLC Continuous Monitoring of Fraud Detection Organization should develop ongoing monitoring and measurements Establish measurement criteria (and communicate to Board) Measurable criteria include: Managing the business risk of fraud EZ-R Stats, LLC Measurable Criteria – number of fraud allegations fraud investigations resolved Employees attending annual ethics course Whistle blower allegations Messages supporting ethical behavior delivered by executives Vendors signing ethical behavior standards Managing the business risk of fraud EZ-R Stats, LLC Management ownership of each technique implemented Each process owner should: – – – – Evaluate effectiveness of technique regularly Adjust technique as required Document adjustments Report modifications needed for techniques which become less effective Managing the business risk of fraud EZ-R Stats, LLC Practice Advisory 2320-1 Analysis and Evaluation International standards for the professional practice of Internal Auditing Analytical audit procedures – – Efficient and effective Useful in detecting Differences that are not expected Potential errors Potential irregularities Managing the business risk of fraud EZ-R Stats, LLC Analytical Audit Procedures May include – Study of relationships – Comparison of amounts with similar information in the organization – Comparison of amounts with similar information in the industry Managing the business risk of fraud EZ-R Stats, LLC Analytical audit procedures Performed using monetary amounts, physical quantities, ratios or percentages Ratio, trend and regression analysis Period to period comparisons Auditors should use analytical audit procedures in planning the engagement Managing the business risk of fraud EZ-R Stats, LLC Factors to consider Significance of the area being audited Assessment of risk Adequacy of system of internal control Availability and reliability of information Extent to which procedures provide support for engagement results Managing the business risk of fraud EZ-R Stats, LLC Objective 1c Peeling the Onion Fraud Items Possible Error Conditions Population as Whole Managing the business risk of fraud EZ-R Stats, LLC Objective 1d Fraud Pattern Detection Round Numbers Market Basket Benford’s Law Stratification Gaps Target Group Trend Line Univariate Duplicates Holiday Day of Week Managing the business risk of fraud EZ-R Stats, LLC Objective 1e Digital Analysis (5W) A little about the basics of digital analysis…. Who What Why Where When Managing the business risk of fraud EZ-R Stats, LLC Objective 1e Who Uses Digital Analysis Traditionally, IT specialists With appropriate tools, audit generalists (CAATs) Growing trend of business analytics Essential component of continuous monitoring Managing the business risk of fraud EZ-R Stats, LLC Objective 1e What - Digital Analysis Using software to: – – – Classify Quantify Compare Both numeric and non-numeric data Managing the business risk of fraud EZ-R Stats, LLC Objective 1e How - Assessing fraud risk Basis is quantification Software can do the “leg work” Statistical measures of difference – Chi square – Kolmogorov-Smirnov – D-statistic Specific approaches Managing the business risk of fraud EZ-R Stats, LLC Objective 1e Why - Advantages Automated process Handle large data populations Objective, quantifiable metrics Can be part of continuous monitoring Can produce useful business analytics 100% testing is possible Quantify risk Repeatable process Managing the business risk of fraud EZ-R Stats, LLC Objective 1e Why - Disadvantages Costly (time and software costs) Learning curve Requires specialized knowledge Managing the business risk of fraud EZ-R Stats, LLC Objective 1e When to Use Digital Analysis Traditional – intermittent (one off) Trend is to use it as often as possible Continuous monitoring Scheduled processing Managing the business risk of fraud EZ-R Stats, LLC Objective 1e Where Is It Applicable? Any organization with data in digital format, and especially if: – – – Volumes are large Data structures are complex Potential for fraud exists Managing the business risk of fraud EZ-R Stats, LLC Disadvantages of digital analysis Cost – – – Software Training Skills not widely available Time consuming – – Development costs Testing resources Managing the business risk of fraud EZ-R Stats, LLC Objective 1 Objective 1 Summarized Three brief examples CFE Guidance Paper “Top 10” Metrics Process Overview Who, What, Why, When & Where Managing the business risk of fraud EZ-R Stats, LLC Objective 1 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Next is plan, perform … Managing the business risk of fraud EZ-R Stats, LLC Statistical Sampling Brief History / Timeline Overview Attribute Sampling – Compliance Variable Sampling – Numeric Estimates Managing the business risk of fraud EZ-R Stats, LLC History of Sampling Basis is two laws/theorems of probability Law of Large Numbers Central Limit Theorem Managing the business risk of fraud EZ-R Stats, LLC Law of large numbers Simulated rolling of dice 7 6 Value 5 4 Result Average 3 Linear (Result) 2 1 0 1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 Observation Managing the business risk of fraud EZ-R Stats, LLC Time Line - LLN Indian mathematician Bramagupta 600 AD Italian mathematician Cardon 1500’s Statement without proof that empirical statistics improve with more trials Managing the business risk of fraud EZ-R Stats, LLC Time line LLN (continued) Jacob Bernoulli first to prove in 1713 Foundation for central limit theorem Managing the business risk of fraud EZ-R Stats, LLC Central limit theorem Classic measure Mean of a sufficiently large number of random samples will be approximately normally distributed. Managing the business risk of fraud EZ-R Stats, LLC The traditional explanation Managing the business risk of fraud EZ-R Stats, LLC Central Limit Theorem See it in action today Any population Large number of samples Average is “normally” distributed Managing the business risk of fraud EZ-R Stats, LLC History of Central Limit Theorem French mathematician Abraham de Moivre 1733 – approximate distribution from tossing coin (heads/tails) Ho hum reaction French Mathematician LaPlace – expanded it Ho hum reaction Managing the business risk of fraud EZ-R Stats, LLC History of CLT (cont’d) Russian mathematician Lyapunov Proof in 1901 Same reaction Managing the business risk of fraud EZ-R Stats, LLC Industrial revolution Manufacturing Engineering Excitement! Managing the business risk of fraud EZ-R Stats, LLC Student’s T William Gosset - 1908 Guinness Brewery Managing the business risk of fraud EZ-R Stats, LLC SAS 39 Effective June, 1983 Exposure draft for revision in 2009 Managing the business risk of fraud EZ-R Stats, LLC Attribute sampling Buonaccorsi (1987) Refined calculations Few software packages use it Managing the business risk of fraud EZ-R Stats, LLC Overview Sample size calculations Attribute sampling Variable sampling Random number generators Managing the business risk of fraud EZ-R Stats, LLC Sample size calculation It’s a guess… Every package – different answer Need to know the population But that’s why you’re taking a sample! Managing the business risk of fraud EZ-R Stats, LLC Attribute Sampling Using RAT-STATS Unrestricted populations Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC Session Objectives 1. 2. 3. 4. Understand what is attribute sampling and when to use it Understand unrestricted populations Overview of the process using RAT-STATS Understand the formula behind the computations Managing the business risk of fraud EZ-R Stats, LLC Attribute sampling “Attribute” Compliance testing Signatures on approval documents, attachment of supporting documentation, etc. Managing the business risk of fraud EZ-R Stats, LLC Statistical approach Recommended Economical Efficient Requires determination of a sample size Managing the business risk of fraud EZ-R Stats, LLC Overview of process Determine the sampling objective – – Confidence Precision Determine required sample size Identify samples to be selected based upon random numbers Pull the sample and test Compute the sampling results (i.e. estimate of range) Managing the business risk of fraud EZ-R Stats, LLC How this is done in RAT-STATS The sampling parameters are first developed by the auditor RAT-STATS is used to compute sample size RAT-STATS used to generate random numbers Pull the sample and test Enter results in RAT-STATS to compute estimates Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. 4. Size of population Expected error rate Required confidence Required precision Managing the business risk of fraud EZ-R Stats, LLC Step 2 – Obtain the random numbers Done by entering info into RAT-STATS Output can be a variety of sources: – – – – Text File Excel Microsoft Access Print File Managing the business risk of fraud EZ-R Stats, LLC Step 3 – Pull the sample Each random number selected corresponds with an item Put the selected item on a separate schedule Managing the business risk of fraud EZ-R Stats, LLC Step 4 - Test each selected item Generally requires reviewing documents Managing the business risk of fraud EZ-R Stats, LLC Step 5 – Compute the results Enter summary information into RAT-STATS Output can be in a variety of formats: – – – – – Excel Microsoft Access Text File Print File Printer Managing the business risk of fraud EZ-R Stats, LLC That’s It! Now we’ll see an actual demo using the RATSTATS software Excel population of 5,000 invoices Results of test of attributes stored in the worksheet Managing the business risk of fraud EZ-R Stats, LLC Variable Sampling Using RAT-STATS Unrestricted populations Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC Session Objectives 1. 2. 3. 4. Understand what variable sampling is and when to use it Understand unrestricted populations Overview of the process using RATSTATS Understand the formula behind the computations Managing the business risk of fraud EZ-R Stats, LLC Variable sampling “Variable” Estimating account balances Estimating transaction totals Managing the business risk of fraud EZ-R Stats, LLC Statistical approach Recommended Economical Efficient Requires determination of a sample size Managing the business risk of fraud EZ-R Stats, LLC Overview of process Determine the sampling objective – – Confidence Precision Determine required sample size Identify samples to be selected based upon random numbers Pull the sample and test Compute the sampling results (i.e. estimate of range) Managing the business risk of fraud EZ-R Stats, LLC How this is done in RAT-STATS The sampling parameters are first developed by the auditor RAT-STATS is used to compute sample size RAT-STATS used to generate random numbers Pull the sample and test Enter results in RAT-STATS to compute estimates Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. Probe sample Statistical measure Excel formula Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. Size of population Average value Standard deviation Managing the business risk of fraud EZ-R Stats, LLC Step 2 – Obtain the random numbers Done by entering info into RAT-STATS Output can be a variety of sources: – Text File – Excel – Microsoft Access – Print File Managing the business risk of fraud EZ-R Stats, LLC Step 3 – Pull the sample Each random number selected corresponds with an item Put the selected item on a separate schedule Managing the business risk of fraud EZ-R Stats, LLC Step 4 - Test each selected item Generally requires reviewing documents Example data contains both “examined” and “audited” value. Managing the business risk of fraud EZ-R Stats, LLC Step 5 – Compute the results Enter summary information into RAT-STATS Output can be in a variety of formats: – – – – – Excel Microsoft Access Text File Print File Printer Managing the business risk of fraud EZ-R Stats, LLC That’s It! Now we’ll see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Audited values stored in the worksheet Managing the business risk of fraud EZ-R Stats, LLC Attribute Sampling Using RAT-STATS Stratified populations Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC Session Objectives 1. 2. Understand what is stratification and when to use it Overview of the process using RAT-STATS Managing the business risk of fraud EZ-R Stats, LLC Stratified sampling “Strata” Homogenous More efficient in some instances Managing the business risk of fraud EZ-R Stats, LLC Overview of process Separation into strata Determine the sampling objective – – Confidence Precision Determine required sample size Identify samples to be selected based upon random numbers Pull the sample and test Compute the sampling results (i.e. estimate of range) Managing the business risk of fraud EZ-R Stats, LLC How this is done in RAT-STATS The sampling parameters are first developed by the auditor RAT-STATS is used to compute sample size RAT-STATS used to generate random numbers Pull the sample and test Enter results in RAT-STATS to compute estimates Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. 4. Size of population Expected error rate Required confidence Required precision Managing the business risk of fraud EZ-R Stats, LLC Step 2 – Obtain the random numbers Done by entering info into RAT-STATS Output can be a variety of sources: – – – – Text File Excel Microsoft Access Print File Managing the business risk of fraud EZ-R Stats, LLC Step 3 – Pull the sample Each random number selected corresponds with an item Put the selected item on a separate schedule Managing the business risk of fraud EZ-R Stats, LLC Step 4 - Test each selected item Generally requires reviewing documents Managing the business risk of fraud EZ-R Stats, LLC Step 5 – Compute the results Enter summary information into RAT-STATS Output can be in a variety of formats: – – – – – Excel Microsoft Access Text File Print File Printer Managing the business risk of fraud EZ-R Stats, LLC That’s It! Now we’ll see an actual demo using the RATSTATS software Excel population of 5,000 invoices Results of test of attributes stored in the worksheet Managing the business risk of fraud EZ-R Stats, LLC Variable Sampling Using RAT-STATS Stratified populations Managing the business risk of fraud using sampling and data mining EZ-R Stats, LLC Session Objectives 1. 2. 3. 4. Understand what stratified sampling is and when to use it Populations benefiting from stratified sampling Overview of the process using RATSTATS Understand the formula behind the computations Managing the business risk of fraud EZ-R Stats, LLC Stratified variable sampling “Stratified” “Variable” Estimating amounts Narrower standard deviation Managing the business risk of fraud EZ-R Stats, LLC Overview of process Determine the sampling objective – – Confidence Precision Determine required sample size Identify samples to be selected based upon random numbers Pull the sample and test Compute the sampling results (i.e. estimate of range) Managing the business risk of fraud EZ-R Stats, LLC How this is done in RAT-STATS The sampling parameters are first developed by the auditor RAT-STATS is used to compute sample size RAT-STATS used to generate random numbers Pull the sample and test Enter results in RAT-STATS to compute estimates Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. Probe sample Statistical measure Excel formula Managing the business risk of fraud EZ-R Stats, LLC Step 1 – Develop sampling parameters 1. 2. 3. 4. Number of strata Size of population Average value Standard deviation Managing the business risk of fraud EZ-R Stats, LLC Step 2 – Obtain the random numbers Done by entering info into RAT-STATS Multi-stage random numbers Output can be a variety of sources: – Text File – Excel – Microsoft Access – Print File Managing the business risk of fraud EZ-R Stats, LLC Step 3 – Pull the sample Each random number selected corresponds with an item in a strata Put the selected item on a separate schedule Managing the business risk of fraud EZ-R Stats, LLC Step 4 - Test each selected item Generally requires reviewing documents Example data contains both “examined” and “audited” value. Managing the business risk of fraud EZ-R Stats, LLC Step 5 – Compute the results Enter summary information into RAT-STATS Output can be in a variety of formats: – – – – – Excel Microsoft Access Text File Print File Printer Managing the business risk of fraud EZ-R Stats, LLC That’s It! Now we’ll see an actual demo using the RAT-STATS software Excel population of 5,000 invoices Divided into three strata Audited values stored in the worksheet Managing the business risk of fraud EZ-R Stats, LLC Objective 2 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Next is cost reduction … Managing the business risk of fraud EZ-R Stats, LLC Techniques for cost reduction Optimize sample size (most “bang” for the buck) Skip sampling – review 100% of transactions using computer assisted audit techniques (CAATs) Managing the business risk of fraud EZ-R Stats, LLC Sample optimization Sequential sampling Managing the business risk of fraud EZ-R Stats, LLC University of Hawaii Banana aphids Managing the business risk of fraud EZ-R Stats, LLC Sequential sampling Banana aphids Managing the business risk of fraud EZ-R Stats, LLC 100% test using CAATs Provides complete coverage Best practice Basis for continuous monitoring Repeatable process Managing the business risk of fraud EZ-R Stats, LLC Objective 3 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Next is Yellow Book and SAS 56 … Managing the business risk of fraud EZ-R Stats, LLC Yellow book standards Standards regarding statistical sampling and IT Managing the business risk of fraud EZ-R Stats, LLC General standards 3.43 Technical Knowledge and competence “The staff assigned to conduct an audit or attestation engagement under GAGAS must collectively possess the technical knowledge, skills, and experience necessary to be competent for the type of work being performed before beginning work on that assignment. The staff assigned to a GAGAS audit or attestation engagement should collectively possess: “ Managing the business risk of fraud EZ-R Stats, LLC Stat sampling and IT Skills appropriate for the work being performed. For example, staff or specialist skills in (1) statistical sampling if the work involves use of statistical sampling; (2) information technology Managing the business risk of fraud EZ-R Stats, LLC SAS 56 – Analytical procedures Requires use of analytic review procedures for: Audit planning Overall review stages Managing the business risk of fraud EZ-R Stats, LLC SAS 56 – Analytical Review Encourages use of analytical review Provides guidance “A wide variety of analytical procedures may be useful for this purpose.” Managing the business risk of fraud EZ-R Stats, LLC Objective 4 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Next is linear regression … Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 7 - Trends Trend Busters Does the pattern make sense? 30,000 25,000 20,000 15,000 10,000 5,000 0 Sales M ar n0 Ja -0 7 M ay -0 7 Ju l-0 Se 7 p07 N ov -0 7 Ja n08 M ar -0 M 8 ay -0 8 Employee Count 7 Amount ACME Technology Date Managing the business risk of fraud EZ-R Stats, LLC 7 – Trends Trend Busters Linear regression Sales are up, but cost of goods sold is down “Spikes” Managing the business risk of fraud EZ-R Stats, LLC 7 – Trends Purpose / Type of Errors Identify trend lines, slopes, etc. Correlate trends Identify anomalies Key punch errors where amount is order of magnitude Managing the business risk of fraud EZ-R Stats, LLC 7 – Trends Linear Regression Test relationships (e.g. invoice amount and sales tax) Perform multi-variable analysis Managing the business risk of fraud EZ-R Stats, LLC 7 – Trends How is it done? Estimate linear trends using “best fit” Measure variability (standard errors) Measure slope Sort descending by slope, variability, etc. Managing the business risk of fraud EZ-R Stats, LLC 7 – Trends Trend Lines by Account - Example Results Account N Slope Std Err 32451 18 1.230 0.87 43517 17 1.070 4.3 32451 27 1.023 0.85 43517 32 1.010 0.36 43870 23 0.340 2.36 54630 56 -0.560 1.89 Generally the trend is gently sloping up, but two accounts (43870 and 54630) are different. Managing the business risk of fraud EZ-R Stats, LLC Scatter plot with prediction and confidence intervals Managing the business risk of fraud EZ-R Stats, LLC Objective 5 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Next is data mining … Managing the business risk of fraud EZ-R Stats, LLC Objective 6 Basis for Pattern Detection Analytical review Isolate the “significant few” Detection of errors Quantified approach Managing the business risk of fraud EZ-R Stats, LLC Objective 2 Understanding the Basis Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi Square and K-S The metrics used Managing the business risk of fraud EZ-R Stats, LLC Objective 2a Quantified Approach Based on measureable differences Population vs. Group “Shotgun” technique Managing the business risk of fraud EZ-R Stats, LLC Objective 2a Detection of Fraud Characteristics Something is different than expected Managing the business risk of fraud EZ-R Stats, LLC Objective 2b Fraud patterns Common theme – “something is different” Groups Group pattern is different than overall population Managing the business risk of fraud EZ-R Stats, LLC Objective 2c Measurement Basis Transaction counts Transaction amounts Managing the business risk of fraud EZ-R Stats, LLC Objective 2d A few words about statistics (the “s” word) Detailed knowledge of statistics not necessary Software packages do the “numbercrunching” Statistics used only to highlight potential errors/frauds Not used for quantification Managing the business risk of fraud EZ-R Stats, LLC Objective 2d How is digital analysis done? Comparison of group with population as a whole Can be based on either counts or amounts Difference is measured Groups can then be ranked using a selected measure High difference = possible error/fraud Managing the business risk of fraud EZ-R Stats, LLC Demo in Excel of the process Based roughly on the Wake County Transportation fraud Illustrates how the process works, using Excel Managing the business risk of fraud EZ-R Stats, LLC Objective 2d Histograms Attributes tallied and categorized into “bins” Counts or sums of amounts Managing the business risk of fraud EZ-R Stats, LLC Objective 2d Two histograms obtained Population and group Population 700 Group 80 70 60 50 40 30 20 10 0 600 500 400 300 200 100 0 Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07 Managing the business risk of fraud Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec07 07 07 07 07 07 07 07 07 07 07 07 EZ-R Stats, LLC Objective 2d Compute Cumulative Amount for each Count by Month Cum Pct 80 120.0% 70 100.0% 60 Count 50 80.0% 40 60.0% 30 20 40.0% 10 20.0% Managing the business risk of fraud ov -0 7 N Se p07 Ju l-0 7 07 M ay - 07 M ar - Month 0.0% Ja n07 Ja n0 Fe 7 bM 07 ar -0 Ap 7 r-0 M 7 ay -0 Ju 7 n0 Ju 7 lAu 07 g0 Se 7 p0 O 7 ct0 No 7 v0 De 7 c07 0 EZ-R Stats, LLC Objective 2d Are the histograms different? Two statistical measures of difference Chi Squared (counts) K-S (distribution) Both yield a difference metric Managing the business risk of fraud EZ-R Stats, LLC Objective 2d Chi Squared Classic test on data in a table Answers the question – are the rows/columns different Some limitations on when it can be applied Managing the business risk of fraud EZ-R Stats, LLC Objective 2d Chi Squared Table of Counts Degrees of Freedom Chi Squared Value P-statistic Computationally intensive Managing the business risk of fraud EZ-R Stats, LLC Objective 2d Kolmogorov-Smirnov Two Russian mathematicians Comparison of distributions Metric is the “d-statistic” Managing the business risk of fraud EZ-R Stats, LLC Objective 2d How is K-S test done? Four step process 1. 2. 3. 4. For each cluster element determine percentage Then calculate cumulative percentage Compare the differences in cumulative percentages Identify the largest difference Managing the business risk of fraud EZ-R Stats, LLC Objective 2d - KS Kolmogorov-Smirnov Managing the business risk of fraud EZ-R Stats, LLC Objective 2e Classification by metrics Stratification Day of week Happens on holiday Round numbers Variability Benford’s Law Trend lines Relationships (market basket) Gaps Duplicates Managing the business risk of fraud EZ-R Stats, LLC Objective e Auditor’s “Top 10” Metrics 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers / Variability Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC Objective 2 Understanding the Basis Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi Square and K-S The metrics used Managing the business risk of fraud EZ-R Stats, LLC Objective 2 - Summarized 1. 2. 3. 4. 5. Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How pattern detection fits in Next are the metrics … Managing the business risk of fraud EZ-R Stats, LLC It’s that time! Session Break! Managing the business risk of fraud EZ-R Stats, LLC Objective 3 The “Top 10” Metrics Overview Explain Each Metric Examples of what it can detect How to assess results Managing the business risk of fraud EZ-R Stats, LLC Objective 3 Trapping anomalies Managing the business risk of fraud EZ-R Stats, LLC Objective 3 Fraud Pattern Detection Round Numbers Market Basket Benford’s Law Stratification Gaps Target Group Trend Line Univariate Duplicates Holiday Day of Week Managing the business risk of fraud EZ-R Stats, LLC 1 - Outliers Outliers / Variability Outliers are amounts which are significantly different from the rest of the population Managing the business risk of fraud EZ-R Stats, LLC 1 - Outliers Outliers / Variability Charting (visual) Software to analyze “z-scores” Top and Bottom 10, 20 etc. High and low variability (coefficient of variation) Managing the business risk of fraud EZ-R Stats, LLC 1 - Outliers Drill down to the group level Basic statistics – Minimum, maximum and average – Variability Sort by statistic of interest – Variability (coefficient of variation) – Maximum, etc. Managing the business risk of fraud EZ-R Stats, LLC 1 - Outliers Example Results Provider N Coeff Var 3478421 3,243 342.23 2356721 4,536 87.23 3546789 3,421 23.25 5463122 2,311 18.54 Two providers (3478421 and 2356721) had significantly more variability in the amounts of their claims than all the rest. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 2 - Stratification Unusual stratification patterns Do you know how your data looks? Managing the business risk of fraud EZ-R Stats, LLC 2 - Stratification Stratification - How Charting (visual) Chi Squared Kolmogorov-Smirnov By groups Managing the business risk of fraud EZ-R Stats, LLC 2 – Stratification Purpose / types of errors Transactions out of the ordinary “Up-coding” insurance claims “Skewed” groupings Based on either count or amount Managing the business risk of fraud EZ-R Stats, LLC 2 – Stratification The process? 1. 2. 3. 4. 5. Stratify the entire population into “bins” specified by auditor Same stratification on each group (e.g. vendor) Compare the group tested to the population Obtain measure of difference for each group Sort descending on difference measure Managing the business risk of fraud EZ-R Stats, LLC 2 – Stratification Units of Service Stratified Example Results Provider N Chi Sq D-stat 2735211 6,011 7,453 0.8453 4562134 8,913 5,234 0.7453 4321089 3,410 342 0.5231 4237869 2,503 298 0.4632 Two providers (2735211 and 4562134) are shown to be much different from the overall population (as measured by Chi Square). Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 3 – Day of Week Day of Week Activity on weekdays Activity on weekends Peak activity mid to late week Managing the business risk of fraud EZ-R Stats, LLC 3 – Day of Week Purpose / Type of Errors Identify unusually high/low activity on one or more days of week Dentist who only handled Medicaid on Tuesday Office is empty on Friday Managing the business risk of fraud EZ-R Stats, LLC How it is done? Programmatically check entire population Obtain counts and sums by day of week (1-7) Prepare histogram For each group do the same procedure Compare the two histograms Sort descending by metric (chi square/dstat) Managing the business risk of fraud EZ-R Stats, LLC 3 – Day of Week Day of Week - Example Results Provider N Chi Sq D-stat 2735211 5,404 12,435 0.9802 4562134 5,182 7,746 0.8472 4321089 5,162 87 0.321 4237869 7,905 56 0.2189 Provider 2735211 only provided service for Medicaid on Tuesdays. Provider 4562134 was closed on Thursdays and Fridays. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 4 – Round Numbers Round Numbers It’s about…. Estimates! Managing the business risk of fraud EZ-R Stats, LLC 4 – Round Numbers Purpose / Type of Errors Isolate estimates Highlight account numbers in journal entries with round numbers Split purchases (“under the radar”) Which groups have the most estimates Managing the business risk of fraud EZ-R Stats, LLC 4 – Round Numbers Round numbers Classify population amounts – $1,375.23 is not round – $5,000 is a round number – type 3 (3 zeros) – $10,200 is a round number type 2 (2 zeros) Quantify expected vs. actual (d-statistic) Generally represents an estimate Journal entries Managing the business risk of fraud EZ-R Stats, LLC 4 – Round Numbers Round Numbers in Journal Entries - Example Results Account N Chi Sq D-stat 2735211 4,136 54,637 0.9802 4562134 833 35,324 0.97023 4321089 8,318 768 0.321 4237869 9,549 546 0.2189 Two accounts, 2735211 and 4562134 have significantly more round number postings than any other posting account in the journal entries. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 5 – Made up numbers Made up Numbers Curb stoning Imaginary numbers Benford’s Law Managing the business risk of fraud EZ-R Stats, LLC 5 – Made Up Numbers What can be detected Made up numbers – e.g. falsified inventory counts, tax return schedules Managing the business risk of fraud EZ-R Stats, LLC 5 – Made Up Numbers Benford’s Law using Excel Basic formula is “=log(1+(1/N))” Workbook with formulae available at http://tinyurl.com/4vmcfs Obtain leading digits using “Left” function, e.g. left(Cell,1) Managing the business risk of fraud EZ-R Stats, LLC 5 – Made Up Numbers Made up numbers Benford’s Law Check Chi Square and d-statistic First 1,2,3 digits Last 1,2 digits Second digit Sources for more info Managing the business risk of fraud EZ-R Stats, LLC 5 – Made Up Numbers How is it done? Decide type of test – (first 1-3 digits, last 1-2 digit etc) For each group, count number of observations for each digit pattern Prepare histogram Based on total count, compute expected values For the group, compute Chi Square and d-stat Sort descending by metric (chi square/dstat) Managing the business risk of fraud EZ-R Stats, LLC 5 – Made Up Numbers Invoice Amounts tested with Benford’s law - Example Results Store Hi Digit Chi Sq D-stat 324 79 5,234 0.9802 563 89 4,735 0.97023 432 23 476 0.321 217 74 312 0.2189 During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benford’s Law. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 6 – Market Basket Market Basket Medical “Ping ponging” Pattern associations Apriori program References at end of slides Apriori – Latin a (from) priori (former) Deduction from the known Managing the business risk of fraud EZ-R Stats, LLC 6 – Market basket Purpose / Type of Errors Unexpected patterns and associations Based on “market basket” concept Unusual combinations of diagnosis code on medical insurance claim Managing the business risk of fraud EZ-R Stats, LLC 6 – Market basket Market Basket JE Accounts JE Approvals Credit card fraud in Japan – taxi and ATM Managing the business risk of fraud EZ-R Stats, LLC 6 – Market basket How is it done? First, identify groups, e.g. all medical providers for a patient Next, for each provider, assign a unique integer value Create a text file containing the values Run “apriori” analysis Managing the business risk of fraud EZ-R Stats, LLC 6 – Market basket Apriori outputs For each unique value, probability of other values If you see Dr. Jones, you will also see Dr. Smith (80% probability) If you see a JE to account ABC, there will also an entry to account XYZ (30%) Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 8 - Gaps Numeric Sequence Gaps What’s there is interesting, what’s not there is critical … Managing the business risk of fraud EZ-R Stats, LLC 8 – Gaps Purpose / Type of Errors Missing documents (sales, cash, etc.) Inventory losses (missing receiving reports) Items that “walked off” Managing the business risk of fraud EZ-R Stats, LLC 8 – Gaps How is it done? Check any sequence of numbers supposed to be complete, e.g. Cash receipts Sales slips Purchase orders Managing the business risk of fraud EZ-R Stats, LLC 8 – Gaps Gaps Using Excel Excel – sort and check Excel formula Sequential numbers and dates Managing the business risk of fraud EZ-R Stats, LLC 8 – Gaps Gap Testing - Example Results Start End Missing 10789 10791 1 12523 12526 2 17546 17548 1 Four check numbers are missing. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 9 - Duplicates Duplicates Why is there more than one? Same, Same, Same, and Same, Same, Different Managing the business risk of fraud EZ-R Stats, LLC 9 – Duplicates Two types of (related) tests Same items – same vendor, same invoice number, same invoice date, same amount Different items – same employee name, same city, different social security number Managing the business risk of fraud EZ-R Stats, LLC 9 - Duplicates Duplicate Payments High payback area “Fuzzy” logic Overriding software controls Managing the business risk of fraud EZ-R Stats, LLC Fuzzy matching with software Levenshtein distance Soundex “Like” clause in SQL Regular expression testing in SQL Vendor/employee situations Managing the business risk of fraud 9 - Duplicates Russian physicist EZ-R Stats, LLC 9 - Duplicates How is it done? First, sort file in sequence for testing Compare items in consecutive rows Extract exceptions for follow-up Managing the business risk of fraud EZ-R Stats, LLC 9 - Duplicates Possible Duplicates - Example Results Vendor Invoice Date Invoice Amount Count 10245 6/15/2007 3,544.78 4 10245 8/31/2007 2,010.37 2 17546 2/12/2007 1,500.00 2 Five invoices may be duplicates. Managing the business risk of fraud EZ-R Stats, LLC Next Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Outliers Stratification Day of Week Round Numbers Made Up Numbers Market basket Trends Gaps Duplicates Dates Managing the business risk of fraud EZ-R Stats, LLC 10 - Dates Date Checking If we’re closed, why is there … Adjusting journal entry? Receiving report? Payment issued? Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Holiday Date Testing Red Flag indicator Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Date Testing challenges Difficult to determine Floating holidays – Friday, Saturday, Sunday, Monday Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Typical audit areas Journal entries Employee expense reports Business telephone calls Invoices Receiving reports Purchase orders Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Determination of Dates Transactions when business is closed Federal Office of Budget Management An excellent fraud indicator in some cases Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Holiday Date Testing Identifying holiday dates: – Error prone – Tedious U.S. only Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Federal Holidays Established by Law Ten dates Specific date (unless weekend), OR Floating holiday Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Federal Holiday Schedule Office of Personnel Management Example of specific date – Independence Day, July 4th (unless weekend) Example of floating date – Martin Luther King’s birthday (3rd Monday in January) Floating – Thanksgiving – 4th Thursday in November Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates How it is done? Programmatically count holidays for entire population For each group, count holidays Compare the two histograms (group and population) Sort descending by metric (chi square/d-stat) Managing the business risk of fraud EZ-R Stats, LLC 10 – Dates Holiday Counts - Example Results Employee Number N Chi Sq D-stat 10245 37 5,234 0.9802 32325 23 4,735 0.97023 17546 18 476 0.321 24135 34 312 0.2189 Two employees (10245 and 32325) were “off the chart” in terms of expense amounts incurred on a Federal Holiday. Managing the business risk of fraud EZ-R Stats, LLC Objective 3 The “Top 10” Metrics Overview Explain Each Metric Examples of what it can detect How to assess results Managing the business risk of fraud EZ-R Stats, LLC Objective 3 - Summarized 1. 2. 3. 4. 5. Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How pattern detection fits in Next – using Excel … Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Use of Excel Built-in functions Add-ins Macros Database access Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Excel templates Variety of tests – – – – Round numbers Benford’s Law Outliers Etc. Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Excel – Univariate statistics Work with Ranges =sum, =average, =stdevp =largest(Range,1), =smallest(Range,1) =min, =max, =count Tools | Data Analysis | Descriptive Statistics Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Excel Histograms Tools | Data Analysis | Histogram Bin Range Data Range Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Excel Gaps testing Sort by sequential value =if(thiscell-lastcell <> 1,thiscell-lastcell,0) Copy/paste special Sort Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Detecting duplicates with Excel Sort by sort values =if testing =if(=and(thiscell=lastcell, etc.)) Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Performing audit tests with macros Repeatable process Audit standardization Learning curve Streamlining of tests More efficient and effective Examples http://ezrstats.com/Macros/home.html Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Using database audit software Many “built-in” functions right off the shelf with SQL Control totals Exception identification “Drill down” Quantification June 2008 article in the EDP Audit & Control Journal (EDPACS) “SQL as an audit tool” http://ezrstats.com/doc/SQL_As_An_Audit_Tool.pdf Managing the business risk of fraud EZ-R Stats, LLC Objective 4 Use of Excel Built-in functions Add-ins Macros Database access Managing the business risk of fraud EZ-R Stats, LLC Objective 4 - Summarized 1. 2. 3. 4. 5. Understand why and how Understand statistical basis for quantifying differences Identify ten general tools and techniques Understand examples done using Excel How Pattern Detection fits in Next – Fit … Managing the business risk of fraud EZ-R Stats, LLC Objective 5 How Pattern Detection Fits In Business Analytics Fraud Pattern Detection Continuous monitoring Managing the business risk of fraud EZ-R Stats, LLC Objective 5 Where does Fraud Pattern Detection fit in? Right in the middle Business Analytics Fraud Pattern Detection Continuous fraud pattern detection Continuous Monitoring Managing the business risk of fraud EZ-R Stats, LLC Objective 5 Business Analytics Fraud analytics -> business analytics Business analytics -> fraud analytics Managing the business risk of fraud EZ-R Stats, LLC Objective 5 Role in Continuous Monitoring (CM) Fraud analytics can feed (CM) Continuous fraud pattern detection Use output from CM to tune fraud pattern detection Managing the business risk of fraud EZ-R Stats, LLC Objective 6 - Summarized Understand the framework for managing the business risk of fraud Plan, perform and explain statistical sampling in audits Reduce audit costs using data mining, sequential sampling and other sampling techniques Apply SAS 56, the new SAS suite and the revised (2007) Yellow Book. Run, hands-on, the most productive analytic technique (regression analysis). Use data mining to introduce greater efficiency into the audit process, without losing effectiveness. Managing the business risk of fraud EZ-R Stats, LLC Links for more information Kolmogorov-Smirnov http://tinyurl.com/y49sec Benford’s Law http://tinyurl.com/3qapzu Chi Square tests http://tinyurl.com/43nkdh Continuous monitoring http://tinyurl.com/3pltdl Managing the business risk of fraud EZ-R Stats, LLC Market Basket Apriori testing for “ping ponging” Temple University http://tinyurl.com/5vax7r Apriori program (“open source”) http://tinyurl.com/5qehd5 Article – “Medical ping ponging” http://tinyurl.com/5pzbh4 Managing the business risk of fraud EZ-R Stats, LLC Excel macros used in auditing Excel as an audit software http://tinyurl.com/6h3ye7 Selected macros http://ezrstats.com/Macros/home.html Spreadsheets forever http://tinyurl.com/5ppl7t Managing the business risk of fraud EZ-R Stats, LLC Questions? Managing the business risk of fraud EZ-R Stats, LLC Contact info Phone: (919)-219-1622 E-mail: Mike.Blakley@ezrstats.com Blog: http://blog.ezrstats.com Managing the business risk of fraud EZ-R Stats, LLC