Continuous Assurance Best Practices: Laying the Roadmap to Success BA’s in History and Religion Studied for an Master’s Degree in Church History Got a job twisting balloons while in grad school. Worked for Babies R Us as an Assistant Manager. Worked Customer Service while working on an MBA. Took Intro to Pascal in college to fulfill a distribution requirement! Because I am not a techie. Prior to learning ACL: I was answering phones And twisting balloons If you can take some clown off the street and teach them ACL, then you can teach anybody ACL. 2007 – ACL’s Seven Habits of Successful Auditor’s Campaign Founder and first president of TexasACL User Group Second person ever recognized as “ACL Super User” 2008 – Spoke at ACL’s Annual Convention 2008 – Participated on ACL Round Table 2009 – Keynote speaker at ACL Annual Conference 2009 – Asked by David Coderre to endorse book 2011 – Spoke at ACL Business Assurance Seminar 2012 – Presented Webinar for ACL 6 Major Projects Projects range from 40-100+ scripts New projects are built upon a common “skeleton”. New projects re-use same scripts! Projects may perform over 100 different tests Continuous Assurance results are sent to one of several standardized report formats Some projects produce over 10K worth of ACL Logging per day! Commands Comments End of Scripts Field Names In Scripts Naming Conventions Opening Sections Project Names Reports Script Library Script Names Start of Scripts Table Names Variable names Naming Conventions Comments Start of Script In Scripts End of Scripts Opening Sections Project Names Script Names Table Names Variable Names Field Names Comments Commands Reports All projects begin with two or three letters followed by two numbers. The letters organize the scripts by type/purpose/objective/process. The numbers organize the script within the process. We use alpha-numeric combinations: 2-3 letters to describe the nature of the project AP --- Accounts Payable AR --- Account Receivable GL --- General Ledger GLT --- General Ledger Transactions ACS --- Account Code Structure 2 numbers o1 -99 The alpha-numeric construct used to identify projects can be used throughout the project to perform various tasks. Again a four digit alpha-numeric construct can be used to define and organize script names. Use the Alpha characters to represent the same processes in every project. For example: AX## = Start up scripts BX## = repeating scripts within the project CX## = Communal scripts DX## = Data Load Scripts EX## -WX## = Processing/Analytic Scripts YX## = Administrative Functions ZX## = Shut down/Clean Up Scripts With both scripts and projects, I will have a second part and possibly 3rd part that describes what the script does. The second part is usually what the script/project does and the third part may describe or qualify the second. But when referring to the project or script, I always do so via the alpha-numeric constructs. Naming Conventions Script Names Table Names Data Tables Results Tables Working Tables Temp Tables Variable Names Field Names Project Names Comments Start of Script In Scripts End of Scripts Opening Sections Comments Commands Other Issues to Consider AA01Naming Conventions AA02 Script Names AA03 Table Names AA04 Data Tables AA05 Results Tables AA06 Working Tables AA07 Temp Tables AA08 Variable Names AA09 Field Names AA10Project Names BA01Comments BA02 Start of Script BA03 In Scripts BA04 End of Scripts CA01 Opening Sections CA02 Comments CA03 Commands DA01 Other Issues to Consider There are 4 types of tables: 1) Original Data 2) Computed data/working tables 3) Temporary files 4) Final Product(s)/Reports Original data files---Naming should include 3 parts: – – – A prefix to identify the file as original data. The original table name without any non-alpha-numeric characters. The date of the data---consider YYMMDD v MMDDYY AAA_Employeefile_111013 AP_VendorFile_111013 STD_memberfile_111013 MSTR_Hrfile_121212 {corporate initials}_Employeefile_111013 Temporary files are labeled TEMP001-TEMPXXX. This allows for Easy trouble shooting Helps to identify where issues exist Helps to determine if expected results flowed through temp files Clean up of project via a standardized clean up script Reports begin with the word “REPORT” followed by the alpha numeric script identifier that generated the report and a short description. So one might have a REPORT_DA01_DuplicateAddress. Reports begin with the word “REPORT” followed by the alpha numeric project identifier that generated the report and a short description with dates. So one might have a REPORT_ACS01_091812_092012. Working tables are generally one of two types of tables: 1) Data derived from basic data tables. 2) Data contained within the project, but not derived from current data tables. These tables should be descriptive and contained in a special folder. Two or three part name in creating variables. Part 1 – the v_ that identifies the value as a variable. Part 2 – the type of variable being defined Part 3 – a short description/qualifier for the variable (optional) User defined values should: Not be unique and not potentially used by the client. Poor values would include “Amount”, “Company”, “Name”, etc. They should be descriptive enough to remember and to understand. Variable “A_1” would not be a good option. Ideally they should mirror common or anticipated names. • V_Path V_path_bat – V_path_data – V_path_results • – • – • V_date V_date_run – V_date_DOS – V_date1 • V_table • – V_table_AP • V_cnt_test/v_cntr_tes t V_table – • V_com_test1 V_cnt/v_cntr/V_cou nt – – • V_com v_table_list V_define_tablename V_Drive V_type Field Names Field names generated by ACL should always be readily identified as such. Amount vs amt? Dept vs Department? Use a prefix “F_”ield or something comparable to identify a field name. Best practice is to put the COM at the start of every record which should be commented out. Comments that go at the start of a script should be highlighted with two rows of COM followed by asterisks. E.g.: COM****************** COM****************** COM Written by Porter Broyles Date: 10/13/11 COM Description: This makes it easier to find the start of COM subscripts in the log. COM****************** COM****************** Comments should be used at the start of every script to: 1) Identify who wrote the script 2) When the script was written 3) When the script was last modified (by whom) 4) What the intent of the script is 5) Any special considerations When writing comments they should stand out. To help them stand out add a line before and after the comment itself. This should include the word COM with a series of asterisks after it: COM********************* COM details of the comment COM********************* When using scripts and subscripts, it can often be a challenge to determine where one is when reviewing the project log. Whenever a subscript is called, the last command in the subscript should be a note indicating that the subscript is completed: DO AA01_subscript_A ~~~~ COM******************* COM Returning from AA01_Subscript_A COM******************* Before changing logs, always indicate what the next log is going to be. After changing logs, always indicate what the previous log was. COM************* COM Next Log [new logname] COM************* SET LOG [new logname] COM************* COM Old Log [old logname] COM************* CAPITALIZE ACL commands and functions() Lowercase for field names DELETE FIELDS before DEFINing them DELETE tables both .fil and format before creating Don’t abbreviate functions or commands. Functions use the thousandths separator to separate parts. In the US this is the COMMA: SUBSTRING(fieldname,start,length) In most countries, this the is period: SUBSTRING(fieldname.start.length) Generic option, use a blank space: SUBSTRING(fieldname start length Development v Production Change Control Version Archiving Crash Log Diagramming a CAMS via flow charting is an excellent way to understand how a script operates and which scripts/routines are called from different locations. You can indicate what scripts are shared scripts, scripts called via a DO-WHILE sequence, scripts called via a DO IF sequence, etc. Skeleton Load Process Load Process CA01Master Script Script controls load process This script is only called when a project/table requires special treatement 3 1 2 BC01_Select_Table Process to pick tables used in project CA30 Verify Fields Control Script accumulating results from load CA02 Load Data Loop Series of Script to Load data files Process repeated Until options selected 1 3x BE01 Error Extracxt 2 1 3 2 - Repeated for each table being loaded CA03_Select DatesGLT Defines v_new_dateGLT BC02 Table Select User selects tables 1 - Process repeated to create dialog bod ca40_dATAnORMALIZATI ON_%v_type% Script normalizes data fields. Currenly only used in IA01---which is a project that hasn’t been worked much. CA31 Bars Err Identify Bar differences CA32 Load Err Load Errors CA33 Verify ERR Identify invalid fields BE01 Error Extracxt BE01 Error Extracxt BE01 Error Extracxt 2 - Repeat to create list BC03_Define_Diabx Creating Dialog Box CA06_Load_SQL_IMP Import Files selected in BC01 BC04 Create List Creating list of items 7 6 5 --- If table is GLTTRNSDT 4 1 2 CA36 Date_ERR Checks to see if create date of folder is the same as data date CA21 BarDelimi Clnup Checks to see if table has extra bars that needs to be cleaned up BE01 Error Extracxt Process is repeated if there are records with extra bar delimiters CA22 BarDelimit2 CLNUP2 3 CA07_Loaddetail Identifies syntax for loading each table CA23 Format STB Takes care of issue with the SYSTRANSBODY table BE02_NoLoad_Extract Extracts record if no tables are loaded CA08_Communal Cumulative file for GLTTRNSDT KEY CA34 Field Err CA35 record Err BE01 Error Extracxt BE01 Error Extracxt Yellow boxes indicate shared scrpts Green boxes indicate processes that are repeated via a DO WHILE Sequence All changes to scripts in production should be first implemented in a separate development area. The scripts in production should not be changed until and unless vetted in development first. All script changes should be approved by the manager/director over the project area. When changes are made, do not simply overwrite the old scripts, but copy them to an archive. That way if questions arise as to logic at a previous date, the exact scripts can be revisited. The crash log is one of my favorite best practices. It does several things: 1) Serves as documentation should a external reviewer ask about a process on a date where the script crashed or had to be reperformed. 2) Serves as a source to identify issues with the system/IT. 3) Serve as a means to evaluate one’s own script. 4) Serve as a means to identify bugs (both with the software and with your scripts.) Date Issue Worked Resolve by d Reference 5/12/1 2 Data from Accounts Payable was incomplete PDB 5/13/12 Link to email from AP Manager 9/1/12 ACL could not connect to data folder PDB 9/1/12 10/1/1 2 ACL could not connect to data folder PDB 10/1/12 Link to saved email from IT 11/1/1 2 ACL could not connect to data folder PDB 11/1/12 Link to saved email from IT Link to saved email from IT Porter Broyles porter@someclownconsulting.com www.someclownconsulting.com www.texasacl.com