By Glenn L. Helms
In Brief
Electronic Testing and Evidence Gathering
Advances in computer technology have made more timely and detailed financial and operational information available; interested parties no longer have to wait until historical financial statements are published. Assurance providers must keep pace with this demand for real-time information while dealing with information systems that require new testing techniques and new evidence-gathering procedures. Traditional electronic assurance methods may not be as relevant in the increasingly paperless environment, where an audit trail is primarily electronic. The development of a truly continuous auditing approach requires a combination of techniques in order to ensure that sufficient evidence exists to assure the integrity of the system.
The advances made in computer technology during the past several decades have had a significant impact on how accounting systems process financial transactions. One implication of these advances is that users have more timely, detailed financial and operational information about an entity. Users no longer need to wait until the publication of quarterly or annual financial statements in order to assess performance. Advances in enterprise resource planning (ERP), extensible business reporting language (XBRL), and other software has enabled companies to report information on a weekly, daily, or even instantaneous basis. In some cases, users can even access the entity’s financial and operating information databases directly and select the information they consider relevant.
Much like users of audited historical financial statements, users of electronic accounting systems might need assurance that the financial and operating data has integrity. For example, a day trader might want assurance that the stock prices and operating ratios provided by an online stockbroker’s website are accurate. Other users, such as a bank closely monitoring the collateral on its loan to a car dealership, might want assurance about the reporting of financial or other data triggered by an economic event. The bank

would want to be advised by the car dealership’s system when a vehicle is sold
(economic event) so that the loan can be collected.
Two assurance services, WebTrust and SysTrust, and other assurance services have already been developed (see www.aicpa.org). In WebTrust, an assurance provider reports on an entity’s electronic storefront; in SysTrust, on a particular system. The time period covered by the assurance provider’s report is typically shorter than that covered by reports on historical financial statements. Traditional audit procedures can clearly not be completed in the time required by many assurance services. For example, confirmations sent via U.S. mail might not be received and returned by a third party in time to meet the assurance provider’s deadlines. Improvements to existing procedures or new evidencegathering procedures need to be developed in order to obtain sufficient competent evidential matter to satisfy the frequency of many assurance service engagements.
Assurance providers typically encounter a broad range of accounting systems. For example, a single company might have its payroll accounting outsourced to a third-party service provider alongside a major revenue subsystem that uses ERP e-commerce software conducted over virtual private networks and a LAN containing shared resources of printers, data files, and software. Newer systems might record data differently from older systems or might delete old data more quickly. Assurance providers must be competent in a broad range of established and current systems and cognizant of what electronic data testing techniques are appropriate in each environment.
Traditional Accounting Systems
Traditional accounting systems (sometimes called legacy systems) generally consist of various stand-alone subsystems (payroll, purchasing) that produce printed output.
Bookkeepers take the totals from these subsystems and prepare general journal entries for posting to a general ledger system. A financial reporting system then produces financial statements and other performance reports. These systems are batch oriented—data is accumulated in a transaction file and posted periodically to a master file. Batch control totals are established and reconciled from input to processing to output. Many such systems remain in a wide range of business and not-for-profit entities.
Assurance providers can use some of the following established evidence-gathering techniques, identified in the AICPA’s
Auditing with Computers , to test the data integrity in traditional systems:


Audit software can foot, crossfoot, select samples, and analyze results; compare data between multiple files for consistency; import and export data; and perform analytical review procedures.
Job accounting data or operating systems logs provide a plethora of information concerning resource utilization. Software that compares budgeted software processing times with actual can identify any significant variances for investigation.

 Library management software logs all changes to programs, program modules, and operating systems. Reviewing these logs to detect unauthorized changes could result in discovering compromised data integrity.
 Access control and security software (e.g., firewalls) restricts system access to authorized personnel. Review of access control tables to determine the access provided to various users, and comparison to security logs could detect attempts
 to breach the security software.
Comparison programs are used to ensure that the auditor has the same version of the program as the company. Comparing the source program against a controlled version, or comparing the production object program with an audit version compiled from the previously reviewed source program, would highlight any

 changes to the preceding version.
Flowcharting software that uses the source version of the program to document program logic, processing logic, and input and output files facilitates comparison with other documentation (e.g., decision tables).
Program tracing, essentially a walk-through of the computer’s logic, produces computer output that indicates which instructions were executed, in which sequence. For example, this procedure might reveal that customer credit is approved after the goods are shipped, a discovery that could lead to expanded substantive tests concerning the net realizable value of accounts receivable.
 Mapping identifies sections of a program that were not executed during program operation. Dormant sections of program code may be a potential source of unauthorized use. For example, a section of code in a payroll program might not
 execute during routine processing. Upon further investigation, the auditor might discover that this section of code contains logic that infects the system with a computer virus if the programmer does not have a payroll transaction in the current pay period.
A snapshot documents the status of program execution, intermediate results, or transaction data at a specified point. Programmers often use this procedure as a debugging tool. The event triggering the snapshot can be a specific data condition, the execution of a specific instruction, or the occurrence of a particular transaction
(tagging). The tagged transaction permits examining the impact of specified transactions on other files. For example, comparing the results from known test data at various stages of processing, such as gross pay/net pay, could identify problems.
Program Testing
Program testing uses auditor-controlled actual or simulated data to test programs and related procedures and provide direct evidence about the operation of programs and programmed controls. Program testing techniques include the following:
 Test data is useful when control procedures are pervasive throughout programs or when the volume of processing is so great that verifying system performance and related controls otherwise would require a high volume of manual verification.




The test data are processed through the application programs and the actual results are compared with the auditor’s expected results. If the test results match actual results, then the test provides assurance that the program is functioning as described, and control risk may be assessed at less than maximum. The auditor can use software that develops test data (test data generators).
Base case system evaluation is a special case of test data that requires an allinclusive set of test data in order to test every possible data and processing condition. This method is time-consuming and expensive and best developed by an internal audit staff. The external auditor should refer to professional standards in deciding whether to rely upon the work of an internal audit department (see
SAS 65,
The Auditor’s Consideration of the Internal Audit Function in an Audit of Financial Statements ).
In a parallel simulation, the auditor processes live data through auditor-developed software that is supposed to duplicate the logic in the live program and compares the outputs. This method eliminates the need to prepare test data and allows the auditor to test unannounced and more frequently without disrupting the operational system or possibly modifying the files.
Continuous Testing
Continuous testing techniques are particularly appropriate in systems that leave electronic trails of evidence, such as e-commerce systems. Continuous monitoring should allow the auditor to adopt a lower control risk assessment approach in a financial statement audit.
Many auditors believe that a continuous auditing approach is necessary for paperless systems, because transaction and other files might not be retained for the entire period under audit. For example, some e-commerce systems might use a web hosting provider that retains transaction data for a limited period of time. If the data is not reviewed continuously, it might not be available to the auditor.
The following are some major types of continuous auditing techniques:


Embedded audit modules filter transaction files for data or relationships that are considered anomalies. For example, they could inform the auditor when a credit card is used in twelve countries within two hours.
An integrated test facility (ITF) allows for testing of the entire system, both manual and computer processes. The auditor establishes a fictitious entity through which auditor-submitted transaction data can be processed with live data. For example, an auditor-established fictitious vendor or customer could enter transactions via a VAN in a traditional EDI environment. The actual results could be compared with expected results, and the differences, if any, would be investigated by the auditor. ITF is most effective in a stable, legacy environment; more modern, open networks are nearly impossible to accurately replicate in an
ITF.
Emerging Systems

An example of an emerging system is one that involves business-to-consumer ecommerce on the Internet. E-commerce is marked by electronic, nearly instantaneous transactions and increased challenges to electronic security and integrity. Though it continues to grow, the future of e-commerce is uncertain; currently, it presents new challenges to businesses, consumers, and auditors.
An assurance provider can employ both emerging and traditional evidence-gathering techniques in this paperless processing environment. Many traditional evidence-gathering and assurance techniques that can increase the likelihood that the system will possess integrity are appropriate in this type of electronic system. For example, auditors can use job accounting data or operating systems logs, library management software, comparison programs, flowcharting software, snapshots, program tracing and mapping, test data, ITF, and parallel simulation to provide assurance that the software works as intended and has not been modified without authorization.
The auditor could employ audit software to discover anomalies in data files of deleted transactions (e.g., payment of the same invoice number twice). Embedded audit modules can be used to provide real-time notification of a variety of events, such as a denial of service attack.
The auditor can also employ emerging electronic assurance techniques in paperless systems. One of these techniques, identified in the AICPA and CICA’s
Continuous
Auditing , is the use of digital agents. Digital agents are data and code that act on the behalf of the user. A reactive digital agent filters incoming information, such as an order for goods that exceeds a certain dollar amount and is sent to a manager for approval. A proactive digital agent searches the system for the existence of prespecified conditions and takes specific actions upon discovery or nondiscovery.
Reactive digital agents are static and remain in one location in the system. For example, the agent could notify the auditor if the purchase price of an inventory item fell outside of a prespecified range. Mobile agents are proactive and move through networks. For example, the agent could search the web for specific information that would impact inventory marketability and net realizable value. This information could be stored in a database for the auditor’s review.
Mobile agents can subscribe to specific types of updated information within an internal or web-based system. The agent could be programmed to take appropriate action upon notification of specified events. For example, a day trader could subscribe to an online service that advises when a company’s stock price reaches a certain level and then issues a buy or sale order.
Embedded audit modules and digital agents can only be implemented with extensive assistance from management and internal audit staff. This degree of involvement in the design and implementation of the audit tool might raise questions concerning auditor independence.

Another emerging assurance technique utilizes data provided by sensors in analytical review procedures. Sensors measure a physical process, such as the amount of oil that flows through a pipeline, the amount of water used as measured by meter, or the number of rotations of a turnstile. The auditor could obtain operational data provided by the sensors and perform analytical review procedures to compare expected results with recorded amounts; for example, multiplying the gallons of water actually used at a car wash by an average revenue per gallon. This analytical review procedure is based upon objectively obtained data and yields a fairly precise estimate of gross revenue.
Another technique utilizes electronic confirmations (e-mail) to obtain thirdparty confirmation of amounts on the books. The assurance of the true identity of the sender and the recipient is critical to the integrity of the electronic confirmation process. This assurance can be obtained if both the sender and recipient utilize the services of a digital certificate authority. This is an authentication control: It ensures that the individuals are who they purport to be, not impersonators. An analogy is the customer who purchases goods with a check and produces a driver’s license (independent authentication) as proof of identity. See “Electronic Signatures and Encryption” (
The CPA Journal , August 2001), along with www.verisign.com
, for further information about certificate authorities, public and private key encryption, and digital signatures.
Glenn L. Helms, PhD, CIA, CISA, CPA, is a professor of accounting in the department of management and accountancy at the University of North Carolina at Asheville.
This Month | About Us | Archives | Advertise | NYSSCPA
The CPA Journal is broadly recognized as an outstanding, technical-refereed publication aimed at public practitioners, management, educators, and other accounting professionals. It is edited by CPAs for CPAs. Our goal is to provide CPAs and other accounting professionals with the information and news to enable them to be successful accountants, managers, and executives in today's practice environments.
©2002 CPA Journal. Legal Notices