Reports
Leading Practices in Report
Testing
1
Agenda
• What are reports, really?
• Why is report testing important?
• Which reports should be tested?
• How is report testing performed?
• How often should reports be tested?
PwC
2
What are Reports, really?
High level or detailed analysis snapshot of business operations
•
Subledgers and Registers (e.g., Fixed Asset Listing, Sales Register, Payroll Register)
•
Agings (e.g., A/R, A/P)
•
Analyses (e.g., Excess & Obsolete Inventory, Cycle Count Variances)
•
Transactional Analysis (e.g., Salesperson Commission Report, Sunshine Act Payments)
Types of Reports
•
Canned/Standard vs. Custom
•
Automated vs. Manual
•
Ad hoc
•
Extracted to spreadsheets or other tools
Report Uses
•
PwC
Used in reporting to external and internal audiences
-
Financial
-
Operations
-
Compliance (e.g. HIPAA, SOX, PCI, Nevada Gaming Board, FDA, etc.)
3
Why is Report Testing Important?
Testing a report is the process of assessing the reliability of
information on key reports. This is an important process
because:
• Reports are often used as a key basis for management’s decisions.
• Users of the report assume that the information on the report is
representative of the source data.
• Gives assurance needed to rely on reports regardless of the use:
- Compliance
- Controls Reliance
- Audit Procedures
- Etc.
PwC
4
Which Reports Should Be Tested?
The scope of your audit drives the reports to be tested. In
determining the reports which should be subject to
evaluation, it is useful to evaluate the following:
• Usage of the report: Does the report drive the achievement of the
business, operational or audit objective?
• Impact of the report: Would a mistake in the operation of the report
present potential financial or operational risk to the organization?
• Controls considerations: Is this report used in the execution of a key
control?
• Compliance requirements: Is this report used as evidence in part of a
compliance program?
• Supports other testing: Is this report used as a population for
substantive or control testing samples?
PwC
5
Which Reports Should Be Tested?
What is the Risk-Based Approach to Scoping
Risk factors to consider while determining approach:
• Report's complexity
- Canned/Standard vs. Custom
- Automated vs. Manual
- Ad hoc
• Frequency of changes to the report
• Number and nature of users relying on the reports
• Downstream reliance on reports
• Materiality of the transactions or values included in the report
PwC
6
How are Reports Tested?
Reliability is achieved by ensuring the report data is complete
and accurate.
• Completeness - all relevant transactions that occurred (or relevant
information) are captured on the report once and only once
• Accuracy – transactions/information are reported at the correct
amount, reflecting appropriate calculations, in correct accounts (or
other categorizations)
PwC
7
How are Reports Tested?
Pathways for achieving comfort over reports
• Substantive Procedures - Sampling Approach
• Tested Via Performance of a Control – Management
• Automated Report Test - Tests of One
• Last Change Date & Query Parameter Inspection
PwC
8
How are Reports Tested?
Automated Reports
Steps for testing Accuracy and Completeness of an Automated
Report:
1.
Obtain a recent copy of the report.
2.
Select a sample from the report and tie it back to the source system to
verify that the report contains accurate data; perform recalculation of
key components to ensure report calculates accurately
3.
Two avenues for Completeness:
PwC
1.
Full and False Inclusion - select a sample from the source system
that would be expected to be included in the report and verify that it
is indeed included in the report; select a sample that should be
excluded from the report and verify that it is indeed excluded from
the report
2.
Total Reconciliation - Tie the total of the report back to the relevant
reference (e.g., account balance) in the source system
9
How are Reports Tested?
Automated Reports
Accuracy
Completeness
PwC
10
How are Reports Tested?
Automated Reports
Challenges encountered in Automated Reports Testing:
1. Timing - If the underlying data within the report is changing
frequently, it may be advisable to have the contact run an ad-hoc
iterationof the report to compare to real-time data rather than
using a recent version of the report.
2. Difficulty in Completeness Assertion - It may be difficult to
either capture a total or even a population of items from the system
to tie back to the report for completeness. In these instances, it may
be necessary to select a specific item to subselect from, or select a
sample from a different population. Judgment required.
PwC
11
How are Reports Tested?
Automated Reports
Challenges encountered in Automated Reports Testing:
3. Multitude of Client Contacts - Suggest organizing reports by
key contact and walking through all reports with a specific contact
at one time if possible
4. Assessing inventory of Reports to test - Suggest confirming
the inventory of reports early in the audit process
5. Report extracted to spreadsheet - If it is determined that a
report could be manipulated after created (as when exported to a
spreadsheet), the report will not be able to be tested via samples of 1
and will need Spreadsheet Testing (more involved) or require a
more substantive audit approach to assess Accuracy and
Completeness
PwC
12
How are Reports Tested?
Automated Reports
Steps for testing an Automated Report based on last change:
1.
Obtain comfort over the query parameters used if applicable.
2. Verify that the last change to the underlying program was before
the date this report was last tested, or
3. Verify that the last change was performed by the program vendor.
This provides comfort that the program is default program
functionality and has not been customized by company personnel.
4. If the report has been last changed by someone other than program
vendor, inquire with appropriate personnel regarding the nature of
the change and perform additional test procedures.
PwC
13
How are Reports Tested?
Automated Reports
Testing an Automated Report based on last change and query parameters
PwC
14
How are Reports Tested?
Automated Reports
Testing an Automated Report based on last change:
PwC
15
How often should reports be tested?
Consider the following when determining when to test the
reports and with what frequency:
• Policy (e.g., audit guidance, corporate policy)
• Risk criteria of the report (e.g., canned vs. custom)
• Reliability on the underlying system (e.g., are ITGCs operating
effectively?)
• Changes in the environment
• Base-lining
PwC
16
FAQ
• How can we be sure source data is correct?
• What do we do if the report ties, but the data is bad (i.e. garbage in,
garbage out)?
• Why not just look at the query?
• Why is a sample of 1 acceptable? When is a sample of 1 not acceptable?
• What if there are no totals to tie report back to for the testing of
completeness?
• When should testing be performed using source documents versus using
system data?
• What if the testing of a report fails?
• Is a query run on the system considered a report? How do you gain
comfort around this?
PwC
17
Thank You!
This publication has been prepared for general guidance on matters of interest only, and does
not constitute professional advice. You should not act upon the information contained in this
publication without obtaining specific professional advice. No representation or warranty
(express or implied) is given as to the accuracy or completeness of the information contained
in this publication, and, to the extent permitted by law, PwC, its members, employees and
agents do not accept or assume any liability, responsibility or duty of care for any
consequences of you or anyone else acting, or refraining to act, in reliance on the information
contained in this publication or for any decision based on it.
© 2012 PwC, Irvine. All rights reserved. In this document, “PwC” refers to PwC, Irvine which is
a member firm of PricewaterhouseCoopers International Limited, each member firm of which is
a separate legal entity.