Internal Audit Testing
and Sampling Techniques
Chartered Institute of
Internal Auditors – May 2014
Controls Testing
PwC
Slide 1
Testing Priorities
Risk B1
Risk B2
Risk C2
Risk A2
Risk C1
Risk A1
Controls testing
Testing techniques
Inquiry
Observation
Inspection/
Examination
Re-performance
PwC
Slide 3
Controls testing
Control testing
Tests of controls are
designed to obtain
evidence to assess
their operating
effectiveness.
Operating
effectiveness means
that the controls are
functioning as
designed on a
consistent basis over
the period under
examination.
PwC
- Inquiry – consists of seeking information of
knowledgeable people within the client
- Observation – consists of looking at a process
being performed by others
- Examination
◦ inspection of information or data
◦ walkthrough – confirming our
understanding of a process by tracing
individual transactions from beginning to
end
- Re-performance – independent execution of
procedures that were originally performed as
part of management’s internal controls
Slide 4
Controls testing
Determining which Testing technique
to use
Re-performance
Level of
Comfort
Inspection/
Examination
Observation
Inquiry
PwC
Slide 5
Controls testing
Determining which testing technique
to use
Considerations:
• The susceptibility of the control to change.
• The frequency and extent of the control.
• Our initial view of the likelihood of control weakness.
• Significance of the control to the control environment and how much
reliance is being placed on it.
PwC
Slide 6
Value Protection - execute
Sampling
PwC
• Sampling is the application of
auditing procedures to a
representative group of less than
100% of the items within a
homogenous population
3 Steps to follow:
1.
Determine the control test
objective, population and
sampling unit
• We use non-statistical sampling
2.
Determining the sample size
3.
Selecting the sample for testing
Slide 7
Value protection – Execute
Sampling
Manual Controls
Depends on:
• Frequency of control or population size
• Level of evidence that is judged to be necessary
The table below, can be used as a general rule; however, we may use a
smaller sampling size:
Frequency
of Control
Assumed population
size
Sample Size
Annual
1
1
Quarterly
4
2
Monthly
12
2 (minimum) to 5 (maximum), Select 3 if you require
a mid-range.
Weekly
52
5 to 15. Select 10 if you require a mid-range
Daily
250
20 to 40. Select 30 if you require a mid-range
Multiple
times per day
Over 250
25 to 60. Select 30 or 45 if you require a mid-range
PwC
Slide 8
Value protection – Execute
Sampling
Manual Controls
Following factors may indicate that sample sizes should be selected at the
higher end of the ranges:
- The greater the potential financial loss or adverse event to the company if
the control is not effective or fails:
- The more complex the control
- The greater the degree of judgment in control operation
PwC
Slide 9
Value protection – Execute
Sampling
Automated Controls
If IT General Controls have been tested and found to be effective, it may be
sufficient to only test one operation of the Automated Control
PwC
Slide 10
Documentation
Audit documentation
Audit documentation must contain
sufficient information to enable an
experienced auditor, having no previous
connection with the engagement to:
Remember: if what
you did isn’t
documented, it’s the
equivalent of not
performed!
- Understand the nature, timing, extent and results of the procedures
performed, evidence obtained, and conclusions reached
- Determine who performed the work and the date such work was
completed, as well as the person who reviewed the work and the date of
such review.
- Understand the linkage between conclusions and facts
- Document what you have done and how you reached your conclusions
PwC
Slide 11
Confidential
The changing shape of internal audit
Increased use of technology
Drivers for change (top 3):
1. Complexity
increased use of technology within the
business | higher volume of transactions |
increased automation | businesses driven by
data | devil is in the detail | how do you find a
needle in the hay stack?
2. More for less
pressure to deliver more with less | value |
quality | efficiency | insight | pressure to
deliver with less resource and using samples?
3. Resources
skills sets | innovation | technologically
minded team | reduced fear factor |
development opportunities for your people?
May 2014
PwC
12
CIIA - 14 May 2014
Confidential
May 2014
PwC
13
CIIA - 14 May 2014
Confidential
What are CAATs?
Computer Assisted Audit Techniques
A means of accessing
large amounts of data in
a format that can provide
transparency not
attainable through other
auditing procedures.
The results may be used to identify areas of
key risk, fraud, errors or misuse; improve
business efficiencies; verify process
effectiveness; or influence
business decisions.
(ISACA August 2011)
May 2014
PwC
14
CIIA - 14 May 2014
Confidential
Data analytics - methodology
Extract and
upload raw data
May 2014
PwC
Map and
organise data
Analyse and
visualise data
Finalise audit
evidence, identify
anomalies and
insight
15
CIIA - 14 May 2014
Computer Assisted Audit Techniques
Advantages
How can you ever pick a sample that
is representative?
Expandable model, allowing tests to
be refined, tuned, added, removed
Standing still or moving with the
times?
You can quickly identify and address
emerging issues and risks
In the future it will allow audit tests
to be “pushed” into the organisation
as monitoring controls
May 2014
PwC
1
2
3
4
5
Increased coverage – 100% of
transactions
Efficiency – repeatable and
automated
Value and insight – improve the
perception of IA
Basis for prioritisation of where
to look next in the organisation
Climb the maturity curve –
predictive business enabler
16
CIIA - 14 May 2014
Confidential
Data analytics on vendor standing data
Identify duplicate vendors based on the same or similar (fuzzy match) vendor name.
Identifying and resolving duplicate vendor records is important as otherwise this could lead to loss, error or fraud. For
example: loss of purchasing volume discounts available where spend with a specific supplier is recorded across two or
more records for the same supplier, error if one vendor record is updated but the duplicate vendor record is not
resulting in incorrect and inconsistent records, and fraud for example where duplicate vendor records are used to
process payments below a review threshold.
12,253
vendors listed in standing data
1,031
perfect duplicates
May 2014
PwC
46
96
fuzzy match with 1
character difference
fuzzy match with 2
character difference
231
fuzzy match with 3
character difference
17
CIIA - 14 May 2014
Exercise
You are the internal auditors to an NHS Trust. You have been asked to
undertake a review to assess the accuracy of the information used to support the
KPIs that are reported to the Board on a monthly basis and to external
regulators quarterly. What would you consider in devising a testing approach?
PwC
18
Exercise
You are the internal auditors to an NHS Trust. You have been asked to
undertake a review to assess the accuracy of the information used to support the
KPIs that are reported to the Board on a monthly basis and to external
regulators quarterly.
You are driving to work and hear on the radio that a NHS Trust in another part
of the country has got into serious trouble for mis-reporting cancer waiting
times data. There seems to be an issue in distinguishing between cancellations
and DNAs. Would you do anything differently.
PwC
May 2014
19
Exercise
You are the internal auditors to an NHS Trust. You have been asked to
undertake a review to assess the accuracy of the information used to support the
KPIs that are reported to the Board on a monthly basis and to external
regulators quarterly.
You are driving to work and hear on the radio that a NHS Trust in another part
of the country has got into serious trouble for mis-reporting cancer waiting
times data. There seems to be an issue in distinguishing between cancellations
and DNAs.
In checking the above with the client you realise that they may have innocently
mis-interpreted the above and that this might mean that they have been misreporting data to their external regulators. What would you do?
PwC
May 2014
20
This publication has been prepared for general guidance on matters of interest only, and does not
constitute professional advice. You should not act upon the information contained in this publication
without obtaining specific professional advice. No representation or warranty (express or implied) is
given as to the accuracy or completeness of the information contained in this publication, and, to the
extent permitted by law, PricewaterhouseCoopers LLP, its members, employees and agents do not
accept or assume any liability, responsibility or duty of care for any consequences of you or anyone
else acting, or refraining to act, in reliance on the information contained in this publication or for any
decision based on it.
© 2014 PricewaterhouseCoopers LLP. All rights reserved. In this document, “PwC” refers to
PricewaterhouseCoopers LLP (a limited liability partnership in the United Kingdom) which is a
member firm of PricewaterhouseCoopers International Limited, each member firm of which is a
separate legal entity.