INTRODUCTION TO NONSTATISTICAL
SAMPLING FOR AUDITORS
Jeanne H. Yamamura
CPA, MIM, PhD
APIPA 2010
1
SITUATION
You are auditing the Dept. of Admissions
& Records for Micronesia College.
 One of your objectives is to verify that
student records are being updated
correctly and timely.
 You decide to select a sample of grades
posted from the most recent semester
completed.

APIPA 2010
2
SITUATION

What would you normally document
about this sample?
Sample size
 Selection method
 Population
 Procedures to be performed
 Purpose of test


What kind of test is this?
APIPA 2010
3
OBJECTIVES
Review of sampling concepts
 Types of sampling - overview
 Nonstatistical attribute sampling

Steps in applying
 Additional coverage of:



Sampling methods
Compliance auditing
APIPA 2010
4
Applicable Professional Standards
SAS 39 Audit Sampling
 SAS 111 Amendment to SAS 39 Audit
Sampling
 ISA 530 Audit Sampling

APIPA 2010
5
AUDIT SAMPLING

Application of an audit procedure to less than
100% of the items in a population


Account balance
Class of transactions
Examination “on a test basis”
 Key: Sample is intended to be representative
of the population.
 Objective: To reach a conclusion about the
population based on the sample items tested.

APIPA 2010
6
SAMPLING RISK
Possibility that the sample is NOT
representative of the population
 As a result, auditor will reach WRONG
conclusion
 Decision errors

Type I – Risk of incorrect rejection
 Type II – Risk of incorrect acceptance

APIPA 2010
7
TYPE I – RISK OF INCORRECT
REJECTION

Internal control: Risk that sample
supports conclusion that control is NOT
operating effectively when it really is


AKA – Risk of underreliance, risk of
assessing control risk too high
Substantive testing: Risk that sample
supports conclusion that balance is NOT
properly stated when it really is
APIPA 2010
8
TYPE II – RISK OF INCORRECT
ACCEPTANCE

Internal control: Risk that sample
supports conclusion that control is
operating effectively when it really isn’t


AKA – Risk of overreliance, risk of
assessing control risk too low
Substantive testing: Risk that sample
supports conclusion that balance is
properly stated when it really isn’t
APIPA 2010
9
WHICH RISK POSES THE GREATER
DANGER TO AN AUDITOR?

Type I - Risk of incorrect rejection


Type II - Risk of incorrect acceptance


Efficiency
Effectiveness
Auditor focus on Type II

APIPA 2010
Also provides coverage for Type I
10
NONSAMPLING RISK

Risk of auditor error
Sample wrong population
 Fail to detect a misstatement when applying
audit procedure
 Misinterpret audit result


Controlled through
Adequate training
 Proper planning
 Effective supervision

APIPA 2010
11
SAMPLE SIZE FACTORS
 Desired
level of assurance
(confidence level)
 Acceptable defect rate (tolerable
error)
 Historical defect rate (expected
error)
APIPA 2010
12
CONFIDENCE LEVEL

Complement of sampling risk

5% sampling risk, 95% confidence level

How much reliance will be placed on test
results
 The greater the reliance and the more severe
the consequences of Type II error, the higher
the confidence level needed
 Sample size increases with confidence level
(decreases with sampling risk)
APIPA 2010
13
TOLERABLE ERROR AND
EXPECTED ERROR

“Precision” – the gap between tolerable
error and expected error
Expected population error rate = 1%
 Auditor’s tolerable error rate = 3%

AKA Allowance for sampling risk
 Sample size increases as precision
decreases

APIPA 2010
14
WHEN DO YOU SAMPLE?
Inspection of tangible assets, e.g.,
inventory observation
 Inspection of records or documents, e.g.,
internal control testing
 Reperformance, e.g., internal control
testing
 Confirmation, e.g., verification of AR
balances

APIPA 2010
15
WHEN IS SAMPLING
INAPPROPRIATE?






Selection of all items with a particular
characteristic, e.g., all disbursements >
$100,000
Testing only one or a few items, e.g.,
automated IT controls, walk throughs
Analytical procedures
Scanning
Inquiry
Observation
APIPA 2010
16
WALKTHROUGHS

Designed to provide evidence regarding the
design and implementation of controls
 Can provide some assurance of operating
effectiveness BUT



Depends on nature of control (automated or
manual)
Depends on nature of auditor’s procedures to test
control (also includes inquiry and observation
combined with strong control environment and
adequate monitoring)
Walkthough = sample of 1
APIPA 2010
17
STATISTICAL VS
NONSTATISTICAL SAMPLING

Statistical sampling


Statistical computation of sample size
Statistical evaluation of results

Nonstatistical sampling
 Sample sizes should be approximately the
same (AU 350.22)
 Sample sizes must be sufficient to support
reliance on controls and assertions being
tested
APIPA 2010
18
WHEN IS SAMPLING
NONSTATISTICAL?
If sample size determined judgmentally
 If sample selected haphazardly
 If sample results evaluated judgmentally

APIPA 2010
19
TYPES OF SAMPLING
Attribute sampling
 Monetary unit sampling
 Classical variables sampling

APIPA 2010
20
ATTRIBUTE SAMPLING
Used to estimate proportion of a
population that possesses a specific
characteristic
 Most commonly used for T of C
 Can also be used for dual purpose
testing (T of C and Substantive T of T)

APIPA 2010
21
MONETARY-UNIT SAMPLING
AKA probability proportional to size
(PPS) sampling, cumulative monetary
unit sampling
 Used to estimate dollar amount of
misstatement

APIPA 2010
22
CLASSICAL VARIABLES SAMPLING
Uses normal distribution theory to
identify amount of misstatement
 Useful when large number of differences
expected


Smaller sample size than MUS
Effective for both overstatements and
understatements
 Can easily incorporate zero balances

APIPA 2010
23
STEPS IN NONSTATISTICAL ATTRIBUTE
SAMPLING APPLICATION
 Planning
1. Determine the test objectives
2. Define the population characteristics
3. Determine the sample size
 Performance
4. Select sample items
5. Perform the auditing procedures
 Evaluation
6. Calculate the results
7. Draw conclusions
APIPA 2010
24
STEP 1: DETERMINE THE TEST
OBJECTIVES

Objective for T of C: To determine the
operating effectiveness of the internal
control


Support control risk assessment below
maximum (FS audit)
Identify controls to be tested and
understand why they are to be tested
APIPA 2010
25
TESTS OF CONTROLS

Concerned primarily with
Were the necessary controls performed?
 How were they performed?
 By whom were they performed?


Appropriate when documentary evidence
of performance exists
APIPA 2010
26
SUBSTANTIVE TEST OF
TRANSACTIONS

Objective for S T of T: To determine
whether the transactions contain
monetary misstatements


Alternatively, to determine whether the
system is operating as designed
Identify transactions to be tested and
understand why they are to be tested
APIPA 2010
27
STEP 2: DEFINE THE POPULATION
CHARACTERISTICS
 Define the sampling population
 Can be defined however desired BUT must include
entire population as defined
 Test population for completeness
 Define the sampling unit
 Determined by available records
 Based on definition of population and audit
objective
 Define the control deviation conditions
APIPA 2010
28
STEP 3: DETERMINE THE SAMPLE
SIZE
Consider desired confidence level,
tolerable deviation rate, and expected
population deviation rate
 Judgmentally determine sample size
 NOTE: Check against statistical sample
size tables to verify adequacy

APIPA 2010
29
TOLERABLE RATE GUIDELINES
Significance of the transactions and related account balances
that the IC are intended to affect
Highly significant balances
Tolerable Rate of 4%
Significant balances
Tolerable Rate of 5%
Less significant balances
Tolerable Rate of 6%
Preliminary
Assessment of CR
Low
< = 5%
Moderate
< = 10%
High
APIPA 2010
Tolerable Rate
Do not test controls
30
TOLERABLE RATE GUIDELINES
Assessed importance
of the control
Highly important
Moderately important
APIPA 2010
Tolerable Rate
3 - 5%
6 – 10%
31
ESTIMATE OF POPULATION ERROR
RATE
Prior
year results
Preliminary sample
Should be low – 0, 1%
Higher rates increase sample
size
APIPA 2010
32
STEP 3: DETERMINE THE SAMPLE
SIZE


Guidelines for nonstatistical sample sizes for tests
of controls
If any errors found, increase sample size or
increase control risk (Probably not applicable to
Public Auditor)
Desired level of controls reliance (how
important is the control/process)
Sample size
Low
15-20
Moderate
25-35
High
40-60
APIPA 2010
33
SMALL POPULATIONS AND INFREQUENTLY
OPERATING CONTROLS
Small Population Sample Size Table
Control Frequency and
Population Size
Sample Size
Quarterly (4)
2
Monthly (12)
2-4
Semimonthly (24)
3-8
Weekly (52)
5-9
APIPA 2010
34
STEP 4: SELECT SAMPLE ITEMS
Random sample
 Systematic sample (with random start)
 Haphazard selection

APIPA 2010
35
RANDOM SELECTION
Every possible combination of population
items has an equal chance of being
included in the sample
 Random number tables
 Computer generation of random
numbers

APIPA 2010
36
SYSTEMATIC SELECTION

Interval calculated and items selected
based on size of interval

Interval = Population / Desired Sample Size
Starting point is random number within
interval
 Need to consider if bias present due to
patterns in data

APIPA 2010
37
HAPHAZARD SELECTION

Selection by auditor without any conscious
bias

If you select large, risky, or unusual items, it is NOT
haphazard selection and it is NOT audit sampling.
Instead – targeted or directed selection

Still desire representative sample
 Avoid unusual, large, first or last
 Useful for certain situations


APIPA 2010
Example: Tracing credits from AR to CR/other
sources looking for fictitious credits
Less costly and simpler
38
STEP 5: PERFORM THE AUDITING
PROCEDURES
Conduct planned audit procedures
 What if?

Voided documents - if properly voided, not
a deviation; replace with new sample item
 Unused or inapplicable documents –
replace with new sample item
 Inability to examine sample item – deviation
 Stopping test before completion – large
number of deviations detected

APIPA 2010
39
STEP 5: PERFORM THE AUDITING
PROCEDURES

Deviations observed

Investigate nature, cause, and consequence
of every exception
Unintentional error? Or fraud?
 Monetary misstatement resulted?
 Cause – misunderstanding of instructions?
Carelessness?
 Effect on other areas?

APIPA 2010
40
STEP 6: CALCULATE THE RESULTS
No computed upper deviation rate (per
table in statistical sampling)
 Compute Calculated Sampling Error =
Tolerable Error Rate – Sample Error
Rate.

APIPA 2010
41
STEP 7: DRAW CONCLUSIONS

Control not effective (system not working
as designed) if

Calculated Sampling Error too small

Depends on sample size used
Sample Error Rate > Tolerable Error Rate
 Sample Error Rate > Expected Population
Error Rate

APIPA 2010
42
COMPLIANCE AUDITING

Performance of auditing procedures to
determine whether an entity is complying with
specific requirements of laws, regulations, or
agreements
 Governmental entities and other recipients of
governmental financial assistance

APIPA 2010
Compliance with laws and regulations that
materially affect each major federal assistance
program
43
COMPLIANCE AUDITING OF FEDERAL
ASSISTANCE PROGRAMS

Definition of population for testing of an
internal control procedure that applies to
more than one program
Define items from each major program as a
separate population, OR
 Define all items to which control is
applicable as a single population
 Second choice usually more efficient

APIPA 2010
44
COMPLIANCE AUDITING EXAMPLE

Federal financial assistance for Island
City
Three major federal financial assistance
programs
 Four nonmajor programs


Control: Transaction review to ensure
that only legally allowable costs are
charged to each program
APIPA 2010
45
COMPLIANCE AUDITING - EXAMPLE
More efficient to select one sample from
population of all transactions (major and
nonmajor programs)
 Confidence level = 95%
 Tolerable deviation rate = 9%
 Expected population deviation rate = 1%
 Sample size: 51
 1 allowable deviation

APIPA 2010
46
T of C versus S T of T

Test of Control


Verifies that a control is operating effectively
Substantive Test of Transactions

APIPA 2010
Verified that a transaction does not contain
a misstatement
47
ASSERTIONS FOR CLASSES OF
TRANSACTIONS
Occurrence: Transaction actually
occurred and pertains to the entity
(existence/validity)
 Completeness: All transactions have
been recorded
 Accuracy: Amounts and other data have
been recorded correctly

APIPA 2010
48
ASSERTIONS FOR CLASSES OF
TRANSACTIONS
Cutoff: Transactions have been
recorded in the correct accounting period
 Classification: Transactions have been
recorded in the proper accounts

APIPA 2010
49
CALCULATED SAMPLING ERROR
Tolerable error rate – Sample error rate =
Calculated sampling error
 Sample error rate = Population error rate
due to sampling error
 Auditor must evaluate calculated
sampling error to see if it is big enough
(sufficiently large to allow for sampling
error in population)

APIPA 2010
50
CALCULATED SAMPLING ERROR
If Sample error rate > Tolerable error rate
= REJECT – CONTROL NOT
WORKING or PROCEDURE NOT
BEING FOLLOWED
 If Sample error rate > Expected
population error rate, REJECT –
CONTROL NOT WORKING OR
PROCEDURE NOT BEING FOLLOWED

APIPA 2010
51
STEPS IN NONSTATISTICAL SUBSTANTIVE
SAMPLING APPLICATION
 Planning
1. Determine the test objectives
2. Define the population characteristics
3. Determine the sample size
 Performance
4. Select sample items
5. Perform the auditing procedures
 Evaluation
6. Calculate the results
7. Draw conclusions
APIPA 2010
52
STEP 2: DEFINE THE POPULATION
CHARACTERISTICS
 Identify individually significant items
 Some items too risky, must be audited, OR
 Easier to pull out and test large items
 Stratify population
 Divide population into homogeneous units
 For example, all items > $10,000
 Items tested 100% are not part of the sample
APIPA 2010
53
STEP 2: DEFINE THE POPULATION
CHARACTERISTICS
 Define the sampling population
 Consists of an account balance or class of
transactions
 Will project sample results to population
 Must be sure to adequately identify population
 For example: Accounts Receivable could be
defined as
 All accounts
 Accounts with zero balances
 Accounts with debit balances
 Accounts with credit balances
APIPA 2010
54
STEP 2: DEFINE THE POPULATION
CHARACTERISTICS
 Define the sampling unit
 Any item in the defined population
 Could be an account or a transaction
APIPA 2010
55
STEP 3: DETERMINE THE SAMPLE
SIZE

Subjective determination OK
 Factors to consider:

Amounts of individual items



Variability and size of population


APIPA 2010
Accounting populations usually include a few
very large items, a number of moderately large
amounts, and a large number of small amounts
If not stratified, will need larger sample
The greater the variability, the larger the sample
size needed
Population size – little effect on sample size so
usually ignored
56
STEP 3: DETERMINE THE SAMPLE
SIZE

Factors to consider:

Risk of incorrect acceptance (RIA)



Tolerable misstatement and expected
misstatement


APIPA 2010
As RIA increased, sample size decreases
If controls good, can accept larger RIA for
substantive testing
Larger tolerable misstatement, smaller sample
size
Larger expected misstatement, larger sample
size
57
STEP 4: SELECT SAMPLE ITEMS

Any method that will result in
representative sample
Random sample
 Systematic sample (with random start)
 Haphazard selection

APIPA 2010
58
STEP 5: PERFORM THE AUDITING
PROCEDURES

Deviations observed

Investigate nature, cause, and consequence
of every exception
Unintentional error? Or fraud?
 Monetary misstatement resulted?
 Cause – misunderstanding? Carelessness?
 Effect on other areas?

APIPA 2010
59
STEP 6: CALCULATE THE RESULTS


Compute sample error amount or
sample error rate
Project to population

Projected misstatement


APIPA 2010
Error * number of items in population
Error rate * dollar population value
60
STEP 7: DRAW CONCLUSIONS

APIPA 2010
Compare projected misstatement to
tolerable misstatement

If projected misstatement < tolerable
misstatement, population OK

If projected misstatement > tolerable
misstatement, population misstated
61
STEP 7: DRAW CONCLUSIONS

Consider sampling risk

If projected misstatement < expected misstatement,
probably safe to conclude that population is OK
(i.e., there is an acceptably LOW risk that the true
misstatement exceeds the
tolerable misstatement)

If projected misstatement > expected misstatement,
greater risk present (i.e., there is an
UNACCEPTABLY HIGH risk that the true
misstatement exceeds the tolerable
misstatement).
APIPA 2010
62
STEP 7: DRAW CONCLUSIONS

If recorded amount believed to be
misstated, need more work!
Investigate misstatements
 Adjust recorded amounts

APIPA 2010
63
RESOURCES
Audit Sampling: An Introduction, 3rd
Edition, Guy, Carmichael & Whittington
 Audit Guide: Audit Sampling, New
Edition as of May 1, 2008, AICPA
 Auditing & Assurance Services, 6th
Edition, Messier, Glover, & Prawitt
 Auditing & Assurance Services, 12th
Edition, Arens, Elder & Beasley

APIPA 2010
64
QUESTIONS?
APIPA 2010
65