Why Do Auditors Over-rely on Weak
Analytical Procedures? The Role of
Outcome Bias and Insensitivity to Precision
Steve Glover
Doug Prawitt
Jeff Wilks
Brigham Young University
Importance of Analytical Procedures

Frequent and increasing use of analytical
procedures (Ameen & Stawser 1994; McDaniel & Kinney
1995; Hirst & Koonce 1996; Kinney & McDaniel 1996; Bell et al.
1997)

Findings of the POB (2000)
– Analytical procedures used for substantive
testing provided insufficient assurance 20%
of the time.
– Insufficient procedures typically related to
income statement accounts.
Reliance on Audit Evidence

Auditors sensitive to
– Source competence (Bamber 1983)
– Objectivity (Hirst 1994, Caster & Pincus 1996)
– Amount, composition, directness, and
deviations from expectations (Caster & Pincus 1996)

What might explain over-reliance on audit
evidence, and analytical procedures in
particular?
Favorable Outcome Effect

When analytical procedures produce
expected or even desired results, auditors
are less critical of the procedure.
– Motivated reasoning (Kunda 1990)
– Inherited hypotheses (Swann & Giuliano 1987)
– Biased hypothesis testing (Pyszczynski & Greenberg
1987)
Sensitivity to Precision

Precision
– The expected closeness of an observed statistic to the
“correct” amount
– A critical determinant of the quality of evidence
produced by substantive analytical procedures
– SAS No. 56 suggests that
• Disaggregated data produces more precise expectations than
aggregated data
• Precision also affected by reliability of source data,
predictability of account, and type of procedure
Sensitivity (continued)

Contingent auditor sensitivity to precision
– Auditors sensitive to precision when testing a
hard account, but not estimates (McDaniel &
Simmons 2003)
– We argue “no-significant-difference”
outcomes diminish auditor sensitivity to
precision
Experiment 1
Aggregate
Procedure
No Significant
Difference
Significant
Difference
Assess
Evidential
Strength
H1: Outcome Bias
The assessed strength of audit evidence
obtained from an aggregate analytical
procedure will be greater for auditors
who observe no significant difference (a
favorable outcome) than for auditors who
observe a significant difference.
Experiment 1
Aggregate
Procedure
Disaggregate
Procedure
No Significant
Difference
Significant
Difference
Significant
Difference
Assess
Evidential
Strength
Assess
Evidential
Strength
Reassess
Strength of
Part A
H2: Sensitivity to Precision
After evaluating the disaggregated
procedure, auditors who initially observe
no significant difference will lower their
assessments of the aggregate procedure’s
evidential strength more than will
auditors who initially observed a
significant difference.
Expected Results
Evidential Strength
Originally
Observed
Significant
Difference
No
Yes
Initial
Assessment of
Aggregate
Procedure
Reassessment
of Aggregate
Procedure
Experiment 1: Method

Task
– Determine appropriateness of interest income
– Compute expectation for current year based on
annual loan balances and rates
– Compute new expectation based on quarterly and
categorized loan balances and rates

Participants
– 67 supervising seniors from one Big Four firm
participated during national training sessions
– 34.5 months of audit experience (standard deviation
12.8)
Experiment 1: Method

Design (2x2 mixed design)
– Favorable outcome: Yes or no (between
subjects)
– Precision of analytic: Aggregate or
disaggregate (within-subject)

Primary dependent measure
– Evidential strength: 1 (extremely weak) to 7
(extremely strong)
Results (Table 1)
Estimates of Evidential Strength for
Part A-Aggregate
Procedure (Initial
Assessment)
Part B-Disaggregated
Procedure – Suggests
Material Misstatement
Aggregate
Procedure
(Reassessment)
No Significant
Difference
3.9
(0.81)
[29]
3.7
(0.99)
[29]
2.7
(1.16)
[29]
Significant Difference
2.9
(1.02)
[36]
3.8
(1.03)
[36]
2.8
(1.21)
[36]
3.4
3.8
2.8
Outcome of Aggregate
Analytical Procedure
Combined
Control Conditions
Evidential Strength
Originally
Observed
Significant
Difference
No
Yes
Initial
Assessment of
Aggregate
Procedure
Reassessment
of Aggregate
Procedure
Cueing Sensitivity to Precision

Ex ante warning to consider weaknesses
can lead to more careful scrutiny (Petty &
Cacioppo 1986)
– Evidenced by lower assessments of strength
– Possible demand effect

Include two types of participants
– Auditors with relevant task knowledge
– Students without such knowledge
H3: Cueing Sensitivity to Precision
Part 1: The perceived strength of evidence
obtained from a weak, high-level analytical
procedure will be lower for auditors who are
cued ex ante to critically evaluate the quality of
the analytical procedure than for auditors who
are not cued to critically evaluate the procedure.
Part 2: An ex ante cue to critically evaluate the
quality of a procedure will make no difference in
students’ assessments of evidential strength.
Evidential Strength
Expected Results
Students
Auditors
No Cue
Cued to
Critically
Evaluate
Experiment 2: Method

Participants
– 84 supervising seniors from one Big Four firm
participate during national training sessions (34.5
months of audit experience, standard deviation 12.8
months)
– 90 graduate students in a second auditing course



Design (2x2 between subjects)
Task and materials same as Part A, Exp. 1 (ND)
Dependent measures same as Part A, Exp. 1 (ND)
Experiment 2: Method

The text of the cue read as follows:
Before computing an expectation of
interest income for the current year, please
consider the strength (quality and
sufficiency) of the evidence provided by
the interest income analytical procedure
used by the audit team last year. Please list
in the space provided below one or more
weaknesses of this analytical procedure.
Results
Participant
Professional
Participants
Student
Participants
Combined
No
4.1
(0.64)
[42]
2.9
(1.27)
[45]
3.5
(1.18)
[87]
Yes
3.3
(1.05)
[42]
2.8
(1.13)
[45]
3.0
(1.11)
[87]
3.7
2.9
3.3
Explicit Guidance to
Consider Weaknesses
Combined
Conclusions & Implications

An outcome effect can explain in part why
auditors may over rely on weak analytical
procedures.
– Hinders consideration of precision
– May also hinder consideration of…?
• Source competence (Bamber 1983)
• Objectivity (Hirst 1994, Caster & Pincus 1996)
• Amount, composition, directness, and deviations
from expectations (Caster & Pincus 1996)
Conclusions & Implications

Ex ante prompt to consider weaknesses of
an analytical procedure appears to
– Produce no demand effect
– Lead to more careful scrutiny by those who
have relevant task knowledge
Limitations & Future Research


All auditors from one Big Four firm
The impact of teamwork, the review
process, and accountability not examined