“The Reports of My Death Are Greatly
Exaggerated” – Expert Systems Research
in Accounting
Steve G.Sutton
KPMG Professor of Accounting
Matt Holt
Ph.D. Student
Vicky Arnold
EY Professor of Accounting
Kenneth G. Dixon School of Accounting
Gray et al. (IJAIS, 2014)
• Looked at ES/AI research in terms of the
Gartner Hype Curve
• AI/ES research from an ES centric lens has
waned since the late 1990s
• AI/ES practice became disillusioned and
abandoned their use in audit firms
• But, I’m reading 2nd Machine Age
(Brynjolfsson and McAfee 2014) and it’s all AI
• Am I biased because I’m still researching a
dead field?
2
Artificial Intelligence
Supervised
Learners
Classifiers
Nearest Neighbor
Naïve Bayes
Decision Trees
Regression Trees
Model Trees
Neural Networks
Dual Use
Support Vector Mach.
Pattern Detection
Associative
Learners
Clustering
Predictors
Machine
Learning
Artificial Intelligence
And
Intelligent Systems
Unsupervised
Learners
K-means
Expert Systems
Knowledge-Based
Systems
Intelligent DA
Intelligent DSS
Intelligent Agents
3
Artificial Intelligence
Supervised
Learners
Classifiers
Nearest Neighbor
Naïve Bayes
Decision Trees
Regression Trees
Model Trees
Neural Networks
Dual Use
Support Vector Mach.
Pattern Detection
Associative
Learners
Clustering
Predictors
Machine
Learning
Artificial Intelligence
And
Intelligent Systems
Unsupervised
Learners
K-means
Expert Systems
Knowledge-Based
Systems
Intelligent DA
Intelligent DSS
Intelligent Agents
4
Artificial Intelligence
Supervised
Learners
Classifiers
Nearest Neighbor
Naïve Bayes
Decision Trees
Regression Trees
Model Trees
Neural Networks
Dual Use
Support Vector Mach.
Pattern Detection
Associative
Learners
Clustering
Predictors
Machine
Learning
Artificial Intelligence
And
Intelligent Systems
Unsupervised
Learners
K-means
Expert Systems
Knowledge-Based
Systems
Intelligent DA
Intelligent DSS
Intelligent Agents
5
Are AI and ES Dead?
• Audit (Full-Time)- Orlando, Winter/Spring 2016
• Deloitte - Orlando, FL
• You will use our cutting-edge audit tools and technologies that
use artificial intelligence, advanced analytics, data
visualizations and process flow automation...
• 3:07 PM
• Audit (Intern)- Orlando, Summer 2016
• Deloitte - Orlando, FL
• You will execute these audit procedures using our cuttingedge audit tools and technologies that use artificial
intelligence, advanced analytics, data...
6
• 3:07 PM
Is Practice that Different Today?
Brown (1991)
Dowling & Leech (2007)
Work Program Development
Program Development and Test Planning
Internal Control Evaluation
Internal Control Assessment
IT Control Assessment
Risk Analysis and Assessment
Risk Identification & Assessment
Inherent Risk Analysis
Tax Accrual and Deferral
(Unclear of differences potentially from Australian firm
implementations of audit support systems).
Disclosure Compliance
Disclosure Compliance
Need for Second Partner Review
Loan Analysis
Bank Failure Prediction
SEC AID
Compliance with GAAP
Sample Size
Sample Size Calculator
Automated Analytics
Automated Analytics
Client Acceptance and Retention
Materiality Calculator
Need for Specialists to be Involved
Need for IT Specialist for Control Risk Assessment
Incomplete Work Identifier
Review Risk Analysis to Assist Reviewer of Auditor’s Work
7
• Gregor and Benbasat’s (1999, 498) seminal
paper on explanation facilities:
“[K]nowledge-based (expert) systems (KBS) and
intelligent systems in general, are important components
of an organization’s information systems portfolio… what
we will label generically “intelligent systems” to indicate
a broader focus than that of traditional KBS. The
distinguishing feature of intelligent systems is that they
commonly contain a knowledge component—a
computerized version of human tacit and explicit
knowledge. Such systems are based on the basic
elements of artificial intelligence: knowledge
representation, inference and control.”
8
Artificial Intelligence
Supervised
Learners
Classifiers
Nearest Neighbor
Naïve Bayes
Decision Trees
Regression Trees
Model Trees
Neural Networks
Dual Use
Support Vector Mach.
Pattern Detection
Associative
Learners
Clustering
Predictors
Machine
Learning
Artificial Intelligence
And
Intelligent Systems
Unsupervised
Learners
K-means
Expert Systems
Knowledge-Based
Systems
Intelligent DA
Intelligent DSS
Intelligent Agents
9
Changing the Search Terms
• expert system, artificial intelligence, intelligent
system, knowledge based system, intelligent
decision aid, intelligent decision support
system, intelligent agent, audit support system
• The parameter added to each artificial
intelligence term was the extension “AND (tax
OR audit* OR accounting)”. An extended set of
artificial intelligence terms were paired with
the AND Boolean logic for the accounting
terms.
10
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
5-year Article Totals
250
200
150
5 Yr-Current Study
100
5 Yr-Gray et al. 2014
50
0
11
Publication Type
250
200
150
Academic
Professional
Education
100
50
0
1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2013 2015
12
More Importantly: Research Needs
• One lesson we have learned over the past decades is
that building complete systems are not necessarily
AIS researchers’ competitive advantage (Alles et al.
2008)
• Original editorial of original form of IJAIS (AiAIS)
suggests focus on pieces “better knowledge
representation capabilities, better knowledge
acquisition capabilities, better validation techniques,
and methodologies for selecting among alternative
techniques for a given domain based on attributes of
the domain”
• Complementarity of Design and Behavioral Science 13
Where is AI Today?
• The discussion of data analytics needs to
become real.
• CM/CA needs AI to solve many of the
problems experienced today.
• Calls for machine learning understanding, but
how used most effectively? Will managers use
when they cannot understand the logic?
• Acceptance of explanatory versus predictive?
• Where does natural language fit into the
equation? (U AZ & Homeland on interviews)
14
We Also Need to be the Conscious
of Society
• Australia Government report suggests 40% of
jobs could be made redundant by technology
by 2025.
• Is the goal completely automated systems?
• Are auditors and accountants immune?
• Should we care?
• Should we be doing AI research?
15
Theory of Technology Dominance
• Prop 5: Novices may make worse decisions
with smart technologies.
• Prop 6: Value of smart technologies is with
smart users—expert working with expert.
– We need more work on collaborative systems–
how to maximize decision making by leveraging
both the user and the system.
16
Theory of Technology Dominance
• Prop 7: Deskilling of professionals through use
of technology (lost or non-acquired skill).
– Google effect – I know more than I know
– Google effect – I am losing my ability to store
knowledge
– Can we reverse this through design?
• Prop 8: Stagnation of Epistemology
– Are our experts auditors or data scientists?
17
“The Reports of My Death Are Greatly
Exaggerated” – Expert Systems Research
in Accounting
Steve G.Sutton
KPMG Professor of Accounting
Matt Holt
Ph.D. Student
Vicky Arnold
EY Professor of Accounting
Kenneth G. Dixon School of Accounting
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
Area of Accounting
90
80
70
60
Accounting
50
Financial Accounting
Managerial Accounting
40
Auditing
Tax
30
Education
Multiple Disciplines
20
10
0
19
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
1981
Type of Journal
160
140
120
100
ES/AI
80
AIS
Accounting
60
IS
Other
40
20
0
20