Part 2: Practice of Internal Auditing
Welcome to Part 2 of The IIA’s CIA Learning System®.
The self-study text for the learning system includes the content addressed in
The IIA’s CIA syllabus. (You can download the syllabus from the online
Resource Center or from The IIA’s website.) However, in some cases, the
content has been reorganized to facilitate instruction and understanding.
Refer to the Table of Contents for an outline of the content.
To get the most out of the course materials, complete the course in this
order:
1. Begin by accessing the course at www.learncia.com.
2. Read the overview and return to the menu. Select Part 2 from the menu.
3. Complete the pre-test and view the report to help focus your study efforts.
4. Read each section and follow the Next Steps directions included at the
end of the section.
5. Complete Part 2 as outlined in the online overview.
Note that Part 2 of the CIA exam will consist of 100 multiple-choice
questions and test takers are given 120 minutes to complete this portion of
the exam. You can go to https://na.theiia.org/certification/CIACertification/Pages/CIA-Certification.aspx to register for the exam
separately.
Study Support
The IIA’s CIA Learning System includes online tools to support your study.
These tools may be accessed from the menu at any time.
• Glossary—Refer to the glossary for definitions of terms used in all three
parts of The IIA’s CIA syllabus.
• Reports—Refer to the reports to review your most recent test scores and
progress through the learning system.
• Resource Center—Refer to the Resource Center to access information
about The IIA’s International Professional Practices Framework, updates,
test-taking tips, printable flashcards, related links, and reference material
and to provide feedback to The IIA regarding the learning system.
The IIA’s CIA Learning System®
The IIA’s CIA Learning System® is based on the Certified Internal Auditor®
(CIA®) syllabus developed by The IIA. However, program developers do
not have access to the exam questions. Therefore, while the learning system
is a good tool for study, reading the text does not guarantee a passing score
on the CIA exam.
Every effort has been made to ensure that all information is current and
correct. However, laws and regulations change, and these materials are not
intended to offer legal or professional services or advice. This material is
consistent with the revised Standards of the International Professional
Practices Framework (IPPF) introduced in July 2015, effective in 2017.
Copyright
These materials are copyrighted; it is unlawful to copy all or any portion.
Sharing your materials with someone else will limit the program’s
usefulness. The IIA invests significant resources to create quality
professional opportunities for its members. Please do not violate the
copyright.
Acknowledgments
The IIA would like to thank the following dedicated subject matter experts
who shared their time, experience, and insights during the development and
subsequent updates of The IIA’s CIA Learning System.
Pat Adams, CIA
Terry Bingham, CIA, CISA, CCSA
Al Marcella, PhD, CISA, CCSA
Raven Catlin, CIA, CPA, CFSA
Markus Mayer, CIA
Patrick Copeland, CIA, CRMA, CISA, CPA
Vicki A. McIntyre, CIA, CFSA, CRMA, CPA
Don Espersen, CIA
Gary Mitten, CIA, CCSA
Michael J. Fucilli, CIA, QIAL, CRMA, CGAP,
CFE
Lynn Morley, CIA, CGA
Lyndon Remias, CIA
James D. Hallinan, CIA, CPA, CFSA, CBA
James Roth, PhD, CIA, CCSA
Larry Hubbard, CIA, CCSA, CPA, CISA
Brad Schwieger, CPA, DBA
Jim Key, CIA
Doug Ziegenfuss, PhD, CIA, CCSA, CPA,
CMA, CFE, CISA, CGFM, CR.FA., CITP
David Mancina, CIA, CPA
Part 2 Overview
Part 2 of The IIA’s CIA Learning System focuses on the auditor’s abilities
related to the Performance Standards (series 2000, 2200, 2300, 2400, 2500,
and 2600). Performance Standards describe the nature of internal auditing
and provide quality criteria against which the performance of internal
auditing services can be measured. Note that Standard 2100 (“Nature of
Work”) is addressed in Part 1, Section V, “Governance, Risk Management,
and Control.”
Part 2 is made up of four sections:
I. Managing the Internal Audit Activity. The chief audit executive (CAE)
must effectively manage the internal audit activity to ensure that it adds
value to the organization (Standard 2000).
II. Planning the Engagement. Internal auditors must develop and document
a plan for each engagement, including the engagement’s objectives, scope,
timing, and resource allocations. The plan must consider the organization’s
strategies, objectives, and risks relevant to the engagement (Standard
2200).
III. Performing the Engagement. Internal auditors must identify, analyze,
evaluate, and document sufficient information to achieve the engagement’s
objectives (Standard 2300).
IV. Communicating Engagement Results and Monitoring
Progress. Internal auditors must communicate the results of engagements
(Standard 2400). The CAE must establish and maintain a system to
monitor the disposition of results communicated to management (Standard
2500), discuss any remaining unacceptable risk levels with senior
management, and communicate any unresolved issues related to
unacceptable risk levels to the board (Standard 2600).
Those managing engagements must ensure that engagements are conducted
in a professional manner—from planning through supervision and
communication to monitoring engagement outcomes—and with a continuous
awareness of risk.
Section I: Managing the Internal
Audit Activity
This section is designed to help you:
•
•
•
•
•
•
•
•
Describe policies and procedures of internal auditing operations.
•
•
Describe potential reliance on other assurance providers.
•
•
Identify how the CAE seeks board approval of the annual audit plan.
•
Recognize that the CAE reports on the overall effectiveness of the organization’s internal
control and risk management processes to senior management and the board.
•
Identify internal audit key performance indicators that the CAE communicates to senior
management and the board periodically.
Interpret administrative activities of internal audit.
Identify sources of potential engagements.
Identify a risk management framework to assess risks.
Prioritize audit engagements based on the results of a risk assessment.
Interpret the types of assurance engagements.
Interpret the types of consulting engagements.
Describe coordination of internal audit efforts with external auditors, regulatory oversight
bodies, and other internal assurance functions.
Describe how the chief audit executive (CAE) communicates the annual audit plan and
its results to senior management and the board.
Identify significant risk exposures and control and governance issues for the CAE to
report to the board.
The Certified Internal Auditor (CIA) exam questions based on content from this section make
up approximately 20% of the total number of questions for Part 2. Most of the topics are
covered at the “B—Basic” level, meaning that you are responsible for comprehension and recall
of information. (Note that this refers to the difficulty level of questions you may see on the
exam; the content in these areas may still be complex.) A few topics are covered at the “P—
Proficient” level, meaning that you are responsible not only for comprehension and recall but
also for higher-level mastery, including application, analysis, synthesis, and evaluation.
Section Introduction
Performance Standard 2000, “Managing the Internal Audit Activity,” states
that the chief audit executive must effectively manage the internal audit
activity to ensure that it adds value to the organization. Interpretation tells
us that “the internal audit activity adds value to the organization and its
stakeholders when it considers strategies, objectives, and risks; strives to
offer ways to enhance governance, risk management and control processes;
and objectively provides relevant assurance.”
This section focuses on the criteria for effectively managing the internal
audit function at both strategic and operational levels.
From a strategic perspective, the CAE must ensure the establishment of a
risk-based plan for managing the function’s activity. This will require that
internal audit leaders:
• Manage changes needed to implement and support the organization’s
strategy.
• Establish relationships throughout the organization to foster communication
and cooperation.
• Assess and promote an ethical climate and good governance.
• Develop an appropriate system to measure the efficiency and effectiveness
of the internal audit function and report performance to senior
management and the board.
• Manage interactions with external auditors, regulatory bodies, and other
internal assurance functions.
From an operational perspective, the CAE must ensure that the function is
managed in a professional manner and that:
• Policies and procedures are in place to plan, organize, direct, and monitor
internal audit operations.
• The function is administered to make the best use of internal audit
resources.
• The function is staffed appropriately for its tasks.
• A risk-based audit plan is used to identify potential engagements and
prioritize engagements.
• Management is informed about the effectiveness of the organization’s
internal control and risk management frameworks.
• The quality of internal audit work is monitored, assessed, and reported to
management, and a quality assurance and improvement program is in
place.
Chapter 1: Internal Audit Operations
Chapter Introduction
Strategically managed organizations recognize the need to operate not just
as organizationally connected functions but as fully integrated, often
interdependent parts of a whole. Functional and operational strategies must
be aligned with the organizational strategy. The organization’s risk
management approach must be enterprise-wide.
The internal audit activity plays a critical role in ensuring that the
organization’s resources are being used efficiently and effectively toward
accomplishing organizational objectives and that the organization’s internal
control framework is adequate for controlling the variety of internal and
external risks to which the organization is vulnerable.
The topics in this chapter focus on the role of internal audit at an
operational level, including how the CAE ensures that the activity can fulfill
its role and responsibilities. This includes:
• Formulating policies and procedures that support the activity’s
independence, objectivity, proficiency, and due professional care.
• Directing administrative functions that allow the activity to operate
efficiently and effectively.
Topic A: Policies and Procedures for Internal Audit
Operations (Level B)
Policies and Procedures
Engagement management is the process of planning, organizing, directing,
and monitoring an internal audit activity’s resources (people, equipment and
technology, time, and money) so that objectives can be met within the
defined scope, time, and cost constraints of assurance and consulting
engagements.
• Planning, which is at a strategic level, includes activities such as
developing a risk-based audit plan and reviewing staff competency needs
and planning for hiring and development. The audit plan is discussed in
the next chapter.
• Organizing, which is at an operational level, involves designing structures
and processes aimed at achieving activity objectives and overall goals of
efficiency and effectiveness. This may include assigning auditors to
specific engagements on the basis of their experience with similar
engagements and their business experience. It may also involve allocating
time for engagement activities like planning, developing and implementing
the audit program, conducting fieldwork, and writing reports.
• Directing includes the many tasks in leading internal audit.
Communication and coordination must be maintained within the
organization, with the board, and with external bodies, as applicable. New
staff members must be interviewed and hired or contracted with.
Performance management systems should be implemented.
• Monitoring involves activities such as ensuring that budgets are monitored
and assessed; that the audit committee, senior management, and
engagement clients are receiving value-added services; and that the activity
is meeting its strategic objectives, including the requirements of the audit
plan.
Audit policies and procedures help the chief audit executive in carrying out
these management activities. Establishing policies and procedures might
entail developing processes to support engagement work, such as
engagement initiation/transition meetings and report review processes,
processes for qualifying and contracting with external service providers,
structures for communicating different types of activity information,
monitoring processes aimed at maintaining quality and budget adherence
(e.g., dashboards), and channels for gathering this data.
Internal audit policies typically include guidance on:
• The overall purpose and responsibilities of the internal audit activity.
• Adherence to the mandatory guidance of the International Professional
Practices Framework (IPPF).
• Independence and objectivity.
• Ethics.
• Protecting confidential information.
• Record retention.
• Staff training.
• Establishing a quality assurance and improvement program.
Internal audit procedures typically include guidance on:
• Preparing a risk-based audit plan.
• Planning an audit and preparing the engagement work program.
• Performing audit engagements.
• Documenting audit engagements.
• Communicating results/reporting.
• Monitoring and follow-up processes.
Interpretation of Standard 2040 stipulates that “the form and content of
policies and procedures are dependent upon the size and structure of the
internal audit activity and the complexity of its work.”
Audit Manuals
The audit manual provides a guide to existing and new members of the
internal auditing activity about the activity’s objectives, the way these
objectives will be accomplished (the policies and procedures), and the use
of internal audit standards. The documents may include methods and tools
for training and may require internal auditors to provide acknowledgment
by signature that they have read and understood the policies and procedures.
The CAE is responsible for ensuring that an audit manual is created and
maintained, that it is distributed throughout the internal auditing activity,
and that the policies and procedures contained in the audit manual are
consistently and continually enforced.
The purpose of the audit manual is, in general, to:
• Provide guidance to activity members that will support adherence to the
profession’s code of ethics and professional standards.
• Define a high level of performance expectations for staff that will enable
the activity to fulfill its role in supporting the organization’s governance,
risk management, and control objectives and to fulfill the activity’s own
strategic objectives.
• Focus activity members on key objectives and values. For example, an
activity may focus on assuring controls or adding value to the organization
by identifying opportunities for greater efficiency and quality—or it may
balance both roles.
• Coordinate roles and responsibilities within the activity and in relation to
other internal and external bodies.
• Codify critical processes, such as the steps involved in performing
different types of engagements, and policies, such as protection of
confidential information and communication and monitoring of engagement
results.
• Provide the basis on which to evaluate the internal auditing activity’s
performance.
As suggested in Implementation Guide 2040, audit manuals can vary in
content and format. Exhibit I-1 lists possible topic headings for audit
manuals.
Exhibit I-1: Sample Audit Manual Content
Topic
Description
Internal audit charter
Establishes the purpose, scope of authority, and responsibilities of the CAE
and the internal audit activity, including professional standards,
responsibilities, and ethics/code of conduct
Internal audit
organization
Reporting structures, services provided by internal audit
Internal audit strategic
plan
Process and schedule for developing strategic goals and objectives
Annual audit plan
Process for developing and modifying the annual audit plan, from
identifying audit universe through risk analysis and allocation of resources,
including risk management tools to be used
Personnel
Roles and responsibilities, training and career development, certification
opportunities, continuing education requirements, and performance
management system
Communication
Guidelines for communicating with internal clients and external bodies;
handling of confidential information
Audit engagement
procedures
Procedures to be followed, from planning through reporting; requirements
regarding workpapers; report template
Quality assurance and
improvement program
(QAIP)
Description of QAIP requirements; evaluation processes
Administration
Policies regarding tracking of time, documentation, and document retention
Implementation Guide 2040 explains that in small internal audit activities,
close and daily supervision may take the place of formal internal audit
operations manuals. However, in large audit activities, more formal and
comprehensive policies and procedures may be essential to guide the
internal audit staff in the execution of the internal audit plan.
Audit Activity Organizational Charts
It is important that internal auditors understand the roles and responsibilities
of each layer of the internal audit organization’s reporting structures (the
second item listed in Exhibit I-1). Exhibit I-2 shows an example of an
organizational chart for an internal audit activity. Note that CAEs have
significant flexibility in determining job titles and the organizational
structure of the internal audit activity. Also, the number of persons on staff
in the internal audit activity may influence the type of structure. Therefore,
treat this as one example of how the activity might be organized.
Exhibit I-2: Sample Internal Audit Activity Organizational Chart
Exhibit I-3 explores these internal auditor authority levels.
Exhibit I-3: Internal Audit Activity Authority Levels
Title
Chief audit
executive
Description
The CAE is a corporate executive in charge of internal auditing. This individual
typically has a dual reporting structure: first, reporting functionally to the audit
committee of the board of directors to ensure independence from management
and, second, reporting administratively to the chief executive officer (CEO). The
CAE ensures that a risk assessment is performed periodically and uses this, plus
knowledge of corporate objectives, to establish risk-based audit plans and internal
audit priorities. The CAE coordinates with other internal and external assurance
providers to minimize duplicate work and ensure proper and efficient audit
coverage in accord with the approved annual audit plan. He or she monitors
engagements for timely completion and provides an annual holistic opinion on the
adequacy and effectiveness of governance, risk management, and control
processes.
Audit
manager (for
a specific
area)
An audit manager directs the performance of assurance and consulting
engagements and interacts with functional area managers in areas being audited to
fully understand the activities being performed and any relevant issues while
remaining independent from these areas. For example, an audit manager for
operations, compliance, and reporting (this mirrors the three control objectives in
the COSO internal control framework) needs to have a good understanding of the
organization’s operations. An audit manager for information technology (IT)
would need competencies in understanding IT risk and control and the impact of
technology on organizational objectives. Audit managers for any area will also
coach, counsel, and direct auditors in their area in accordance with the IPPF to
ensure that these professionals get the development they need to meet changing
needs.
Senior auditor
A senior auditor conducts assurance and consulting engagements of up to
moderate complexity and exercises good business judgment and skills to develop
appropriate audit recommendations in accordance with the IPPF. Senior auditors
are expected to understand client business operations, develop and maintain
professional working relationships with clients, and work in a professional
manner. A senior auditor may also be charged with providing staff auditors with
day-to-day direction and guidance under the overall supervision of the audit
manager.
Auditor (staff
member)
An internal auditor staff member works under supervision to conduct assurance
audits and develop recommendations in accordance with the IPPF. Auditors are
expected to work alongside senior auditors to learn professional practices on
audits, gain exposure to various types of projects, learn about operations in
various functional areas, build a knowledge base of organizational operations,
perform general research, and provide support.
Topic B: Administrative Activities of Internal Audit
(Level B)
As the leader of internal audit, the CAE is tasked with many aspects of
functional management, including budgeting and staffing. (Staffing includes
things like workforce planning or resourcing, creating or revising position
descriptions or organizational charts, recruiting, recruit selection, and
contractor management.) These administrative activities are key components
of successful internal auditing.
Budgeting
Effective budgeting depends on a sound organizational structure, meaning a
budget in which authority and responsibility for each operational phase are
clearly defined. Budgets built on a solid foundation of research and analysis
are better equipped to produce realistic goals that aid in achieving the
organization’s desired growth and profitability. Here are some of the main
benefits of budgeting:
• Planning ahead. A solid budget plan requires all levels of management to
be involved and to formalize goals on an annual or more frequent basis.
• Definite objectives. Identifying definite objectives provides for
performance evaluation at each responsibility level.
• Early warning system. Working from an established budget allows for
early identification of potential sources of conflict or issues so that these
situations can be addressed by management in a timely manner.
• Coordination of activities. A budget links segmented goals within the
company’s overall list of objectives so that the organization as a whole
can incorporate multiple facets of its various departments.
• Management awareness. Budgeting results in a greater awareness of the
organization’s overall operations at the management level, including the
impact that external factors might have on the operation as a whole.
• Personnel motivation. Meeting budget objectives can motivate personnel
through use of various incentive or reward mechanisms that are also tied
to maintaining high quality work rather than being based solely on
meeting the budget.
A well-developed budget is the key component of planning that enables the
internal audit activity to perform its mission on time and within established
financial parameters. The CAE will also create a schedule budget, aligning
the number of available audit personnel against available work hours to
determine the amount of coverage that can be provided during a fiscal year
as well as within each audit project.
Staffing
Staffing begins with workforce planning, followed by creating or revising
position descriptions. It then proceeds to recruiting and screening applicants
or contractors with the desired skills, knowledge, and characteristics;
interviewing applicants or contractors to confirm that they possess the
necessary qualifications; and selecting and hiring those applicants who can
succeed in the job and the organization. In large organizations, the CAE or
designee may work with the human resources department, who can provide
invaluable assistance and reduce staffing risks by ensuring that the process
conforms to employment laws and regulations (and, in the case of
contractors, tax laws) and organizational hiring and contracting policies. HR
can also offer experience in the selection process to reduce the risk of a bad
hire or an ineffective contractor. In smaller organizations, the CAE or
designee may do the interviewing and hiring themselves or use external
service providers.
Workforce Planning (Resourcing)
Workforce planning involves determining the number of employees and/or
contractors that are required in each position for the internal audit activity.
HR may already have performed workforce planning for the entire
organization, and, if so, the CAE will need to conform to the limits
established in the workforce plan as well as in the budget. If the CAE feels
that the workforce plans are unbalanced or insufficient, he or she can raise
the issue with HR and the appropriate executives.
Position Descriptions and Organizational Charts
Before recruiting applicants, the CAE or designee should establish and
update existing job descriptions for the position(s) being filled. If it is a
new position or if the position has changed (for example, has a different
authority level or a different location in the chain of command), the
organizational chart may also need revising. A good position description
should accurately and specifically reflect the requirements for the position
but should also be in alignment with the organization’s and the internal
audit activity’s strategic objectives. For example, an IT auditor may be
expected to know specific platforms and applications, but, as an
organization begins to expand its online presence, the auditor may also need
an understanding of distributed or web-based computing and associated
security controls.
Recruiting and Contractor Sourcing
Recruiting starts with advertising the open position and spreading the word
through various formal and informal networks. HR functional areas often
have considerable resources available to devote to this endeavor as well as
multiple methods at their disposal, from memberships on recruiting websites
to associations with external recruiters, campus recruitment services, and so
on. If such resources do not exist, internal auditors should weigh the cost of
a given method against its benefits to ensure that the best value is gained
from each investment. HR (or internal audit) will also need to keep detailed
records of applications received in order to fulfill diversity or other
compliance requirements and may also need to submit such records to
government bodies as required.
Recruit and Contractor Selection
Recruit selection involves narrowing the choice down to those applicants
who have the requisite qualifications and then conducting one or more
rounds of interviews with those applicants. Background checks and other
procedures may also be needed.
It is critical that anyone conducting interviews be trained or receive training
from HR to reduce the risk of interviewers asking illegal questions,
requesting that applicants take illegal or invalid tests, or being inconsistent
in the use of allowed questions or tests. Managers will also need training in
valid reasons for selection or nonselection so they do not use (or record)
reasons that are illegal. Failure to follow such practices creates serious
noncompliance risk, including the risk of losses from lawsuits and
reputation risk.
Contractor selection will involve many of the same steps as recruit
selection, but a key factor here is ensuring compliance with national tax
laws related to use of contractors. Since some organizations have overused
contractors to avoid paying benefits and taxes, many nations have created
tax and employment regulations related to contractor duties and how they
are managed.
Applicants may be interviewed with the intent of confirming what has been
said on the application and what the applicant understands about the job’s
requirements and conditions. Screening calls may be held with HR initially
and later with the CAE or designee.
Applicants are then selected for more in-depth interviews, which are usually
conducted on site. Interviewers must prepare for the meeting by carefully
reviewing the application against the job description and identifying critical
areas that must be confirmed or explained. If multiple interviewers will be
involved, their questions should be coordinated.
There are different approaches for interviewing applicants, which can be
combined in a single interview:
• Structured interviews follow an interview guide that has been developed to
focus on necessary skills, knowledge, experience, and attitudes. The guide
helps ensure consistency and completeness in the interviewing process and
also supports legal compliance. Applicants are asked the same questions,
with follow-up questions as needed.
• Behavioral interviews focus on obtaining feedback and indicators of past
behavior, considered a predictor of future job performance. Applicants may
be asked how they handled a specific situation in a previous position, such
as coordinating with an engagement client to ensure access and efficient
practices.
• Situational interviews are similar to behavioral interviews in that they try
to obtain more concrete information about possible job performance, but in
this case the applicants are asked about hypothetical situations rather than
real experiences. For example, applicants might be asked how they would
handle a client who would not accept audit findings and recommendations.
The interview should be a conversation that allows both the CAE or
designee and the applicant to get to know one another and to determine if
this working relationship will meet the needs of both the internal audit
activity and the applicant. There are social dimensions to these needs as
well as work dimensions. For example, those involved in interviewing
applicants should consider whether an individual will thrive in the
organization’s culture. An applicant that tends toward abrupt, abrasive
behavior may not work well in an organization that values positive
relationships among employees.
For this reason, effective interviewing includes skills beyond asking the
right questions. Effective interviewing skills include:
• Establishing a relaxed and open atmosphere that is more likely to produce
honest and complete answers.
• Listening actively—asking open-ended and/or follow-up questions that
encourage the applicant to talk openly or that confirm the interviewer’s
understanding.
• Observing nonverbal behaviors and identifying red flags (signs that
indicate that an applicant may not be telling the truth or the whole story)
as one would in an engagement interview.
• Taking notes. Note taking should not interfere with the discussion, but
notes will be invaluable later in remembering key points and supporting
hiring decisions.
When ready to make a selection, it is important to check with HR regarding
updates to regulations or laws and to ensure that any discriminatory hiring
decisions are avoided.
Employee and Contractor On-Boarding, Training, and
Management
Once new internal auditors are hired or contracted with, they need to be led
and managed, and the CAE may need to work with HR to ensure that new
employee orientation is conducted as appropriate. The CAE will also need
to ensure that specific technical training or other ongoing training is
provided as necessary and appropriate. See Section I, Chapter 1, of Part 3
of this learning system for more information on leadership and management.
Chapter 2: Establishing a Risk-Based Internal
Audit Plan
Chapter Introduction
Management is responsible for establishing and maintaining a system of
internal controls in an organization. The Standards Glossary defines control
as follows:
Any action taken by management, the board, and other parties to manage risk and
increase the likelihood that established objectives and goals will be achieved.
Management plans, organizes, and directs the performance of sufficient actions to
provide reasonable assurance that objectives and goals will be achieved.
Structures, activities, processes, and systems that help management
effectively mitigate risk are all examples of internal controls. Internal
controls are an integral component in risk management.
Because internal auditors are experts in understanding organizational risks
and internal controls available to mitigate these risks, they are in a unique
position to help management protect their organizations from risk exposures
—present and future—ranging from minor disruptions to major catastrophes.
The internal audit activity assists both management and the oversight body
(the board or its audit committee) in enterprise risk management (ERM) by:
• Helping management to understand governance, internal controls, and risk
management processes.
• Developing and implementing a risk assessment framework for internal
audit planning.
• Bringing a systematic, disciplined auditing approach to assessing the
effectiveness of internal controls and risk management processes.
• Providing objective and independent assurance that the organization’s risks
have been appropriately mitigated.
• Making recommendations for improvements as warranted.
Risk is a part of conducting business. Companies must take risks to pursue
their strategies and objectives. The key is how to do this while also
mitigating or reducing risk. Ideally, ERM is a robust process that identifies
and mitigates threats and/or occurrences that can thwart organizational
success. The internal auditing function’s risk assessment role plays an
important part in confirming management successes and identifying
exceptions for management actions.
While helping an organization to embrace a framework of internal control
and an ERM framework are critical for organizational governance and are
integral to most controls, internal auditing itself needs to incorporate the
same ERM techniques into its audit planning procedures. To be truly valueadded to the organization, the annual audit plan and specific engagements
must focus on significant risks. What is considered significant can be
defined as those risks that are considered likely (highly probable) and/or
would have a real impact (highly damaging even if less probable) on the
achievement of the organization’s objectives or goals for that area.
Standard 2010, “Considerations,” notes that risk is central to planning
prioritization, stating that the CAE “must establish a risk-based plan to
determine the priorities of the internal audit activity, consistent with the
organization’s goals.” For assurance engagements, planning “must be based
on a documented risk assessment, undertaken at least annually,” according
to Standard 2010.A1. For consulting engagements, “improving management
of risk” is also a consideration in what consulting engagements to accept,
per Standard 2010.C1.
Auditing Maturity Level
When developing a risk-based internal audit plan, it is important for
auditors to assess the internal audit activity’s maturity level. Consider the
following maturity levels of annual audit planning and audit engagement
objective setting:
• Controls-based auditing. Prior to the 1980s, controls-based internal
auditing was the standard. This method is basically an extension of
external audit procedures, consisting primarily of providing assurance of
the validity of various account balances and other financial details; audits
of compliance with laws, regulations, policies, and procedures; and audits
of specific transaction controls from initiation to reporting. The focus was
to understand the laws, regulations, policies, and procedures in the area
and then to identify and correct exceptions and errors.
• Process-based auditing. Process-based audits were developed in the 1980s
to address some flaws with controls-based auditing such as its low value
to decision makers. Process-based audits look at processes as a whole and
evaluate their design, efficiency, and effectiveness. These audits began to
stress measuring gaps between prescribed and actual processes and how
they relate to achievement of business objectives, but the primary focus of
the audits was often still controls-based.
• Risk-based auditing. Risk-based auditing was developed in the 1990s to
demonstrate further added value, especially as more consulting firms
entered into co-sourcing arrangements for internal auditing and had to
justify their fees. The intent was to limit the audit engagement to
significant risks, starting by developing a thorough understanding of the
organization and its risks. Relatively low-risk controls could be omitted
from engagements (or included in scope infrequently as part of a regular
rotation) to ensure a greater return on the investment in auditing. This
auditing maturity level satisfies the mandates of the Standards to be riskbased in selecting engagements, audit objectives, and specific audit tests. It
is a method that is intuitive for management to understand and endorse.
However, organizations that have relatively mature risk-based processes
can move to an even higher auditing maturity level—ERM-based auditing.
• ERM-based auditing. ERM-based auditing was developed in the late
1990s as a counterpart to the organization-wide use of ERM for holistic
risk-based assessment and decision making. In addition to setting project
priorities based on perceived risk to key business objectives, it focuses
strongly on measuring risk based on relevant key performance indicators
(KPIs), considering risk appetite and risk tolerance levels, and planning
responses based on what ERM capabilities already exist. Rather than
focusing just on mitigating risks to an acceptable level, ERM-based
auditing assesses how well ERM activities are supporting organizational
objectives by managing risks to an acceptable level within a risk
appetite/tolerance. Thus, the focus is on the gaps in ERM effectiveness,
based not only on the auditor’s objective assessment of what risks are
significant but also on management’s assessment of those risks.
Advantages of maturing to an ERM-based auditing methodology include:
• Creating a foundation for audit judgments based on organizational
strategy and objectives, risk appetite, and governance maturity.
• Developing an assurance framework for assessing the adequacy of ERM
and governance activities.
• Synchronizing the auditor’s tolerance for risk with management’s
tolerance for risk, rather than focusing solely on the auditor’s, as in
prior audit methods.
• Emphasizing the critical need to base performance measurements on
what will provide real incentives to accomplish organizational
objectives.
• Focusing on the organization’s future capability to assess and manage
risk rather than on just its historical risk response track record.
The remainder of this chapter refers to risk-based auditing, which should be
taken as a generalization that could refer to either risk- or ERM-based
audits, with the goal of achieving the maturity level of an ERM-based audit.
The chapter starts by identifying sources of potential engagements,
highlighting the importance of understanding the organization and its
industry and market when considering audit priorities. The second topic
discusses the need to establish a framework for assessing risk and how to
rank and validate risks by priority. The third and fourth topics provide
detailed descriptions of various types of assurance and consulting
engagements. Although all audit engagements have common elements and
factors, such as developing an audit plan or putting together an audit team,
each type of engagement also has individual and specialized components. It
is critical that internal auditors can identify these differences and apply the
correct methods of performing the engagements. The last topic describes
coordinating internal audit efforts with the work of other internal and
external assurance and consulting service providers.
Topic A: Sources of Potential Engagements (Level B)
The Audit Universe
In most organizations, the potential audit universe is vast. Identifying the
audit universe is vital in laying the foundation for internal audit’s annual
risk assessment and planning process. The internal audit plan is derived
from the annual risk assessment process. This plan outlines the priority of
audit cycle requirements for the internal audit activity, including the
objectives, scope, and frequency of audit engagements to be performed in
the coming year. It also contains any annual portions of engagements that
are cycled out over a period longer than 12 months (e.g., 24 months, 36
months).
The audit universe includes the major functions, operations, operating units,
subsidiaries, and business, service, and product lines of the organization,
each of which is considered an “auditable unit.” It also includes any
applicable areas (e.g., financial reporting or compliance) that have a
pervasive, organization-wide impact and fall under the internal audit
“umbrella” from an assurance coverage perspective.
In highly regulated industries, such as financial services, health care,
insurance, oil and gas, or brokers/dealers, the audit universe will include
audits of the relevant regulatory mandates. In addition to this, government
agency examiners often set expectations around independent compliance
assessments of high-risk areas, even in the absence of specific laws and
regulations requiring independent audits. An example is internal auditors for
insured depository institutions conducting reviews related to the U.S. Bank
Secrecy Act and related anti-money-laundering laws. Another example
might be internal auditors for a federal government defense contractor
performing independent assessments of organizational compliance with
contract provisions.
A major area of financial reporting and legal compliance impact for U.S.
organizations (or foreign organizations traded on U.S.–based exchanges)
involves the U.S. Sarbanes-Oxley Act (SOX) and the internal control over
financial reporting (ICFR) assessment requirements contained in Section 404
of SOX. Internal audit often plays an integral assurance coordination and
reliance role in conjunction with an organization’s external auditors in
reviewing, testing, and assessing (and/or helping support management’s
assessment of) the effectiveness of an organization’s ICFR. ICFR and other
major or relevant assurance responsibilities of internal audit—beyond the
relatively more traditional auditable units identified—should be included or
considered in the audit universe of the organization for risk assessment and
planning purposes each year (as appropriate and applicable), along with all
other defined auditable units.
There will be a number of functional areas or auditable units that may or
may not need auditing in a given audit cycle, as follows:
• Accounts payable
• Accounts receivable
• Cash management
• Customer service
• Environmental
• Finance
• General services
• Health and safety
• Human resources
• Inventory management
• Legal
• Locations
• Manufacturing
• Marketing
• Payroll
• Production/operations
• Products and services
• Procurement/purchasing
• Research and development
• Sales and collections
• Security
• Supply chain and vendor management
• Technology
The audit universe is not defined solely by operating entities, their
overarching processes, and their related functional activities. It also
encompasses the organization’s strategic plan and the controls management
has in place to mitigate risks, achieve organizational goals and objectives,
and ensure that customer needs are being met. The internal auditor looks to
evaluate and report on the efficiency and effectiveness of management
governance, ERM, and controls and their likelihood of helping to achieve
the organization’s strategic goals and objectives.
In the rest of this topic, we’ll take a closer look at some of the sources of
engagements.
The Organization’s Strategic Plan
Implementation Guide 2010, “Planning,” states:
The audit universe includes projects and initiatives related to the organization’s
strategic plan, and it may be organized by business unit, product or service lines,
processes, programs, systems, or controls.
Strategic plans are based on some degree of environmental analysis
(environmental scanning) that provides intelligence on what is and what will
potentially be happening inside and outside the organization.
Organizations may use a strengths, weaknesses, opportunities, and threats
analysis (SWOT analysis) to identify and classify elements that can help or
hinder the organization or its strategic plans or activities. Strength and
weakness reviews in SWOT analysis look at the organization’s internal
capabilities (or lack thereof). This type of internal capability analysis can
also be done as a stand-alone study. It involves a review of current
infrastructure and technologies and employee and process capabilities that
can support or impede organizational activities and goals. Opportunities and
threats in a SWOT analysis are then focused mostly on external factors that
can impact organizational success for good or for ill (perhaps also
considering some related internal opportunities or risks). Opportunity and
threat reviews may look at the following factors:
• Legal factors. The laws, law-making activities, and litigation promulgated
by governmental entities (e.g., federal, state, county/provincial, or city) and
enforced by punishment that can impact the success of an organization’s
products or services.
• Regulatory factors. The regulations, principles, and rules promulgated by
governmental and nongovernmental entities (such as self-regulating bodies
and professional societies) designed to control or govern behavior and that
can result in some form of punishment or disenfranchisement.
• Market forces, industry trends, and the competition. The environment
in which the organization competes for employees, customers, goods and
services, etc.
• Stakeholder groups. The wide array of people, departments, and other
organizations that have an investment or interest in the success of or
actions taken by the organization.
• Technology trends and related internal capabilities. Key technologies
critical to competitive advantage and base technologies that are necessary
to compete as well as the organization’s technical strengths, weaknesses,
and priorities relative to these trends and the competition’s capabilities.
• Customers. Assessments of both internal and external customers to
understand their needs, preferences, behaviors, expectations, etc.
Internal audit consideration of environmental analysis data can surface many
potential risks. By incorporating components of the organization’s strategic
plan, the audit universe will likely reflect the organization’s attitude toward
risk and the degree of difficulty in achieving planned objectives. The audit
universe will normally be influenced by the results of the risk management
process. Implementation Guide 2010 tells us that:
[Reviewing] the organization’s approach to risk management may help the CAE
decide how to identify, organize, or update the audit universe, which consists of all
risk areas that could be subject to audit, resulting in a list of possible audit
engagements that could be performed.
Management and Employees
Beyond functional activities, operating entities, and the strategic plan, the
potential audit universe also includes organizational management and
employees. The risk perspective of executives and key operational managers
is important, as they are responsible for establishing plans, defining risk
tolerances, allocating resources to achieve the plans, monitoring the
activities being done to achieve the plans, and reviewing results. The
employees’ perspectives are also important, as employees are closest to the
business activities. Both parties can offer valuable insights on the risks the
organization faces.
Management may have special projects that should be included in the audit
universe. However, the internal audit function must have the competencies
and resources required to perform such work for it to be accepted. Standard
2010.C1 states:
The chief audit executive should consider accepting proposed consulting engagements
based on the engagement’s potential to improve management of risks, add value, and
improve the organization’s operations. Accepted engagements must be included in the
plan.
Special requests can come in many forms. For example, the chief
information security officer may request that internal audit perform
penetration testing to identify possible IT security vulnerabilities. Or perhaps
the director of human resources may request internal audit’s assistance to
benchmark industry best practices for statistics and time lines for employee
retention and recruiting and on-boarding new employees.
Information can be solicited from management and employees in different
ways. Exhibit I-4 provides an overview of some of the most common
methods.
Exhibit I-4: Management and Staff Information-Gathering Techniques
Technique
Interviews
Description
• A structured discussion between two
parties—one representing internal audit
and the other representing an engagement
client or information source—about
business risks.
• Intended to solicit an unbiased view from
the individual being interviewed.
• Typically a precursor to focus groups and
surveys.
Examples:
• The CAE meets annually with the board
chair, the audit committee chair, general
counsel, and others who have identified
organizational risks and mitigating
Considerations
• Must be conducted in an
objective manner without any
preconceptions about the
outcome.
• Most effective when the
interviewer builds rapport early,
probes for additional
information as necessary, and
summarizes throughout to
facilitate a substantive
discussion.
• Generates qualitative
information that can be explored
further in a focus group or
quantified by a survey.
controls.
• Internal audit managers meet with
division managers and other managers to
get their perspectives on organizational
objectives and related risks.
Focus groups
• Small groups of current managers or
employees (normally six to 12 people)
are invited to participate in a structured
discussion facilitated by a representative
of the internal audit function.
• Usually lasts two to three hours.
Examples:
• The CAE or internal audit manager meets
with business unit leaders and facilitates
activities in which participants compile,
discuss, and prioritize risks based on the
magnitude and probability of occurrence.
Or the group may explore risks
articulated in management interviews in
greater depth.
• Requires effective planning,
clearly defined objectives, and a
skilled facilitator to get the
most value out of the time
spent.
• Can include group brainstorming
and prioritization, resulting in
improved decision making.
• Can incur conformity
(“groupthink”) if individuals are
easily influenced by what others
say.
• Provides a good foundation for
and supplement to quantitative
surveys.
• A control self-assessment (CSA) meeting
may be initiated for a combined
assessment on certain risks and as an
opportunity to get an organizational
evaluation of a process.
Questionnaires/
surveys
• A way to quantify management and
• Easy way to obtain and compile
employee attitudes and perceptions.
information from any number of
respondents.
Example:
• A “top ten” list of risks is compiled from
interview and focus group data.
Participants are surveyed about their
perceptions of the adequacy and
effectiveness of internal controls and
actions for each risk.
• Can help to identify serious
risks.
• May not get honest and/or
complete answers; information
received may be too general
and, therefore, not very useful if
people don’t take the time to
provide thoughtful responses
and write out details to any
open-ended questions.
Regulatory Mandates
While compliance with some regulations is voluntary, many regulations
have the force of law. (Many laws are implemented using regulations
developed and administered by governmental regulatory bodies.) An
organization may not be able to compete in an industry if it does not
comply with regulatory mandates. Some regulatory mandates cut across a
variety of industries (such as environmental protection regulations restricting
pollution or occupational safety and health regulations protecting workers).
Industries may also have unique regulations (such as aviation, banking, or
forestry). Regulatory mandates relevant to an organization should be
considered part of the audit universe.
External Business Relationships and Third-Party
Risk
Organizations may out-source business activities, contract with individuals
who act in quasi-employee roles, or be involved in joint ventures or other
relationships that present risks. These third parties are acting on the
organization’s behalf to provide a complementary good or service in the
organization’s name, and they could be in violation of regulations or the
organization’s code of ethics or take other actions that reflect poorly on the
organization. Such third-party risks are hard to manage. While the
organization may manage the risks through contracts, it retains responsibility
and must monitor the risks. “Internal auditing plays a key role in assisting
management and validating management’s efforts,” as noted in the Practice
Guide “Auditing External Business Relationships.”
Information Technology (IT)
IT risks and controls are an important consideration for most organizations
and for the CAE to include when identifying the audit universe and
developing the annual plan. Results from several IIA external quality
assessment reviews reveal that the IT audit plan is one of the weakest links
in internal audit activities. Many times, internal auditors simply review what
they know or co-source IT auditing with external service providers, letting
them help to decide what to audit. The IIA’s Practice Guide “Developing
the IT Audit Plan” (previously GTAG 11) can help the CAE define and
understand the IT environment, identify the role of risk assessments in
determining the IT audit universe, and formalize the annual IT audit plan.
Examples of IT areas that might require consideration for inclusion in the
audit universe include information security and user access, vendor risk
management, disaster recovery and business continuity planning, and change
control and change management. These subjects are explored further in Part
3 of this learning system.
Relevant Market and Industry Trends
Risk issues posed by current industry or economic situations could be valid
sources for potential engagements. For example, organizations that are
investing heavily in new technologies are also creating new risks. Artificial
intelligence for decision support and the use of robotics in manufacturing or
supply chain processes (such as warehouse robots that move whole shelves
to warehouse pickers or the use of drones for delivery or self-driving cars)
are examples of disruptive technology trends that are creating both
opportunities and risks. Operational audits could focus on whether these
innovations are adding the value promised relative to their cost and payback
periods and the degree to which they are introducing new risks or
unintended consequences. In another example, a market trend that developed
in 2007 was the subprime mortgage and banking liquidity crisis that caused
the Great Recession in the United States (and elsewhere to a lesser degree).
This led to internal audits of organizational investments in mortgage-backed
securities and collateralized debt obligations over the next few years.
The market for a given product or service has a life cycle, and an industry
that produces the product or service will be facing certain trends depending
on whether demand needs to be built up, is growing rapidly, is steady, or is
in decline. Technology changes, changes in customer preferences, and
societal shifts may be driving these trends. Of course, organizations
diversify in order to protect themselves to some extent from being entirely
driven by a product’s life cycle, and the organization’s risk appetite and
residual risk levels can also be taken into account. Overall, however, each
market and industry will be facing its own trends, and internal auditors need
to understand the root causes of these changes (such as online ordering
putting pressure on retail profitability and demand) as well as what types of
pressures these are creating for the organization, especially in times of rapid
growth or times of decline. Rapid growth could promote a fast and loose
mentality around governance, risks, and controls; decline could put pressure
on the organization to cut costs or maintain profits, and these same areas
may suffer.
Emerging Issues
The 2018 executive summary titled “Enterprise Risk Management—
Applying Enterprise Risk Management to Environmental, Social, and
Governance-Related Risks” (see the link in the Resource Center) states the
following:
From small startups to large multinationals, companies have experienced instances of
product safety recalls, worker fatalities, child labor, forced labor, polluting spills, and
weather-related supply chain disruptions. Failure to manage these emerging risks
proactively has translated into extensive financial and reputational harm—in some
cases to the point of no recovery. For some companies, failure to take advantage of
opportunities arising from the changing risk landscape has resulted in lost revenue or
even obsolescence.
This report refers to information from the Global Risks Report 2018 of the
World Economic Forum, which compares global risks in terms of both
likelihood and impact for 2008, 2013, and 2018. In 2008, global pandemics
were the only societal risk that made the impact list, and there were no
environmental risks in either list. In 2018, the environmental risks of
extreme weather, natural disasters, and failure of climate change mitigation
and adaptation made up three of the top five global risks in terms of both
impact and likelihood. Water crises were a societal risk that made the 2018
impact list, and two technological risks—cyberattacks and data fraud or
theft—made the top five of the likelihood list. (There were no technological
risks in either list in 2008). Given the significance of these technological,
environmental, social, and governance-related risk trends, when identifying
sources of potential engagements, the CAE should consider whether
emerging issues such as these are a factor in the organization and its
industry.
Other Sources
In some organizations, internal assurance functions (e.g., security, quality,
health and safety) or external assurance providers (e.g., external auditors,
regulators, partners) may be sources of potential engagements. Internal audit
may review areas of weakness identified by these assurance functions and
may also evaluate the quality of the assurance functions as part of the audit
universe.
As we have seen here, the sources for the audit universe are many and
varied. The point of this step in risk assessment is to identify a
comprehensive list of all potential engagements for further consideration and
prioritization.
Topic B: Using a Risk Management Framework
to Assess Risks and Prioritize Audit
Engagements (Level B)
Risk frameworks can help an organization filter the audit universe down to
selected targets based on the areas of significant risk. Internal auditors can
use the risk framework the organization has adopted (or adopt their own
framework if the organization does not have one) to identify sources of
potential engagements. CAEs generally follow a risk assessment process
that includes phases such as risk identification, risk measurement, and risk
prioritization.
This topic starts with a review of two common risk frameworks. It then
moves to a discussion of the risk assessment process, strategic risk
responses, and prioritizing and selecting engagements.
Risk Management Frameworks
COSO’s Enterprise Risk Management—Integrated Framework and ISO
31000 are two of the risk frameworks auditors can use. (Note: The
following text introduces these frameworks. For a more complete review,
refer to Section V of Part 1 of this learning system or to The IIA’s website.
Students already familiar with these frameworks can skip to “Risk
Assessment” later in this topic.)
COSO’s ERM Framework
COSO (The Committee of Sponsoring Organizations of the Treadway
Commission) published its Enterprise Risk Management—Integrated
Framework in 2004, with an update in 2017, Enterprise Risk Management—
Integrating with Strategy and Performance, in an effort to help
organizations design and implement effective enterprise-wide approaches to
risk management. The framework introduces key ERM concepts and a
common ERM language, provides principles-based guidance, and applies
ERM in a strategic setting.
The COSO framework is applicable to all industries and all types of risk.
Starting at the top and supporting an organization’s mission, vision, and
core values is what differentiates COSO from most other risk models. The
model describes the connection between strategy, business objectives, and
performance (what the organization strives to achieve) and the ERM
components (what is needed to achieve the objectives).
Components of COSO’s ERM Framework
COSO’s ERM framework consists of five interrelated components, shown in
Exhibit I-5.
Exhibit I-5: Components of COSO’s ERM Framework
Component
Description
Governance
and culture
Governance sets the organization’s tone, reinforcing the importance of, and
establishing oversight responsibilities for, enterprise risk management. Culture
pertains to ethical values, desired behaviors, and understanding of risk in the
entity.
Strategy and
objective
setting
Enterprise risk management, strategy, and objective setting work together in the
strategic planning process. A risk appetite is established and aligned with strategy;
business objectives put strategy into practice while serving as a basis for
identifying, assessing, and responding to risk.
Performance
Risks that may impact the achievement of strategy and business objectives need
to be identified and assessed. Risks are prioritized by severity in the context of
risk appetite. The organization then selects risk responses and takes a portfolio
view of the amount of risk it has assumed. The results of this process are
reported to key risk stakeholders.
Review and
revision
By reviewing entity performance, an organization can consider how well the
enterprise risk management components are functioning over time and in light of
substantial changes and what revisions are needed.
Information,
communication,
and reporting
Enterprise risk management requires a continual process of obtaining and sharing
necessary information, from both internal and external sources, which flows up,
down, and across the organization.
Source: Enterprise Risk Management—Integrating with Strategy and Performance, © 2017
Committee of Sponsoring Organizations of the Treadway Commission (COSO). All rights reserved.
Used with permission.
Principles of COSO’s ERM Framework
These five components are supported by a set of 20 principles—the things
the organization would do as part of the enterprise risk management
process. The principles provide senior management and the board with a
reasonable expectation that the organization understands and strives to
manage the risks associated with its strategy and business objectives. The
principles are listed in Exhibit I-6.
Exhibit I-6: Principles of COSO’s ERM Framework
Component
Principles
Governance
and culture
1. Exercises board risk oversight—The board of directors provides oversight of
the strategy and carries out governance responsibilities to support management
in achieving strategy and business objectives.
2. Establishes operating structures—The organization establishes operating
structures in the pursuit of strategy and business objectives.
3. Defines desired culture—The organization defines the desired behaviors that
characterize the entity’s desired culture.
4. Demonstrates commitment to core values—The organization demonstrates a
commitment to the entity’s core values.
5. Attracts, develops, and retains capable individuals—The organization is
committed to building human capital in alignment with the strategy and
business objectives.
Strategy and
objective
setting
6. Analyzes business context—The organization considers potential effects of
business context on risk profile.
7. Defines risk appetite—The organization defines risk appetite in the context of
creating, preserving, and realizing value.
8. Evaluates alternative strategies—The organization evaluates alternative strategies
and potential impact on risk profile.
9. Formulates business objectives—The organization considers risk while
establishing the business objectives at various levels that align and support
strategy.
Performance
10.Identifies risk—The organization identifies risk that impacts the performance of
strategy and business objectives.
11.Assesses severity of risk—The organization assesses the severity of risk.
12.Prioritizes risks—The organization prioritizes risks as a basis for selecting
responses to risks.
13.Implements risk responses—The organization identifies and selects risk
responses.
14.Develops portfolio view—The organization develops and evaluates a portfolio
view of risk.
Review and
revision
15.Assesses substantial change—The organization identifies and assesses changes
that may substantially affect strategy and business objectives.
16.Reviews risk and performance—The organization reviews entity performance
and considers risk.
17.Pursues improvement in enterprise risk management—The organization pursues
improvement of enterprise risk management.
Information,
communication,
and reporting
18.Leverages information and technology—The organization leverages the entity’s
information and technology systems to support enterprise risk management.
19.Communicates risk information—The organization uses communication
channels to support enterprise risk management.
20.Reports on risk, culture, and performance—The organization reports on risk,
culture, and performance at multiple levels and across the entity.
Source: Enterprise Risk Management—Integrating with Strategy and Performance, © 2017
Committee of Sponsoring Organizations of the Treadway Commission (COSO). All rights reserved.
Used with permission.
The components and principles of the framework do not represent isolated,
stand-alone concepts. COSO states that enterprise risk management is not
static. It is integrated into the development of strategy, the formulation of
business objectives, and the implementation of those objectives through dayto-day decision making.
ISO 31000 Framework
ISO 31000:2018, “Risk management—Guidelines,” is an international
standard for risk management and a framework for the systematic
development of enterprise risk management for any size or type of
organization. ISO has also published two complementary resources, ISO
Guide 73:2009, “Risk management—Vocabulary,” which helps organizations
discuss risks using a common set of risk management terms, and ISO
31010:2009, “Risk management—Risk assessment techniques,” which
focuses on risk assessment concepts, processes, and the selection of risk
assessment techniques.
The purpose of ISO 31000 is to help organizations manage uncertainty. An
organization that can manage uncertainty and adapt quickly to change will
not only be better able to achieve its objectives but will be more attractive
to investors. ISO 31000 also helps organizations benchmark their own risk
management practices against those of other organizations adopting ISO
31000. It provides a guide for managing risk based on key principles, a
framework, and a process.
ISO 31000 Principles
ISO 31000 is a principles-based standard intended to generate transparency
and credibility within the risk management function. The principles describe
characteristics of effective and efficient risk management and should be
used as a foundation for establishing an organization’s ERM processes.
These principles state that risk management:
• Is an integral part of all activities in an organization.
• Should follow a structured and comprehensive approach to provide
consistent results.
• Is customized to the organization’s operating environment, culture, and
objectives.
• Is transparent, auditable, and inclusive of all stakeholders, providing
improved communications and awareness.
• Addresses uncertainty in a structured, orderly, unambiguous, and timely
fashion.
• Makes use of the best information available.
• Is influenced by organizational culture and staff behavior.
• Uses an iterative cycle to generate continual improvement, organizational
learning, and the ability to quickly respond to changing environments.
ISO 31000 Framework Components
The ISO 31000 framework components assist in integrating risk
management into all organizational activities and functions. These
components, which should work together and be customized as needed to
achieve the organization’s own objectives, include the following:
• Leadership and commitment. Oversight by top management ensures that
a risk management approach is integrated into all activities, promoting the
value to the organization and stakeholders.
• Integration. Risk management should be a key aspect of governance. It
should be aligned to the organizational purpose, strategy, objectives, and
operations.
• Design. The framework should be designed to fit the context of the
organization and demonstrate the commitment to risk management.
• Implementation. Success requires stakeholder engagement and awareness.
The framework ensures that a risk management process is included in all
activities.
• Evaluation. To evaluate the effectiveness of the framework, auditors
should measure performance against indicators and expected behaviors.
• Improvement. Organizations should continually monitor and adapt the
framework to address identified gaps and incorporate enhancements.
ISO 31000 Cycles
At a high level, the ISO 31000 framework is a cyclical process that begins
with top executives expressing a strong commitment to risk management
and mandating its adoption based upon the principles described above. The
framework is then designed and customized. Once implemented, it is
monitored and reviewed to enable continual improvement and further
customization.
The implementation phase has its own cycle, as shown in Exhibit I-7.
Exhibit I-7: ISO 31000 Implementation Phase Process Framework
Risk Assessment
Now we turn to the risk assessment phases noted at the beginning of this
topic: risk identification, risk measurement (or analysis), and risk
prioritization. COSO ERM identifies similar phases in the principles for its
performance component—“identifies risk,” “assesses severity of risk,” and
“prioritizes risk.” Under ISO 31000, these are the implementation phase
processes of risk identification, risk analysis, and risk evaluation.
But first, note that gathering comprehensive information from a variety of
sources is of paramount importance during risk assessment. The internal
auditor should use techniques that solicit both qualitative data and
quantitative data, as described in Exhibit I-8.
Exhibit I-8: Qualitative and Quantitative Data
Description
Examples
Considerations
Qualitative data
Subjective, or
soft, measures
focusing on
perceptions and
attitudes
• Interviews/walkthroughs
• Focus groups
• Observations
• Meetings
• Allows individuals the opportunity
to provide information using their
own words
• Can identify critical risks and
provide insights about
organizational priorities
• May be open to interpretation
Quantitative data
Measures
derived from
concrete,
objective
criteria
• Studies (e.g., quality and productivity
measurement)
• Reports (e.g., market share and market
share growth, revenue, and profit)
• Surveys (e.g., statistical data)
• Provides scorecards, trend data,
data that can be benchmarked, or
other useful numbers that can be
readily averaged
• May indicate symptoms rather
than causes
Why are both types of data important? Objective criteria are not always
applicable when assessing certain risks such as board concerns. Combined,
soft and hard data facilitate a more comprehensive understanding of the
risks facing the organization. This knowledge, in turn, enhances the firm’s
opportunity to make better business decisions.
Risk Identification
Risk identification takes a systematic look at the nature of risks and
opportunities facing the organization. Risks and opportunities are often
grouped in strategic, project/program/process, and operations categories, as
shown in Exhibit I-9.
Exhibit I-9: Types of Organizational Risks
Description
Sources
Outcomes
Strategic risks
Reflect
elements of the
strategic plan
Review of organizational documents such
as mission, vision, values, and strategic
plan in order to establish a clear
understanding of organizational goals and
objectives
• Development of matrices that
classify strategic risks according
to their time horizon so that shortterm issues are distinguishable
from medium- and long-term
issues
• Use of qualitative tools (e.g.,
brainstorming, multi-voting, whatif scenario analysis) to identify
the most significant strategic risks
• Determination of which risks are
likely to have the greatest impact
(e.g., financial impact, asset
liquidity, reputation) on the
achievement of organizational
goals and objectives
Project/program/process risks
Examine
specific
projects,
programs, and
processes
Derived from functional areas as well as
external customers
Establishment of risk factor criteria
(e.g., financial impact, brand,
management competence, quality of
internal controls, complexity,
client/customer satisfaction)
• Health
• Safety
• Physical (environmental)
• Security and system safety functions
• Transaction/processing
Determination of risks to employees,
visitors to the organization, and
physical assets (e.g., equipment and
buildings)
Operations risks
Day-to-day
operational
risks to the
organization
and its
stakeholders
Risk Analysis (Measurement)
Analyzing risks involves taking a closer look at the key risks the
organization is facing, whether identified by the ERM process, by the board
or management, or during consulting engagements. This step involves
evaluating such categories as monetary, brand, past audit, or regulatory risks
or risks related to the quality of internal controls (or specific individual
risks as warranted). Risk measurement is based on the likelihood and
impact of risk occurrence, where likelihood is the probability that a specific
risk event will occur and impact is the magnitude of an effect or
consequence if a risk event were to occur. Impact is sometimes also
expressed as materiality, significance, severity, or magnitude. The
combination of these elements is an assessment of the severity of the risk,
or the degree to which the risk will result in a consequence that could
materially impact the organization’s ability to achieve goals and objectives.
Spending more time measuring key risks can provide additional information
on the likelihood or impact of the key risks.
Approaches include probability estimates (e.g., percent chance of a given
annual loss), risk factor measures (e.g., statistical or subjective), and
weighted matrices.
Risk analysis often requires understanding the sources and drivers of risks
and designing metrics that will help show when the risk may be occurring
or is about to occur.
Risk Source Analysis
A root cause analysis for risk starts with analyzing the source of the risk. In
other words, where does the source of the risk occur?
A basic categorization is internal versus external.
• For internal sources, it is important to identify the specific department,
business unit, function, process, or job role that is the true source of the
risk. This allows the organization to deploy targeted resources in the
correct area. The true source of an internal risk may not be obvious. For
example, if an organization is having trouble matching competitors’ low
prices due to its high cost of goods sold, the source may be poor transfer
pricing agreements with subsidiaries, a poor cost allocation model that
loads too many costs on the key product, or a bottleneck in the production
process. Designating the risk source as a sales or marketing problem
would be a poor way to address this price risk.
• For external sources, the organization can use the information to direct
energies in different ways, since the risk may not necessarily be directly
managed and a focus that treats it as an internal risk will likely be
ineffective. Instead, the organization can focus on diligence in monitoring
the external risk, such as a competitor’s actions. A specific strategy around
risk mitigation may also be possible, such as lobbying to change
regulations or laws.
Risk Driver Analysis
Another key step in a root cause analysis is to determine the drivers of the
risk. Risk drivers are the forces that address why a risk is occurring, such
as a market bubble, an internal control weakness, or poor worker morale.
They can be broadly categorized as either events that can be specifically
identified or pervasive risks.
• Events that can be specifically identified can be tracked and may have a
window in which they could occur. This could be a monsoon or hurricane
season, a workplace accident or equipment failure, or a market downturn.
• Pervasive risks may be caused by a single significant control deficiency or
a number of small control deficiencies that collectively increase the
likelihood or impact of a failure. Pervasive problems may also involve a
control environment weakness, such as a “tone at the top” that permits
ethical violations in pursuit of profits or a lack of scheduled routine
maintenance for IT systems and applications.
Understanding the type of driver can provide important benefits. First, it
helps organizations prepare for the risk by illustrating how it will likely
develop. Knowing the drivers can help infer what effect the driver is likely
to produce so plans can also address these secondary effects. In addition to
the benefit of more complete plans, the drivers themselves can be
prioritized based on just those most likely to trigger a key risk event or to
be leading indicators. For example, loss of Internet connectivity may lead to
the loss of connection to the accounting department’s cloud-based
accounting system. This loss can be tolerated for a certain window of time
before it creates problems for customers. A specific response can then be
designed and tested in advance.
Remeasuring Key Risks and Developing Metrics
While a broad brush may have been used to measure risks while ranking
and prioritizing them, those risks identified as significant can be further
measured in terms of impact and likelihood. Organizations may perform
sensitivity analyses, create simulations or models, and use other analytical
techniques to better understand a risk, for example, in terms of best- and
worst-case scenarios. Often organizations use historical data to test these
models. The results may lead to identification of key performance
indicators, leading economic indicators, or other metrics that can be used to
monitor the risk. The results may also be helpful in determining the best
risk response and often can show where certain risks can be aggregated or
bundled so they can be dealt with using the same response. Other risks will
not lend themselves to quantitative analysis, and these can be further
analyzed to develop a consensus of opinion regarding their significance,
how to measure them, and the appropriate response.
To reduce risk and improve efficiency, the CAE must take a closer look at
the risk assessment data. Implementation Guide 2010 recommends steps to
help ensure that proposed engagements are aligned to organizational
objectives:
This standard directs the CAE to start preparing the internal audit plan by consulting
with senior management and the board to understand the organization’s strategies,
business objectives, risks, and risk management processes. Thus, the CAE considers
the maturity of the organization’s risk management processes, including whether the
organization uses a formal risk management framework to assess, document, and
manage risk.
It continues with additional considerations for implementation by stating
that:
Linking critical risks to specific objectives and business processes helps the CAE
organize the audit universe and prioritize the risks.
Risk measurement scores and CAE guidance are then used in risk
prioritization.
Risk Prioritization
Risk prioritization uses various methods to rank risks and establish the
relative strength of each risk and the potential consequences of each.
Methods include the following:
• Absolute ranking. This method ranks risk management scores and places
the risks in order of magnitude.
• Relative ranking. This method groups risk measurement scores into
natural clusters and assigns relative values such as low, medium, or high.
• Matrices ranking. This method further analyzes risk probability and
consequences. It places risks in a matrix, with low, medium, or high
impact on one axis and low, medium, or high likelihood on the other axis
(creating nine possible combinations), or in a simple quadrant of low
likelihood/low impact, low likelihood/high impact, high likelihood/low
impact, and high likelihood/high impact (omitting medium
likelihood/impact levels).
Strategic Risk Responses
Organizations can choose to manage identified and prioritized risks in a
number of different ways, including:
• Avoidance. Identifying ways to prevent risk exposure.
• Reduction or control. Establishing internal controls for reducing the
potential negative impact of risk and uncertainty or training employees in
how to recognize potential risks and respond to prevent damage and
reduce the effects.
• Sharing or transfer. Sharing or transferring the risk to insurance or to
other parties (through a contractual arrangement).
• Acceptance. Accepting the risk because a response would not be costeffective or identifying alternate ways to manage the risk such as
establishing contingency plans.
• Pursue. Accepting increased risk to increase the likelihood of opportunity
or goal realization.
Terminology may vary among organizations. For example, some use the
terms “terminate,” “treat,” “transfer,” and “tolerate” to describe the above
activities.
CAE Assessment of Risk Responses
Once the organization has selected strategic responses to significant risks,
the CAE should balance the quantified risk priorities with the organization’s
risk response strategy. The CAE makes two main assessments at this point:
• What is the organization’s ERM maturity level related to its response?
• What is the residual risk after the risk response is taken into account
relative to the organization’s risk appetite or tolerance to absorb the risk?
Assessing Organization’s ERM Maturity
The maturity level of an organization’s ERM processes affects how much
weight the CAE should give to a selected risk response. An organization
may have the intent—but not the ability—to effectively address a risk. The
enabling processes to address risks include people, processes, and
technology.
• People. People include the leaders of the organization and whether they
are developing and communicating strategies and the risk appetite clearly
and effectively. This also include all persons directly responsible for
managing and owning specific risks. The organization must have the
proper accountability structures in place, diligent hiring procedures, and
training.
• Processes. Processes include policies, procedures, and tasks that must be
performed as intended and must be audited to ensure that they are
executed, efficient, and effective.
• Technology. Technology includes information timeliness, availability,
completeness, and relevance as well as the security and level of integration
of the technology itself. This includes not only information systems but
also production line technology, radio frequency identification (RFID), and
other enabling technologies.
The organization’s relative maturity level in each of these areas for its ERM
capabilities will result in an overall organizational maturity level for ERM,
as shown in Exhibit I-10. (Different sources may use different names for
stages.)
Exhibit I-10: Assessing the Organization’s ERM Maturity Level
Lack of organizational maturity for the ERM function may result in ERM
processes becoming one of the areas in an upcoming audit plan. The
organization’s relative level of ERM maturity should be taken into account
when assessing the likelihood that a risk response will be adequate. More
precisely, CAEs assess the degree of residual risk that remains for each
significant risk given the organization’s ERM maturity. Residual risk is
discussed next.
Assessing Residual Risk versus Risk Appetite
Residual risk is the risk remaining after management takes action to reduce
the impact and likelihood of an adverse event, including control activities in
responding to a risk. It is contrasted with inherent risk, or the risk derived
from the environment without the mitigating effects of internal controls.
ERM professionals may calculate or estimate residual risk, and the CAE
may start with the results of this assessment. The CAE would then make
his or her own independent assessment of significant residual risks,
factoring in the organization’s ERM maturity level and the CAE’s
assessment of the strength of relevant controls in the area. If the
organization’s ERM maturity level is very low or there have been control
weaknesses in a given area in the past, the auditor may decide to audit an
area based on its inherent risks rather than assuming that internal controls
are properly designed and will perform as expected. When ERM maturity is
high and controls appear adequate, it may be enough to test the sufficiency
and reliability of the risk-monitoring techniques. Audit frequency may also
be adjusted based on such considerations.
Significant residual and/or inherent risks must also be measured against the
organization’s risk appetite. A risk response that leaves significant residual
risk relative to risk appetite levels will be a higher priority than a risk that
has been reduced to below the risk appetite level in a reliable manner, such
as the use of insurance from a solvent and reputable insurer. In other words,
the CAE should measure the type of risk response and its reliability against
the risk appetite and tolerance levels when determining the priorities of the
annual audit plan. For example, the organization may have chosen a risk
acceptance strategy for the possibility of poor returns on a new investment
because it has sufficient cash reserves to absorb the loss, but the CAE may
determine that more substantive audit techniques are needed to prove that
the actual risk impact and likelihood remain within acceptable risk tolerance
and risk appetite policies.
Prioritizing Potential Engagements
Three standards apply when assessing and prioritizing potential audit
engagements.
• Implementation Standard 2010.A1 (Assurance Engagements):
The internal audit activity’s plan of engagements must be based on a documented
risk assessment, undertaken at least annually. The input of senior management and
the board must be considered in this process.
• Implementation Standard 2010.A2 (Assurance Engagements):
The chief audit executive must identify and consider the expectations of senior
management, the board, and other stakeholders for internal audit opinions and other
conclusions.
• Implementation Standard 2010.C1 (Consulting Engagements):
The chief audit executive should consider accepting proposed consulting engagements
based on the engagement’s potential to improve management of risks, add value, and
improve the organization’s operations. Accepted engagements must be included in the
plan or be separately approved if accepted in the interim (during the year)
subsequent to the most recent annual plan approval process and in advance of the
next annual planning process time frame.
As these standards state, management and board requests should be
considered. Regulatory mandates are also applicable.
CAE Validation of Risk Priorities
The internal audit activity assists both management and the oversight body
(the board or its audit committee) and helps protect the organization from
risk exposures. As the discussion of risk-based prioritization has shown, a
thorough risk assessment executed in a timely manner should:
• Produce credible results about engagements.
• Establish buy-in through its participatory processes.
• Help management and the oversight body focus on top risks.
One way to ensure that the board and management have a clear
understanding of the connection between significant risks and the
achievement of business objectives is to link risks to specific processes.
These can be processes that could be interrupted by a risk event or a
process used to manage and control a risk. Tracing the links between key
processes and related risks can help illustrate why the risk is significant and
why a specific related process should be audited. A matrix showing
significant risks in the columns and processes in the rows (or vice versa)
can be used to assess the strength of the correlation and can show where
more than one risk could be addressed by one process audit engagement.
Strongly correlated risks and processes can be targeted.
The CAE needs to make decisions for applying audit activity resources
based on the significance of risk and exposure related to achievement of
organizational strategy and objectives. In addition to the analysis of risks
and responses discussed above, other factors to establish the priority of
engagements include financial impact, asset liquidity, management
competence, quality of internal controls, degree of change or stability,
timing and results of the last audit engagement, complexity, employee and
government relations, regulatory environment, etc.
A credible risk-based assessment should build respect for internal audit plan
recommendations and legitimize the internal audit activity’s involvement in
the engagement selection process.
In conducting audit engagements, methods and techniques for testing and
validating exposures should also reflect the risk impact and likelihood of
occurrence.
For additional information on coordinated approaches applied to leverage
synergies between the organization’s risk management and internal audit
processes, consult Implementation Guide 2010, “Planning.”
Selecting Engagements
Aligning internal audit activities with strategic and operational goals and
objectives through an internal audit risk assessment helps to ensure efficient
use of internal audit resources while providing management with valuable
insights on risk management activities. While risk analysis and assessment
are not foolproof, the processes are better than relying on intuition.
Educated decisions can be made about the selection of internal audit
engagements.
A general rule of thumb for engagement selection is to recommend auditing
just the risk management activities for those risks that are rated as high
impact but low likelihood, under the assumption that they are low
likelihood as a result of effective risk management actions. For high to
medium impact and high to medium likelihood risks, audit engagement
objectives should be to identify the root cause(s) of the increased
impact/likelihood if not already known and to produce actionable
recommendations for positively influencing the root cause(s) once known.
The output of the audit activity’s risk assessment process and selection of
engagements is a proposed annual audit plan, which is explored in detail
later in this section in Chapter 3, Topic A, “Communicating and Seeking
Approval of the Annual Audit Plan.”
Topic C: Assurance Engagements (Level P)
Assurance engagements are conducted for a variety of purposes. Some focus
on assessing major areas of controls (governance, operations, financial
reporting); others focus on the efficiency, effectiveness, and compliance of
certain business processes. All are intended to add value to the organization
by strengthening governance, reducing risk, and/or improving control.
Features of Assurance Engagements
An assurance engagement has three parts:
• Planning. During this phase, the engagement objectives and scope are
established. The audit team seeks a more complete understanding of the
process being audited. This includes the function’s business objectives, its
assertions (the accomplishments it reports), the risks that are unique to the
process, and the controls that have been designed to manage these risks.
Based on this information and an assessment of the adequacy of the
controls, the team develops a plan for testing the controls and a work
program that lists specific audit procedures. The team identifies the
resources and expertise required to implement the audit plan, possibly
including external resources. Work is assigned to team members, and a
schedule is prepared.
• Performance. During this phase, the audit team gathers evidence sufficient
to support the achievement of engagement objectives, evaluates data, and
develops observations (or findings) and recommendations (or corrective
actions).
• Communication. If necessary, the audit team will bring critical issues to
management’s attention immediately so that prompt corrective actions can
be taken. The team prepares a draft report and reviews it with
management to clarify conclusions and recommendations. Final reports are
then distributed to the appropriate parties.
Audit Team Composition
Performing audit work with proficiency and due professional care is the
responsibility of every internal auditor. As stated in Implementation Guide
1200, when the CAE is developing the internal audit plan and assembling
audit team resources, he or she:
Generally thinks about the alignment between the knowledge, skills, and other
competencies needed to complete the plan and the resources available among the
internal audit activity and other providers of assurance and consulting services.
The 2000 series of standards address additional details of managing the
internal audit activity and audit resources.
Engagement teams may include internal and external auditors, lawyers, and
experts such as IT specialists or privacy professionals. The audit team
composition should include auditors familiar with the business process being
audited. The required competencies will vary and must be assessed by the
CAE depending on the situation.
Data Gathering and Evidence
The data that needs to be gathered for assurance engagements will vary
depending on the objectives and the methods of assessment. The
information and documentation that needs to be gathered may also vary
from country to country. In most assurance engagements, the team will need
to review the organization’s internal records. In cases where this data
includes private or sensitive information of individuals or other entities, the
information is usually protected by confidentiality agreements and/or
government regulations. When in doubt about privacy implications, the
auditor should have legal counsel review the information before disclosing
it as evidence in official audit communications, especially if there may have
been potential privacy violations. This will balance the auditor’s need to
disclose findings against the counsel’s legal requirement to defend the
organization. In some types of assurance engagements, public documents
such as SEC filings, court records, and press releases should also be
reviewed.
The internal auditor will likely be able to present objective evidence of
discrepancies between intended governance, risk management, and control
(including policies, procedures, and systems) versus how these things are
actually being done. Audit evidence may demonstrate physical or logical
control weaknesses that have been observed or evidence of past control
failures. Similarly, the internal auditor should be able to gather and present
objective evidence on the use and safeguarding of assets. For example, to
determine whether assets are being used efficiently, the internal auditor may
prepare asset usage ratios and compare them to historical trends or
benchmark them against those of a competitor or an industry average to put
them in context. In a performance assurance audit, for example, data
gathering and analysis may start by determining whether key performance
indicators are being used in a given area for measurement, analysis, and
feedback. Analysis of KPIs should show a cause-and-effect link between the
KPIs and the actual behavior they promote, using real examples of observed
behavior when feasible.
In some cases, such as a control self-assessment (CSA) or a review of
management performance, audit data and evidence will be qualitative and
subjective to some degree. In these cases, data gathering should find ways
to corroborate the information. In a CSA, for example, areas of consensus
among a sufficient number of respondents can be used to balance the effects
of low levels of response or bias. Evidence on the quality of management
may need to be somewhat subjective but can be related to the degree to
which management decisions have supported organizational goals and
objectives.
Some data gathering and analysis needs to occur on a continual basis over
the course of a year or other period. For example, for internal control over
financial reporting assurance, the audit committee and senior management
typically request that the internal audit activity perform sufficient audit work
and gather other available information during the year to form an opinion
on the adequacy and effectiveness of the internal control processes. The
CAE normally communicates that overall evaluation, on a timely basis, to
the committee. The committee will evaluate the coverage and adequacy of
the CAE’s report and may incorporate its conclusion in the committee’s
report to the governing board.
Types of Assurance Audits
The rest of this topic looks at the following types of assurance audit
engagements in greater detail. (Note that any of these could be alternately
designed as a consulting engagement given a management request for
consulting services in the given area.)
For each type of audit, the following aspects are discussed:
• Objectives
• Stakeholders
• Risks
• Audit team
• Audit methods
• Controls
In some cases, the discussions that follow also provide information on
recommendations, common management responses, or follow-up procedures.
Operational (Efficiency and Effectiveness) Audit
Engagements
Operational audits are focused on providing assurance on governance, risk
management, and controls in regard to the effectiveness and efficiency of
operations. They are not focused on finance or compliance in particular but
may include some financial or compliance risks. Such engagements may be
referred to as management audits in government environments.
Objectives
There are three key considerations in reaching an evaluation of the overall
effectiveness of the organization’s risk management and control processes:
• Were significant discrepancies or weaknesses discovered from the audit
work performed and other assessment information gathered?
• If so, were corrections or improvements made after the discoveries?
• Do the discoveries and their consequences lead to the conclusion that a
pervasive condition exists resulting in an unacceptable level of business
risk?
Stakeholders
Stakeholders include the board and management who are ultimately
responsible for oversight and specific business process owners who will be
responsible for addressing audit recommendations.
Risks
Risks related to operational effectiveness include business processes that fail
to work toward or are counterproductive to organizational objectives. Risks
related to inefficiency involve achieving goals in a manner that is more
costly than the value that is added or a selected benchmark.
Suboptimization can affect both efficiency and effectiveness.
Suboptimization involves focusing on optimizing a particular business
process or unit at the expense of the overall organizational goals, often
caused by a departmental “silo” mentality. For example, a repair department
could decide to ground an airplane for several days so that the repair
department could avoid the travel cost of rushing personnel to the site. The
repair department may meet its budget, but the overall organizational
opportunity cost may be much greater. The event is also ineffective because
it may inconvenience passengers. An audit recommendation focused on root
causes would recommend redesigning the repair department manager’s
incentives (altering the current overemphasis on meeting the department
budget).
Audit Team
Team members need specialized knowledge and experience in the business
unit(s) or functional areas being audited and an understanding of the
organizational strategy as well as any complementary unit or area strategic
goals.
Audit Methods
Operational audit subjects could include manufacturing plants, marketing
activities, human resources, engineering, cafeteria services, housekeeping,
fleet management, facility management, etc. Operational information may
include production volumes, reserves, personnel head counts, emission or
effluent statistics, productivity rates, weights and measures, etc.
The temporary existence of a significant risk management or control
weakness does not necessarily lead to the judgment that it is pervasive and
poses an unacceptable residual risk. The pattern of discoveries, the degree
of intrusion, and the level of consequences and exposures are factors to be
considered in determining whether the effectiveness of the whole system of
controls is jeopardized and unacceptable risks exist.
Controls
COSO’s Internal Control—Integrated Framework describes internal control
as a process designed to provide reasonable assurance of achieving
objectives in three areas:
• Effectiveness and efficiency of operations
• Reliability of reporting
• Compliance with applicable laws and regulations
Therefore, evaluating the internal control framework for the area being
audited is an important audit step, considering “soft” controls such as the
“tone at the top,” the ethical environment, or the corporate culture.
Operational auditing goes beyond traditional concerns with the reliability of
financial statements to consider how efficiently and effectively the various
operations in an organization are meeting their objectives. This may include,
among other goals, review of:
• Policies, procedures, and systems.
• The quality of management.
• The use of resources to achieve organizational goals efficiently and
effectively and the safeguarding of assets.
Security Audit Engagements
Security aspects of a business are critical and an important part of internal
auditing. Standard 2120.A1 states that:
The internal audit activity must evaluate risk exposures relating to the organization’s
governance, operations, and information systems regarding the:
•
•
•
•
•
Achievement of the organization’s strategic objectives.
Reliability and integrity of financial and operational information.
Effectiveness and efficiency of operations and programs.
Safeguarding of assets.
Compliance with laws, regulations, policies, procedures, and contracts.
Objectives
Security audits primarily focus on governance, risks, and controls related to
the safeguarding of assets and the reliability and integrity of information.
Stakeholders
Security audits can span the operations and facilities or can be focused on
one subject, such as information technology security or head office security.
Therefore, stakeholders will include all parties directly responsible for the
security of the area to be audited, including security guards, if any, and IT
professionals. The board and management have ultimate responsibility and
are the stakeholders to whom the CAE reports all security issues and
recommendations.
Risks
A security policy might include control systems and standards to manage
security within acceptable risk limits in regard to the following risks:
• Unauthorized physical access to or attack on a facility or organizational
personnel
• Theft of or willful damage to products, inventory, supplies, assets, or
information
• Fraud by employees or third parties
These concerns can be influenced by other industry- or region-specific risks:
• Inherent industry risks
• Inherent social and political risks
• Market and economy pressures
• Location and facility risks
• Natural hazards (such as fires, floods, earthquakes, or animals) and
biohazards
The policy could include standards and guidance in regard to risks and
issues such as:
• Terrorism, including active shooters, bomb threats, and so on.
• Addictions (e.g., drugs, alcohol, gambling).
• Weapons.
• Travel risk.
• Executive protection, including kidnapping or extortion.
• Access and egress.
• Crisis management.
• Cameras and electronic monitoring.
• Intrusion detection.
• Facility design and construction.
• Investigations or searches.
• Use of third-party security services.
• Relationship with law enforcement.
• Other issues specific to the risks of the organization.
Audit Team
Physical security audits should be conducted by internal auditors who have
experience in physical access control and the provisioning of rights to
various job roles or individuals. Logical or other information system
security audits should be conducted by team members who are generally
conversant with information systems as well as specialists in information
systems auditing, if the project complexities require it.
Audit Methods
Internal auditing for security differs, depending on whether it is done as
part of the design of a facility or information system or whether it is
conducted for ongoing operations and systems.
Auditing Security for New Facilities and/or Information Systems
Facility design is often developed from an operating need, and security
features may be added to rather than integrated into the design. Internal
auditors can help organizational design teams consider items such as video
surveillance; clear lines of sight; electronic controls over access, egress, and
movement (e.g., elevators); lighting; heating, ventilation, and air
conditioning systems; patrols; emergency response; and weapons
management.
As noted in Sawyer’s Internal Auditing (Sawyer et al. wrote the definitive
Sawyer’s Internal Auditing, published by The IIA), physical security in
regard to information technology can be achieved, in most cases, by careful
planning. Factors that impact security can be mitigated by:
• Selecting an inconspicuous location for data centers.
• Providing for continued operation during floods, fires, earthquakes, and
other disasters.
• Locating data centers and computers in rooms with only interior walls.
• Establishing and ensuring adherence to virtual and physical data storage
protocols.
• Minimizing entry points to data centers. (However, these limited entry
points must meet fire codes.)
• Eliminating entry points such as false ceilings and crawl spaces by using
slab-to-slab construction in highly sensitive areas.
• Locking emergency exits from the outside and installing alarms.
• Using closed-circuit TV to monitor visitors in sensitive areas.
• Installing motion sensors in areas such as tape vaults to monitor access
before and after business hours.
• Ensuring that air temperature and humidity are controlled and monitored.
• Minimizing exposed wiring.
Auditing Security for Existing Facilities and/or Information Systems
Internal auditors must evaluate the effectiveness of the organization’s
security controls and must be able to explain, in reports to management, the
extent of the risks or exposures resulting from any deficiencies found in the
security controls. The auditor should be able to recommend alternatives to
the current system or control. These alternatives may include, but are not
limited to, a different security system, changes to procedures or controls,
and enhanced training for relevant employees.
IT security is so important for financial information and general privacy
concerns that many organizations audit IT security every year. The
increasing use and constant release of new hardware and software require
continuous training for the auditor to remain proficient in security issues.
The GTAG series of Practice Guides and GAIT methodologies are important
learning tools for this type of audit.
For other records, the internal auditor should consider both on- and off-site
storage equipment and facilities, record retention and destruction policies
and procedures, and recycling activities (to ensure that recycled documents
do not contain confidential information).
Internal auditors are also expected to comply with security requirements
when working at various locations and while traveling in order to protect
themselves and the information they have. Laptops and cell phones are
highly vulnerable to theft.
Controls
Controls include the following types:
Physical Security Controls
Physical security of assets is one of the most basic and important
considerations for an organization, and it is the responsibility of
management to ensure that controls are appropriate and complied with.
Physical security includes manual measures, such as doors, locks, and
guards, as well as policies and procedures.
Within each area, many types of risks must be addressed, and controls
should include both preventive and detective measures.
Organizations have building entrances, common areas, data centers, storage
sites, and hazardous areas where access should be restricted in layers of
higher and higher security based on role, with visitors being one of the
designated roles (with minimal access). There are several ways to limit
access to authorized personnel. Locks and keys are an obvious example, but
swipe or proximity cards/badges can be used as well as biometric access
systems, for example, fingerprint scanners. Physical barriers such as
bulletproof glass or concrete barriers for cars are also access controls.
Access control requires effective monitoring controls, including cameras and
good lighting and lines of sight for security guards.
Logical Security Controls
Logical security is designed to achieve the same results as physical controls
but is electronic in nature. This includes controls to ensure that:
• Only authorized users have access to data.
• The level of access is appropriate to the need and the role.
• Modifications to data leave a complete audit trail.
• Unauthorized access is denied and the attempt is reported.
Passwords are the most common means of authenticating users. They are
also the most common way for unauthorized persons to access electronic
information, usually because employees fail to protect their passwords. For
example, the employee may use a personal piece of information (home
address or spouse’s name) or write the password down and leaves it in a
location where others may see it. Some organizations use software that
encrypts passwords, forces a periodic change of passwords, or requires a
particular password structure (for example, a designated number and type of
characters).
Data security software may also control access to resources by, for example,
permitting access only to certain departmental users or suspending a user ID
after a set number of invalid entry attempts.
In addition, security codes can be required for various levels of access. For
example, one ID code may be necessary to access all data except salary and
a second ID code may be necessary to access salary information. Biometric
systems can be used for high-security information system access control.
Hazard Controls
For many hazards, security is primarily a factor of adequate control design
and crisis management programs. Some of these risks and controls may also
be evaluated as part of a health and safety audit.
Fire, smoke, and water are frequent sources of damage for organizations.
Because disaster prevention is preferable to disaster recovery, an
organization needs to develop fire and flood prevention techniques that will
mitigate their risks. Some examples of safeguards are:
• Early warning systems, such as fire alarms and smoke detectors throughout
the facility.
• Installing appropriate fire suppression systems in facilities and/or vehicles.
• Conducting drills.
• Acquiring fire- and water-resistant storage for important documents.
• Ensuring that relevant third parties have adequate controls.
These systems must be constantly monitored and periodically tested.
Earthquake loss prevention involves proper construction of facilities, quality
control over materials, site location, emergency preparedness and drills,
insurance coverage, and business interruption plans.
Power Supply Controls
Organizations that could be severely damaged by a power shortage or
outage (for example, through loss of valuable data or by providing a
window of opportunity for security breaches) can provide alternative power
sources.
Organizations should consider their needs for both long- and short-term
solutions. A long-term solution such as a generator will provide power for
longer periods of time. Generators tend to have a higher costs, both in
installation and maintenance, and are most commonly used for large
systems and critical applications or for facilities in countries with unreliable
electrical grids. Short-term solutions such as uninterruptible power supply
(UPS) systems and surge protectors are less costly sources of temporary
power that allow an orderly shutdown of electronic equipment, such as
computers, during brownouts or blackouts.
Data Storage Controls
Organizations are continuously generating data; enormous amounts of data
are produced and retained. There are a number of ways to store data. If the
media are kept only on site, or only one copy is kept, there is tremendous
exposure for it to be damaged or lost. Most organizations have a system
where a backup copy of the media is also kept off site or data is routinely
backed up to the cloud (which is typically also off site).
Data can be lost through mislabeling, mishandling, repeated use, magnetic
disruption, or natural disasters. Organizations have different ways of dealing
with these kinds of exposures, such as using systems that automatically
label a file with an internal code. This mitigates the risk of external labels
being lost or removed or becoming unreadable through time.
For electronic data storage, the auditor might:
• Determine that internal and external labels are used appropriately.
• Determine whether or not tape, disk, and file management systems are
available.
• Audit third-party distributed storage methods such as a cloud, including
that the organization has done due diligence for any third-party storage
organization, understands the intellectual property risks of the countries in
which the data may be stored, and has appropriate contractual controls
over these solutions.
• Assess the frequency of abnormal terminations of stored data because of
inadequate disk space.
• Ensure that data has been sufficiently backed up to allow recovery of lost
files.
• Evaluate the housing of data and determine whether there is both off- and
on-site storage.
• Assess temperature and humidity controls.
• Evaluate file-naming conventions to determine whether they provide
sufficient information to allow proper handling.
Physical records may be stored in locked file rooms; fire-, smoke-, or
water-resistant cabinets; rooms using waterless fire suppression techniques;
safes; etc. There may be duplicate imaged or microfilmed copies stored as a
backup should the original be damaged. Off-site storage of archived records
should be evaluated for similar risks, as should controls over information
held by third parties.
Financial and Financial Reporting Audit
Engagements
Financial audits performed by external auditors focus on the fairness of an
organization’s financial statements and internal controls over financial
reporting (ICFR), if applicable (i.e, for audits of public companies).
Financial audits performed by internal auditors primarily focus on assurance
over internal controls but may also review the quality and usefulness of the
organization’s managerial accounting and internal reporting systems.
Assurance on internal controls may be needed because of rules over the
quality of those controls, such as those found in the U.S. Sarbanes-Oxley
Act (SOX) or those of market regulators such as the U.S. Securities and
Exchange Commission (SEC). Internal and external audit coordination and
reliance efforts can play a major role in providing the most effective and
efficient overall assurance coverage for public-company ICFR audits.
Objectives
External financial reporting is historical in nature, so the objective of
external financial reporting is to prepare relevant and reliable financial
statements that fairly and accurately represent the recent historical activities
of the organization. Financial reporting objectives should form the basis for
the majority of internal controls. Internal controls set reliable financial
reporting as a key objective because of the importance of these reports to
lenders (primarily bankers) and investors and because of their role in
satisfying legal and regulatory requirements and in ensuring efficiency and
stewardship over the organization’s resources.
The objective of assurance audits of internal control over financial reporting
is to provide assurance regarding the effectiveness of the processes and
procedures (controls) supporting the reliability, timeliness, transparency, and
completeness of the organization’s financial reporting. The objective of
assurance audits of managerial accounting and reporting systems is the
same, except that the audience will be management rather than external
parties. Management needs reliable, timely, transparent, and complete
information to enable effective decision making.
In the U.S., SOX and related SEC rules provide considerable incentive for
principal executives and financial officers to seek assurance that controls are
in place to support their certifications in regard to financial statements,
controls, and disclosures, because the laws make them personally
responsible for these matters. Internal auditors can provide value-added
services to those executives (and, of course, to the organization and the
investing public).
Regulations around the world have increased focus on internal auditor
responsibilities in the area of financial audits.
Stakeholders
Stakeholders include the board, the audit committee, and senior
management, especially the CEO and the CFO due to their personal
accountability. Senior management is the owner of the control environment
and financial information, including the footnotes and disclosures, which are
integral to the financial statements. Other stakeholders include regulators
such as the SEC, rating agencies, current and potential stockholders,
investors, bankers, external auditors, and interested parties.
Risks
The internal audit activity’s work plans and specific assurance engagements
begin with a careful identification of the exposures facing the organization.
The work plan is based on the risks and the assessment of the risk
management and control processes maintained by management to mitigate
those risks.
Among the events and transactions included in the identification of risks
are:
• New businesses—including mergers and acquisitions.
• New products and systems.
• Joint ventures and partnerships.
• Restructuring.
• Management estimates, budgets, and forecasts.
• Environmental matters.
• Regulatory compliance.
• Fraud risks—often from overstating revenues or assets and/or understating
expenses or liabilities.
Audit Team
For financial/financial reporting audits, internal auditors need to be
knowledgeable in procedures that provide a level of assurance to senior
management and the audit committee that internal controls surrounding the
processes supporting the development of the financial statements are
effective. Where applicable, internal and external auditors need sufficient
interpersonal skills to closely manage and coordinate efforts related to ICFR
assurance to maximize coverage and efficiency. Internal auditors may also
need managerial accounting skills to provide assurance related to the
effectiveness of managerial accounting. External auditors need experience in
either the Generally Accepted Accounting Principles (GAAP) in the U.S. or
the International Financial Reporting Standards (IFRS) in most other
countries, or both, to assess whether the reported information fairly presents
the financial condition and result of operations of the organization in
accordance with GAAP or IFRS. If mandated by certain public company
audit requirements, the external auditor may also need the experience to
provide independent assurance regarding the effectiveness of an
organization’s ICFR.
Audit Methods
The IIA believes that internal control should be broadly defined and that
excellent guidance on how to do this is to be found in COSO’s Internal
Control—Integrated Framework. While use of the COSO framework is
widely accepted, there may be a valid reason to use some other recognized
and credible model. Sometimes, regulatory or legal requirements will
specify the use of a particular model or control design for an organization
or industry within a country.
Several conclusions in the Internal Control—Integrated Framework are
relevant to this discussion:
• Broadly defining internal control means not limiting assurance to
accounting controls or narrowly restricting it to financial reporting.
• While accounting and financial reports are important issues, there are other
important aspects of the business, such as resource protection, operational
efficiency and effectiveness, and compliance with rules, regulations, and
organization policies. These factors also have an impact on financial
reporting.
• Internal control is management’s responsibility and requires the
participation of all persons within an organization if it is to be effective.
• The control framework is tied to the business objectives and is flexible
enough to be adaptable.
The CAE should provide internal audit’s assessment of controls, including
the design or model, to the audit committee. The governing board must rely
on management to maintain effective controls, and it (or the audit or other
designated committee) should ask questions such as the following (and
obtain sufficient, reasonable, and acceptable assurance thereto).
• Is there a strong ethical environment and culture in the organization?
• Do board members and senior executives set examples of high
integrity?
• Are performance and incentive targets realistic, or do they create
excessive pressure for short-term results?
• Is the organization’s code of conduct reinforced with training and topdown communication? Does the message reach the employees in the
field?
• Are the organization’s communication channels open? Do all levels of
management get the information they need? (For example, are
management accounting and reporting systems useful?)
• Is there zero tolerance for fraudulent financial reporting at any level?
• How does the organization identify and manage risks?
• Is there a risk management process, and is it effective?
• Is risk managed throughout the organization?
• Are major risks candidly discussed with the board?
• Is the control system effective?
• Are the organization’s controls over the financial reporting process
comprehensive, including preparation of financial statements, related
notes, and other required and discretionary disclosures that are an
integral part of the financial reports?
• Do senior and line management demonstrate that they accept control
responsibility?
• Is there an increasing frequency of “surprises” occurring at the senior
management, board, or public levels from the organization’s reported
financial results or in the accompanying financial disclosures?
• Is there good communication and reporting throughout the organization,
especially for timely disclosure of bad news?
• Are controls seen as enhancing the achievement of objectives or as a
“necessary evil”?
• Are qualified people hired promptly, and do they receive adequate
training?
• Are problem areas fixed quickly and completely?
• Is there strong monitoring?
• Is the board independent of management, free of conflicts of interest,
well informed, and inquisitive?
• Does internal audit have the support of senior management and the
audit committee?
• Do the internal and external auditors have and use open lines of
communication and private access to all members of senior
management and the audit committee?
• Is line management monitoring the control process?
• Is there a program to monitor out-sourced processes?
Controls
Best practices indicate that internal auditors recommend improvements to
policies, procedures, and the process for quarterly reporting and suggest that
internal controls include:
• Properly documented policies, procedures, controls, and monitoring reports.
• Quarterly checklists of procedures and key control elements.
• Standardized control reports on key disclosure controls.
• Management self-assessments (such as CSA).
• Review of draft regulatory filings prior to submission.
• Process maps to document the source of data elements for regulatory
filings, key controls, and responsible parties for each element.
• Follow-up on previously reported outstanding items.
• Consideration of internal audit reports issued during the period.
• Special or specifically targeted reviews of high-risk, complex, and problem
areas, including material accounting estimates, reserve valuations, offbalance-sheet activities, major substitutions, joint ventures, and special
purpose entities.
• Observation of the closing process for the financial statements and related
adjusting entries, including waived adjustments.
• Conference calls with key management from remote locations to ensure
appropriate consideration of and participation by all major components of
the organization.
• Review of potential and pending litigation and contingent liabilities.
• CAE reports on internal control, issued at least annually and possibly
quarterly.
• Regularly scheduled disclosure and audit committee meetings.
No matter how effective, internal controls cannot ensure organizational
success. Bad decisions, poor managers, or environmental factors can negate
controls. Also, dishonest management may override controls and ignore or
stifle communications from subordinates. An active and independent
governing board that is coupled with open and truthful communications
from all levels of management and is assisted by capable financial, legal,
and internal audit functions can identify problems and provide effective
oversight.
Ratio Analysis
Ratio analysis—the comparison of two or more elements of data—is an
important part of financial analysis. Ratios, by definition, measure
relationships between quantities. One objective of ratio analysis is to detect
unexpected relationships compared to what the internal auditor would
normally expect to see, based on historical trends, expected cause-and-effect
relationships among several ratios or inputs and outputs, or various
benchmarks. Another objective of ratio analysis is to validate what the
auditor already knows about the business, or expects to see, as confirmed
by the numbers.
Stakeholders for ratio analysis include the board and management, who
want analysis results to highlight just the key drivers that indicate
significant risk or impact achievement of organizational objectives. Other
stakeholders include various analysts who produce and analyze ratios as
well as any persons who have their performance assessed by or provide
inputs to ratios.
Risks relate to the limitations of financial ratio analysis. These limitations
include:
• Difficulty of comparing financial statements between organizations due to
differences in accounting methods.
• Management’s ability to adjust estimates and assumptions.
• Difficulty in comparing long-term historical information due to inflation.
• Misinterpretation of some ratios because the wrong ratio is being used or
the ratio fails to show other contributing factors to a relationship, such as
a ratio that fails to account for the time value of money.
Audit team composition will vary depending on the complexity of the ratios
being analyzed. Specialists for a given area may be needed to provide
sufficient expertise.
Targets for financial or operating ratios can be used as KPIs to show when
controls are operating effectively and when they are out of control. Audit
evidence related to ratio analysis can be used to show whether the current
set of KPIs is driving the business toward achievement of its objectives.
Recommendations might suggest the use of different ratios, adding ratios,
reducing the number of ratios used, or correcting misapplied or erroneously
calculated ratio calculations. The set as a whole should enable timely
management decision making without providing too much information,
which can often obscure key relationships.
Auditors employ several types of ratio analysis, including common-size
statements, nonfinancial ratios, and financial ratios.
Common-Size Statements
Common-size statements express all account balances as percentages of one
relevant aggregate balance, usually sales revenues for the income statement
or total assets for the balance sheet. For instance, expressing all items on
the income statement as percentages of sales revenues makes all income
statements to be compared a common size, no matter how large or small
particular values might be. This enables comparisons among statements
containing very different absolute values. If marketing expenses, for
example, are expressed as 0.50% of sales revenues, it doesn’t matter if
expenses are U.S. $1,000 and revenues are U.S. $200,000 or expenses are
U.S. $10,000 and revenues are U.S. $2,000,000. On both statements, sales
revenues will be 100% and marketing expenses will be 0.50%. Thus, the
two can be compared to look for trends or variances. Converting all income
statement items to percentages of sales revenues is also called vertical
analysis.
Nonfinancial Ratios
Nonfinancial ratios compare relationships between two measurable and
correlated business elements. For example, internal auditors could calculate
the ratio of sick days taken to the total accrual of sick days and compare
these ratios by department. As with any ratio, nonfinancial ratios make data
comparable or of a common size. These ratios are often used as KPIs to
measure and manage achievement of objectives and mitigation of key risks.
For example, the ratio of orders processed without error or delay to the
total number of orders shipped might be a supply chain management KPI.
Nonfinancial ratios can be reviewed from period to period or assessed
against an internal or external benchmark. Exception reports may highlight
aspects of business activities that require scrutiny.
Financial Ratios
Financial ratios are used to compare the relationships between various items
in financial statements or other financial accounts. Numerous specific ratios
are used by auditors, managers, lenders, and investors to determine the
health of an organization. Auditors will use financial ratios in various
phases of audit engagements. These ratios can compare performance of
units within an organization, and they can be used to compare organizations
to one another or to a common standard, such as overall ratios in an
industry.
Financial ratios are often categorized according to the type of information
they provide as activity, liquidity, leverage, or profitability ratios.
• Activity ratios. Activity ratios measure an organization’s efficiency in
turning various balance sheet accounts, such as inventory or accounts
receivable, into sales or cash. One common activity ratio is the inventory
turnover ratio, which measures the number of times per year inventory is
sold and then replaced.
• Liquidity ratios. Liquidity is an indication of how quickly an asset can be
converted into cash. Liquidity ratios measure an organization’s ability to
pay its obligations in a timely fashion and are therefore a measure of the
organization’s financial health. A common liquidity ratio is the current
ratio, which divides current assets by current liabilities. A larger value for
this ratio is generally considered to be more favorable because it indicates
that there are more liquid assets available to pay the organization’s shortterm obligations and thus honor its commitments. Too high a ratio
(relative to the industry), however, may indicate that assets are being
insufficiently utilized.
• Leverage ratios. Financial leverage ratios (also called debt ratios) measure
an organization’s assets against its liabilities to ascertain ability to pay
down debt. Compared to liquidity ratios, leverage is a longer-term measure
of the organization’s solvency. Financial leverage ratios can track an
organization’s change in solvency from period to period. While an
organization can sustain a high rate of short-term indebtedness (start-ups
are invariably in this position), a high level of indebtedness in relation to
equity cannot be sustained over the long term without reducing the
organization’s ability to attract lenders and investors. A commonly used
leverage ratio is the debt ratio, which is calculated by dividing total
liabilities by total assets. The higher the value for this ratio, the greater the
organization’s financial leverage. Reasonable debt ratios vary among
industries and with the age and risk appetite of the organization. Generally,
higher debt ratios indicate greater risk (and reward) for investors and
creditors. Shareholders in particular tend to prefer higher debt ratios
because they can magnify gains (but also losses) without the dilution of
ownership control that additional equity may entail.
• Profitability ratios. Profitability ratios measure an organization’s ability to
earn a profit (earnings in excess of the expenses used to generate and sell
goods and services). They involve various comparisons of earnings
(revenues reduced by expenses) in the numerator to revenues in the
denominator. There are numerous ways to measure profitability, but a
common profitability ratio is the gross profit margin, which is calculated
by dividing gross profit (or net sales minus the cost of goods sold) by net
sales. Note that net sales is sales (also called revenue) minus any sales
returns. When used as a comparison between organizations, a higher gross
profit ratio indicates more effective management of pricing and control of
costs that can be directly associated with the goods (or services) sold.
When used to track an organization’s profitability over time, a rising trend
for the gross profit ratio indicates increases in operational efficiency.
Note that more examples of ratios are provided in Section IV of Part 3 of
this learning system.
Compliance Audit Engagements
According to the Open Compliance and Ethics Group:
Compliance is the act of adhering to, and the ability to demonstrate adherence to,
mandated requirements as defined by laws and regulations as well as voluntary
requirements resulting from contractual obligations and internal policies.
Compliance audits evaluate the adequacy and effectiveness of controls that
keep the organization in compliance with applicable laws and regulations,
contracts, and the organization’s own policies.
Objectives
The objectives of an effective compliance program are to:
• Identify and discourage intentional and unintentional violations.
• Detect illegal activities.
• Ensure that adequate organization-wide compliance training programs are
in place.
• Assist in proving insurance claims.
• Encourage proper behavior by providing incentives.
• Enhance and create corporate identity.
The organization should establish compliance standards and procedures that
are reasonably capable of reducing the prospect of criminal conduct by
employees and other agents, and compliance audits should review and
assess them.
Internal audit scope may include a review of the compliance programs to
see if written materials are effective, employees have received
communications, detected violations have been handled appropriately,
discipline has been even-handed, whistleblowers have not suffered
retaliation, and the overall compliance function has fulfilled its
responsibilities.
Stakeholders
Stakeholders include the board and management, compliance professionals,
and process owners and workers who are responsible for day-to-day
compliance. Large organizations, especially those in heavily regulated
industries, often have a chief compliance officer.
Risks
While there are many types of regulations and policies, each with their own
risks, environmental compliance risks can be used as a representative
example. Environmental health and safety (EH&S) risks to consider in audit
planning include ineffective organizational reporting structures; the
likelihood of causing environmental harm; damage to the health and safety
of workers, customers, or the community; fines and penalties; expenditures
mandated by environmental or health and safety agencies; and negative
publicity and loss of reputation and public image.
Audit Teams
Compliance professionals need to make up the bulk of the team for this
type of assurance engagement. The team should not be composed solely of
internal auditors.
Audit Methods
The role of internal auditing in compliance is to provide assurance that
compliance professionals, processes, and systems are effective.
Internal auditors should start an audit of the compliance process by forming
a basic knowledge of the laws and regulations pertaining to their areas of
assurance coverage and the roles and responsibilities of compliance
professionals and the frameworks and systems they use. Two compliance
frameworks have become generally accepted:
• ISO 19600:2014, “Compliance management systems—Guidelines.” This
framework provides guidance to establish, develop, implement, evaluate,
maintain, and improve a compliance management program and can be
combined with other management program standards like ISO 9001,
“Quality management.” Rather than targeting a specific risk area, ISO
19600 helps ensure that compliance programs are more comprehensive, in
part by using the plan-do-check-act continuous improvement methodology:
• Plan involves identifying issues and stakeholders, establishing good
governance principles, setting scope, establishing compliance policy, and
identifying compliance obligations and risks.
• Do involves establishing leadership and support functions, performing
operational planning and compliance risk control, and reporting on
compliance and performance.
• Check involves determining areas of noncompliance using independent
compliance functions.
• Act involves continuous improvement.
(Note that this standard supersedes Australian Standard AS 3806, a widely
accepted compliance framework.)
• U.S. Federal Sentencing Guidelines for Organizations (USFGO). This is
a principles-based framework, originally intended to guide U.S. federal
judges when imposing sentences on organizational defendants, but it has
become a de facto standard for compliance. It has seven principles that are
intended to guide human behavior toward clear accountability and ethical
conduct. It also promotes compliance training and leadership.
The internal auditors should then determine how well the compliance
professionals are using the frameworks to ensure compliance with the
specific laws, regulations, and policies that they are responsible for
assessing.
Internal auditors should verify that written policies clearly identify required
and prohibited activities. There should be an organizational chart that
identifies personnel who are responsible for implementing compliance
programs. Responsibility for oversight of regulatory compliance programs
should be assigned to high-level personnel in the organization. Due care
should be taken not to assign compliance responsibilities to persons who are
or should be known to have a tendency to break the law.
International companies should create a global compliance program that
reflects the laws, regulations, and local conditions of all geographies where
the organization operates.
The organization should take steps to communicate standards and
procedures to all employees through training, publications, etc. The
development of an effective “tone at the top” governance program will help
in the implementation of standards and procedures.
There should be hotlines for reporting suspect activities (without fear of
reprisal). A good way to accomplish this is to use an outside service
provider for whistleblower hotline intake calls; this practice is becoming
quite common. If the hotlines are internal, the in-house representative
should not be a member of the legal department. Hotlines should be backed
by nonretaliation policies, and there should be options for providing
information anonymously. Note that whistleblower guidelines are outlined in
the U.S. Sarbanes-Oxley Act.
The organization should take responsible steps to achieve compliance
through monitoring and auditing systems that have a reasonable chance of
detecting noncompliance and encouraging the reporting of inappropriate
behavior without fear of retribution.
Note that resources devoted to internal audit should be proportional to the
size of the organization and the demands of the annual audit plan.
Controls
Controls will differ depending on the area for which the rules, regulations,
and policies apply. Controls related to compliance with environmental laws
and regulations are used here as a representative example.
Types of environmental audits include the following:
• Environmental management systems audits that focus on systems in place
to ensure that they are operating properly to manage future environmental
risks
• Due diligence audits used as a risk management tool by organizations
purchasing land
• Treatment, storage, and disposal facility audits that track hazardous
substances “from cradle to grave”
• Pollution prevention audits that identify ways to minimize waste and
pollution at the source
• Environmental liability accrual audits that quantify and report accrued
liabilities for environmental issues
• Product audits that assess a facility’s production process to ensure
compliance with standards
The majority of environmental compliance audit functions report to the
organization’s environmental health and safety (EH&S) executive or the
general counsel rather than to the CAE. Reporting to the EH&S executive
may result in a loss of the independence required for an effective function.
EH&S executives are typically responsible for the facilities being reviewed
and therefore have a potential conflict of interest that may tempt them to
suppress information about problems.
IIA studies have shown that environmental auditors seldom meet with
governing boards and a majority of them have no contact with the CAE.
Also, a substantial majority of organizations never include environmental
issues in their agendas, and a significant number have incurred penalties
recently and describe their environmental risks as material.
The CAE should foster a close working relationship with the chief
environmental officer and coordinate the audit plan with environmental
auditing activities. Periodically, the CAE should schedule an EH&S audit,
which could focus on compliance, management systems, or both.
The CAE should evaluate whether environmental auditors outside the CAE’s
organization are in compliance with audit standards and/or a code of ethics.
The CAE should evaluate the organizational placement and independence of
the environmental audit function to ensure that significant information about
risks is being reported to the audit committee or other board committee.
Recommendations and Follow-Up
After an offense has been detected, the internal auditor should observe
whether the organization takes all reasonable steps to prevent further
offenses of the same type. This might include appropriate discipline or a
requirement to self-report to a government agency. Detecting a violation that
was not prevented or detected by the compliance program’s internal controls
signals a need to review the program to see if it needs improvement.
Discipline of violators is necessary and should be appropriate to the
individual case. If the CAE uncovers exposures that are not properly
managed, he or she would normally change the schedule of engagements to
evaluate those risks further prior to presenting recommendations. The CAE
should discuss needed improvements and solicit management contributions
to improve internal controls.
Internal auditors should make timely recommendations if they identify a
significant risk of a large control failure, such as the theft of data from a
database containing customers’ credit card information. This type of breach
may have immediate severe consequences such as negative press and
scrutiny from regulators and privacy watchdogs. More information on
auditing privacy can be found in The IIA’s Practice Guide “Auditing
Privacy Risks,” second edition.
Performance Audit Engagements
Performance audit engagements assess whether management has appropriate,
necessary, and sufficient monitoring and controlling activities in place to
assess the performance of the organization as a whole, of specific units or
functional areas, or of specific job roles or individuals toward meeting
strategic, tactical, and/or operational objectives and goals. The audits can
also determine whether the information is gathered and analyzed in a timely
enough fashion to be useful and whether the information is being leveraged
for informed decision making and management control. Performance
reporting functions can also be audited.
It is important to realize that not all aspects of an organization can be
efficiently and effectively tracked. Standards should be designed and written
in a way that will allow personnel to measure progress toward meeting the
organization’s most important objectives. As noted earlier, these standards
are often called KPIs, or key performance indicators. They are also referred
to as metrics, targets, goals, or business objectives.
Objectives
The objectives of performance monitoring, reporting, and controlling are to
accurately measure performance in areas that relate to key business
objectives, to gather and prepare sufficient and useful information efficiently
and in a timely fashion, and to use the information effectively for
management control. For example, a data analyst might identify deficiencies
in progress toward a business objective, analyze the root causes of the
deficiencies, and prepare a report that effectively conveys this information
to management while there is still time to influence the outcome.
Management would then take this and other applicable information into
account and may take corrective actions as appropriate.
Performance audit engagements might determine whether the right things
are being measured; whether the measurement process is efficient and is
being performed correctly; whether data is collected and analyzed per the
desired schedule; whether reports highlight the critical information needed
for control (versus being hidden by too much unnecessary information);
and/or whether the information is being used to make informed decisions.
Stakeholders
Stakeholders for performance audits include the board and management who
require accurate and timely performance information to make corrections as
well as any internal or external party or entity that is being measured.
Stakeholders who are being measured for performance want to be measured
against criteria that they can personally control to some degree. They also
want to know how they are being measured so they can have a reasonable
chance of success and improvement.
Risks
What can be measured can be managed, so failure to measure performance
introduces the risk that performance cannot be managed. Other risks include
measuring the wrong KPIs so that workers or processes fail to work toward
organizational goals or objectives, receiving information too late to be of
use, and measuring too many performance indicators rather than just the
key ones. Measuring too many indicators can increase measurement costs
while reducing the effectiveness of the indicators as a management tool.
Persons and processes that are not managed can quickly get out of control,
and the results can include missed budgets or deadlines, accidents, lawsuits,
increased insurance premiums, and loss of worker productivity.
Audit Team
Team members need knowledge and experience similar to that for
operational audits. They also need knowledge of measurement systems and
what makes them effective at promoting organizational objectives while
minimizing unintended consequences, such as the right selection and
number of KPIs and consistent enforcement.
Audit Methods
While management should exercise control by creating and applying KPIs,
auditors should perform efficient and cost-effective audits by focusing on
the KPIs. KPIs are based on plans and objectives and not on policies or
rules. Types of KPIs include the following:
• Quantity of output standards measure quantitative performance, such as
units produced per day or week.
• Accuracy of quality standards measure quality performance, such as the
number of sold items returned.
• Cost standards specify benchmarks, such as material costs per unit.
• Timeliness standards are associated with things such as production
schedules or project completion.
• Capital standards deal with capital investment such as return on investment
(not operating costs).
• Revenue standards measure the monetary values assigned to sales, such as
revenue per airplane passenger mile.
KPIs might include accident reduction, reduced machine downtime caused
by accidents, or lower workers’ compensation insurance premiums.
Controls
The auditor should assess considerations such as the following:
• Does the organization have KPIs?
• Are the KPIs appropriate, i.e., can they really measure the organization’s
success in meeting its objectives?
• Do the KPIs include the human factor, i.e., will they create frustration or
confusion for employees?
• Are measurements taken and reported at appropriate times, i.e., early
enough in the process to correct course?
• Are measurements used effectively in controlling performance by
identifying deficiencies and correcting them?
Recommendations
Recommendations should be linked to organizational objectives by
indicating whether the existing KPIs are adequate to promote achievement
of these objectives or if a different set of KPIs would be more efficient or
effective.
Audits of External Business Relationships
The IIA Practice Guide “Auditing External Business Relationships” explains
that:
“External business partners,” “extended relationships,” and “contractual relationships”
are among the numerous names by which today’s organizations define their external
business relationships.
External business relationships (EBRs) include joint venture partners, outsourced service providers, agents, contract workers, vendors, franchisees,
etc.
When contemplating the internal audit activity’s EBR responsibilities,
consider the following:
• Organizations have multiple EBRs that satisfy a number of business
requirements.
• Each relationship generates risks.
• It is management’s responsibility to manage these risks and achieve the
benefits of the relationship.
• Internal auditing plays an important role in helping management and
validating their efforts.
Internal audits of EBRs range from an audit of a single contract or
relationship to an audit of an overall process that includes some
organizational processes and some EBR processes. Audits of EBRs often
take the form of contract assurance. A contract is an agreement between
parties, with terms and conditions that describe the agreement and constitute
a legal obligation.
Objectives
The IIA Practice Guide on EBRs states:
Internal auditors need to understand all the elements associated with EBRs, from
initiating a relationship, contracting and defining a relationship, procurement,
managing and monitoring the continued relationship (including control environment
considerations of objectivity and independence of those responsible for managing and
monitoring), and finally discontinuing the relationship. After understanding the
expectations of both parties, along with the appropriate processes to manage and
monitor the relationship, the internal auditor develops an appropriate internal audit
program with relevant audit objectives for internal audits of external relationships. In
addition, internal audit procedures may include elements of evaluating adherence to
(and compliance with) contractual terms to determine whether monetary and nonmonetary obligations are met.
Third and independent parties may audit the organization for the same
purposes.
Stakeholders
Stakeholders for audits of third parties include the organization and the third
party as well as individuals responsible for approving and signing contracts.
Those responsible for managing the EBRs and monitoring and enforcing
contract compliance need to be specified in each organization. Other
stakeholders include internal auditors, legal counsel, and individuals
responsible for providing inputs to or receiving outputs from the third party
under contract. Courts of law may be considered another stakeholder, since
this is the ultimate forum for interpreting contract language and compliance.
Risks
General EBR Risks
Risks for external business relationships include all of the risks of the
business process that is being out-sourced, since the end result is still the
organization’s responsibility. The organization will be held responsible for
the actions of its partners and perhaps even for the partners of those
partners (i.e., the third tier in the supply chain). Contracts can help transfer
some of this risk, but other risks, such as reputation risk, cannot be
transferred.
From a supplier’s perspective, there is a risk that the buyer could attempt to
create an unfair business relationship in contracts, which is more likely
when working with large organizations with strong purchasing power.
Unfair treatment of vendors may provide only a short-term gain for the
buyer because that supplier may not be able to sustain operations if it is
unprofitable. This can create supply instability.
Organizations monitor and manage EBR risks, and failure to do so properly
is another risk. Other risks revolve around the process of finding the most
appropriate partners, establishing controls over partners and contract
management, contract compliance auditing, and customer and supplier
relationship management. These are the risks of having ineffective,
inefficient, or negative business relationships. For example, an EBR could
violate laws or regulations or misrepresent organizational values. Internal
auditors can perform due diligence audits at the start of a relationship to
determine the risks of the EBR misrepresenting the organization’s values.
Another risk is that not all EBRs are formally arranged and documented.
For example, a procurement professional could have a relationship with an
unofficial supplier that weakens the official purchasing contract
relationships. Poor partner accounting or reporting is also a risk; this could
impact the organization’s required accounting (e.g., there could be
uncollected revenues) and reporting (e.g., the organization could be unable
to verify if a certain toxic substance is found in supplier subcomponents).
Internal auditors also have a role to play in verifying that the EBR has
sufficient and effective insurance to address insurable risks. This may
include workers’ compensation, liability to the public or of professionals,
and vehicle insurance. When partnerships are formed, there is a risk that not
all partners will be included in insurance coverage or that it may not be in
effect in certain countries. Another risk is that the actions of one partner
could void the insurance of another partner. Also, insurance could expire
after contracts are signed, so there may be an ongoing need to receive
insurance certificates as evidence of continued coverage.
EBRs may have conflicts of interest such as also working with a
competitor. A control for this risk is to ensure that the EBR reports any
actual, potential, or perceived conflicts of interest. Requiring such
disclosures may allow some relationships to continue if they are within the
organization’s risk tolerance level.
Intellectual property (IP) may also be at risk in any EBR relationship in
which the organization must share confidential information. Clear contracts
can reduce the risks of theft of IP or the associated revenue streams, but the
contracts may not be enforceable in some countries. Contracts can be
designed to share the risk of poor IP control with the EBR, such as a
mutual loss of revenue.
Contract-Specific EBR Risks
A major risk of contracts is the risk of lawsuits related to perceived contract
breach on the part of one party or the other. Major misunderstandings occur
when contracts are worded in a way that allows product or service
requirements to be interpreted differently by different parties. Lawsuits are
expensive, and even a successful outcome may be more costly than the
benefit gained. They can also result in significant delays or damage to
reputation.
Contracts are classified in a variety of ways; the following classifications
can be used to describe some inherent risks:
• Express and implied. An express contract is one in which the terms are
expressed verbally, either orally or in writing. Implied contracts are not
expressed in words. An informal verbal agreement can be as binding and
legally valid as a written contract. The risk is that an organization can be
found to have unwittingly entered into an express or implied contract.
• Bilateral and unilateral. In bilateral contracts, which are the most
common, both parties make a promise. In unilateral contracts, one party
makes a promise (such as an insurance or reward contract). Risks involve
being liable for the performance of promised work that is more costly than
the agreed-upon payment or that cannot be supplied, such as in the case of
a disaster; receiving products or services of unacceptable quality; or the
other party defaulting on or delaying delivery or payment. Other risks
specific to particular contract types are discussed later in this topic.
• Void, voidable, and unenforceable. Void contracts are considered never to
have come into existence (such as a contract based on an illegal purpose).
A voidable contract is one in which one of the parties has the option to
terminate it (such as a contract with a minor). An unenforceable contract
is one in which neither party may enforce the other’s obligations (if it
violates the statute of frauds, for example). The risks here involve
developing a contract that is void or unenforceable. One control for this
risk involves including contract language to the effect that if one element
is found to be unenforceable, the rest of the contract remains in force.
(Legal wording will differ.) Voidable contracts should be entered into
knowingly and willingly rather than being a loophole.
Audit Team
It is important that the audit team possess some understanding of the EBR
organization to be audited, including its operating environment, business
model, organizational structure, strategic goals, risks, and key controls. Onsite audit staff may be needed to achieve this understanding. The CAE may
decide to rely on the work of the EBR organization’s internal auditors if he
or she determines that they are independent, objective, competent, and on
topic. For contract audits, the audit team—not surprisingly—needs
individuals with some experience auditing contracts. However, since this is
an increasingly common requirement for internal auditors, it is likely that
the staff will have this experience. Including a team member with extensive
legal experience may not be necessary if the team has access to or includes
a lawyer to provide examples of standardized contract language, perform
legal reviews, and give advice as needed.
Audit Methods
Audit methods include audits of certification to standards and audits of
contracts.
Audits of Certification to Standards
Audits of certification to standards ensure that partners have the proper
quality controls, corporate social responsibility policies, or other standards.
Certification requires testing by an accredited third-party testing
organization. Internal auditors can provide assurance but not certification.
The American Institute of Certified Public Accountants (AICPA) published
its Statement on Standards for Attestation Engagements (SSAE) No. 18,
Attestation Standards: Clarification and Recodification, to provide
consolidated guidance for independent audits for certification to standards.
This became effective on May 1, 2017. Note that SSAE 18 supersedes all
prior SSAEs except AT 701 and AT 501; this includes SSAE 16 (which
itself superseded SAS 70).
SSAE 18 was developed as part of the AICPA’s System and Organization
Controls (SOC) related examinations and report guidance for service
organizations (the provider of a contracted service) and for use by user
organizations (the customer of a contracted service).
SSAE 18 is widely recognized as authoritative guidance that allows service
organizations to disclose their control activities and processes to their
customers and their customers’ auditors in a uniform reporting format. In
other words, the organization contracts with an independent accounting and
auditing firm to perform an audit in accordance with SSAE 18 and is able
to produce the certification document for multiple parties that want
assurance rather than being audited by all of them. Service organizations
issue their reports as a SOC 1 report under the SSAE 18 guidance, while
user organizations can just request the latest SOC 1 report from their
service provider and, if it was prepared after May 1, 2017, the user
organization can have assurance that it was prepared in accordance with
SSAE 18. Note that a SOC 1 report is concerned primarily with internal
controls over financial reporting; there are also SOC 2 and SOC 3 reports
that focus on data center security and privacy protections.
Another type of independent audit is for certification to standards developed
by organizations such as the International Standards Organization. These
audits are performed by registered auditors whose task is to ensure that the
organization conforms to the relevant standards, such as ISO 9000 or ISO
14000.
Audits of Contracts
Organizations use contracts for many purposes, from simple purchase orders
to capital construction projects or out-sourcing a non-core-competency
service such as human resources. Evaluating the soundness of these
contracts from cost and contract compliance standpoints is an increasingly
important aspect of an internal auditor’s job. Appropriate contracts will help
ensure that an organization successfully meets its strategic objectives and
avoid the risks associated with excessive costs, project delays, and quality
issues.
A valid contract typically requires the following elements:
• Mutual agreement—There must be an express or implied agreement with
evidence that the parties understand and agree to the details, rights, and
obligations of the contract.
• Consideration—The parties exchange something of value (cash, goods, or
a promise to do something).
• Competent parties—The parties must have the capacity to understand the
terms of the contract. Minors and mentally disabled people do not have
this capacity.
• Proper subject matter—The contract must have a lawful purpose.
• Mutual right to remedy—Both parties must have an equal right to remedy
a breach of terms by the other party.
While even a verbal contract can be enforceable, ensuring that these
elements exist reduces the risk of a contract being successfully contested.
Rather than auditing every page of a contract (legal language can get quite
lengthy), a best practice is to begin by determining audit objectives and
then to search the contract for expected clauses and details related to those
objectives. A common factor in most contract types is to look for instances
of poor control over costs for people, material, equipment, and supplies.
Control over results is always important. Control over processes and
methods could be warranted, such as in a corporate social responsibility
enforcement contract.
Contracts fall into general categories. The following types are discussed
next.
Fixed-Price (Lump-Sum) Contracts
A fixed-price contract (lump-sum contract) requires a contractor to
successfully perform the contract and deliver supplies or services for a price
agreed to up front. A firm fixed-price contract is appropriate when
goods/services can be described in sufficient detail to ensure that both
parties fully understand the contract requirements and inherent performance
risks.
Fixed-price contracts often include methods of reducing risks:
• Economic price adjustment factors to allow for volatile market prices
• Escalation clauses to increase prices per a schedule or against an economic
index
• Re-pricing provisions to permit fixed-price orders (However,
reasonableness checks should be performed later.)
• Incentives for good performance or penalties for poor performance
• A specified level of effort
Fixed-price contracts are commonly used if the work required is
uncomplicated. If completed as agreed upon, there is little reason for an
audit of the contract. If these contracts have a change of scope and
additional expenses while the work is occurring (who bears the risk of these
additional expenses must be specified in the contract), an audit may be
warranted.
The major risk of using fixed-price contracts is of receiving inferior-quality
goods or services. Expectations of quality need to be explicit in the form of
acceptance criteria or specific materials to be used, or the contractor could
substitute materials of lower quality. Fixed-price contract audit review areas
also include:
• Charges for equipment not received or activities not completed.
• Escalation clauses or re-pricing provisions.
• Authorization for extras, revisions, or change orders.
• Overhead expenses charged separately.
• Certification of completion before work has actually been completed.
• Inadequate inspection relative to specifications or inadequate completion.
Cost Reimbursement (Cost-Plus) Contracts
A cost reimbursement contract (cost-plus contract) is a way of handling
pricing difficulties when there are numerous unknown factors. It is
appropriate when the uncertainties of performance will not permit a fixed
price to be estimated with sufficient accuracy. In a cost reimbursement
contract, the contractor is reimbursed for costs above what was specified.
Additional costs are usually based on the initial costs plus a fixed fee or a
fee based on a percentage of costs. In the latter type, a risk is that there is
usually an incentive for the contractor to escalate costs.
This type of contract places the least cost and performance risk on the
contractor and requires the contractor’s “best efforts” to complete the
contract. Significant risks of cost reimbursement contracts include being
charged over the market value or for goods that were not actually delivered.
Common controls for these risks are to set a predetermined ceiling on costs
that can be allocated to the contract, are allowable within cost standards,
and are reasonable. This cost ceiling is a key control that should be audited
to ensure that such contracts cannot be used to overcharge or underdeliver
to the organization.
Other risks for auditors to consider when auditing cost reimbursement
contracts include:
• Direct billing of overhead costs.
• Inadequate cost controls on the contractor’s part and no effort to obtain
best prices.
• Unreasonable charges for contractor-owned equipment or idle rented
equipment.
• Excessive hiring, poor work practices (e.g., absences, excessive overtime).
• Excess billing over contractor costs.
• Failure to pass along discounts, refunds, salvage, etc.
• Duplication of effort between headquarters and field offices.
• Inadequate job site supervision, inspection, follow-up from headquarters,
etc.
• Unreliable cost accounting (e.g., billing supervision as labor in violation of
contract).
• Extravagant use or early arrival of material and supplies.
• Quality or grade issues: excessively high or low standards for materials or
equipment.
• Poor physical protection of materials or equipment.
Unit-Price Contracts
In unit-price contracts, a price per unit of work is agreed upon. These
contracts are best for a large number of identical products or services. Total
cost is the per-unit price times the number of units (e.g., number of
brochures printed).
The following risks are important for the auditor to consider:
• Excessive progress payments
• Improper reporting of units completed
• Prices unrelated to actual costs or improper extension or escalation of unit
prices
• Improper changes to the original contract
Joint Venture Contracts
Joint venture contracts are often based on cost-, revenue-, or profit-sharing
or profit-and-loss-sharing arrangements. The engagement objective is often
to evaluate compliance with financial and nonfinancial terms and conditions.
Financial terms may include:
• Reliability of cost allocation and billing systems and data.
• Reliability of revenue management and distribution.
Nonfinancial terms may include:
• Safeguarding of assets, including information, reputations, and brands.
• Proper governance and compliance with laws, regulations, and contractual
obligations with third parties such as corporate social responsibility
policies and procedures.
• Reliability of nonfinancial information.
• Reasonableness of budgets and forecasts.
Additional Contract Types
Additional contract types include:
• Time and materials—Fixed rate for services; materials at cost plus a
handling fee.
• Letter contracts—A preliminary instrument letting a contractor begin work
prior to contract finalization (only for circumstances of unusual and
compelling urgency).
• Indefinite delivery contracts—A delivery or task order initiates delivery.
Controls
The following items may be sources of risks, weaknesses, and control
issues:
• Price competition, price analysis, and cost analysis
• Type, complexity, and urgency of contract
• Contract period, payment terms, cost sharing, and delivery dates
• Property rights
• Adequacy of contractor’s technical capability and financial responsibility
• Performance measurement methodology, project reporting, and work
acceptance
• Change orders, cost accounting, billing, insurance coverage, audit rights,
substantial completion, and dissolution terms
• Regulatory compliance
• Documentation requirements
The type of contract is important, and internal auditors should review the
basis for selection and advise management on any inconsistencies with
organizational strategies and objectives. However, contract clauses largely
determine project or service outcomes.
A primary set of clauses, often called the scope of services or the scope of
work, spells out what the contractor is expected to deliver. Any undefined
detail could be exploited. A vaguely worded contract clause such as “for
consulting services” can add scope creep risk (unwanted billable work).
Properly documenting invoices received from the EBR partner to show what
requests are in and out of scope is one control. The scope may also set for
acceptable minimum quality levels and maximum cost. Scope and quality
should be defined and measured using acceptance criteria that both parties
know in advance. A critical control is ensuring that a specific individual
tests final deliverables against these acceptance criteria.
Lack of a response prior to an acceptance deadline could indicate automatic
acceptance. Warranties and guarantees may also expire, and product/service
audits need to be timely enough to ensure the ability to exercise such rights
if needed. Customer contracts also need to specify the same dates to limit
rework, repair, return, and replacement liabilities.
Contracts may need to specify a clear duty to report interim KPIs on a set
schedule, especially if linked to compensation payments and specific
deliverables. Contracts that authorize periodic payments without timely KPI
reporting and hand-off of specific deliverables create risks of overcharges or
missed deadlines with no legal recourse.
Auditing other contract clauses involves determining existence and
completeness based on audit objectives. For example, a right-to-audit clause
with a business partner is needed to allow internal auditors to audit the
third party’s documentation; for completeness, the clause may need language
to enable addressing broader relationship risks plus the right to audit books
and records. To audit for completeness, internal auditors compare the
language of a contract clause against a representative source document that
contains the official wording of the clause as reviewed and approved by
legal counsel. Any variations from the standard wording may need
justification and legal review. These reviews may address compliance with
laws and regulations, ethical standards, and organizational values.
Complex contracts go through multiple versions tracked by a contract date,
and proper version control is vital, including auditing the most current
version and ensuring that relevant parties are using this version. To ensure
validity, dates for deadlines in contracts must be reviewed, including when
the contract becomes and ceases to be in force, especially if there are
delays in contract signing. Specific verbiage outlining formal authorization
signature requirements by all parties can help prevent unauthorized changes.
Finally, contract termination may be allowed for only poor performance.
This creates the risk of needing to remain in an undesirable arrangement. To
avoid this possibility, organizations can specify a “right to terminate for
convenience,” allowing either party to terminate the contract without cause,
given sufficient notice. Clear time frames for what “sufficient notice” means
must be included, as well as the means of confirming that notice has been
received. Internal auditors can recommend a termination-for-convenience
clause for certain contracts, but only if the desire to be able to end the
contract is greater than the business need for that contract to remain in
force until it expires.
Recommendations
Assurance engagements for business relationships or contracts may reveal
unfair business practices, fraud, ineffective controls, ineffective monitoring,
or noncompliance risks. Recommendations reflect constructive changes to
increase fairness and controls and limit risks. Recommendations may apply
to a contract currently in force, which may require renegotiation for
changes, and/or to future contracts in general, which may involve changing
standardized templates, policies, and procedures. In either case, getting legal
review and approval of recommended changes is a critical control step.
Audit reports for EBR often have two formats:
• The audit engagement report to senior management and the board,
following the guidance provided in the IPPF
• A report to the organization being audited (The CAE can develop
appropriate reporting protocols for such reviews, in consultation with
management, since management is responsible for handling the business
relationship.)
Privacy Audit Engagements
The issues of privacy and the protection of private information are critical
to effective governance, risk management, and control and as such require a
proactive, future-impact oriented approach as addressed by Standard 2100:
The internal audit activity must evaluate and contribute to the improvement of the
organization’s governance, risk management, and control processes using a
systematic, disciplined, and risk-based approach. Internal audit credibility and value
are enhanced when auditors are proactive and their evaluations offer new insights
and consider future impact.
As noted in the Practice Guide “Auditing Privacy Risks,” second edition,
privacy can mean many things to many people.
Privacy definitions in the business environment vary widely depending upon the
country, culture, political environment, and legal framework. . . . Simply defining
privacy as the protection of the collection, storage, processing, dissemination, and
destruction of personal information, the many definitions of privacy can be used by
any organization to guide its privacy program.
Personal information generally refers to information that is associated with a
specific individual or that has identifying characteristics that, when
combined with other information, can be associated with a specific
individual. It can include any factual or subjective information—recorded or
not—in any form of media. Personal information could include:
• Names, addresses, identification numbers, family relationships.
• Employee files, evaluations, comments, social status, or disciplinary
actions.
• Credit records, income, financial status.
• Medical status.
Objectives
There are some overall expectations for performing a privacy audit. The
Practice Guide states that:
Auditing the organization’s privacy practices involves risk assessment, engagement
planning and performance, and communication of results. However, there are
additional aspects the CAE should take into account, including possible privacy
breaches, staff management and record retention issues, and privacy assessments
performed by other assurance providers. Many of these aspects are covered by
practices of the internal and external audit professions.
The Practice Guide continues by stating that:
An organization’s governing body is responsible for deciding the risk it is willing to
take and to ensure that resources are in place to manage risk according to that
appetite. Addressing privacy risks includes establishing an appropriate privacy
framework consisting of policies, procedures, and controls. Internal audit can
evaluate that framework, identify significant risks, and make appropriate
recommendations to enhance the privacy framework.
Stakeholders
Stakeholders for privacy issues start with the board and its audit committee.
These parties are responsible for oversight of all aspects of privacy relevant
to the organization. Next, senior and operating management are ultimately
accountable for assessing risks and implementing privacy controls. Other
stakeholders include anyone who could be affected by a breach in privacy
controls, including customers, employees, the organization, and business
partners. Other oversight groups may exist.
Risks
Standard 2120, “Risk Management,” states that “the internal audit activity
must evaluate the effectiveness and contribute to the improvement of risk
management processes.”
Protection of privacy is a very serious risk management issue for
organizations. The failure to protect personal information with appropriate
controls can have significant consequences. The reputation of individuals
and/or the organization could be damaged, and the organization could be
exposed to legal liability and diminished consumer and/or employee trust.
Risks associated with the privacy of information encompass personal
privacy (physical and psychological); privacy of space (freedom from
surveillance); privacy of communication (freedom from monitoring); and
privacy of information (collection, use, and disclosure of personal
information by others).
Privacy risks may also include impairment of the organization’s brand and
public image, followed by potential losses of market share and customers,
leading to potential losses for investors or the organization. Also, in
addition to legal liability, regulators could impose sanctions and they or
other parties could allege that the organization uses deceptive practices.
Organizations could lose the trust of customers, employees, or society or
damage relationships with business partners.
Audit Team
Internal auditors who conduct privacy audits need training in the highly
technical and legal nature of privacy issues.
Audit Methods
When conducting an evaluation of an organization’s privacy framework, the
internal auditor:
• Works with in-house legal counsel to determine the laws, regulations, and
other standards and practices relating to privacy that are applicable to the
organization and the countries in which it operates.
• Works with information technology specialists to determine that
information security and data protection controls are in place and regularly
reviewed and assessed for appropriateness.
An important step in evaluating an organization’s privacy framework is to
determine the maturity level of the organization. This is often done using a
Capability Maturity Model (CMM). As described in Privacy: Assessing the
Risk by Hargraves, Lione, Shackelford, and Tilton, there are five levels of
an organization’s maturity concerning privacy protection:
• Level 1—Initial. Organizations at this level have no defined policies or
procedures that control the security of private information. There is a lack
of interest in or involvement by management and employees.
• Level 2—Repeatable. These organizations have a defined policy and at
least some commitment on the part of management and employees.
• Level 3—Defined. A privacy policy has been established and senior
management is committed to it. Risk assessments, priorities, resource
allocation, and activities have been performed or developed to ensure
consistent privacy controls.
• Level 4—Managed. Privacy requirements and controls are an integral part
of the organization’s framework. There is commitment by all parties in the
organization.
• Level 5—Optimizing. Continuous improvement is achieved through
monitoring of the privacy framework. Any corrective action is addressed
by all relevant parties in the organization, and no change is made without
extensive coordination.
Depending on the level, the internal auditor may have differing roles. The
auditor may facilitate the development and implementation of the privacy
program. (Note, however, that if the internal auditor assumes responsibility
for developing and implementing a privacy program, his or her
independence will be impaired.) The auditor may evaluate management’s
privacy risk assessment to determine the needs and risk exposures of the
organization or provide assurance on the effectiveness of the privacy
policies, practices, and controls across the organization. The internal auditor
can help the organization to achieve the next level of maturity by
identifying significant risks and making recommendations to mitigate them.
Additional audit procedures include working with privacy professionals to
help understand internal and customer-oriented privacy policies and
organizational maturity in these areas. When working with IT specialists,
audit areas may include system and access controls, information flows,
storage controls, and incident response programs.
According to the “Auditing Privacy Risks” Practice Guide, privacy audits
may also include the following topics:
• Governance/management oversight
• Privacy policies and controls
• Applicable privacy notices
• Types and appropriateness of information collected
• Systems that process, store, and transmit personal information
• Collection methodologies
• Consent and opt-in/opt-out management
• Use of personal information for compliance with stated intent, applicable
laws, and other regulations
• Security practices, operations, and technical controls in place to protect
personal information
• Retention and disposal practices for personal information
Controls
The Practice Guide states that effective control over the protection of
personal information is an essential component of the governance, risk
management, and control processes of an organization. The board is
ultimately accountable for identifying the principal risks to the organization
and implementing appropriate control processes to mitigate those risks. This
includes establishing the necessary privacy framework for the organization
and monitoring its implementation.
While there are many privacy frameworks an organization can adopt, a best
practice is to adopt a framework that is principles-based and that balances
the individual right to privacy against the legitimate information rights of
the organization. A principles-based framework usually starts with a set of
generally accepted privacy principles and works to make them easier to
understand and implement. For example, the AICPA and the Canadian
Institute of Chartered Accountants has issued a set of ten Generally
Accepted Privacy Principles (GAPP), as shown in Exhibit I-11.
Exhibit I-11: AICPA/CICA Generally Accepted Privacy Principles
AICPA/CICA Generally Accepted Privacy Principles
Management
Privacy policies and procedures are defined, documented, communicated, and
accountable.
Notice
Policies, procedures, and purpose of collection, use, retention, and disclosure are
provided in a notice.
Choice and
consent
Individuals are told available choices, and the organization gets explicit or implicit
consent to collect, use, and disclose personal information.
Collection
Collection is limited to the purpose identified in the notice.
Use,
retention,
and disposal
Use is limited to the purposes in the notice. Retention is limited to the need stated
in the notice or as required by law.
Access
Individuals can access and correct personal information.
Disclosure to
third parties
Disclosure is limited to parties in the notice.
Security for
privacy
Physical and logical access controls exist.
Quality
Personal information is maintained, complete, and accurate.
Monitoring
and
enforcement
Compliance and complaint resolution systems exist.
The privacy framework should also enable easier compliance with privacyrelated laws, regulations, and policies. An internal auditor must be aware of
and up-to-date on such laws, regulations, and policies in jurisdictions in
which the organization operates. The following are some examples:
• The European Union’s Global Data Protection Regulation (GDPR) (This
binding regulation became effective on May 25, 2018.)
• The European Union’s E-Privacy Directive (This and the GDPR are
complementary, and they are both administered by the same regulatory
body. The GDPR is related to Article 8 of the European Charter of
Human Rights on personal data protection; the E-Privacy Directive is
related to Article 7 of the same charter dealing with a person’s private
life.)
• The OECD Guidelines (of the Organisation for Economic Co-operation
and Development)
• The Canadian Personal Information Protection and Electronic Documents
Act
• The U.S. Health Insurance Portability and Accountability Act of 1996
(with modifications to its Privacy Rule which were published in 2002)
• The U.S. Gramm-Leach-Bliley Act (GLBA) of 1999 (requires financial or
insurance organizations to explain their information-sharing practices and
protect sensitive data)
• Japan’s Personal Information Protection Act of 2003 (amended in 2015
and fully in force as of September 2017)
• Australia’s Privacy Act of 1988 (amended in 2015 and fully in force as of
2014)
Note that some of the above directives or acts contain their own versions of
generally accepted privacy principles, and most are very similar to the
AICPA/CICA principles listed above. The internal auditor should work with
either in-house or external counsel to determine the exact nature of such
laws or regulations and whether they impact the organization or expose it to
risks.
Recommendations
Internal auditors can provide value in the area of privacy by describing risk
and control implications and potential losses from risk exposures, quantified
in monetary terms when this is possible. For example, in a privacy
assurance engagement, observed control weaknesses could include
inadequate access controls for sensitive data; excessive collection, sharing,
disclosure, or retention of data; incomplete, outdated, or damaged data; poor
or incorrect data processing; or inappropriate use of data.
Quality Audit Engagements
Achieving and continually improving quality is a significant endeavor for an
organization. Auditors measure an organization’s current operations against a
set of standards or other criteria. Essentially, they assess the quality of the
organization’s controls and determine if controls are being updated and
enhanced as organizational activities and industry practices change and as
technology is enhanced over time.
Objectives
The objective of a quality audit engagement is to help an organization
improve its quality and productivity by providing assurance that the
organization’s quality and continuous improvement plans are such that, if
followed, the desired quality will be attained. Quality needs to start at the
beginning of a process and needs to be considered throughout, until
completion. This may take the form of conformance to a methodology such
as total quality management (TQM). The term total quality
management was first used by the U.S. Naval Air Systems Command to
describe the Japanese-style management approach to improving quality. The
military was looking for a systematic method that would provide the ability
to improve overall quality and comprehensively identify areas for corrective
action.
A related objective is to provide assurance that the internal audit function is
functioning at the desired quality level for the board and management, as
was discussed in Part 1, Section IV, on quality assurance and improvement
programs.
A quality audit provides management with the necessary information to:
• Recognize actual or potential risks.
• Make appropriate decisions so the costs of quality can be prevented or
rectified.
• Identify areas of opportunity for continuous improvement.
• Assess the quality of staff training.
• Verify compliance with the organization’s processes and procedures as well
as any regulatory or legal requirements.
• Justify the expenditures on quality activities by assessing the actual
savings achieved (which are more difficult to identify and track).
• Eliminate outdated activities and unnecessary controls.
The organization uses this information for continuous improvement of its
controls and standards.
Stakeholders
As with TQM, all members of the organization must be involved for the
organization to achieve long-term success, customer satisfaction, and
positive benefits for its members and society at large. However, specific
persons should be responsible and accountable for specific quality aspects.
Management and the board are ultimately responsible for control and
oversight.
Risks
The risks of poor quality are often called the costs of quality. Costs of
quality are the activities associated with the prevention, identification,
repair, and rectification of poor quality. They also include the opportunity
costs from lost production time and lost sales as a result of poor quality.
Exhibit I-12 lists some of the most common costs of quality.
Exhibit I-12: Costs of Quality
Quality Cost Type
Quality Cost Components
Prevention costs (incurred to eliminate defective
products before they are produced)
• Reengineering and design
• Use of high-quality parts
• Improved processes
• Employee training
Appraisal costs (incurred to evaluate purchased
materials, processes, products, and services to
ensure conformance to specifications)
• Assessment and approval of suppliers of
products and services
• Inspecting and testing raw materials received
from suppliers and work-in-process (WIP)
inventory
• Inspecting WIP at vulnerable, high-risk points
in the production process
Internal failure costs (incurred when defects are
discovered before sending products to
customers)
• Handling and fixing products or disposing of
them
• Opportunity cost of not being able to sell
disposed-of products
External failure costs (incurred when customer
receives defective products)
• Cost of returns
• Warranty work
• Product liability claims
• Opportunity cost of lost sales from damaged
reputation
Audit Team
Internal auditors need formal training in the organization’s quality control
system, such as TQM or Six Sigma, so they can converse intelligently with
persons directly responsible for the quality methodology.
Audit Methods
Effective quality systems consist of checks and tests that provide a way to
identify needed corrective actions. The internal auditor should focus on the
organization’s procedures; these procedures must be defined, controlled,
communicated to all relevant parties, and followed by all appropriate
employees.
Controls
The auditor must ensure that controls and any relevant laws or regulations
are adhered to—that procedures are adequate and are being followed and
that there is conformance to specifications. In addition, data systems must
maintain and be able to convey accurate and adequate information on
quality for the organization.
If tasks such as these are followed, the organization will more easily be
able to identify deficiencies, required corrective actions, and opportunities
for continuous improvement.
Due Diligence Audits
Due diligence is the process of investigating a person, business, or financial
transaction to establish the value of the entity or transaction and the cost of
any associated liabilities. The investigation should identify the presence of
certain risks or confirm the absence of such risks. A due diligence audit
may refer to either an investigation of an entity/transaction or an audit of
the due diligence investigation process itself.
The most common situations for performing due diligence audits are those
involving financial activities (banking, securities, mergers or acquisitions),
real estate (property, structures), and intellectual property.
Objectives
Due diligence investigations are often undertaken by persons or
organizations when they are interested in acquiring another business or
property or are otherwise becoming involved in a financial transaction. The
results of the investigation are used to decide whether or not to purchase
real estate, enter into a lease, or enter into a business partnership, joint
venture, merger, consolidation, or other similar arrangement. Therefore,
common objectives of a due diligence audit are to ensure that a proposed
action such as a merger will enhance the value of the organization, will be
consistent with strategic objectives, and will avoid hidden liabilities.
Stakeholders
Stakeholders to a due diligence audit include the person or entity being
reviewed, because the process could reveal potentially damaging information
that should be handled confidentially so as not to cause harm to that
person’s or entity’s reputation. Other key stakeholders include the
management decision maker involved in the potential new relationship or
merger/acquisition, the board, the legal department, related business process
owners, and any consultants or advisors.
Risks
Whenever due diligence is discussed, “standard of” or “due care” is also
mentioned. Due care is the level of caution exercised when performing the
due diligence audit and reporting the results. Basically, did the internal
auditor do what any reasonable person would do? This concept is especially
important when the risk of civil litigation is involved. Such litigation could
occur if due diligence reveals information that the other party denies to be
true and claims is damaging to its livelihood or reputation (assuming the
information was disclosed in some way).
Other risks related to poor due diligence include harm to the organization’s
reputation if it is associated with an individual or entity that is later found
to have engaged in illegal or unethical activities that should have been
discovered earlier. The acquiring organization’s revenue and profits may be
harmed if the acquired organization has inflated its revenues and profits.
This allegedly occurred in November 2012 when Hewlett Packard (HP)
accused the organization it acquired, Autonomy, of inflating its revenues
and profits prior to the acquisition. HP has claimed that a U.S. $5 billion
write-down is required despite having conducted a due diligence audit with
300 team members. Autonomy has denied the allegations.
Audit Team
Internal auditors may need experience in the organization’s strategic goals
and objectives to assess whether these goals are likely to be better achieved
if a deal such as a facility purchase or a business acquisition goes through.
For a merger example, external auditors would need experience in firm
valuation. Merger and acquisition lawyers would be needed to look for risks
related to client conflicts, trademarks and patents, title searches, workplace
safety, and so on.
Audit Methods
A due diligence audit can be as complex as purchasing another business or
as simple as a home buyer performing a title search. Therefore, these audits
will have a broad range in scope. For example, methods used in audits of
mergers or acquisitions include pre-acquisition audits from the perspectives
of the buyer and the seller and post-acquisition audits:
• Pre-acquisition due diligence (buyer perspective). Due diligence audits
from the perspective of the buyer start by ensuring that a key executive is
sponsoring the acquisition and is promoting buy-in and the due diligence
process itself. Internal auditors often follow checklists for this complex
activity, including a controls checklist and an accounting checklist.
A controls checklist may include the following analyses related to the
potential merger or acquisition:
• Analysis of the internal control environment and risk appetite (actual
versus expected)
• Any significant inherent/residual risks that exceed the acquirer’s risk
appetite
• The cost of improving controls to address significant inherent/residual
risks
• Relative difficulty of integration with the candidate’s information
systems
An accounting checklist may include analysis of the following items:
• Earnings quality
• Cash flow
• Quality of assets
• Valuation of liabilities and potential for unrecorded or underestimated
liabilities
• How the organization accounts for any business losses
• Pro forma financial statements (analysis of whether financial projections
are likely to be achieved)
• Potential for fraud
Organizations use the results of these analyses to adjust the relative value
of the potential acquisition or merger. Added costs such as for improving
controls are accounted for as a cost of acquisition.
• Pre-acquisition due diligence (seller perspective). An organization selling a
business may want to conduct its own due diligence prior to attempting to
find a buyer, especially if the unit was recently acquired and is not
fulfilling its potential. The purpose of this type of audit is to avoid a
situation where a business is rejected by potential buyers after their own
due diligence and the business becomes difficult to sell at a fair price. In
general, organizations conducting this type of audit should adopt the
perspective of a buyer. Therefore, the methods are the same as previously
described from the buyer perspective. However, the internal auditor is
expected to provide management with any information that could cause
due diligence concerns as well as recommendations for remediation.
• Post-acquisition due diligence. Once the transaction is completed, it often
becomes possible to review more detailed performance and financial
records than was possible prior to the acquisition. Therefore, it is
important to perform a post-acquisition due diligence audit to check for
internal control weaknesses, financial reporting fraud, and actual versus
planned progress toward organizational objectives. This type of audit
should be conducted as soon as possible because it could reveal areas
where immediate action could be taken to correct the course or stop
losses.
Post-acquisition audits are run similarly to an audit of any business unit,
except that any observations indicating negative or unexpected results
(from expectations developed at the time of acquisition) should be closely
examined to determine how long these control deficiencies existed without
being addressed by the acquired organization’s management. This may
result in disciplinary actions.
Key stakeholders for this type of audit are a transition manager and
various experts in relevant business processes. The transition manager, a
financial expert who reports to the CFO, does not run the business unit
but works to achieve financial integration. The other experts also help with
integration of their business functions. Internal audit works with these
individuals to perform a due diligence audit of internal controls and
business process integration.
Controls
The entire spectrum of controls is considered in a due diligence audit of a
merger or acquisition, but key controls related to financial accounting
accuracy (especially business valuation) and prevention of fraud are the first
concern. IT controls and integration are also key concerns.
Recommendations and Response
The final report to management should focus only on facts and maintain a
neutral, objective tone. In an article in Internal Auditor, “The Art of
Coordination,” Charles Zhang provides some suggestions for avoiding
subjectivity in the report:
• Include an executive summary with bullet points highlighting aspects that
could favor the negotiation of a better deal.
• Structure the report by cycles of business as defined by the acquiring
organization. For example, categories could include finance and
administration, sales and marketing, human resources, management,
purchasing, production, and treasury.
• Index all supporting documents and workpapers.
A number of different formats can be used for final reports of due diligence
engagements. The key is determining how to convey the information and
conclusions in a way that will best communicate the results.
Once the report is presented to senior management, an organization can
decide if they want to:
• Continue with the deal—No problems or liabilities have been discovered.
• Revalue the transaction—The price is adjusted, usually downward, based
on the internal auditor’s findings.
• Correct the problem—There is a way to resolve an issue before the final
deal is concluded.
• Cancel the deal—There is no remedy for the problems or liabilities, or the
remedy may be too costly.
Control Self-Assessment (CSA)
In 1987, the internal audit team for Gulf Canada Resources Ltd. developed
an approach to auditing called control self-assessment (CSA), now also
referred to as control/risk self-assessment, or CRSA. This experiment
revealed that a broader approach, based upon employee self-assessment
workshops that were facilitated by senior internal auditors, resulted in more
honest disclosures concerning significant factors affecting the success or
failure of the organization. Further, the workshops created a sense of
commitment to improving performance.
Sawyer’s Internal Auditing defines CSA as:
A process whereby employee teams and management, at local and at executive
levels, continuously maintain awareness of all material factors affecting the likelihood
of achieving the organization’s objectives, thereby enabling them to make appropriate
adjustments. To promote independence, objectivity, and quality within the process, as
well as effective governance, it is desirable that internal auditors are involved in the
process and that they independently report results to senior management and board
committees.
CSA is a useful and efficient approach for managers and internal auditors to
use in collaborating to assess and evaluate control procedures. In its purest
form, CSA integrates business objectives and risks with control processes.
However, keep in mind that the assessment will be only as good as the
evaluators’ transparency and commitment levels make it.
Objectives
Although CSA practitioners use a number of differing techniques and
formats, most programs share some key features and objectives. An
organization that uses self-assessment will have a formal, documented
process that allows management and work teams who are directly involved
in a business unit, function, or process to participate in a structured manner
for the purpose of:
• Identifying risks and exposures.
• Assessing the control processes that mitigate or manage those risks.
• Developing action plans to reduce risks to acceptable levels.
• Determining the likelihood of achieving the business objectives.
Stakeholders
Stakeholders to a CSA include the board and senior management in their
monitoring and oversight roles; internal auditors in facilitation, oversight,
analysis, and interpretation roles; and the managers and work teams directly
involved in a business unit, function, or process to be audited using CSA.
These latter stakeholders participate directly in the CSA process. Other
stakeholders may include persons or functions that provide inputs to the
process, those affected by the process, or those affected by or who use its
outputs, which could include other internal users or external customers,
regulators, and society.
Risks
Risks related to CSA include that the process could be unlikely to reveal
significant issues because of a lack of commitment or transparency/honesty
on the part of persons participating in the assessment. Participants who lack
commitment might fail to contribute in a meaningful way, such as by failing
to fill out open answer fields in a questionnaire or survey. An example of a
transparency or honesty issue is participants having some form of bias, such
as an overly positive view of an area or process, and therefore describing
how a process or control is supposed to be done rather than how it is
actually being done. Therefore, the CSA process may itself be the subject of
an audit, and a distinction should be made between an audit of the CSA
process and the use of CSA as an audit tool within an engagement. For
example, an audit of the CSA process could reveal ways to structure
questionnaires so that they reduce the risks of incomplete information or
bias on the part of respondents. An audit that uses CSA as a tool may
include the risk that this is not the most efficient or effective tool that could
be included in the audit approach. For example, perhaps a walkthrough
would be a more effective audit procedure.
Audit Team
The audit team should include auditors who can be objective regarding the
area in question and who are experienced in facilitation and analysis of risk
and control. The internal auditor’s role in CSA varies widely between two
extremes:
• Intense involvement, with internal audit undertaking to sponsor, design,
implement, and effectively own the process; conduct training; supply
facilitators, scribes, and reporters; and orchestrate participation of
management and work teams
• Minimal involvement, with internal audit serving as an interested party,
consultant, and verifier of the team’s evaluations
Depending on the level of involvement, auditors should be alert to anything
that will affect their objectivity. Standard 1120 notes, “Internal auditors must
have an impartial, unbiased attitude and avoid any conflict of interest.”
In most programs, internal audit’s investment in the organization’s CSA
efforts is somewhere between the two extremes described above. As the
level of internal audit’s involvement in the CSA program and individual
workshop deliberations increases, the CAE should monitor the objectivity of
the internal audit staff, take steps to manage that objectivity (if necessary),
and augment internal audit testing to ensure that bias or partiality do not
affect the final judgments of the staff.
Using a CSA in addition to traditional audit methods and practices does not
change the responsibilities themselves. Rather, it shifts some of the
responsibilities to other personnel such as client work teams.
Exhibit I-13 illustrates some of the differences between CSA roles and
responsibilities and a traditional internal audit approach.
Exhibit I-13: Roles and Responsibilities
Traditional
Approach
CSA
Approach
Setting business objectives
Management
Management
Assessing risks
Management
Management
Adequacy of internal controls
Management
Management
Evaluating risks and controls
Auditors
Work teams
Reporting
Auditors
Work teams
Validating evaluation of risks and controls
Auditors
Auditors
Objectives used
Audit’s
Management’s
Responsibilities
Source: Control Self-Assessment: A Practical Guide by Larry Hubbard.
Audit Methods
Several principles underlie valid approaches to control self-assessment:
• “Control” is a broad framework that integrates all the factors that bear
upon achievement of an organization’s objectives, with the people in an
organization being the most significant factor. This perception parallels the
findings of the Treadway Commission and the philosophical underpinnings
of the COSO Internal Control—Integrated Framework.
• The auditor cannot adequately assess such broad-based controls alone.
CSA is unique among audit approaches in assuming that an adequate
assessment of controls requires contributions from all who perform the
relevant tasks. This is a major departure from traditional practice.
• Measurement is a necessary but not sufficient basis for control assessment.
In the end, judgment has the final say in assessment, not statistical
precision.
Although there are many approaches to control self-assessment,
organizations often use the COSO or CoCo (Criteria of Control) models for
CSA training sessions and workshops.
Three major methods are especially significant:
Facilitated Team Workshop Approach
Facilitated team workshops gather information from work teams
representing different levels of the business unit or function. Facilitation
may be provided either by the client or by internal audit staff.
Categories of workshops include the following:
• Objective-based workshops focus on the best way to accomplish a
business objective. Teams identify controls currently being used and then
identify any residual or unmitigated risks. The goal is to determine if the
controls are working effectively.
• Risk-based workshops focus on identifying the risks of achieving an
objective. Teams identify the risks or roadblocks to success and then
determine if the controls are adequate for mitigating these risks.
• Control-based workshops focus on how well current controls are working.
In this approach, the facilitator identifies the controls and key risks, not
the participants. The goal is to determine if the controls are working the
way management intends for them to work.
• Process-based workshops focus on selected activities that are elements of a
process, usually a series of related activities with a beginning and an end,
such as the various steps in purchasing or product development. The goal
is to analyze, revise, or verify the effectiveness of a particular process.
Questionnaire Approach
The survey form of CSA uses a questionnaire that asks mostly simple
yes/no or have/have not questions that are carefully written to be understood
by the target recipients. Surveys are often used if the desired respondents
are too numerous or too widely dispersed to participate in a workshop.
They are also preferred if the culture in the organization may hinder open,
candid discussions in workshop settings or if management desires to
minimize the time spent and the costs incurred in gathering the information.
Control Self-Assessment: A Practical Guide by Larry Hubbard lists some
factors to consider when creating a questionnaire:
• Use the recipient’s language.
• Use one topic per question.
• Use words with clear meaning to the recipients.
• Ask easy-to-answer questions first.
• Keep the questionnaire short and simple.
• Address the questionnaire in a personal manner.
• Personally distribute and collect the survey.
• Use the questionnaire as a conversation tool in an interview.
Management-Produced Analyses Approach
This form of self-assessment covers most other approaches by management
groups to produce information about selected business processes, risk
management activities, and control procedures. The analysis is often
intended to reach an informed and timely judgment about specific
characteristics of control procedures and is commonly prepared by a team in
a staff or support role. The internal auditor may synthesize this analysis
with other information to enhance the understanding about controls and to
share the knowledge with managers in business or functional units as part
of the organization’s CSA program.
Hubbard provides some examples of management-produced analyses:
• A questionnaire developed and administered by management to support an
opinion about internal controls required by a law or regulation
• A discussion among senior financial management to support the annual
representation letter required by external accountants
• An investigation into the reasons why a particular control breakdown or
fraud occurred
• A review of the internal control implications of a new system being
developed or the combination of business units/organizations
Management-produced analyses are not used as often as workshops or
questionnaires in the CSA framework.
Controls
CSA can result in the following types of improvements to controls:
• People in business units become trained and experienced in assessing risks
and associating control processes with managing those risks and improving
the chances of achieving business objectives.
• Informal, “soft” controls are more easily identified and evaluated.
• People are motivated to take ownership of the control processes in their
units, and corrective actions taken by the work teams are often more
effective and timely.
• The entire objectives-risks-controls infrastructure of an organization is
subject to greater monitoring and continuous improvement.
• Internal auditors become involved in and knowledgeable about the selfassessment process by serving as facilitators, scribes, and reporters for the
work teams and as trainers of risk and control concepts supporting the
CSA program.
• The internal audit activity acquires more information about the control
processes within the organization and can leverage that additional
information in allocating its scarce resources. The result is an improved
focus on investigating and performing tests of business units or functions
that have significant control weaknesses or high residual risks.
• Management’s responsibility for the risk management and control
processes of the organization is reinforced, and managers will be less
tempted to abdicate those activities to specialists such as auditors.
• The primary role of the internal audit activity will continue to include the
validation of the evaluation process by performing tests and the expression
of its professional judgment on the adequacy and effectiveness of the
entire risk management and control system.
Topic D: Consulting Engagements (Level P)
Consulting involves internal auditors providing value-added advice and
suggestions related to future-oriented decisions upon management request.
Auditors can provide insight to decision makers as processes are being
developed so that the proper controls are built into a new project or process
from the start. Internal auditors may consult with teams working on
business process reengineering, new business development, post-investment
reviews, and similar projects, providing advice on risk management,
governance, and controls. So long as internal auditors make it clear that
they are not making or implementing any decisions themselves,
independence or objectivity is not compromised when they provide advice
or suggestions.
This topic examines several types of consulting engagements: internal
control training, systems development life cycle review, benchmarking, and
business process mapping. Privacy and due diligence engagements and
control self-assessment are revisited, this time from the perspective of
consulting engagements.
Internal Control Training
Internal controls are fundamental to risk mitigation. They are used to
standardize and guide operations and are designed to improve performance
in order to accomplish an organization’s goals and objectives. Internal
auditors are natural facilitators of internal control presentations and classes.
The auditors themselves should have ongoing training in controls.
From the perspective of COSO’s Internal Control—Integrated Framework,
internal control naturally involves everyone in an organization; therefore,
everyone (and the organization itself) can benefit from wider awareness of
control procedures and their importance.
Relationship of Training and the Engagement Process
Audits go more smoothly if those being interviewed during the engagement
are willing and cooperative participants. Providing audit clients with the
opportunity to attend a well-structured workshop on internal controls or the
COSO internal control framework can help them understand the importance
of audits of internal controls and, perhaps, make them more comfortable
with the process and more willing to provide useful, complete information.
COSO training would give audit clients a high-level understanding of the
five components of the framework:
• Control environment. Tone at and from the top is set, and the
organization’s control consciousness and the foundational framework upon
which the overall system of internal control is built and maintained are
established.
• Risk assessment. Risks are considered, identified, assessed, and prioritized
for all objectives.
• Control activities. Controls are in place to address identified risks.
• Information and communication. Data is available and discussed.
• Monitoring activities. Achievement of objectives is monitored.
Knowledge of these components and the related principles help clients to
understand the necessary management activities to be evaluated in making a
conclusion on the quality of internal controls.
Delivering this type of training helps audit clients better understand the
importance of internal control related to their job responsibilities, which
helps the organization achieve its objectives. The training also helps internal
audit activities to be understood and better received by the clients being
audited.
Systems Development Life Cycle Review
Organizations need to control information system resources as they do
human resources or any other precious organizational resource. For this
reason, the process of systems development has a unique methodology and
discipline.
Sawyer’s Internal Auditing notes that the systems development life cycle
(SDLC) should involve all stakeholders in the system. Stakeholders
encompass all those who have an organizational interest in the day-to-day
operations of the system. The auditor has significant responsibilities during
the SDLC:
• Ensuring that stakeholder interests are at the forefront of the development
objectives
• Ensuring that the development project follows the organization’s standards
for systems development
• Ensuring that the IT activity adheres to a framework or methodology such
as the SDLC
Exhibit I-14 shows a typical systems development life cycle. (Other models
exist.)
Exhibit I-14: Systems Development Life Cycle
Auditors could be involved in a design review at several places:
• During systems analysis as a project team member to define the goals or
purposes of a procedure or business function and identify ways to
accomplish those goals most efficiently, to evaluate the feasibility of
proposed systems (to determine if a project will add value and satisfy
objectives at a reasonable cost), or to evaluate the feasibility assessment
process itself
• During system design or system selection as a project team member to
ensure that controls are designed in
• During conversion and implementation to ensure that the project meets
objectives and acceptance criteria
• During feedback as part of a post-project design or acquisition review for
continuous improvement of the system and/or the process in general
Systems analysis is a key phase of the SDLC, and this is often where
internal auditors play a key role in a consulting engagement. Systems
analysis may involve applying problem-solving methodologies and a systemwide perspective and/or deconstructing the parts and subparts of the system
to gain an understanding of how the system or process works in detail. In a
different example, the internal auditor’s consulting role for a feasibility
study could include ensuring that:
• The team includes appropriate stakeholder representation.
• The team has sufficient hardware and software expertise.
• A thorough analysis of the preexisting manual or automated system is
conducted.
• Control deficiencies identified through audits are considered.
• Specifications for the new system consider anticipated growth, not just
current volume.
• The risk of fraud or the loss of control is considered.
• Users agree on the proposed system.
• Budget estimates are reasonable and supportable.
• Input and output requirements are clearly defined.
• Reasonable conversion plans are formulated.
• Proper written authorization is obtained for each phase of the SDLC.
Systems design is the process of defining the architecture, modules,
interfaces, and data for a system to satisfy the organization’s requirements
for the system. Internal auditors may be able to take their holistic view of
organizational processes and the overall goals of the process identified in
systems analysis to help ensure that systems design is comprehensive and
that the overall architecture or framework is sound. That is, systems design
can be seen as an extension of systems theory (understanding the system as
a whole, the cyclic nature of many processes, and the role of its inputs and
outputs) into the realm of product development.
A more detailed review of the components of the SDLC is presented in Part
3, Section III.
Due Diligence Engagements
Since due diligence was addressed in the prior topic, it is not covered in
detail here. Auditors could be involved in a review of the financial,
governance, risk management, control, and other aspects of a potential
acquisition to aid in the purchase decision. In a consultative capacity,
internal audit may provide advice and insight regarding a proposed
transaction’s contributions to the organization’s strategic objectives and the
transaction’s impact on ongoing core business activities.
Privacy Engagements
Like due diligence, privacy was addressed in detail in the previous topic. A
privacy consulting engagement will have considerable overlap with a
privacy assurance engagement. In today’s business environment, privacy
controls are legal and business requirements, and generally accepted policies
and practices are evolving. Internal audit can help keep the organization upto-date on the latest trends, regulations, and controls. Internal audit is also
uniquely positioned to help with the overarching issue of privacy because
auditors have a holistic perspective of the organization. They are aware of
the different regions and regulatory jurisdictions in which the organization
operates, and they understand the needs of the organization’s industry and
its business partners. Internal audit can leverage this perspective to provide
valuable advice to management and the board regarding the most
appropriate privacy framework and the most cost-effective investments in
privacy such as information systems security. When studying an
organization’s privacy framework, auditors should consider the following:
• Laws and regulations in all jurisdictions in which business is conducted
• Internal privacy policies and guidelines
• Privacy policies intended for customers and the public
• Working with in-house legal counsel to understand legal implications
• Working with information technology specialists and business process
owners to understand information security implications
• The maturity of the organization’s privacy controls
Control Self-Assessment (CSA)
Control self-assessment was primarily addressed in the prior topic in terms
of assurance engagements, but it can be conducted as a consulting
engagement as well. While the term “assessment” is generally associated
with assurance more than consulting, internal auditors may play a role in
these self-checks, primarily in the role of facilitators, which is a type of
consulting activity. Internal auditors may also be called on to provide
subject mater knowledge in the areas of internal controls and control-related
considerations.
Benchmarking
A benchmark is simply a goal that an organization (or person) aims to
achieve. In benchmarking, a benchmark is measured against an internal or
external group for the purpose of determining areas for potential
improvement and to identify best practices. Internal benchmarks include
historical data as well as goals and objectives; external benchmarks include
industry standards or best practices. Regulatory requirements are another
external standard of comparison.
Effective benchmarking depends upon the care and intelligence invested in
selecting the goal. A benchmark has little or no value if it can’t be
measured, can’t be reached, or can be reached too easily. Evaluating the
benchmarks set by clients within the organization is a service appropriate
for internal auditors to provide. Benchmarking is especially appropriate in
performance and quality audits. (Benchmarking is in fact associated with
total quality management.)
Benchmarking assumes that reaching the set benchmark will help the
organization grow, gain market share, improve customer satisfaction, or
otherwise achieve a significant goal. A department might benchmark the
performance of its employee rugby team, but that’s unlikely to be a
mission-critical matter.
Classifications of Benchmarking
There are several widely accepted ways of selecting benchmarks that are
measurable, precise, meaningful, and realistic. Exhibit I-15 describes
common classifications for benchmarking activities.
Exhibit I-15: Classifications of Benchmarking
Description
Example
Advantages
Historical comparison
of manufacturing
processes, services, or
types of reports:
Fairly easy to collect
data.
Disadvantages
Internal benchmarking
Comparing similar
information within
process or entity,
either achievable
performance above
current baseline or
stellar practice.
• Across divisions.
• Between different
locations.
• With another
function.
Focus is somewhat
limited and possibly
biased.
Competitive benchmarking
Comparing measures
with similar
measures of direct
competitors, locally,
nationally, or
worldwide.
Review of practices,
policies, procedures,
services, and
timeliness of market
share leader in hightech electronics
industry with goal of
surpassing that leader.
Comparisons with
organizations in same
industry typically yield
valuable, relevant
information.
• Data may be difficult
to collect; may
encounter resistance
with direct
competitors.
• Ethical/legal issues
may be concern.
Industry benchmarking
Comparing processes
to those with similar
processes in the
same industry.
Steel manufacturer
reviewing other steel
manufacturers to
determine average
spending on pollution
control equipment.
Likely to create
realistic benchmark.
Can be limiting or
unambitious.
Comparison of billing
and collection
procedures between a
private hospital and
government-operated
walk-in urgent care
clinic.
Typically little
resistance encountered
in partnering and
collecting data.
Relevance across
industries may be
limited.
Comparison of
customer service wait
times in U.S. post
offices to wait times
at a major retail store
chain.
Spurs innovation.
May be problems
and/or high costs
associated with
translating practices
from one organization
to another.
• Spurs innovation.
• Typically little
May be problems
and/or high costs
associated with
translating practices
from one organization
to another.
Functional benchmarking
Compares
organizations with
related functions in
same technical area
to show what is
being achieved in
other industries.
Generic benchmarking
Comparing process
in one operation
against process with
similar features but
in another industry.
Best-in-class benchmarking
Comparing measures
with those of
organizations that are
best in class for
function.
Comparison of
process management
between American
companies and a
Malcolm Baldrige
National Quality
Award winner.
resistance
encountered in
partnering and
collecting data.
Business Process Mapping
Business process mapping is often used in consulting engagements as a
method of understanding what is really needed to make a business process
function versus what is being done but isn’t adding any value to the end
customer and continues to get done simply out of inertia. Business process
mapping often begins with a process owner leading the internal auditor on a
walkthrough. Then a flowcharting activity is conducted to map the process
and identify where value is added and where business process
improvements could be made, for example, by simultaneously performing
some activities or by eliminating non-value-added activities. It can also
reveal where controls are used and whether they are functioning as
intended. Many other methods of business process mapping can also be
performed. See Section III of this part of the learning system for additional
details on process mapping, including performing walkthroughs and
flowcharting.
Topic E: Coordinating Audit Efforts with Those of
Other Audit/Oversight Bodies (Level B)
Internal Audit Activity’s Role in Response to
Agency Comments/External Audits
The internal auditing activity may be involved in coordinating with and
supporting both internal and external groups engaged in assuring compliance
with laws and regulations or organizational policies. These groups could
include other functions in the organization, such as quality assurance or
regulatory affairs, or external groups, such as regulatory agencies and
external auditors.
The goal for the CAE is to support a level of accuracy, transparency, and
integrity consistent with good governance. In addition, internal auditing
should seek ways to make its own work more efficient through coordination
with these other groups.
This topic will focus primarily on the coordination of internal audit with
regulatory oversight bodies, internal assurance functions, and external
auditors.
Relevant Standards and Implementation Guides
Performance Standard 2050, “Coordination and Reliance,” states:
The chief audit executive should share information, coordinate activities, and
consider relying upon the work of other internal and external assurance and
consulting service providers to ensure proper coverage and minimize duplication of
efforts.
The interpretation of this standard states that:
A consistent process for the basis of reliance should be established, and the chief
audit executive should consider the competency, objectivity, and due professional care
of the assurance and consulting service providers. The chief audit executive should
also have a clear understanding of the scope, objectives, and results of the work
performed by other providers of assurance and consulting services. Where reliance is
placed on the work of others, the chief audit executive is still accountable and
responsible for ensuring adequate support for conclusions and opinions reached by
the internal audit activity.
Implementation Guide 2050 states:
The roles of assurance and consulting service providers vary by organization. Thus,
to start the task of coordinating their efforts, the chief audit executive…identifies the
various roles of existing…providers.
The CAE meets with each of the providers to gather sufficient information so that
the organization’s assurance and consulting activities may be coordinated.
The process of coordinating assurance activities also varies by organization;
however, the CAE retains ultimate responsibility for determining whether
the internal audit activity may rely on the work of another provider. The
CAE is responsible for internal audit conclusions and opinions. The CAE
obtains the support of the board to coordinate audit work effectively.
Without effective coordination and reporting, work can be duplicated or key
risks may be missed or misjudged. The Implementation Guide goes on to
state:
One way to coordinate assurance coverage is to create an assurance map by linking
identified significant risk categories with relevant sources of assurance and rating the
level of assurance provided for each risk category. Because the map is
comprehensive, it exposes gaps and duplications in assurance coverage, enabling the
CAE to evaluate the sufficiency of assurance services in each risk area.
The results can be discussed to determine how best to coordinate to
maximize the efficiency and effectiveness of assurance coverage.
Coordinating with Regulatory Body Reviews
Ensuring compliance with legal or regulatory requirements has become a
commonplace activity in conducting business worldwide. Organizations must
be sure that they meet the compliance requirements imposed by stock
exchanges (for example, NASDAQ, the New York Stock Exchange, the
London Stock Exchange, and others), industry regulators (for example,
banks and insurance companies), legislative bodies (for example, the U.S.
Congress with the Sarbanes-Oxley Act of 2002), and myriad other agencies.
Most organizations have responded by implementing certain structures and
processes to ensure compliance.
Internal audit activities should coordinate with regulatory bodies having
relevant oversight responsibilities. Different industries have different
oversight bodies and different requirements, so it is not realistic to examine
specifics here. But the primary goal is to exchange information that could
minimize duplicate efforts and/or focus engagement activities on the most
significant areas. Internal auditors, for example, can supply regulatory
examiners with internal audit documentation and reports as evidence of
regulatory compliance in an effort to reduce their workload.
Practical examples demonstrating coordination between internal audit
activities and external regulatory auditors include:
• Internal auditors reviewing copies of regulatory reports in planning related
internal engagements.
• Regulatory auditors sharing their perspective with the internal audit
activity regarding organizational conformance to the regulations as well as
organizational risk management, control, and governance.
Coordinating with Internal Assurance Functions
Coordination with other internal assurance functions can further reduce
redundancies and optimize the effectiveness and efficiency of the internal
audit activity. Internal assurance functions will vary from enterprise to
enterprise, depending on size, industry, and other variables. Common
examples found in many organizations include the following.
• Security. Potential coordination with security could be achieved by
holding periodic meetings with security personnel to keep them apprised
of ongoing audit projects. These meetings can also provide a forum to
gather security input on potential risks, ongoing security investigations that
may be related to control breakdowns, and any areas where past
improprieties have occurred.
• Health and safety. Potential coordination with health and safety should
establish mechanisms so that dangers are immediately reported for
investigation and necessary corrective actions can be evaluated.
• Enterprise risk management. The ongoing identification and evaluation
of risks by the organization helps identify the most important areas for
management to focus on in order to achieve the organization’s objectives.
This information helps internal audit in establishing appropriate priorities
for the audit activity and in performing internal audit evaluations.
• Quality control. A quality control function often conducts a variety of
audits, for example, product quality audits, process quality audits, and
quality system audits. The internal audit activity and the quality control
department should exchange audit schedules and reports.
• Compliance. A compliance function conducts audits to ensure adherence
to laws and regulations related to an organization’s scope of operations.
Issues such as protection of personal information (e.g., medical, financial,
personal preferences), governance structure and activities, public reporting
of an organization’s financial and nonfinancial information, environmental
and animal protection, and consumer protection are receiving significant
attention in many organizations. The compliance focus can be specific by
industry. For example, banks will ensure compliance with moneylaundering laws and regulations. Internal audit needs to be aware of
relevant laws and regulations when conducting a review of an area that
may be affected by them.
The compliance function will be involved in communication and training
programs as well as assurance reviews and special investigations, if
required. The compliance officer will typically be a senior-level executive
reporting to the chief executive officer or chief legal counsel. A committee
of the board of directors will usually provide oversight for the activities of
the compliance function. This could be part of the audit committee’s
charter or of a separate compliance committee.
Coordinating with Assurance Functions of Outside
Organizations
The AICPA’s System and Organization Controls (SOC) reports were
introduced in Topic C as part of the discussion of assurance engagements of
external business relationships. This is a key example of how the internal
audit function can leverage the assurance of an independent assessor and, in
so doing, reduce internal audit expense. The SOC reports provide assurance
that the reports have been generated in accordance with the AICPA’s
standards. The SOC 2 and 3 reports focus on standardized benchmarks for
security, confidentiality, privacy, and processing integrity of a service
provider’s data center and information systems. SOC 2 in particular
provides in-depth information on operational effectiveness, data center
testing, organizational oversight (including governance, risk management
and control), vendor management programs, and regulatory oversight. SOC
3 is the public use version of the same report, providing a description of
the system and the auditor’s opinion. The SOC 3 report provides the highest
level of certification and assurance that a data center can be given in
relation to assurance of operational excellence. For more information on the
SOC 1, 2, and 3 reports, go to the Resource Center, where there is a link to
an article on the subject.
Coordinating with External Assurance Providers
Examples of external assurance providers include legal investigators,
government auditors, external auditors, joint venture partners, specialist
reviewers, or third-party audit firms.
Let’s review the basic differences between internal and external auditors:
• They have different objectives and accountability.
• They may possess different qualifications.
• They engage in different auditing activities.
These basic differences are summarized in Exhibit I-16.
Exhibit I-16: Scope for Internal and External Auditors
Scope
Defined by . . .
Internal
auditors
• Apply a systematic, disciplined
Section 2100 of the Standards.
approach to evaluate and improve the
effectiveness of organizational risk
management, control, and governance
processes.
• Concerned with all aspects of the
organization (e.g., financial and
nonfinancial and with the prevention of
fraud in any form).
• Focus on future events as a result of
their continuous review and evaluation
of controls and processes. (Note that
while the focus of external auditors is
primarily historical, internal auditors
add value by taking a multifaceted
historical, present, and future focus as
part of continuous reviews for
assurance and consulting engagements.)
External
auditors
• Ordinary examination is designed to
obtain sufficient evidential matter to
support an opinion on the overall
fairness of the annual financial
statements.
• Approach is primarily historical in
Their professional standards; they are
responsible for judging the adequacy of
procedures performed and evidence
obtained for purposes of expressing their
opinion on the annual financial
statements.
nature.
Acknowledging these differences, Implementation Guide 2050,
“Coordination and Reliance,” includes the following recommended guidance
on coordinating internal audit activity efforts with those of external auditors.
Internal providers include oversight functions that either report to senior management
or are part of senior management. They are often involved in “second line of
defense” activities, according to The IIA’s Three Lines of Defense model.
External assurance providers may report to senior management or external
stakeholders, or they could be hired by and report to the CAE.
It is essential that the CAE establish a consistent process and set of criteria to
determine whether the internal audit activity may rely on the work of another
provider. In this process, the CAE may:
•
Evaluate objectivity by considering whether the provider has, or may appear to
have, any conflicts of interest and whether they have been disclosed.
•
Consider independence by examining the provider’s reporting relationships and
the impact of this arrangement.
•
Confirm competency by verifying whether the provider’s professional experience,
qualifications, certifications, and affiliations are appropriate and current.
•
Assess due professional care by examining elements of the practice the provider
applies to complete the work (i.e., the provider’s methodology and whether the
work was appropriately planned, supervised, documented, and reviewed).
The CAE may also seek to gain an understanding of the scope, objectives, and
results of the actual work performed to determine the extent of reliance that may be
placed on the provider’s work. The CAE typically considers whether the provider’s
findings appear reasonable and are based on sufficient, reliable, and relevant audit
evidence. The CAE determines whether additional work or testing is needed to
obtain sufficient evidence to support or increase the level of reliance desired. If
additional work is needed, the internal audit activity may retest the results of the
other provider.
It is important to ensure that the internal audit activity will have access to
the work of external auditors, that engagement expectations are clearly
defined in a work agreement or contract, and that the external auditor’s
work is performed with attention to independence, objectivity, competencies,
elements of practice, adequacy of execution of audit work, and sufficiency
of audit evidence to support the given level of assurance.
Practical examples demonstrating the coordination of internal audit activity
efforts with external auditors include:
• Comparing annual internal and external audit plans to eliminate duplication
and encourage cooperation in performance of an audit activity where
appropriate.
• Enterprise-wide agreement, so that results of activities (e.g., final reports)
are shared to help the organization achieve objectives and eliminate risks.
• Communication/sharing of the external audit perspective on risk
management, control, and governance processes with the internal audit
activity to help with internal audit planning.
Benefits of Coordination and Cooperation to Auditors
The two parties—internal and external—have mutual interests that make
coordination of their efforts important. Cooperation becomes a key factor in
the process. Without compromising either group’s independence or
objectivity, coordination and cooperation between internal and external
auditors is prudent because of the potential to increase the economy,
efficiency, and effectiveness of the total audit activity for the organization.
• Economy. Basic costs associated with performing both internal and
external audits (time, materials, and resources) rise with inflation.
Financial reporting standards necessitate increased financial audits and
increased fees. Simply put, coordination between internal and external
auditors can help limit the rising costs for both types of engagements.
• Efficiency. The efficiency of the total audit effort is increased if internal
and external auditors share audit results with each other as needed and in
a timely manner. The enterprise benefits when both parties accept each
other’s work. Such cooperation can provide board members and senior
management with further assurance that financial and operational reports
and statements are proper, controls are adequate and effective, and any
weaknesses will be promptly identified.
If the two types of audits are not coordinated, overlaps and duplication of
efforts during engagements are inevitable, which in turn unnecessarily
increase auditing costs and confuse responsibilities. On the other hand,
coordination and information sharing can increase efficiency and minimize
redundancies in audit activities. Testing is a good example. If the external
audit has performed detailed tests and results are shared, the need for the
internal audit activity to repeat such tests can be greatly lessened. External
audit can also potentially reduce the extent of external auditor detail
testing depending on the level of reliance they place on internal audit
testing.
• Effectiveness. Internal and external auditors each have special expertise
they bring to their activities. Coordination and information sharing allows
each party to accumulate useful information and knowledge they otherwise
might not have, and new areas of risk or concern might be identified.
The type of audits to be performed by either internal or external auditors
relates to the training, experience, and organizational knowledge to make
certain that the right audits and audit activities are being performed. The
knowledge of the auditors, the amount of time, and the depth of the
testing will all help in having a positive effect on the process being
audited. Using the right audit organization and the right auditors helps to
provide the most meaningful results to the management of the organization
being audited.
Studies have demonstrated that the benefits of coordination and cooperation
extend beyond the total audit activity. Collaboration can improve internal
and external auditors’ competencies in other ways. Both parties generally
benefit from the interchange of new/different auditing techniques,
procedures, ideas, and information. External auditors gain better insights
into client operations, control systems, and so on, typically much more
quickly than when this is left to independent discovery. Collaboration also
allows both parties to focus on more significant issues during their
respective engagements. A prime example of an area that strongly benefits
from deliberate, up-front collaboration is the testing performed by internal
auditors in support of the U.S. Sarbanes-Oxley Act (Section 404) internal
control over financial reporting assessment work, which can offer significant
potential reliance opportunities for external auditors.
Certainly much of the other work the internal audit activity performs is not
relevant to the efforts of external auditors. For example, internal audit
engagement objectives intended to assess compliance, efficiency, and
effectiveness of operations have little application to external audits focused
on the fairness of the presentation of financial statements. But when
synergies are possible (as in the case of understanding controls, risk
management techniques, and testing in financial reporting areas), everyone
stands to gain from coordination and cooperation. Proper planning provides
the foundation for the success.
Responses to External Audits
Transparency requires full disclosure of relevant financial and operational
information and the internal processes management has put in place for
oversight and control. The Sarbanes-Oxley Act of 2002, the U.S. Securities
and Exchange Commission, and international stock exchanges impose many
additional requirements in these areas for companies.
The audit committee and its external auditors both have key roles related to
transparency and disclosure.
For the audit committee, the primary concerns are financial accuracy,
including the completeness of financial disclosures, significant business and
accounting policy changes, correct and truthful reporting, and interim and
annual reviews of financial statements. The audit committee must have the
financial acumen to assess the significance of complex or unusual
transactions, financial statement presentations, changes in the organization’s
selection or application of accounting principles, and the effect of regulatory
and accounting initiatives, as well as off-balance-sheet structures and
transactions.
External auditors primarily assess the effectiveness of internal controls over
financial reporting. External auditors are independent of the organization. To
preserve this independence, the audit committee should own the relationship
with the external auditors and have oversight responsibilities. General
ownership and oversight responsibilities are listed in Exhibit I-17.
Exhibit I-17: Audit Committee Ownership and Oversight Responsibilities for External Auditors
Ownership Responsibilities
• Direct reporting
• Ongoing open communication
• Frequent meetings
• Robust discussions about audit scope and
results
Oversight Responsibilities
• Compensation
• Engagement scope
• Selection criteria
• Independence
• Rotation
• Monitoring
• Performance assessment
External auditors issue a formal written report at the conclusion of the
engagement including their observations and opinions. If ongoing
communications have been adequate, there should be no surprises in the
final report.
The audit committee should have a clear understanding of all information
reported by the external auditors, including:
• The external auditors’ judgments about the quality of accounting policies.
• The external auditors’ conclusions regarding the reasonableness of
management accounting estimates.
• Significant adjustments arising from the audit that could have a significant
effect on financial statements.
• Reported disagreements with management that could have a significant
effect on financial statements and whether or not they have been
satisfactorily resolved.
• Difficulties encountered with management in performing the audit.
• Significant deficiencies or material weaknesses in internal control.
• Fraud or illegal acts.
Management has the primary responsibility for acting on the external audit
findings. To support management’s follow-up, the audit committee should:
• Discuss the issues with the external auditors to ensure a full understanding
of concerns and acceptable corrective actions.
• Discuss any identified issues with management to ensure a full
understanding of the implications.
• Determine whether additional resources should be consulted.
• Establish a reasonable time line to address the issues.
• Develop a checklist and other necessary tools for monitoring the resolution
of issues.
• Discuss the resolution of issues with management and external auditors.
Exhibit I-18 presents an excerpt from a sample audit committee charter
outlining an audit committee’s responsibilities related to external auditing.
Exhibit I-18: Sample Audit Committee Responsibilities for External Auditing
• Review the external auditors’ proposed audit scope and approach, including coordination of audit
effort with internal auditing.
• Review the performance of the external auditors and exercise final approval on the appointment or
discharge of the auditors. In performing this review, the committee will:
• At least annually, obtain and review a report by the independent auditor describing the firm’s
internal quality control procedures; any material issues raised by the most recent internal
quality control review, or peer review, of the firm, or by any inquiry or investigation by
governmental or professional authorities, within the preceding five years, respecting one or
more independent audits carried out by the firm, and any steps taken to deal with any such
issues; and (to assess the auditor’s independence) all relationships between the independent
auditor and the company.
• Take into account the opinions of management and internal auditing.
• Review and evaluate the lead partner of the independent auditor.
• Present its conclusions with respect to the external auditor to the board.
• Ensure the rotation of the lead audit partner every five years and other audit partners every seven
years, and consider whether there should be regular rotation of the audit firm itself.
• Present its conclusions with respect to the independent auditor to the full board.
• Set clear hiring policies for employees of the independent auditors.
• On a regular basis, meet separately with the external auditors to discuss any matters that the
committee or auditors believe should be discussed privately.
• Ensure effective and efficient coordination between the external and internal audit functions as
related to the relevant assurance coverage they each provide.
Source: “The Audit Committee: Purpose, Process, Professionalism,” www.theiia.org/download.cfm?
file=6676.
Chapter 3: Communicating and Reporting to Senior
Management and the Board
Chapter Introduction
According to Standard 2060, “Reporting to Senior Management and the
Board,” the CAE has the responsibility to:
Report periodically to senior management and the board on the internal audit
activity’s purpose, authority, responsibility, and performance relative to its plan.
Reporting must also include significant risk exposures and control issues, including
fraud risks, governance issues, and other matters needed or requested by senior
management and the board.
The CAE evidences the completion of these professional responsibilities by
periodically reporting the results of ongoing internal audit activities to
senior management and the audit committee during routinely scheduled
meetings throughout the year.
Topic A: Communicating and Seeking Approval of
the Annual Audit Plan (Level B)
Performance Standard 2020, “Communication and Approval,” states:
The chief audit executive must communicate the internal audit activity’s plans and
resource requirements, including significant interim changes, to senior management
and the board for review and approval. The chief audit executive must also
communicate the impact of resource limitations.
Before communicating and presenting the annual audit plan to senior
management and the board, the CAE gathers resource requirements and
determines any issues with priorities and scheduling limitations.
The CAE also typically meets with individual senior executives to solicit
their input regarding the proposed internal audit plan before it is formally
presented to the board for approval. During those meetings, the CAE can
address any concerns that senior executives may express, incorporate their
feedback, and obtain their support. The process may involve gathering
additional information about the timing of proposed audit engagements and
the availability of resources. It might introduce changes that affect the scope
of work. The insight the CAE acquires from these discussions helps
determine whether any adjustments should be made to the internal audit
plan before it is presented to the board for approval.
The CAE’s presentation of the internal audit plan to the board usually
occurs during a meeting, which may include senior management. The
proposed internal audit plan may include but is not limited to:
• A list of proposed audit engagements (identifying whether the engagements
are assurance or consulting in nature).
• The rationale for selecting each proposed engagement (e.g., risk rating,
time since last audit, change in management).
• The objectives and scope of each proposed engagement.
• A list of initiatives or projects that result from the internal audit strategy
but may not be directly related to an audit engagement.
The audit plan is typically developed with some flexibility included to
respond to slight changes from the organization’s business, emerging risks,
operations, programs, and controls. However, if those changes result in
significant deviations to the plan, those should be noted and shared.
Standard 2060 states that the CAE must report on the internal audit
activity’s performance relative to its plan.
Communication should include significant deviations from approved
engagement work schedules, staffing plans, and financial budgets; the
reasons for the deviations; and action taken or needed. Significant
engagement observations and recommendations should also be reported.
In addition to the information identified above, a report typically includes
the results of management’s self-assessment regarding the design adequacy
and operating effectiveness of the organization’s internal controls.
Gaining Board Approval
Once a risk-based audit plan is developed, the CAE communicates the plan
to the board or other appropriate governing body, who will review it and
will either approve it or make suggestions for changes. Showing how the
plan was developed and the reasons for its development can help to gain
buy-in and win approval. Key points to address include:
• Why the area should be audited at this time instead of later.
• What the audit objectives are and how this relates to the audit scope.
• How the process is currently being monitored for performance/control.
• Relevant results or ongoing concerns from past audits.
• Relevant current events.
• Proposed, ongoing, or completed changes related to the process or risks.
• What significant new or ongoing risks or root causes the audit is designed
to address.
• What resource limitations exist and what out-sourcing or co-sourcing
arrangements may be necessary to accommodate them.
Ultimately, the engagement plan should address and support the most
effective use of internal audit resources. Once approved, the CAE should
communicate the plan and resource requirements to senior management to
give them sufficient time to include the plan requirements in the overall
budget and so on.
Keeping the Plan Relevant
Regularly scheduled board meetings provide opportunities to review and
adjust the internal audit plan.
A risk assessment process should be conducted annually. But the resulting
engagement plan cannot be static. Changes in management direction,
objectives, emphasis, and focus as well as other evolving factors such as
emerging trends should be reflected by changes to the audit universe and
the related annual engagement plan. Frequent (quarterly) updating may be
required, and any significant changes should be submitted to the board or
other oversight entities for review and approval.
Topic B: Identifying Risk, Control, and Governance
Issues (Level B)
Performance Standard 2060 states:
The chief audit executive must report periodically to senior management and the
board on the internal audit activity’s purpose, authority, responsibility, and
performance relative to its plan and on its conformance with the Code of Ethics and
the Standards. Reporting must also include significant risk and control issues,
including fraud risks, governance issues, and other matters that require the attention
of senior management and/or the board.
The interpretation of this standard notes that internal auditors determine the
subject matter of these reports and how often to report as part of a
discussion with senior management and the board about “the importance of
the information to be communicated and the urgency of the related actions”
these bodies plan to take.
Getting all stakeholders on the same page with regard to what risks should
be considered significant could be contentious, but once agreement has been
reached in this area, it will naturally lead to easier acceptance of an audit
plan based on those same risk exposures and control issues.
In addition to gaining acceptance of the audit plan to ensure that audits are
based on the most likely and/or significant risks currently facing the
organization, the CAE needs to keep up on emerging risk, control, and
governance issues on an ongoing basis and make routine reports to senior
management and/or the board as appropriate. This is discussed next, in
addition to two specific types of risk, control, and governance issues that
may need follow-up. One is when senior management decides to accept
risks that the CAE believes require some form of resolution; the other is
when various risk, control, and governance stakeholders are in conflict with
each other regarding what to do about audit observations.
Routine Reports on Risk, Control, and Governance
Issues
According to Practice Advisory 2060-1, the CAE needs to report the
content of and reasons for any “significant deviations from approved
engagement work schedules, staffing plans, and financial budgets” and what
he or she is doing to correct the deviations or provide details on the
changes to the plan. The board can decide if these changes create
significant risks and can then advise management.
The board and its audit committee may work with the CAE and senior
management to determine the timing and types of reports related to risk,
control, and governance issues that the CAE and/or management should
prepare and present. Senior management will typically contribute
information on the risk and control activities under their purview, and the
CAE will report on the audit activity’s independent observations.
Routine reports that either or both of these parties may provide could
include activity reports for enterprise risk management (for the entire
organization or for a specific business unit), internal control monitoring,
legal and compliance monitoring, and independent external auditor activity.
The board may also want information on key financial activity and the
quality of the organization’s internal controls over financial reporting
(ICFR). Senior management will typically provide this information, and the
CAE may provide an opinion on ICFR regarding management’s assertions
at these times.
Some CAEs also provide an opinion on the organization’s internal controls
over operations and compliance objectives.
Issues Related to Senior Management’s Acceptance
of Risks
One issue that may require consultation with the board is when senior
management has received a recommendations from the internal audit
activity and has decided to accept the risks rather than formulate a
resolution (neither the suggested one or another option). This may be for
many reasons. For example, senior management may not view the risk as
very significant or highly likely, the resolution may not seem affordable or
cost-effective, or perhaps they are in denial or are rationalizing, such as that
it is a one-time issue that will not recur (even though there is nothing
stopping a recurrence). The CAE will need to take the situation into account
(for example, a recent change in management or in the market or industry)
and assess the significance of the risk being accepted (how it relates to the
organization’s risk appetite and level of residual risk this is creating) when
deciding whether or not to approach the board with this matter.
Audit Resolution Misalignments
The risk, control, and governance objectives of the board, its audit
committee, senior and operating management, internal and external auditors,
external regulators, and other stakeholders may not always align. When
audit reports provide assurance that internal controls are not only effective
in their design but are also operating effectively and there are no significant
audit observations, misalignments between parties are rare. When internal
controls are inadequately designed or ineffectively executed or there are
significant audit observations, the parties involved may be in conflict about
whether there is a problem and what to do about it.
The CAE has a duty to not only report on such observations but also to
coordinate plans for their resolution (and include this in the report). This
may require use of leadership skills to get the parties to agree first on the
significance of the issue and then on a viable solution that all parties can
support and/or take ownership of.
Topic C: Reporting on the Effectiveness of Internal
Control and Risk Management (Level B)
The CAE is responsible for reporting on the overall effectiveness of the
organization’s internal control and risk management processes to both senior
management and to the board. The CAE needs to form a holistic opinion
regarding the general state of internal control in the organization, usually
once each year. The CAE’s assertion is often used as the basis for the chief
executive officer and chief financial officer to express confidence in the
overall quality of the organization’s internal controls over financial
reporting, a requirement under the U.S. Sarbanes-Oxley Act’s Section 404.
Implementation Guide 2130 states the following:
To promote continuous improvement in maintaining effective controls, the internal
audit activity typically provides the board and senior management with an overall
assessment or compiles the results of control evaluations accumulated from individual
audit engagements. The CAE may recommend the implementation of a control
framework if one is not already in place. Additionally, internal auditors may make
recommendations that enhance the control environment (e.g., a tone at the top that
promotes a culture of ethical behavior and a low tolerance for noncompliance).
Standard 2450, “Overall Opinions,” states:
When an overall opinion is issued, it must take into account the strategies,
objectives, and risks of the organization; and the expectations of senior management,
the board and other stakeholders. The overall opinion must be supported by
sufficient, reliable, relevant, and useful information.
The interpretation to this standard indicates that the communication needs to
include information on the scope, scope limitations, and time period that the
opinion pertains to, whether other assurance providers or other projects were
used to provide input, an executive summary of the opinion, reference to
the risk or control framework used to form the opinion, and the overall
opinion that was reached plus any reasons for an unfavorable opinion if
there was one.
In addition to this, or if there are deficiencies, the CAE may wish to
educate the board and senior management on best practices in governance,
risk management, and control as a way for internal auditing to add “value
to the organization” (Performance Standard 2000, “Managing the Internal
Audit Activity”).
In an organization committed to governance and enterprise risk
management, the board and senior management’s ability to provide
oversight and to make sound decisions may be limited by various factors.
Board members, for example, may not have sufficient expertise and
experience in the organization’s business and regulatory environments.
Senior management may be driven by business imperatives and the need to
meet objectives and may “de-prioritize” governance issues. Both the board
and senior management may not be familiar with the principles of risk
management and how that should be affecting both oversight and business
decisions.
The CAE can educate the board and senior management by:
• Reviewing the role of the board, senior management, operations, and
internal auditing in the risk management process. This may be offered as a
tutorial or a workshop during an annual meeting. It can also be required
training for new board members and senior managers. ISO 31000:2018,
“Risk Management,” can provide a basis for this training. The Practice
Guide “Assessing the Adequacy of Risk Management Using ISO 31000”
notes, “Management is responsible for setting the organizational attitude
regarding risk and the board is responsible for determining whether the
risk attitude is aligned with the best interests of shareholders.”
To fulfill these responsibilities, the board and management must
understand the processes for identifying and assessing risk, defining a risk
appetite, developing risk management strategies, and continually
monitoring and improving the organization’s risk management. The CAE
can illustrate this process—or the urgent need for ERM—by organizing
field visits to locations or operations that pose specific risks to the
organization’s health and continued existence.
• Reviewing key, amended, and new laws, regulations, legal decisions, and
standards that affect the organization’s governance and operations.
Periodically the CAE can include this review as an agenda item for a
board/audit committee meeting. It might include, for example, changes in
financial reporting requirements and executive compensation guidelines or
legal decisions about the failure to monitor risks that increase the board’s
and management’s criminal and/or financial liability. It may be necessary
for the CAE to “connect the dots”—to clarify and illustrate the
implications of these laws and regulations.
• Facilitating workshops designed to identify emerging risks associated with
the organization’s business environment.
• Presenting at board/audit committee meetings on best practices in
governance and risk management as practiced in peer organizations.
Topic D: Reporting on Internal Audit Key
Performance Indicators (Level B)
In order to perform its assurance role in governance, risk management, and
operational effectiveness and efficiency, the internal audit activity must
assure its own efficiency and effectiveness and report its performance to
senior management and the board at agreed intervals. Care must be taken to
identify appropriate performance measures—key performance indicators
(KPIs) that are aligned to the organization’s objectives and the internal audit
charter and that target the performance necessary to meet activity objectives.
The IPPF Practice Guide “Measuring Internal Audit Effectiveness and
Efficiency” describes four steps for establishing an effective performance
measurement process:
1. Define internal audit effectiveness. This definition will be based on the
Core Principles for the Professional Practice of Internal Auditing, the
Definition of Internal Auditing, the Code of Ethics, the Standards, existing
charters, agreed internal audit deliverables, and internal consensus on what
constitutes an efficient and effective internal audit activity.
2. Identify key internal and external stakeholders. Internal stakeholders
may include the board or audit committee, senior management, operations
and support management, other internal assurance providers (the second
line of defense), and the internal audit activity itself. External stakeholders
may include regulators and standards-setting bodies, external auditors,
third-party vendors, and customers. In-depth interviews and surveys can be
conducted to develop a clearer understanding of the needs and
expectations of each of these stakeholders.
3. Develop measures, or KPIs, of internal audit effectiveness and
efficiency. KPIs focus on accomplishments or behaviors that are valued by
the organization. They are valid indicators of performance (i.e., they
measure the right target) and are understandable to the internal audit staff
who use them to guide and improve their performance.
KPIs measuring the internal audit activity itself are valuable because they
allow the CAE to detect shortcomings in the activity and plan remedial
action. They also allow the CAE to demonstrate the value of internal
auditing to customers, and they can be used to support requests for
resources needed to support the desired level of performance. Because of
the close relationship between the internal auditing activity’s KPIs and the
expectations of the board and senior management, the CAE should
establish KPIs that consider stakeholders’ needs. In this way the CAE can
ensure that the activity’s KPIs focus on meaningful performance that is
aligned with the organization’s strategic goals.
KPIs may be quantitative (e.g., the percentage of planned audits that have
been completed) or qualitative (e.g., internal customer satisfaction with
audit performance). When establishing KPIs to monitor, measure, and
report, the CAE may consider those areas that need improvement as
identified by the quality assurance and improvement program (QAIP) or
other aspects of internal audit performance.
Implementation Guide 1311 recommends the following ongoing assessment
processes and tools:
• Engagement planning and supervision
• Checklists or automation tools and written procedures (e.g., auditing
manual)
• Feedback from audit customers and other stakeholders
• Staff and engagement KPIs
• Selective peer reviews of workpapers by staff not involved in the
affected audits
• Project budgets, timekeeping systems, audit schedules, projected cost
recoveries (budget-to-actual variances)
• Periodic self-assessments
Occasionally, in-depth interviews and surveys should be conducted with
stakeholders. The CAE should also consider periodically benchmarking the
activity’s KPIs against those of similar peer organizations.
4. Monitor and report results.The CAE must ensure that performance
against agreed KPIs is monitored, considered as the basis for quality
improvement, and reported at a frequency agreed to by the board and
senior management—for example, quarterly—and in the manner desired by
stakeholders (e.g., presentations, automated dashboard, emails).
Standard 1320, “Reporting on the Quality Assurance and Improvement
Program,” requires that the CAE periodically communicate the results of
the entire program—both internal and external quality activities.
Specifically related to internal audit KPI reporting and internal audit
quality activities, the Implementation Guide for Standard 1320 states:
The CAE should establish a means for communicating the results of internal
assessments at least annually to enhance the credibility and objectivity of the internal
audit activity. The Interpretation of Standard 1320 states that the results of periodic
internal assessment should be communicated upon completion of such assessments,
and the results of ongoing monitoring should be completed at least annually...
Ongoing monitoring typically includes reporting on internal audit key performance
indicators. The CAE may provide an annual report to senior management and the
board regarding the results of ongoing monitoring and include any recommendations
for improvement. Generally those assigned responsibility for conducting ongoing
monitoring and periodic assessments communicate the results directly to the CAE
while performing the assessments. In a smaller internal audit activity, the CAE may
take a greater direct role in the internal assessment process. The results of internal
assessments include, where appropriate, corrective action plans and progress against
completion. The CAE may distribute internal assessment reports to various
stakeholders, including senior management, the board, and external auditors.
The appendices to the “Measuring Internal Audit Effectiveness and
Efficiency” Practice Guide contain dashboards and tools for tracking and
reporting internal audit performance.
Next Steps
You have completed Part 2, Section I, of The IIA’s CIA Learning System®.
Next, check your understanding by completing the online section-specific
test(s) to help you identify any content that needs additional study.
Once you have completed the section-specific test(s), a best practice is to
reread content in areas you feel you need to understand better. Then you
should advance to studying Section II.
You may want to return to earlier section-specific tests periodically as you
progress through your studies; this practice will help you absorb the content
more effectively than taking a single test multiple times in a row.
Index
The numbers after each term are links to where the term is indexed and indicate
how many times the term is referenced.
activity ratios 1
assets, physical security controls for 1
assurance engagements 1
audit committees 1
audit managers 1
audit manuals 1
audit plans 1
audit resolution 1
audit teams 1
audit universe 1
auditable units 1
auditing
maturity level 1
auditors 1
external 1
internal 1
audits
compliance 1
control self-assessment 1, 2
due diligence 1, 2
environmental health and safety 1
external 1
financial/financial reporting 1
of certification to standards 1
of contracts 1
of efficiency and effectiveness 1
of external business relationships 1
operational 1
performance 1
privacy 1, 2
quality 1
security 1
See also engagements 1
behavioral interviews 1
benchmarking 1
best-in-class benchmarking 1
bilateral contracts 1
board
approval of audit plan 1
reporting to 1
budgets 1
business process mapping 1
CAEs (chief audit executives) 1
certification to standards 1
chief audit executives 1
Committee of Sponsoring Organizations frameworks
Enterprise Risk Management—Integrated Framework 1
Internal Control—Integrated Framework 1, 2
common-size statements 1
competitive benchmarking 1
compliance
audits 1
coordination with internal assurance function 1
consulting engagements 1
contractors
selection 1
sourcing 1
contracts 1, 2
control 1
control self-assessment 1, 2
controls 1
data storage 1
hazard 1
logical security 1
physical security 1
power supply 1
controls-based auditing 1
coordination with other audit/oversight bodies 1
COSO frameworks
Enterprise Risk Management—Integrated Framework 1
Internal Control—Integrated Framework 1, 2
cost reimbursement contracts 1
cost-plus contracts 1
CSA (control self-assessment) 1, 2
current ratio 1
data
collection 1
qualitative/quantitative 1
storage controls 1
debt ratios 1
directing, as management function 1
driver analysis for risk 1
due diligence audits 1, 2
EH&S (environmental health and safety)
audits 1
coordination with internal assurance function 1
emerging issues, as source of engagements 1
employees, as source of engagements 1
engagements
assurance 1
communicating results 1
consulting 1
managing 1, 2
monitoring progress 1
performing 1
planning 1
prioritization 1
See also audits 1
selection of 1
sources of 1
enterprise risk management 1
coordination with internal assurance function 1
maturity level 1
environmental analysis/scanning 1
environmental health and safety
audits 1
coordination with internal assurance function 1
ERM (enterprise risk management) 1
coordination with internal assurance function 1
maturity level 1
ERM-based auditing 1
evidence 1
express contracts 1
external assurance providers, coordination with internal assurance function
1
external auditors 1
external audits 1
external business relationships
as source of engagements 1
audits of 1
facilitated team workshop approach to control self-assessment 1
financial leverage ratios 1
financial ratios 1
financial/financial reporting audits 1
fire prevention 1
fixed-price contracts 1
flood prevention 1
focus groups 1
frameworks, risk 1
functional benchmarking 1
generic benchmarking 1
governance 1
gross profit margin 1
hazard controls 1
hotlines 1
ICFR (internal control over financial reporting) 1
impact of risk 1
Implementation Guides
1311 1
1320, “Reporting on the Quality Assurance and Improvement Program”
1
2010, “Planning” 1, 2
2050, “Coordination and Reliance” 1, 2
2130 1
implied contracts 1
industry benchmarking 1
industry trends, as source of engagements 1
information gathering 1
information technology
as source of engagements 1
inherent risk 1
intellectual property 1
internal assurance functions, coordination with 1
internal audit activity 1
administrative activities 1
managing 1, 2
internal auditors 1
internal benchmarking 1
internal control over financial reporting 1
internal control training 1
internal control, reporting on effectiveness of 1
Internal Control—Integrated Framework (COSO) 1, 2
International Standards for the Professional Practice of Internal Auditing 1
1320, “Reporting on the 1
2000, “Managing the 1, 2, 3
2010, “Considerations” 1
2010.A1 1, 2
2010.A2 1
2010.C1 1, 2, 3
2020, “Communication and 1
2040 1
2050, “Coordination and 1
2060, “Reporting to 1, 2
2100 1
2120, “Risk Management” 1
2120.A1 1
2200, “Engagement 1
2300, “Performing the 1
2400 1
2450, “Overall Opinions” 1
2500, “Monitoring 1
2600, “Communicating the 1
Performance Standards 1
interviews 1, 2
IP (intellectual property) 1
ISO 19600, “Compliance management systems—Guidelines” 1
ISO 31000, “Risk management—Guidelines” 1
IT (information technology)
as source of engagements 1
job descriptions 1
joint venture contracts 1
key performance indicators 1, 2
KPIs (key performance indicators) 1, 2
leverage ratios 1
likelihood of risk 1
liquidity ratios 1
logical security controls 1
lump-sum contracts 1
management-produced analyses approach to control self-assessment 1
management, as source of engagements 1
manuals, audit 1
market trends, as source of engagements 1
maturity level 1, 2
monitoring
as management function 1
nonfinancial ratios 1
on-boarding 1
operational audits 1
organizational charts 1, 2
organizing, as management function 1
orientation 1
outside organizations’ assurance functions, coordination with 1
performance audits 1
performance measurement 1
Performance Standards 1
See also International Standards for the Professional Practice of Internal
1
physical security controls 1
planning
as management function 1
engagements 1
position descriptions 1
post-acquisition due diligence 1
power supply controls 1
Practice Advisory 2060-1 1
Practice Guides
“Auditing External Business Relationships” 1
“Auditing Privacy Risks” 1
“Measuring Internal Audit Effectiveness and Efficiency” 1
pre-acquisition due diligence 1
prioritization of engagements 1
privacy audits 1, 2
process-based auditing 1
profitability ratios 1
qualitative data 1
quality audits 1
quality control, coordination with internal assurance function 1
quantitative data 1
questionnaire approach to control self-assessment 1
questionnaires 1
ratio analysis 1
recruiting 1
regulatory bodies, coordination with 1
regulatory mandates, as source of engagements 1
reporting
on effectiveness of internal control/risk management 1
on key performance indicators 1
on risk, control, governance issues 1
to board 1
to senior management 1
residual risk 1
resourcing 1
risk 1, 2
analysis 1
appetite 1
assessment 1, 2
driver analysis 1
frameworks 1
identification 1
impact 1
in external business relationships 1
inherent 1
likelihood 1
management 1, 2
measurement 1
prioritization 1, 2
residual 1
responses 1
source analysis 1
third-party 1
risk-based auditing 1
Sarbanes-Oxley Act, U.S. 1
scope
of services 1
of work 1
SDLC (systems development life cycle) review 1
security
audits 1
coordination with internal assurance function 1
data storage controls 1
hazard controls 1
logical controls 1
physical controls 1
power supply controls 1
senior auditors 1
senior management
acceptance of risks 1
reporting to 1
services, scope of 1
situational interviews 1
SOC reports 1, 2
source analysis for risk 1
SOX (Sarbanes-Oxley Act), U.S. 1
SSAE (Standards for Attestation Engagements) 18, Clarification and
Recodification 1
staffing 1
stakeholders 1
standards
certification to 1
See also International Standards for the Professional Practice of Internal
Auditing 1
Standards for Attestation Engagements 18, Clarification and
Recodification 1
strategic plans, as source of engagements 1
strengths, weaknesses, opportunities, and threats (SWOT) analysis 1
structured interviews 1
suboptimization 1
surveys 1
SWOT analysis 1
systems analysis 1
systems design 1
systems development life cycle review 1
third-party risk 1
training 1
U.S. Federal Sentencing Guidelines for Organizations 1
unenforceable contracts 1
unilateral contracts 1
unit-price contracts 1
USFGO (U.S. Federal Sentencing Guidelines for Organizations) 1
void/voidable contracts 1
work
scope of 1
workforce planning 1
Build 08/24/2018 15:40 p.m.
Contents
Part 2: Practice of Internal Auditing
The IIA’s CIA Learning System®
Part 2 Overview
Section I: Managing the Internal Audit Activity
Section Introduction
Chapter 1: Internal Audit Operations
Topic A: Policies and Procedures for Internal Audit Operations (Level
B)
Topic B: Administrative Activities of Internal Audit (Level B)
Chapter 2: Establishing a Risk-Based Internal Audit Plan
Topic A: Sources of Potential Engagements (Level B)
Topic B: Using a Risk Management Framework to Assess Risks and
Prioritize Audit Engagements (Level
Topic C: Assurance Engagements (Level P)
Topic D: Consulting Engagements (Level P)
Topic E: Coordinating Audit Efforts with Those of Other
Audit/Oversight Bodies (Level B)
Chapter 3: Communicating and Reporting to Senior Management and
the Board
Topic A: Communicating and Seeking Approval of the Annual Audit
Plan (Level B)
Topic B: Identifying Risk, Control, and Governance Issues (Level B)
Topic C: Reporting on the Effectiveness of Internal Control and Risk
Management (Level B)
Topic D: Reporting on Internal Audit Key Performance Indicators
(Level B)
Index
Section II: Planning the Engagement
This section is designed to help you:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Establish engagement objectives/criteria.
Differentiate engagement objectives from operational objectives.
Identify the engagement scope.
Define the scope limitations.
Plan engagements to ensure identification of key risks and controls.
Define a risk-based approach.
Define internal controls that are considered likely to be relevant to the engagement.
Complete a detailed risk assessment of each audit area.
Use a risk control matrix to ensure that all significant risks are addressed in the audit.
Define the benefits of a risk control matrix.
Determine engagement procedures.
Prepare an engagement work program.
Describe the approval of the audit activity charter and plan.
Define data collection.
Identify various sources of evidence.
Identify the types of audit tests.
Determine the level of staff and resources needed for the engagement.
Identify budget considerations.
The Certified Internal Auditor (CIA) exam questions based on content from this section make
up approximately 20% of the total number of questions for Part 2. All topics are covered at the
“P—Proficient” level, meaning that you are responsible not only for comprehension and recall
of information but also for higher-level mastery, including application, analysis, synthesis, and
evaluation.
Section Introduction
Section II moves from the general work of internal auditing to the specific
processes used to plan engagements, which includes establishing
engagement objectives, developing an audit program that incorporates
activities aimed at meeting the organization’s risk management objectives,
and allocating staff and resources.
Chapter 1: Engagement Planning
Chapter Introduction
An engagement is “a specific internal audit assignment, task, or review
activity, such as an internal audit, control self-assessment review, fraud
examination, or consultancy” (Standards Glossary). An engagement that is
carefully thought out and planned has a high probability of success. The
planning process helps to ensure that meaningful work is performed, audit
deliverables add value to the organization, and audit resources are used
efficiently and effectively during the engagement.
This chapter focuses on the engagement planning process. Topics include:
• Engagement objectives, criteria, and scope.
• Key risks and controls.
• Detailed risk assessments.
• Engagement procedures and work programs.
• Staff and resources need for engagements.
While many organizations have documented audit programs or suggested
practices for many audit activities, it is important that a work/audit program
be developed for each engagement.
Related Standards and Implementation Guides
The Standards and Implementation Guides related to engagement planning
are listed in Exhibit II-1.
Exhibit II-1: Engagement Planning–Related Guidance
Standard
Performance Standard 2200, “Engagement Planning”
Internal auditors must develop and document a plan for each engagement,
Related Guidance
Implementation Guide
2200, “Engagement
including the engagement’s objectives, scope, timing, and resource
allocations. The plan must consider the organization’s strategies, objectives,
and risks relevant to the engagement.
Planning”
Performance Standard 2201, “Planning Considerations”
Implementation Guide
2201, “Planning
Considerations”
In planning the engagement, internal auditors must consider:
• The strategies and objectives of the activity being reviewed and the
means by which the activity controls its performance.
• The significant risks to the activity’s objectives, resources, and
operations and the means by which the potential impact of risk is kept
to an acceptable level.
• The adequacy and effectiveness of the activity’s governance, risk
management, and control processes compared to a relevant control
framework or model.
• The opportunities for making significant improvements to the activity’s
governance, risk management, and control processes.
Implementation Standard 2201.A1 (Assurance Engagements)
When planning an engagement for parties outside the organization, internal
auditors must establish a written understanding with them about objectives,
scope, respective responsibilities, and other expectations, including
restrictions on distribution of the results of the engagement and access to
engagement records.
Implementation Standard 2201.C1 (Consulting Engagements)
Internal auditors must establish an understanding with consulting
engagement clients about objectives, scope, respective responsibilities, and
other client expectations. For significant engagements, this understanding
must be documented.
Performance Standard 2210, “Engagement Objectives”
Objectives must be established for each engagement.
Implementation Standard 2210.A1 (Assurance Engagements)
Internal auditors must conduct a preliminary assessment of the risks
relevant to the activity under review. Engagement objectives must reflect
the results of this assessment.
Implementation Standard 2210.A2 (Assurance Engagements)
Internal auditors must consider the probability of significant errors, fraud,
noncompliance, and other exposures when developing the engagement
objectives.
Implementation Standard 2210.A3 (Assurance Engagements)
Implementation Guide
2210, “Engagement
Objectives”
Adequate criteria are needed to evaluate controls. Internal auditors must
ascertain the extent to which management has established adequate criteria
to determine whether objectives and goals have been accomplished. If
adequate, internal auditors must use such criteria in their evaluation. If
inadequate, internal auditors must identify appropriate evaluation criteria
through discussion with management and/or the board.
Implementation Standard 2210.C1 (Consulting Engagements)
Consulting engagement objectives must address governance, risk
management, and control processes to the extent agreed upon with the
client.
Implementation Standard 2210.C2 (Consulting Engagements)
Consulting engagement objectives must be consistent with the
organization’s values, strategies, and objectives.
Performance Standard 2220, “Engagement Scope”
The established scope must be sufficient to satisfy the objectives of the
engagement.
Implementation Guide
2220, “Engagement
Scope”
Implementation Standard 2220.A1 (Assurance Engagements)
The scope of the engagement must include consideration of relevant
systems, records, personnel, and physical properties, including those under
the control of third parties.
Implementation Standard 2220.A2 (Assurance Engagements)
If significant consulting opportunities arise during an assurance
engagement, a specific written understanding as to the objectives, scope,
respective responsibilities, and other expectations should be reached and
the results of the consulting engagement communicated in accordance with
consulting standards.
Implementation Standard 2220.C1 (Consulting Engagements)
In performing consulting engagements, internal auditors must ensure that
the scope of the engagement is sufficient to address the agreed-upon
objectives. If internal auditors develop reservations about the scope during
the engagement, these reservations must be discussed with the client to
determine whether to continue with the engagement.
Implementation Standard 2220.C2 (Consulting Engagements)
During consulting engagements, internal auditors must address controls
consistent with the engagement’s objectives and be alert to significant
control issues.
Performance Standard 2230, “Engagement Resource Allocation”
Internal auditors must determine appropriate and sufficient resources to
Implementation Guide
2230, “Engagement
achieve engagement objectives based on an evaluation of the nature and
complexity of each engagement, time constraints, and available resources.
Resource Allocation”
Performance Standard 2240, “Engagement Work Program”
Implementation Guide
2240, “Engagement
Work Program”
Internal auditors must develop work programs that achieve the engagement
objectives.
Implementation Standard 2240.A1 (Assurance Engagements)
Work programs must include the procedures for identifying, analyzing,
evaluating, and documenting information during the engagement. The work
program must be approved prior to its implementation, and any
adjustments approved promptly.
Implementation Standard 2240.C1 (Consulting Engagements)
Work programs for consulting engagements may vary in form and content
depending upon the nature of the engagement.
Topic A: Engagement Objectives, Evaluation Criteria,
and Scope (Level P)
Developing an engagement plan requires defining the objectives, criteria,
and scope of the engagement. Audit engagement objectives answer the
question “Why are we auditing this activity?” Audit scope answers the
questions “What will we review?” and “What are the boundaries of the
audit?” Audit criteria answer the question “Are the requirements being
met?” The audit objectives, criteria, and scope should all be documented.
Engagement Objectives
Performance Standard 2210, “Engagement Objectives,” explicitly states that
“objectives must be established for each engagement.” The
Standards Glossary defines engagement objectives as “broad statements
developed by internal auditors that define intended engagement
accomplishments.” Internal audit engagement objectives should be aligned
with related organizational objectives.
Implementation Guide 2210 tells us:
Internal auditors must establish engagement objectives as part of planning for each
engagement. Objectives typically are developed based on key risks that have been
identified related to the area or process under review.
Additional language in this Implementation Guide states:
Internal auditors can formulate preliminary objectives of engagements through a
review of the annual internal audit plan and prior engagement results, discussions
with stakeholders, and consideration of the mission, vision, and objectives of the area
or process under review. The preliminary objectives are further enhanced through risk
assessment exercises to cover the governance, risk management, and controls of the
area or process under review. The engagement objectives articulate what the
engagement is specifically attempting to accomplish and determine the engagement
scope.
Having clear and agreed-upon engagement objectives is essential to an
effective and efficient audit and to the task of supervising audit
performance. The engagement objectives describe the quantity and quality
of work to be performed, which in turn affect the audit work program,
staffing, scheduling, and communication with clients and stakeholders.
Developing engagement objectives based on a risk assessment of the area to
be audited supports the fundamental risk-based focus of the internal audit
activity.
Differentiating Engagement Objectives from Operational
Objectives
The preliminary survey (discussed more in Chapter 1 of Section III) looks
at the objectives of the operation—what the activity is supposed to achieve.
If the internal auditor finds that the area lacks objectives, time is usually
spent working with operational management to develop appropriate ones.
Engagement objectives are different from management’s operational
objectives. Where operational objectives specify what the client hopes to
accomplish, engagement objectives deal with what the internal auditor hopes
to accomplish. Engagement objectives are the internal auditor’s means for
determining how well the operating objectives are being met. Consider the
following examples of the operating and engagement objectives for different
engagements.
Type of audit:
Operating
objective:
Engagement
objective:
Type of audit:
Operating
objective:
Engagement
objective:
Marketing department
Increase product recognition in China market.
Evaluate controls in place to increase product recognition in China.
Pharmaceutical research and development
Bring three new drugs to market in the next five years.
Evaluate controls in place to bring three new drugs to market in the
next five years.
Type of audit:
Accounts payable department
Operating
objective:
Pay invoices accurately and on a timely basis after verifying receipt of
services or products.
Engagement
objective:
Evaluate accurate and timely payment of invoices based on mitigation
of risks, adequacy of controls, and compliance with financial policies
and procedures.
Type of audit:
Operating
objective:
Engagement
objective:
Manufacturing process
Complete the manufacture of x number of products each day.
Evaluate the accuracy and efficiency of the manufacturing process to
determine that the products manufactured each day are complete,
accurate, and satisfactory for delivery to customers. Evaluate the
accuracy and reliability of the measurement, reporting, and monitoring
systems for production data.
Engagement objectives typically fall into three categories—operations,
reporting, and compliance—which are aligned with the COSO internal
control framework. These categories are at the top of the COSO internal
framework cube, shown in Exhibit II-2.
Exhibit II-2: COSO Internal Control Framework Cube
Operations objectives relate to how an entity strives to ensure that it can
effectively and efficiently manage business operations. Reporting objectives
relate to developing reliable financial and nonfinancial reports. Compliance
objectives relate to determining that an entity is complying with applicable
laws and regulations.
These categories are described further in Exhibit II-3.
Exhibit II-3: Broad Categories of Engagement Objectives
Category
Operations
Description
Measures such as:
• Profitability (maximize revenue or minimize costs).
• Delivery of excellent products and services.
• Reduced processing time.
• Safeguarding of assets (physical, human, information).
• Support of organizational mission and vision.
• Support of appropriate work environment for employees.
Examples:
• Evaluate if controls installed in systems are working to maximize the accuracy
and efficiency of a process.
• Evaluate controls over physical access to a facility.
• Evaluate controls over safeguarding of assets.
• Evaluate if the right personnel have been assigned the proper job responsibilities
and have received appropriate training.
Reporting
Measures such as:
• Maintenance of accurate financial records.
• Collection of useful, reliable, and timely information for decision making.
• External regulatory reporting.
Examples:
• Evaluate controls over timely recognition of revenues.
• Evaluate if the identification and reporting of risks and controls are adequate for
the enterprise risk management reporting process.
• Identify and evaluate if the correct reports are given to the proper managers in
the organization.
• Validate the accuracy of reporting by confirming the accuracy of data provided.
• Evaluate if the personnel responsible for reporting are providing accurate and
timely reports.
Compliance
Measures such as:
• Compliance with applicable laws and regulations.
• Compliance with internal policies and procedures.
Examples:
• Evaluate the process in place to remain updated with legal and regulatory
changes.
• Evaluate how organizational compliance objectives and procedures are
communicated to all employees.
• Evaluate employee perception of the “tone at the top” in the business function
being audited.
• Evaluate controls in place to ensure that the travel expense policy is followed.
Other aspects of the COSO framework are addressed elsewhere in this
learning system.
At this point in the planning process, the internal audit activity may already
have engagement objectives established. For example, if the upcoming audit
is part of a regular rotation, the internal auditor would have engagement
objectives from prior audits. However, these internal audit objectives may
need to be refined, updated, and/or prioritized, taking into account activity
risk assessment results.
Ultimately, engagement objectives become the cornerstone for the work
program and the audit fieldwork.
Engagement Scope
As noted earlier, Performance Standard 2220, “Engagement Scope,” states
that the audit scope must satisfy the objectives of the engagement.
Engagement procedures help the internal auditor to accomplish engagement
objectives. Collectively, engagement objectives and procedures are what
define the engagement scope. For assurance engagements, audit scope and
objectives are defined by the auditor. For consulting engagements, however,
objectives and scope are defined collaboratively by both the auditor and the
engagement client.
Scope establishes the boundaries of the internal audit. Essentially, it
identifies what the internal auditor will do by specifying which activities
will be reviewed and which activities will be excluded from the
engagement.
A scope statement may include a description of the nature and extent of the
audit work as well as supportive information such as the time period
reviewed.
Implementation Standards 2220.A1, 2220.A2, and 2220.C1, which were
described in the introduction to this chapter, apply to establishing audit
scope for assurance and consulting engagements.
Let’s return to our previous example of the accounts payable department.
Here are the objectives.
Operating
objective:
Pay invoices accurately and on a timely basis after verifying receipt of
services or products.
Engagement
objective:
Evaluate accurate and timely payment of invoices based on mitigation
of risks, adequacy of controls, and compliance with financial policies
and procedures.
Based on these objectives, a scope statement should specify the following.
Inclusion:
What areas are within the scope (e.g., invoices, vendor management,
accounts payable application, invoice processing, payments, accounting
journal entries)?
Exclusion:
What are areas that a reasonable person would expect to find
intentionally excluded from the scope?
A plausible scope statement example might be:
The internal audit activity will conduct an operational assessment of the accounts
payable department that will take into account all payments of invoices from January
1, 20xx, to December 31, 20xx.
Scope Limitations
To fulfill the internal audit activity’s professional obligation to maintain
independence and objectivity, any restriction placed on internal audit that
thwarts it from fulfilling the intended scope should be communicated,
preferably in writing, to the board, audit committee, or other appropriate
governing authority. Standard 1110.A1 states:
The internal audit activity must be free from interference in determining the scope of
internal auditing, performing work, and communicating results. The chief audit
executive must disclose such interference to the board and discuss the implications.
See also the interpretation to Standard 1130 and the definition of
impairment, which includes scope limitations as an example of potential
impairment.
Engagement Criteria
Before an audit engagement can be performed, the internal auditor-in-charge
(the lead auditor) must identify the criteria to be used during the
engagement. Internal audit criteria are the benchmarks against which the
subject matter of the engagement can be assessed. They are “measuring
sticks” that the internal audit activity will use in drawing conclusions about
performance relative to audit objectives. Meaningful assurance can be
provided to the board and senior management only if the conclusions
reached are made in relation to suitable criteria.
What constitutes suitable criteria? Generally speaking, criteria should be
consistent with audit engagement objectives and ultimately yield useful
information to the client. The lack of suitable criteria may result in the
internal auditor drawing inappropriate conclusions.
Examples of criteria for assurance engagements could include:
• Acts and regulations.
• Policies and procedures.
• Standards or guidelines.
• Risk management.
• Control frameworks (i.e., COSO, ISO 31000, Control Objectives for
Information and Related Technology [COBIT]).
• Performance indicators.
• Client management roles and responsibilities.
• Industry best practices.
• Guidance provided by recognized bodies of experts.
• Benchmark evidence (benchmark studies).
Using performance indicators as the engagement criteria in an internal audit
intended to provide assurance about process efficiency is an example of
linking criteria with engagement objectives. The lead internal auditor would
communicate this to management and also describe what will be tested,
how much time will be spent testing, what access will be necessary, and so
on.
When there are no generally accepted criteria (such as the examples listed)
consistent with the audit engagement objectives, the lead internal auditor
will need to identify the criteria suitable for the engagement through
consultation with client management.
Topic B: Key Risks and Controls (Level P)
The Definition of Internal Auditing states that a focus of internal audit is to
“add value and improve an organization’s operations.” During engagement
planning, internal auditors need to consider how the individual engagement
can support both goals. A risk-based approach in all activities is an
important component in achieving these goals, as it focuses the audit on the
most important areas.
The Standards Glossary defines risk as “the possibility of an event
occurring that will have an impact on the achievement of objectives.” The
threats to the achievement of the goals and the barriers to success in the
engagement are the risks.
In fact, the Standards specify that an analysis of the risks for the area to be
audited should be a part of the planning process. The description of
Performance Standard 2201, “Planning Considerations,” earlier in this
chapter noted factors internal auditors must consider in planning an
engagement, including risk exposure and control issues. According to
Implementation Guide 2060, “Reporting to Senior Management and the
Board,” significant risk exposures and control issues are “those that would
require the attention of senior management and the board, which may
include conflicts of interest, control weaknesses, errors, fraud, illegal acts,
ineffectiveness, and inefficiency.”
Internal audit should look at all the key areas of risk and how they are
managed. If the internal auditor believes that the organization has risk
exposures that are unacceptable or outside the organization’s risk appetite,
these should be discussed with management and escalated to senior
management and the board as appropriate.
Risk-Based Approach
A risk-based approach requires internal auditors to first understand the
entity and its environment in order to identify risks. An experienced
auditor’s skill and judgment is exercised to put focus on the specific
information that should be obtained through this process. This reduces the
potential for unnecessary information or information overload, saving
critical time and resources.
Understanding the entity involves documenting:
• Operational objectives or goals.
• Level of compliance with laws, policies, and procedures.
• Key processes.
• Organizational structure (e.g., reporting relationships and supervision).
• Information systems.
• Identified key risks.
• Current controls.
Gathering this information can be done in many ways, including:
• Initial client meetings.
• Conducting a preliminary survey.
• Performing analytical review procedures.
• Conducting interviews.
• Observation.
• Inspection of processes and documentation.
• Benchmarking.
• Reviewing prior internal audit reports and those of other assurance
providers.
By assessing identified risks, several factors (such as the nature of the risks,
relevant controls, and level of evidence) are taken into consideration. The
result of a risk assessment categorizes the engagement into areas of
significant risk and areas of normal risk.
Control Types
Control types can be classified in a number of ways. What follows is
information on various types of controls. This information is also presented
in the Part 1 materials; if you are already familiar with the control types,
you can skip to “Understanding Internal Controls.”
Entity-, Process-, and Transaction-Level Controls
Some controls are designed to operate at a high, overarching level, while
others apply to specific processes or transactions.
Entity-Level Controls
Entity-level controls apply to the entire organization and are designed both
to ensure that organizational objectives are achieved and to mitigate risks
that threaten the organization as a whole.
Major subtypes of entity-level controls include the following:
• Governance controls. Governance controls establish the control culture,
clarify organizational expectations, and include organization-wide policies
and procedures. Examples of establishing the culture and clarifying
expectations include instituting audit committee oversight over controls or
communicating the board’s and top management’s risk appetite or attitude
toward financial reporting; examples of policies and procedures include a
code of ethics, compliance policies, IT policies, and management
procedures such as conducting enterprise risk management.
• Management oversight controls. These controls are set at the business
unit or line management level to address achievement of business unit
objectives and mitigation of business unit risks. Examples include risk
committees, some period-end controls, and IT general controls.
Process-Level Controls
Process-level controls are established by a process owner to ensure that the
objectives of the process are achieved and that process-level risks are
addressed. Examples include supervision, monitoring, oversight, processlevel risk assessments, performance evaluations, key account reconciliation,
and inventory counts.
Transaction-Level Controls
Transaction-level controls are specific to individual transactions. They exist
to ensure that the objectives of the transaction are achieved and that
transaction-specific risks are addressed. Examples include documentation
requirements, segregation of duties, required authorizations or approvals,
and IT application controls (input, processing, output).
Key Controls versus Secondary Controls
Controls can also be classified based on their relative importance.
Sawyer’s provides the following definitions:
• Key controls are those “that must operate effectively to reduce a
significant risk to an acceptable level.”
• Secondary controls “help the process run smoothly but are not essential.”
Key controls are those controls that, if omitted, would make it very difficult
to achieve the desired outcome or business objective. Secondary controls
exist either to mitigate risks that are not considered significant or as a
redundant control already addressed by a key control. The clear intent
behind identifying key controls is to ensure that management supervision
and controls testing and other audit procedures are efficient, do not waste
time and resources, and focus on key risks and the achievement of
organizational objectives. Each risk at the entity, process, or transaction
level that has been identified as significant in a risk assessment process will
have one or more key controls associated with it. Secondary controls are the
remaining controls in a system.
Ensuring that the “tone at the top” reinforces rather than undermines
process-level controls is an example of a key governance control at the
entity level. A related secondary control might be ensuring that mission and
vision statements are revisited and recommunicated. At the management
oversight level of entity controls, some key controls exist that can monitor
the effectiveness of several other secondary controls and indicate when
breakdowns in the lower-level controls are occurring, thus providing an
early indicator of a control failure and reducing the amount of testing
needed for the secondary controls. At the process level, reconciliations of
key accounts rather than all accounts (secondary controls) could provide
sufficient evidence of whether the entire process is likely to achieve its
objectives. At the transaction level, a check sum for an accounting entry
could provide evidence that other transaction-level controls are operating
effectively. Controls allowing only a certain range of numeric data in a field
might be secondary.
Controls by Function
Many terms commonly used to describe types of controls are based on their
functions.
• Preventive. These are proactive controls that deter undesirable events from
occurring. An example is a reward mechanism based on a relevant key
performance indicator for an area rather than on achieving an arbitrary
budget number.
• Detective. Detective controls are reactive and detect undesirable events
that have occurred. Examples are account reconciliations or exception
reports.
• Corrective. Corrective controls are reactive controls designed to allow
manual or automated correction of errors or irregularities discovered by
detective controls, including resolution of duplicate payments in a cash
disbursement system, audit trails, or backup and recovery procedures.
• Directive. Directive controls are proactive controls that cause or encourage
a desirable event to occur. Guidelines, training programs, and incentive
plans are examples of directive controls.
• Mitigating. Mitigating controls reduce the potential impact should an
event occur. Insurance is a prime example of a mitigating control.
• Compensating. These controls compensate for the lack of an expected
control. For example, close supervisory review may compensate for a lack
of segregation of duties where a small staff size makes proper segregation
impractical.
• Redundant. Redundant or backup controls duplicate a control objective or
a secondary control that operates only if a key control fails, for example, a
spillover pool below a toxic substance holding tank.
Active/Manual versus Passive/Automated Controls
Controls may be categorized as active (manual) or passive (automated):
• An active or manual control implies a task that prevents or detects a
deviation from the approved procedure. We can think of it as a control
that works by some type of conscious intervention. An example is a
manager’s review of transactions.
• A passive or automated control operates without human intervention. An
example may be controls built into the computer system or a relationship
or process that possesses control implications. We can think of it as a
control that works by just being there. An example is a thermostat set to
maintain the temperature of a room.
Hard versus Soft Controls
The term “soft controls” was first used in a 1980 publication entitled
Internal Control in U.S. Corporations: The State of the Art by Robert K.
Mautz, which has since become a major influence on internal auditors.
Generally speaking, hard controls are more scientific in nature and soft
controls are more humanistic. These terms can be defined as follows:
• Hard controls tend to be quantitative and objective, meaning that
traditional audit tests can be used to test compliance. Inspecting meeting
minutes or performing a monthly budget-to-actual analysis are examples.
• Soft controls tend to be qualitative and subjective and are intended to be
indicative of the culture of an organization, such as states of mind or
perceptions. For example, soft controls may include policies to determine
whether a body of knowledge is sufficient to corroborate results or support
conclusions.
Exhibit II-4 lists common examples of both types.
Exhibit II-4: Hard and Soft Internal Controls
Hard Control Examples
• Policy/procedure
• Organizational structure
• Bureaucracy
• Restrictive formal processes
• Centralized decision making
Soft Control Examples
• Competence
• Trust
• Shared values
• Strong leadership
• High expectations
• Openness
• High ethical standards
The assessment of control efficiency and effectiveness is a time-worn staple
of the internal audit activity. Yet to focus on only hard controls (e.g.,
documented policies and procedures) at the process level yields an
incomplete assessment. To evaluate internal control and provide reasonable
assurance to senior management and the board, the internal audit activity
must include the intangible, inherently subjective soft controls.
IT Controls
Similar to the concept of entity-level versus process- and transaction-level
controls, information technology also has levels of control to address risks
associated with IT systems:
• IT general controls. IT general controls (ITGC) are entity-level controls
that apply to general IT processes such as change management,
deployment, access security, and operations and that can be applied to
most if not all information systems in general. IT general controls consist
of governance controls such as a privacy policy as well as management
oversight controls such as testing standards or segregation of IT duties.
• Application or technical controls. These are process- or transaction-level
controls that are usually specific to a given application but may also
control larger technical processes such as system access rights. Application
controls are sometimes grouped by common function:
• Input controls verify the integrity of data as it is manually or
automatically entered into a system. For example, a control total might
verify that the proper number of records is entered.
• Processing controls check that data processing tasks are accurate,
complete, and valid. For example, a control total might be compared at
various processing points.
• Output controls verify that the data outputs are accurate, complete, and
valid. An example is a control to ensure that output is being sent to
and received by the intended recipients and no other person or system.
An additional IT control tool is an audit trail, which is a permanent record
of every computer input, process, and output activity. Auditors can check
transaction logs, review a list of executed controls, or review error listings.
Understanding Internal Controls
Internal auditors need to understand internal controls that are considered
likely to be relevant to the engagement and its related risks (not all the
controls the organization employs in managing its business).
The COSO Internal Control—Integrated Framework (refer to the COSO
cube shown in the previous topic) assists internal auditors to focus on
gaining an understanding of relevant controls by evaluating the
organization’s internal controls based on five components:
• Control environment. The culture of the organization relative to internal
control.
• Risk assessment. How the organization identifies, assesses, prioritizes, and
responds to its own business risks.
• Control activities (relevant to engagement). The policies and procedures
that help ensure that management directives are carried out. Examples
include physical controls as well as those relating to authorization,
performance reviews, information processing, and segregation of duties.
• Information and communication. Systems related to the capture of
significant transactions, events, conditions or accounting estimates,
procedures, and reporting.
• Monitoring activities. Activities the organization uses to monitor control
activities as well as how it takes action to address any identified
deficiencies.
Understanding internal controls in this way enables the internal auditor to
identify what relevant controls (if any) are in place to test, whether the
absence of controls results in unacceptable levels of inherent risk, how or
when to combine controls testing with substantive testing, and how to test
the operating effectiveness of controls and the extent of reliance that can be
placed on internal controls (thereby reducing the extent of substantive
testing).
Topic C: Detailed Risk Assessments (Level P)
We have learned that internal audit engagements are intended to provide
assurance to senior management and board. Engagements are chosen largely
in response to enterprise-wide risk assessment.
Assessing risk in an engagement area furthers organizational risk
management by helping to ensure that:
• The engagement planning is aligned with the entity-level risk assessment.
• The engagement planning leverages information from the entity-level risk
assessment.
• The engagement is focused on risk.
Additional language about risk assessment for planning purposes is found in
Implementation Standards 2210.A1, 2210.A2, 2210.A3, 2210.C1, and
2210.C2, as described earlier in this chapter, and in Implementation Guide
2210, “Engagement Objectives.” A portion of this recommended guidance
says that internal auditors establish engagement objectives to address the
risks associated with the activity under review and that:
It is helpful to determine whether a risk assessment was performed during the
engagement’s planning phase…. [The auditor should] attain a thorough understanding
of the risks of both the organization and the area or process under review.
Internal auditors need to gain sufficient confidence that relevant risks are
identified for the specific engagement. They should consider management’s
assessment of risks relevant to the activity under review. The internal
auditor also considers:
• The reliability of management’s assessment of risk.
• Management’s process for monitoring, reporting, and resolving risk and
control issues.
• Management’s reporting of events that have exceeded the limits of the
organization’s risk appetite and management’s response to those reports.
• Risks in activities related to the activity under review.
Risk Control Matrix
A risk control matrix (also known as an engagement-level or audit risk
assessment) is a useful tool to help ensure that internal auditors adequately
account for risk at the engagement level and ensure that all significant risks
identified are addressed in subsequent fieldwork. No two matrices are
exactly alike. The most common features are shown in the example in
Exhibit II-5.
Exhibit II-5: Risk Control Matrix Features
Business
Objective(s)
Risks
Likelihood/
Significance
(H, M, L)
Controls
Evaluation
of
Adequacy
Tests of
Effectiveness
Final
Evaluation
Step 1: Identify business objectives. This step considers business objectives for the unit or process,
not the control objectives. This can be accomplished by asking management to discuss their goals
for the activity being reviewed. If no business objectives exist, the internal audit team should work
with management to develop appropriate ones.
Step 2: Identify risks to business objectives. Risks that need to be controlled or mitigated can be
identified in one of two ways: 1) Ask management what events or circumstances could prevent them
from achieving their goals, or 2) walk management though the organizational risks identified at the
macro (entity) level and ask whether or not each of these could impact the achievement of business
objectives.
Step 3: Rate each risk in terms of likelihood and significance (L/S). Risks are usually rated as
high (H), medium (M), or low (L), depending on how likely it is that the risk will occur and how
significant the impact of its occurrence would be. Some ratings combine likelihood and significance,
such as high likelihood/high significance or low likelihood/low significance.
Step 4: Identify the controls. Identify the specific control activity that will address the identified
risk. For example, this could be management approval, reconciliation, or segregation of duties.
Ensure that the response is consistent with the organization’s risk appetite and risk tolerance levels.
Step 5: Evaluate the adequacy of controls. This step asks the question “Are the control processes
for managing this risk well-designed?” Analytic skills and professional judgment are used to
determine the answer. The internal auditor:
• Identifies and documents the controls management says are in place.
• Evaluates how well-designed the controls are—if they are effective, efficient, economical, and
working the way they were designed to work.
Step 6: Test the effectiveness of controls. Controls that are deemed adequate or well-designed are
tested to determine if they are in fact operating as intended.
Step 7: Arrive at the final opinion on adequacy and effectiveness of controls. Final opinions are
often stated in terms of both criteria. For example, a control may be deemed adequate but
ineffective due to lack of compliance with procedures. Or the design of the control may be flawed
because it overcontrols insignificant risks, leading to an opinion of inadequacy.
Source: © James Roth, AuditTrends.
Again, keep in mind that the steps carry over from the planning phase to
audit fieldwork. Flowcharts and narrative techniques are sometimes used in
conjunction with a risk control matrix.
A risk control matrix can be time-consuming to develop. But the benefits of
using the tool are significant. In addition to generally helping to account for
risk at the engagement level and identify all significant risks, developing a
matrix during the planning phase:
• Sets the stage for a more effective and efficient audit by focusing the audit
on the areas of greatest risk.
• Ties the audit together by documenting the complete thought process from
risk identification to audit program development.
• “Teaches” the risk assessment thought process; area management becomes
a more effective “owner” of control.
• Facilitates participatory auditing.
Many organizations use the risk control matrix as a way to develop a list of
all the controls to be tested during the internal audit. By including this level
of detail in the matrix, it can also serve as the work/audit program for the
engagement. Further, incorporating testing results can make the risk control
matrix the major workpaper for each area of the engagement.
Topic D: Engagement Procedures and Work
Programs (Level P)
Engagement objectives help define an audit’s documentation—its
measurable, verifiable work products. Audit documentation lists the work
that will be performed in the course of the engagement, for example,
research activities (process mapping, walkthroughs, interviews), the risk
assessment, the audit program, testing results, audit findings, and the draft
and final reports.
This topic first looks at the internal audit charter and audit activity plan and
then focuses on the audit or work program, which designs the tasks needed
to provide the level and scope of assurance specified in the engagement
objectives.
Audit Activity Charter and Plan
Internal Audit Activity Charter
The internal audit activity operates through an audit charter, which is
prepared by the chief audit executive, approved by senior management and
the board or audit committee, and communicated to engagement clients.
Attribute Standard 1000, “Purpose, Authority, and Responsibility,” addresses
the board’s approval of the audit charter:
The purpose, authority, and responsibility of the internal audit activity must be
formally defined in an internal audit charter, consistent with the Mission of Internal
Audit and the mandatory elements of the International Professional Practices
Framework (the Core Principles for the Professional Practice of Internal Auditing, the
Code of Ethics, the Standards, and the Definition of Internal Auditing). The CAE
must periodically review the internal audit charter and present it to senior
management and the board for approval.
The significance of this approval is that the internal audit activity has the
full authority and backing of the board in carrying out its activities, as long
as it conforms to the charter.
Internal Audit Activity Plan
The CAE presents a plan of audit engagements, usually once a year, to
senior management and the board or audit committee for approval (per
Standard 2020). Just as the charter gives the internal audit activity the
authority to do its work without hindrance by anyone in the organization,
the approval of the audit plan authorizes the specific audit engagements. We
should emphasize, though, that audit plans are flexible and typically change
throughout the year as new risks emerge. The CAE should inform the audit
committee of additions to or deletions from the audit plan but does not
need specific approval of a new project before carrying it out.
In addition to communicating the internal audit activity’s scope of work,
work schedule, staffing plan, and financial budget to the board for review
and approval, updates outlining any significant changes to the plan must be
presented to the board for approval, as warranted. Both the engagement plan
and the interim updates should contain sufficient information to enable the
board to ascertain whether the internal audit activity’s objectives and plans
support those of the organization and the board.
Engagement Work Program
All of the planning activities discussed thus far lead to the preparation of
the engagement work program. The resulting document reflects the
systematic and disciplined approach mentioned in the Definition of Internal
Auditing as it will apply to the audit to be performed. A work program
provides documentation of the planned engagement that can be used to
secure management approval.
The Standards Glossary defines an engagement work program as “a
document that lists the procedures to be followed during an engagement,
designed to achieve the engagement plan.” An engagement work program is
also called an audit program during assurance engagements; thus, the two
terms are often interchanged.
Additional language related to preparation of an engagement work program
is contained in Implementation Standards 2240.A1 and 2240.C1; see the
beginning of this chapter.
Well-crafted, an engagement work program:
• Starts with clearly specifying the engagement work and audit program
objectives.
• Provides an outline of work to be performed and facilitates an
understanding of the audited area.
• Furnishes evidence that the work is adequately planned.
• Provides a record for audit management review.
• Provides assurances that all significant risks have received adequate
consideration.
• Assists in controlling work and assignment responsibilities.
• Gives order and coherence to the audit.
• Addresses the need for conclusions responsive to audit objectives.
Specific time frames for activities are often included in the document. But,
as experience has shown, the work program often needs minor
modifications at the onset of the engagement. For example, the full extent
of testing required may not be known until the testing procedures are
underway. Minor adjustments are to be expected early in the engagement.
Any major modifications should be approved by appropriate internal audit
leadership.
Engagement work plans should be approved in writing by the CAE or
designee prior to the commencement of engagement work, where
practicable. Initially, approval may be obtained orally, if factors preclude
obtaining written approval prior to commencing engagement work.
Adjustments to engagement work plans should be approved in a timely
manner.
An engagement work program becomes guidance for performing the
engagement. A sample work program is shown in Exhibit II-6.
Exhibit II-6: Work Program Example
Audit Objectives:
Obj. 1
To determine that a preventive maintenance schedule has been developed and
adhered to so that vehicles are safe and drivable.
Obj. 2
To determine that the authorized vendor lists and contracts are current and based on
company standards, so that external maintenance and repairs are performed by
qualified vendors.
Obj. 3
To determine that a repair log of in-house and vendor repairs is properly prepared
so that maintenance is performed and documented according to company standards.
Obj. 4
To determine that the policy governing in-house repairs is followed so that minor
mechanical repairs are made in-house.
Obj. 5
To determine that vendor price lists are current, so that cars are repaired by
authorized vendors at reasonable rates.
Obj. 6
To determine that supporting documentation exists and that expenses can be
properly monitored and verified for reasonableness.
Obj. 7
To determine that maintenance expenses are properly recorded in the general ledger
and are monitored.
Test Step
1. Obtain preventive maintenance schedule, a cars inventory listing at
12/31/xx, and a report of car additions and deletions for the audit
period.
A. Verify that all cars owned during the audit period are on the
schedule (compare preventive maintenance schedule to inventory
records, reports).
B. Attest to the completeness of the schedule. Determine that the
schedule contains the VEHNO, VIN, car description, scheduled
maintenance dates, actual maintenance dates, and a description of
the procedures performed. Document any exceptions and perform
follow-up procedures.
2. Select 30 cars from the preventive maintenance schedule for the audit
period and test for the following attributes:
A. Preventive maintenance was performed and documented on
schedule in accordance with company policy. (Belts, hoses, fluids,
WP
Reference
Initial/Date
brakes checked...)
B. Preventive maintenance was performed within established time
frames or mileage parameters.
C. Mechanic’s record was obtained on preventive maintenance
schedule repairs to be performed by outside vendors. Trace repair
to repair log, vendor invoice, and canceled check.
D. Mechanics initialed and dated preventive maintenance schedule.
E. Evidence obtained of supervisory review.
3. Obtain vendor contracts; review for propriety and reasonableness.
Determine that contracts are current and in agreement with the
authorized vendor list.
4. Obtain repair log for the audit period.
A. Determine that work performed on cars is documented on the
repair log and performed by approved vendors.
B. Examine repair log and determine that work performed by outside
vendors is in line with company policy. (Work that can be
performed in-house should not be performed by outside vendors.)
5. Select 30 repairs from the repair log during the audit period and test
for the following attributes:
A. Proper completion of the repair log.
B. Repair work performed by a vendor is adequately described.
C. Mechanics initialed and dated repair log.
D. Evidence of supervisory review.
E. Trace repair to service request form and invoice.
F. Repairs performed by authorized vendors. Trace to authorized
vendor list.
G. If in-house repair, ascertain that current policy authorizes in-house
repair.
6. Obtain current price lists for vendor repairs. Compare to an industry
auto parts and service guide to determine reasonableness of prices.
Source: “Auditing Techniques” course, The Institute of Internal Auditors, 2006 (abridged).
Note that an actual work program would contain additional steps to achieve
the audit objectives. The final steps would be addressing identified potential
risk/control issues with management and adding a concluding statement
responsive to audit program objectives.
Keep in mind that each work program will be unique, depending on the
scope of the engagement. For very high risk audit areas, very complex
business activities or functions, or engagements with numerous audit
objectives, work programs are typically very large. Work programs are less
complex in small organizations and for internal audits being done by one
person or when the budgeted time is relatively short.
Engagement Procedures
Engagement procedures are the means to attain engagement objectives. As
in other areas of engagement planning, the internal auditor’s judgment is
key in determining the specific procedures to be employed and the level of
evaluation necessary. Similar logic applies to determining which procedures
apply to which engagement objectives.
Engagement procedures must be relevant to the selected objectives. A
procedure may be applicable to the internal audit as a whole, but if it is
erroneously applied to an objective, the result will be irrelevant. A simple
example of this is designating interviewing in lieu of testing in a scenario
that clearly necessitates actual testing. The interview results might be
interesting, but they will be irrelevant in providing assurance.
There is no definitive list of internal audit procedures or a road map for
matching procedures to engagement objectives. The internal auditor needs to
discriminate between procedures that may be relevant and those that are
unimportant.
One approach is to ensure that procedures are designed to test that controls
achieve key attributes. For example, in an audit of a purchasing area, an
engagement objective may be to assure that purchase transactions are
authorized. In this case, engagement procedures should include verifying
that the documentation received by purchasing agents, such as a purchase
requisition, contains signed approvals. Tasks such as the review of receiving
memos and vendor invoices would provide no relevant information.
“Authorized” is the attribute deemed to be important in this function.
Other attributes include accurate, reliable, timely, safe, secure, consistent,
appropriate, actual, reasonable, compliant, and complete. For example, when
assessing a budgeting process, it’s more likely that an auditor would look
for “reasonable” processes than for “accurate” processes, because budgets
are only estimates. However, the budget should also be “complete,”
including input from all departments within the organization.
Data Collection
The Standards are very clear on the need for gathering the right facts to
support audit objectives.
• Standard 2300, “Performing the Engagement”: “Internal auditors must
identify, analyze, evaluate, and document sufficient information to achieve
the engagement’s objectives.”
• Standard 2310, “Identifying Information”: “Internal auditors must identify
sufficient, reliable, relevant, and useful information to achieve the
engagement’s objectives.”
Sources of Evidence
The information gathered to support the conclusions and recommendations
in a final audit report—audit evidence—can derive from the following:
• People. This might include management of the area being audited, other
personnel, individuals in other areas that interact with the audited area,
and external individuals or groups (e.g., vendors). Representations or
testimonial evidence include statements or written responses to inquiries or
interviews, questionnaires, etc. They should have documentary support,
such as system flowcharts or written policies and procedures.
• Physical evidence. Physical evidence generally is considered more reliable
than the testimony of a person. It includes photographs, charts, maps,
graphs, pictures, or videos of an act or condition.
• Documentation. Documentation is the most common source of audit
evidence. It can be recorded in media other than paper and includes,
among other examples, letters, memos, emails, invoices (external
documents) and accounting records, program listings, activity and control
logs, and systems development documentation. A caution is that
documents can be forged.
• Operations. This includes activities, systems, work processes, and
functions.
• Measurements. Measurements may be made by the auditor or the audit
client. They can include measurements made on nominal, ordinal, linear,
or ratio scales.
• Software and electronic databases. This includes operating and control
software, applications, and data storage.
The source of evidence can add to, or detract from, its persuasiveness.
Corroborative audit evidence from an independent third party can be more
reliable than audit evidence from the organization being audited. Third-party
evidence may be called external evidence to distinguish it from internal
evidence provided by the audit customer.
The strength or weakness of the evidence depends on how persuasive it is.
Audit evidence is persuasive if it enables the internal auditor to formulate
well-founded conclusions and advice confidently. Recall the characteristics
of information identified in Standard 2310: sufficient, reliable, relevant, and
useful.
• Sufficient means that there should be enough evidence and different but
related pieces of evidence should corroborate each other.
• Reliable implies that the evidence comes from a credible source. This
considers whether or not the internal auditor directly obtained the
evidence.
• Relevance refers to evidence that may be reliable in itself but is of no use
if it does not pertain to the audit objective. Relying on evidence that has
little or no pertinence increases audit risk—the risk of reaching invalid
conclusions and providing faulty advice.
• The usefulness of information is a function of whether the information is
germane to the organization.
The auditor must also consider when the evidence is needed and the
methods used to collect the evidence in determining which engagement
procedures will be performed. Management culture and operating style may
impact evidence-gathering techniques. The internal auditor should consider
what types of evidence will be most persuasive to those managers who will
be making decisions concerning corrective actions.
Audit Tests
Audit tests are a means to obtain the data or evidence described above.
Tests should be chosen according to the type of information that is required.
Typical audit tests include:
• Physical examination of objects. For example, an audit test might entail
verifying that a safety device is attached to a piece of equipment, checking
an inventory listing against the physical items located in an office or
warehouse, or confirming that cash or securities are presented as indicated.
• Documentation/inspection tests. These tests involve scrutiny of
documents used in the organization. For example, the auditor might test
that a business continuity plan has been revised and distributed as
specified in policy. Entrance logs may be checked to test security controls.
Contracts may be examined for proper signatures. Documents may be
traced backward (vouching) or forward (tracing). Board meeting minutes
might be inspected.
• Inquiries. These are appropriate when the auditor is seeking evidence
from people, such as confirmation that employees have received the
necessary training to perform certain tasks.
• Observations. These may be necessary to assure that work processes or
tasks are being performed according to defined procedures. For example,
the auditor may observe a business process to test whether quality controls
are working. Controls may be re-performed and observed to assure that
they are performing effectively and as designed.
• Computations and analysis. These can gather evidence of errors and
possibly fraud.
• Comparisons. These can reveal evidence of procedures that are not being
followed, variances between specifications and actual performance that
indicate lapses in quality controls, or improbable trends or relationships
that can be indicators of red flags for fraud. For example, an increase in
an expensive raw material purchasing quantity can be compared to sales
and beginning and ending inventory levels of end units using that raw
material (plus records on levels of scrap or rework) to test fraud controls.
Topic E: Staff and Resources Needed for
Engagements (Level P)
As noted earlier, Performance Standard 2230, “Engagement Resource
Allocation,” states:
Internal auditors must determine appropriate and sufficient resources to achieve
engagement objectives based on an evaluation of the nature and complexity of each
engagement, time constraints, and available resources.
The nature and complexity of the engagement can be assessed by evaluating
the results of the enterprise-wide risk assessment and the preliminary survey
and risk assessment of the engagement area under review.
The following additional guidance dealing with how to determine the
appropriate level of staff and resources is contained in Implementation
Guide 2230:
Internal auditors consider the following when determining the appropriateness and
sufficiency of resources:
•
How much time each step of the engagement work program is expected to take,
as well as constraints for any relevant party.
•
•
The number and experience level of the internal audit staff.
•
Availability of external resources where additional knowledge and competencies
are required.
•
Training needs of internal auditors as each engagement assignment serves as a
basis for meeting the internal audit activity’s developmental needs.
Knowledge, skills, and other competencies of the internal audit staff when
selecting internal auditors for the engagement.
In determining the level of audit staff for an engagement, the audit leader
should consider:
• The objectives of the audit and their implications in terms of
necessary skills and knowledge. Team leaders should consider the need
for “soft” skills as well, especially in audits of sensitive areas such as
governance. Well-known and respected staff members may be needed to
staff these engagements.
• The availability of staff with the necessary knowledge and skills. If
staff have experience in the audit area, the audit time may be shortened. If
less-experienced auditors are used, more time may have to be allotted to
review their work. (At the same time, providing experience to these
auditors is an essential part of developing the human resources of the
internal audit activity.)
• The nature of the audit’s deadline. This may be a concern if a report is
needed for compliance reasons or due diligence. It may be necessary to
complete this audit in order to proceed to a related audit area.
• The activity’s priorities and resources. Some resources may be necessary
for audits considered to have a higher priority in the audit plan.
A decision may be made that additional knowledge, skills, and
competencies are needed to perform the engagement. External services may
need to be obtained to support or complement the internal audit team.
Information technology is an example of an area that often requires external
expertise.
Budget Considerations
The success of an engagement is often judged by achievement to the level
of standards, fulfillment of engagement objectives, and completion within
budget. Setting a budget is an important consideration in engagement
planning. It is often the principal control over the audit engagement.
Engagement budgets include statements of the time allocated, as time is the
key factor in an internal audit activity. Budgeted hours (or time budgets)
must be realistic and achievable. Some internal audit activities use deadlines
to manage their budgets. They are often based on the level of associated
risk.
The work accomplished in the time scheduled largely reflects the collective
skills and expertise of the internal audit team. Logistics (such as client
availability, site locations, complexity of operations to be examined, security
clearances, and travel time) are additional factors that can impact a time
budget. Setting intermediate target dates or milestones can help teams stay
on target toward a final deadline. If target dates are missed, the root cause
for the delay can be analyzed and possibly addressed by adding resources.
If delays cannot be avoided, adjusting the time budget accordingly at least
allows the audit manager to redeploy resources that are not yet needed to
other engagements.
There are different schools of thought about the execution of time budgets:
• Adhere to the defined number of hours and move from the work once the
time is expired.
• Perform engagement responsibilities to the fullest extent, even if this
means that budgeted hours are exceeded.
• If necessary, add staff and possibly monetary resources to perform the
work within the scheduled time.
Completing an engagement on time and within budget is the ideal scenario.
But no two engagements are exactly the same. Should mitigating
circumstances arise, there is no universal correct position. It is always best
that time budgets be sufficiently flexible to cover unanticipated demands on
the internal audit activity. While time demands might require reducing scope
or quality on recurring audits of lower risk, time demands should not be
allowed to impact work quality in higher-risk areas.
Next Steps
You have completed Part 2, Section II, of The IIA’s CIA Learning
System®. Next, check your understanding by completing the online sectionspecific test(s) to help you identify any content that needs additional study.
Once you have completed the section-specific test(s), a best practice is to
reread content in areas you feel you need to understand better. Then you
should advance to studying Section III.
You may want to return to earlier section-specific tests periodically as you
progress through your studies; this practice will help you absorb the content
more effectively than taking a single test multiple times in a row.
Index
The numbers after each term are links to where the term is indexed and indicate
how many times the term is referenced.
active controls 1
application controls 1
audit charter 1
audit documentation 1
audit plans 1
audit tests 1
audit trails 1
automated controls 1
budgets 1
Committee of Sponsoring Organizations frameworks
Internal Control—Integrated Framework 1
compensating controls 1
controls 1
active 1
application 1
automated 1
by function 1
compensating 1
detective 1, 2
directive 1
entity-level 1
governance 1
hard 1
information technology 1
input 1
key 1
management oversight 1
manual 1
mitigating 1
output 1
passive 1
preventive 1
process-level 1
processing 1
redundant 1
secondary 1
soft 1
technical 1
transaction-level 1
COSO frameworks
Internal Control—Integrated Framework 1
criteria
for engagements 1
data
collection 1
detective controls 1, 2
directive controls 1
engagements 1
budgets 1
criteria 1
objectives 1
planning 1
procedures 1
scope 1
work programs 1
entity-level controls 1
evidence 1
function, controls by 1
governance 1
hard controls 1
Implementation Guides
2060, “Reporting to Senior Management and the Board” 1
2210, “Engagement Objectives” 1, 2
2230, “Engagement Resource Allocation” 1
information gathering 1
information technology
controls 1
input controls 1
International Standards for the Professional Practice of Internal Auditing
1000, “Purpose, 1
1110.A1 1
2200, “Engagement 1
2201, “Planning 1, 2
2201.A1 1
2201.C1 1
2210, “Engagement 1, 2
2210.A1 1
2210.A2 1
2210.A3 1
2210.C1 1
2210.C2 1
2220, “Engagement Scope” 1, 2
2220.A1 1
2220.A2 1
2220.C1 1
2220.C2 1
2230, “Engagement 1, 2
2240, “Engagement Work 1
2240.A1 1
2240.C1 1
2300, “Performing the 1
2310, “Identifying 1
IT (information technology)
controls 1
ITGC (information technology general controls) 1
key controls 1
management oversight controls 1
manual controls 1
mitigating controls 1
objectives
of engagements 1
operational 1
operational objectives 1
output controls 1
passive controls 1
preventive controls 1
process-level controls 1
processing controls 1
redundant controls 1
resource allocation 1
risk 1
assessment 1
control matrix 1
risk-based auditing 1
scope
limitations 1
of engagements 1
secondary controls 1
soft controls 1
staffing 1
technical controls 1
time budgets 1
transaction-level controls 1
work
program 1
Build 08/24/2018 15:40 p.m.
Contents
Section II: Planning the Engagement
Section Introduction
Chapter 1: Engagement Planning
Topic A: Engagement Objectives, Evaluation Criteria, and Scope
(Level P)
Topic B: Key Risks and Controls (Level P)
Topic C: Detailed Risk Assessments (Level P)
Topic D: Engagement Procedures and Work Programs (Level P)
Topic E: Staff and Resources Needed for Engagements (Level P)
Index
Section III: Performing the
Engagement
This section is designed to help you:
•
Perform a preliminary survey of the engagement area, starting with a review of previous
audit reports and other relevant documentation.
•
Show how checklists/internal control questionnaires, interviews, walkthroughs, and
observation can be effectively employed in gathering audit information.
•
•
•
•
Describe various sampling methods and the purpose of each method.
•
•
Evaluate the relevance, sufficiency, and reliability of audit evidence.
•
Apply analytical review techniques such as testing for reasonableness, performing ratio
analysis, analyzing variances, detecting trends, or performing regression analysis.
•
•
Draw conclusions from data analysis efforts.
•
Communicate test results and interim conclusions regarding controls in a professional
manner to the auditor-in-charge to ensure that they receive a full hearing.
•
Base sound conclusions and persuasive recommendations upon well-researched and
logically organized audit findings.
•
•
•
•
Direct/supervise individual engagements.
•
Complete performance appraisals of engagement staff.
Describe commonly used statistical and judgmental (nonstatistical) sampling techniques.
Demonstrate the use of flowcharts to aid in process mapping.
Identify and evaluate various types of audit data and gather the best type to support audit
findings.
Explore available computer software packages used, for example, in continuous auditing,
data extraction, spreadsheet analysis, and workpaper automation.
Develop and review clear, comprehensive documentation/workpapers to support audit
conclusions and recommendations.
Create an audit work plan.
Nurture instrumental relationships, build bonds, and work toward shared goals.
Coordinate work assignments among audit team members when serving as auditor-incharge of a project.
The Certified Internal Auditor (CIA) exam questions based on content from this section make
up approximately 40% of the total number of questions for Part 2. Some topics are covered at
the “B—Basic” level, meaning that you are responsible for comprehension and recall of
information. (Note that this refers to the difficulty level of questions you may see on the exam;
the content in these areas may still be complex.) However, most topics are covered at the “P—
Proficient” level, meaning that you are responsible not only for comprehension and recall but
also for higher-level mastery, including application, analysis, synthesis, and evaluation.
Section Introduction
This section, which covers the tasks that are completed while performing an
engagement, is organized into three chapters:
• Chapter 1, “Information Gathering,” looks at ways to gather information
both during the preliminary survey and during the course of the
engagement, including the use of previous audit reports, interviews,
walkthroughs, observations, checklists, and questionnaires. The chapter
also discusses sampling and statistical analysis techniques.
• Chapter 2, “Analysis and Evaluation,” covers how the internal auditor
compares information gathered during an engagement to the expectations
regarding that information.
• Chapter 3, “Engagement Supervision,” looks at the ways in which the
audit leader manages the audit team and its work product and evaluates
staff performance on audit engagements.
Chapter 1: Information Gathering
Chapter Introduction
This chapter describes the inputs gathered both during the preliminary
survey and during the course of the engagement.
As with any process, the quality of inputs is directly related to the quality
of outputs—in this case, the output of reliable conclusions and
recommendations. When attempting to determine what types and amounts of
data will be sufficient and appropriate, it is important to begin with the end
in mind. First determine the objectives of the engagement, and then work to
find the proper sources of information that will fulfill those objectives.
Topic A: Previous Audit Reports, Interviews,
Walkthroughs, and Observations (Level P)
This topic looks at some of the inputs for the preliminary survey. As stated
in Implementation Guide 2201, “Planning Considerations,” “a preliminary
survey could be a valuable tool to help internal auditors achieve a sufficient
understanding of the area or process to be audited.” Preliminary surveys can
collect information about the objectives, the people, the processes, and/or
the systems involved in an audit engagement.
A competent preliminary survey provides the internal audit activity with
more than general familiarization. Realistic outcomes from a preliminary
survey include clarification of the:
• Purpose of the internal audit.
• Engagement objectives, scope, and timing.
• Area objectives, related risks, and controls.
• Internal audit resources to be used.
• Relevant standards.
The steps in a preliminary survey will depend on the specific nature of the
audit and other factors such as:
• The internal auditor’s training, experience, and knowledge of the activity
being examined.
• The type of engagement being performed.
• Whether the survey is part of a recurring or follow-up assignment.
• The size and complexity of the activity being examined. (The geographical
dispersion of the activity may also influence the duration and time
requirements for the survey.)
For example, an internal audit related to emerging issues will usually
involve different considerations than an engagement that is part of regular
internal audit activities (where processes and standards are known).
Most preliminary surveys include traditional fact-gathering/fact-finding
questions. Organizations are also using techniques such as risk/control
workshops, ongoing client communications, continuous monitoring, and
coordination with enterprise risk management.
The valuable background information collected during a preliminary survey
helps the internal auditor plan the engagement. Preliminary survey results
also help the internal audit activity establish credibility with the engagement
client by demonstrating that the internal auditor takes the engagement
seriously and is thoroughly prepared for the audit.
It is generally advisable to summarize preliminary survey results prior to
finalizing engagement scope and objectives. A one- to two-page report
summarizing the operation reviewed, the work performed, an initial opinion
about the risks and controls, and recommendations for staffing the
engagement suffices.
Having all this information available at the early stages of the engagement
lays the foundation for a solid work program and an efficient audit.
This topic covers four types of preliminary survey inputs: previous audit
reports, interviews, walkthroughs, and observations.
Previous Audit Documentation
Internal auditors can learn a great deal from reviewing audit-related
documentation, including but not limited to prior audit reports and data.
Typically, there’s no shortage of materials to review.
If the audit is part of a regular rotation, the internal auditor should begin by
looking at the permanent files for the operation. A permanent file is a
record of consistent, rarely changing documents. This allows the internal
auditor to review prior audit workpapers, findings, reports, replies, auditor
comments, photographs, and other related information.
It is usually appropriate to review the work of other internal or external
assurance activities that were performed for the activity or are in process.
This may include the work of external auditors, external management
letters, other third-party reports such as an internal compliance review, or
the results of an external regulatory examination.
Note that, in general, evidence obtained from third-party sources is
considered more reliable than evidence obtained from audit client personnel.
“Confirmation” is the term used to describe obtaining direct written
verification of the accuracy of information from independent third parties.
Positive confirmation asks recipients to respond regardless of whether or not
they believe the information is correct; negative confirmation asks recipients
to respond only when they believe the information is incorrect. This type of
evidence is generally considered very reliable because the internal auditor
receives it directly from independent sources. However, the auditor should
be aware that the third-party evidence collection process could be audited as
well.
Audit activity monitoring reports can also be reviewed to understand the
status of any open issues identified in previous internal audits of the area.
One thing to check when reviewing previous audit reports is whether all
prior audit issues have been adequately included in ongoing follow-up
procedures, such as by being included in managements’ tracking process, or
if the issues have already been resolved.
Additional documents that are typically appropriate for review include
relevant:
• Organizational information (e.g., organizational charts, number and names
of employees, key employees).
• Details about recent changes in the organization, including major system
changes.
• Job descriptions.
• Statements of authority and responsibility for the area.
• Objectives and goals.
• Procedural manuals, instructions, and directives.
• Project plans.
• Physical reports.
• Performance reports.
• Certificates of compliance.
• Schedules for production, projects, personnel, etc.
• Budget information, operating results, and financial data of the activity to
be reviewed.
• Correspondence files to determine potential significant engagement issues.
• Input for the organization’s enterprise risk management evaluations.
• Board and committee terms of reference and minutes for relevant
governance activities.
• Internet and intranet sites relevant to the area being audited.
• Public documents produced by the area being audited, such as brochures,
reports, plans, posters, advertisements, and instructions.
Another valuable activity is to research and review authoritative and
technical literature appropriate to the activity, such as internal auditing
literature, industry practices and methods, and trade practices. This is
particularly beneficial if the engagement is a first-time audit or is in
response to an emerging risk. The IIA has a wealth of practical information
available on its website and through The IIA Research Foundation book
store.
Documentation can be in any format, for example, paper, audio, or video.
Increasingly, internal auditors can access information electronically from
remote locations. This capability allows much of the review process to be
completed before the internal auditor arrives at the engagement site. If hard-
copy documentation or tangible materials must be reviewed at a specific
location, they should never be removed. As necessary, copies can be made.
The review of prior audit documentation is important because it:
• Provides familiarity with the area to be audited.
• Overviews what to expect in the activity being audited.
• Shows how other auditors approached the assignment.
• Identifies specific problems found previously and areas likely to have
continuing or repeat problems (e.g., high-risk or repeat findings).
• Reveals the status of promises or actions taken to correct any
nonconformance.
• Reveals strengths that were previously identified that should be verified to
ensure that they have been sustained.
• May identify additional activities for evaluation during the audit.
Study of previous audit files and records does not necessarily mean that the
same approach can be used for the upcoming audit. An internal auditor
needs to evaluate any changes in organizational goals and objectives, the
client operation, risks, and the internal audit activity since the prior audit.
New technologies and any other mitigating factors that can influence the
current engagement must be considered. But even if circumstances differ,
prior engagement documentation still provides a great step forward in the
planning process.
After reviewing prior audit documentation, internal auditors must retain their
focus on the current audit. While these reviews should provide good
information, results and conclusions must be determined during the activity
of the current audit.
Interviews
The auditor also gathers information directly from internal and external
persons. The quality of this information depends upon the skillful use of
tools such as interviews.
An interview is generally described as a structured discussion in which a
person is asked questions about his or her opinions, activities, and other
areas of interest. An audit interview occupies a middle ground between
polite conversation and an interrogation. Like a conversation, an interview
should be pleasant and relaxed. Most of us talk eagerly and openly about
what we do at work, and that’s precisely the focus of an audit interview.
Unlike an ordinary conversation, however, the audit interview has (or
should have) a formal structure, which the auditor provides. Also unlike
most conversations, the audit interview may involve a certain amount of
risk for the person answering the auditor’s questions. The auditor, therefore,
needs to develop considerable skill in putting people at ease during
interviews.
Interviews are typically conducted one on one. Face-to-face interviews are
preferable. But costs and scheduling considerations sometimes necessitate
interviews with small groups by telephone, though an interactive
teleconference, or in a video conference via web technology.
During the engagement planning process, interviews are often conducted to:
• Secure the perspective of management responsible for the activity being
examined.
• Clarify information about the area to be audited.
• Collect additional necessary information.
• Provide an observation of the activities in the organization to be audited.
Input from Engagement Client
Another important use of interviews in engagement planning is to facilitate
a high-level client discussion about the planned internal audit.
The internal audit activity strives to provide audit services designed to help
an organization meet its objectives. With the increased prominence of risk
management and governance, most internal audit functions pursue a valueadded approach and, as Sawyer writes, place a greater emphasis on
satisfying audit customers and demonstrating the benefits the internal audit
activity brings to the organization.
An easy way to demonstrate this commitment to organizational best
interests and gather important engagement background information is to talk
with the client during the preliminary survey. No matter what kind of area
is to be audited or what type of audit is to be performed, such dialogue can
produce valuable results.
Information gathered during preliminary survey interviews can increase the
auditor’s awareness and understanding of:
• Operational objectives or goals.
• Level of compliance with laws, policies, and procedures.
• Key processes.
• Organizational structure (e.g., reporting relationships and supervision).
• Information systems.
• Identified key risks.
• Current controls.
Simple questions such as “What are your objectives?” or “What are your
concerns?” can reveal a lot. For example, appropriate operational objectives
should be in place. The absence of operational objectives or problems with
their alignment to organizational goals and objectives would warrant further
consideration during the engagement planning and fieldwork.
Involving management in the area to be audited during the planning process
in this manner just makes good sense. These individuals are close to the
processes to be examined. They can supply information that will be helpful
with subsequent analytical reviews, testing, and benchmarking. And asking
the engagement client for his or her input early on in the planning process
positions the internal auditor, in Sawyer’s words, as “a seeker of
information rather than an inquisitor.”
Elements of the Interview
An effective interview has a beginning, a middle, and a conclusion. Experts
may not define the steps leading from beginning to conclusion in exactly
the same way, but most would agree that the successful interview contains
these elements:
Plan the interview.
Planning and preparation are critical to interviewing success. Interviews
succeed in gathering complete, relevant, factual, and useful information only
if they have been carefully planned. During planning, the internal auditor
needs to take into account the subject to be discussed in the interview, since
the approach may differ considerably if the purpose is to gather information
about, say, work activities and responsibilities or if it is to confirm or rule
out suspected fraud.
Planning can be broken down into seven steps.
1. Obtain background information.
• Learn about the activities to be discussed by reviewing workpapers
from previous audits, systems documentation, organization charts,
procedure manuals, and other relevant materials.
• Learn about the person to be interviewed, including his or her
personality, experience, knowledge, and job responsibilities.
• Find out about any changes in the organization or operation that may
affect the person.
2. Clearly define the purpose of the interview with specific objectives.
3. Prepare questions designed to achieve the objectives. Use a mix of both
open and closed (yes/no) questions. (The distinction between open and
closed questions is discussed later in this topic.)
4. Organize questions into a logical sequence to stay on track and create a
good impression. Move from general to specific questions; start with easy
questions. (Suspicion of fraud may require a different approach.)
5. Prepare a written agenda for more formal interviews, such as opening
interviews and interviews with senior management.
6. Tailor the objectives and questions to the person’s role in the company.
Questions will differ for senior managers, operating managers, and
operating personnel.
7. Schedule the interview. Agree on a convenient time. State the purpose of
the interview, and state a realistic estimate of the time required for the
interview. Arrive on time.
Open the interview.
The following are guidelines for opening the interview:
• Start on time.
• To build rapport, begin with a warm introduction and a few comments or
questions likely to be of interest to the interviewee. Comments on the
photos in the office may be appropriate. People have different degrees of
interest in such conversational warm-ups, so exercise care to fit the time
and topics to the interviewee’s personality. Above all, be genuine. Phony
friendliness seldom sets an interviewee at ease.
• Be certain to pronounce the interviewee’s name correctly. Write it down,
discreetly. Consider handing a business card to the interviewee.
• After the warm-up, clearly state the objectives of the audit and the specific
purpose of the interview.
• Continue building rapport until (if possible) barriers between auditor and
the interviewee come down.
Conduct the interview.
The following are guidelines for conducting the interview:
• Be confident without being intimidating.
• Avoid sarcasm, overly subtle humor, jargon, or any other language that
might be offensive or confusing.
• Take as few notes as possible and watch the interviewee’s behavior. Notice
gestures that seem inappropriate to what the interviewee is saying or that
indicate discomfort.
• Maintain interest in what the interviewee is saying. Prompt the interviewee
to express concerns about the topics discussed as well as stating facts.
• Use follow-up questioning to have the interviewee clarify overly general or
confusing statements. Ask “Could you provide some examples?” or “Could
you say that another way?” Summarize or restate difficult information in
more familiar words to be sure of getting the facts right. Don’t think this
appears unintelligent, and take care not to make the interviewee feel
inarticulate. When interviewee and auditor have developed rapport, these
techniques will keep both people confident that they are communicating
effectively and will yield more and better information. A productive,
professional interview seldom consists of reading a list of questions and
taking notes on whatever the interviewee says without engaging in
dialogue and making eye contact.
• Distinguish fact from opinion, either during the interview or by performing
audit tests afterward.
• If necessary, bring a second auditor to the interview to assist with note
taking, etc.
• Project a professional bearing.
• Make eye contact without appearing to be staring the other person
down. Maintain peripheral focus to avoid seeming too intense. Be
aware of body language.
• Speak distinctly. Breathing deeply helps the speaker stay relaxed and
maintain a deliberate pace and keeps the voice clear. A strong voice
projects confidence and professionalism.
• Take notes appropriately.
• Begin by asking permission to take notes and explaining your purpose
in doing so.
• Keep note taking to a minimum. Jot down key words and use them to
complete the record immediately after the interview.
• Be sure you have at least a key word for every significant idea. It is
easy to forget and have to ask the same question again.
• Respect the interviewee’s space. Don’t use his or her desk for note
taking.
• Don’t hide your notes. Use them as a basis for paraphrasing what the
interviewee has said. This improves accuracy and helps build rapport.
Interviewers should avoid extended silences and long gaps in eye contact
while they transcribe responses. This is one of the reasons that having two
interviewers in a room may be helpful. While the auditor asks questions,
the second person can record notes and observations.
Close the interview.
The following are suggestions for closing the interview:
• Don’t let the session go beyond the promised time estimate.
• As the end of the allotted time approaches, ask permission to wrap up and
reschedule to cover any remaining questions.
• Summarize the key points from your notes.
• Describe the next step in the process.
• Provide contact information and ask the interviewee to call or email any
additional information he or she might remember.
• Thank the interviewee for his or her time.
Document the interview.
Immediately after the interview, review and organize your notes and write
them up in final form for the workpapers. The longer you wait, the more
key points you will forget.
Evaluate the information gathered in the interview.
Ask yourself how well the interview went.
• Did you meet all your objectives and record all the information required?
If not, what prevented your doing so?
• Was your planning deficient in any way?
• Were there any surprises because you didn’t have sufficient background
information?
• Did you ask the right balance of open and closed questions?
• Were the questions properly organized?
• Was the interview scheduled for the right place, the right time, and the
right amount of time?
• Did you establish rapport with the interviewee early in the interview with
the right amount of small talk? Did you ever establish rapport? If not,
why not?
• Did you explain the purpose of the audit and the objectives of your
interview in a way that put the interviewee at ease?
After you’ve evaluated the information, determine what changes need to be
made in the planning and performance of the next interview in order to
better achieve your objectives.
Interviewing Techniques
The goal of an interview is for the interviewee to answer all questions
freely and expansively and, ideally, to contribute information without being
asked about it. The internal auditor should therefore seek to establish
rapport (a harmonious relationship) with the interviewee—to convey a sense
of shared interest, understanding, lack of judgment, and even empathy.
Verbal and nonverbal skills can be used to establish rapport. In addition,
interviewers must listen and observe carefully and ask the right questions in
the right order.
Establishing Rapport
Rapport is created in an interview situation when the interviewee believes
that the interviewer understands and shares the interviewee’s feelings,
reasoning, and values. It is important then for the auditor to create an
atmosphere characterized by consideration, courtesy, and a sincere desire to
gather information and hear what the interviewee has to say.
Creating rapport can be accomplished through behavior and appearance. An
auditor can begin to establish rapport with an interviewee by:
• Making an appointment at a time that takes into account the interviewee’s
needs, being punctual, and making good use of the interviewee’s time.
• Choosing a site for the interview that is convenient for the interviewee and
allows privacy and confidentiality. (This is an advantage for the auditor as
well, since discretion during interviewing will be less likely to alarm and
warn possible fraud perpetrators and accomplices.)
• Restricting the number of people in the interview. (While it may be a
good idea to have another person assisting in the interview, auditors don’t
want to create a sense of outnumbering the interview subject. Employees
may also have the right to have a union representative present during
interviews.)
• Dressing, as reasonably as possible, in a manner consistent with the
interview subject. (Wearing formal business clothing when interviewing a
management subject conveys respect; however, wearing business-casual
attire may be more suitable when at a plant or interviewing union labor.)
• Having a posture that conveys openness and attention—e.g., sitting
upright, angled directly toward the interview subject.
Internal auditors should be careful not to create a situation in which
interviewees may feel as if they are being detained against their will. This
might include such actions as locking a door or blocking the interviewee’s
exit. These actions could be construed as unlawful detention and become
legal liabilities for the organization.
Active Listening
Effective interviewing uses the basic techniques of active listening. To
encourage subjects to speak freely, auditors should:
• Maintain soft (rather than penetrating) eye contact with the interviewee
when the interviewee is talking.
• Express interest by showing understanding facial gestures, like nodding.
• Allow brief silences after a question is asked or an answer given.
(Sometimes, in the interest of filling a silence, an interviewee will
contribute something new.)
• Paraphrase the interviewee’s statements to clarify them and to show a
commitment to understanding what the interviewee has said.
Open versus Closed Questions
An open question asks for a description, an opinion, or a narrative of some
sort. A closed question can be answered with a one-word response or a
brief, factual statement.
There is a place for both open and closed questions during an interview.
Generally, open questions are a good way to start the conversation, while
closed questions are effective when interviewees don’t seem to understand
the question or are unwilling to answer it or when additional, specific
information is needed as follow-up to a previous answer. An open question
invites the interviewee to say as much as he or she likes; a closed question
limits conversation. Since internal auditors are interviewing to gather and
discover information, it is usually more effective to use open questions.
The difference between open and closed questions is illustrated in Exhibit
III-1.
Exhibit III-1: Open and Closed Questions
Open Questions
Closed Questions
Tell me about the way you document waste left
over from the manufacturing process.
How much waste metal is usually created
during each shift?
Why do you perform the steps in the order you
described rather than in the standard, prescribed
way?
Do you follow the steps in the operating
manual for this task?
Can you explain to me why the figures in the
system don’t match with these receipts? (This is
a good example of a question that appears to be
closed but isn’t really. The significant word in the
question is “explain.”)
Did you falsify this report?
Questioning Style
In an interview, as opposed to an interrogation, the goal is information, not
confession. Avoiding a perception that questions are leading, biased, or
designed to be confusing promotes openness rather than defensiveness.
Asking biased questions may be the most frequent mistake auditors make in
interviews.
So questions are usually asked in a way that promotes neutrality and clarity:
• Questions should be asked in a logical sequence—for example, beginning
with general overviews and then focusing on specific issues. A closed
question can follow an open question to elicit more focused information.
Questions should not be random and haphazard. This could not only be
unproductive in uncovering information, but it could confuse the
interviewee or put the person on the defensive. A question that appears
unrelated may make the interviewee feel manipulated, pressured, or
tricked.
• Questions should not be assumptive, leading, or biased. An unbiased
question provides no clues about what the auditor wants for an answer.
“You’ve been forgetting to lock the files, haven’t you?” is biased. “Tell
me about your daily procedures for securing the files” is unbiased (and
open).
• Questions should not be confusingly constructed—for example, by asking
several questions at once or by using confusing syntax or sentence
structure (e.g., using double negatives).
In sum, the internal auditor’s interviewing skills include the ability to:
• Develop a plan that will make the interview orderly, efficient, and
productive.
• Create rapport with the person being interviewed.
• View the interviewee with empathy—the capacity to participate in
another’s feelings and ideas—and distinguish between facts and opinions
that arise in the discussion.
Walkthroughs
Walkthroughs are step-by-step demonstrations or explanations of a process
or task conducted by the process or task owner in the presence of the
internal auditor. Internal auditors can use walkthroughs to better understand
a process flow. Walkthroughs can also be used to verify the actual state of
controls in an organization—that is, which controls are included in normal
activities because they are efficient and effective to execute in the real
world and which are omitted, altered, or erroneously executed some or all
of the time.
Exhibit III-2 shows that walkthroughs can help reveal the root cause of a
control weakness or failure.
Exhibit III-2: What Walkthroughs May Reveal
Walkthrough Reveals . . .
Potential Root Cause(s)
Potential
Recommendation
The employee is not executing the
control.
The employee has an
incomplete understanding of
the control procedure or its
purpose.
Recommend better
training on the
control.
The employee is deliberately
omitting the control due to
time, cost, or other motives.
Report the issue to
local management.
The employee performs the control
correctly despite evidence of control
failures.
The employee does not usually
perform the control and is
doing it only when observed.
Discuss the potential
problem with local
management.
The employee is attempting to execute
the control but is not following the proper
procedure.
The employee has an
incomplete understanding of
the procedure or is deliberately
modifying it.
Implement better
training or
management
discipline.
The procedure does not work
well in practice despite being
theoretically sound, either
because the environment has
changed since the procedure
was designed or because the
process owner had no input in
control design and it is
impractical to implement as
written.
Recommend that the
control be revised or
redesigned with
process owner input
to be more effective
and/or efficient.
Separating failures in the design of controls from failures in their execution
can help internal auditors add value. When employees are better educated
on controls, they will be more proactive in implementing them. When
controls are designed more effectively, especially when the worker
participates in the design of procedures, it reduces worker frustrations,
promotes buy-in, and helps motivate good behavior.
Observations
The word “observation” carries meaning in ordinary conversation. And
everyone uses his or her senses to note the events and people encountered
in daily life. To the auditor, however, observation implies a stricter
discipline than ordinary “looking around.” The more knowledge, experience,
and skill an auditor acquires, the better able he or she is to gather
significant audit information through purposeful visual examination of,
among other things, people, activities, facilities, inventories, safety systems,
office layouts, and equipment. The auditor as observer not only sees all
these people and things but, at the same time, takes mental notes that will
eventually feed into workpapers and become the basis of findings and
recommendations.
Observation can take many forms. Walkthroughs (described above) are
actually one form. In some types of audits, observations made and recorded
by a camera make up a significant portion of testing. These photographs or
videos may become the audit findings for documentation and reporting. This
approach is particularly appropriate for environmental audits and health and
safety audits.
Effective observation goes beyond merely noticing facts and recording them
on a mental blank slate. The inspector who is prepared knows what to look
for. Observations gain significance when the auditor puts them into context.
This may mean mentally comparing an observed fact to past observations,
to claims made by the audit client, to industry standards, to regulations, and
so on. The more an auditor knows from experience, from study, and from
preparation for the current assignment, the better able he or she is to make
mental comparisons between what is and what ought to be. Disciplined
observation notices what is missing as well as what is present: the lack of
necessary safety devices, the absence of a necessary inspector, equipment
present but not in use.
Observations, by themselves, generally constitute weak evidence. To gain
force in an audit report, observations may need to be backed up by other
evidence and analysis that confirms what the auditor has seen. In some
instances, having the client participate in the observations and agree to the
findings—such as by performing a walkthrough—helps to confirm the
observation. If this is not possible, the report should identify the observed
information as such.
Sawyer provides an example of the use of observation and follow-up
research to develop a money-saving recommendation. In the course of an
audit, an auditor observed a surprising absence of empty wood pallets in a
warehouse. When questioned about this observation, the warehouse manager
confirmed that pallets, when broken, were simply thrown out. The auditor
recommended adoption of recycling and, in a follow-up audit, found that
the warehouse had adopted the recommendation and thereby significantly
reduced the budget for pallets.
While observation yields insights, such as those in the case of the missing
wood pallets, it can also provide misleading data if not done carefully and
cross-checked with other methods. Why? Because the mere fact of being
observed may very well cause people on the job to behave differently. If
the auditor is conspicuously present, the inspector who generally lets a few
too many widgets pass unchecked will become very interested in meeting
the required inspection quotas. Workers and supervisors who usually ignore
safety rules in the interest of convenience or profit will become models of
caution. Nevertheless, observation can add valuable perspective to
complement other information.
Topic B: Checklists and Questionnaires (Level P)
This topic covers two additional types of inputs: checklists and
questionnaires.
Checklists
A checklist is a simple visual tool used to collect, track, and analyze data.
It allows the internal auditor to work in an organized and efficient manner.
Checklists are developed during the planning phase, typically at the end of
the preliminary survey.
A checklist includes items and boxes or spaces for a checkmark to indicate
whether the item is present. A checkmark is, in other words, a yes. A blank
checkbox is a no. Some checklist items are stated as a list of activities such
as “Review of applicable organizational charts.” Items may also be stated as
questions, for example, “Has current research on the subject to be audited
been reviewed?” Space to record notes, evidence, references, and comments
may also be included in the checklist.
Checklists have several uses:
• As a reminder device that helps the auditor be certain that he or she has
asked all the questions or made all the observations intended
• As a quick method of gathering information from a respondent (Doctors
typically use checklists to gather medical histories from new patients, for
example.)
• As controls to be sure that all the correct activities are being performed to
complete the audit with correct and accurate information (The doctor’s
medical checklist is not only an information-gathering device but also a
control technique that helps protect against such mistakes as administering
drugs to which the patient might have a reaction or failing to consider
genetic tendencies that indicate the need for a particular procedure.)
When checklists are used as reminders, they are also useful for tracking
activities from the beginning to the end of an audit engagement. In practice,
they have critical significance in planning an engagement:
• They can be used to support important administrative tasks such as travel
arrangements (travel time, hotels, etc.) and system access.
• They help to establish consistency throughout the audit team and ensure
that the members of the team follow work schedules for testing, reporting
results to the auditor-in-charge or client management, completing
workpapers, and performing other tasks.
• They help to ensure that the internal audit activity addresses all the
appropriate areas and collects data for each when performing the audit.
Ultimately, checklists guide the internal audit activity and help fulfill the
scope of the audit engagement.
Questionnaires
A questionnaire is a tool for documenting information gathered across
multiple survey participants. In some cases, a yes/no questionnaire may be
appropriate. Questionnaires can also test the adequacy of process activities
and existing controls; when used this way, they are called internal control
questionnaires. (These two types of questionnaires are discussed more
below.) Internal auditors also use questionnaires during preliminary surveys
and in control self-assessments.
When using a questionnaire for any of these purposes, the internal auditor
should consider the most effective way to frame the questions and organize
the responses. For example, respondents can be allowed to answer questions
by marking predetermined answers, such as yes or no, or with narratives of
limited or unlimited length. They can also be given statements and asked to
rank them using rating scales. Such scales can be numerical (1 to 5, 1 to
10, etc.), or they can use words to describe categories (always, sometimes,
never; strongly agree, agree, no opinion, disagree, strongly disagree; etc.).
Choosing the proper format for questions and responses can make the
difference between useful and useless information.
Some participants will refuse to respond to questionnaires of any sort,
fearing the consequences of providing their opinion in writing on a
particular topic. Sometimes respondents will provide positive answers
whether they have positive feelings or not, simply to shorten the audit.
Because of their limitations, questionnaires are best used in gathering
information about:
• Multiple units, such as branches, that have the same processes, risks, and
standard operating procedures. In such situations, questionnaires provide
uniform information for use in comparisons.
• Regulatory compliance or other yes/no matters.
Yes/No Questionnaires
Yes/no questionnaires allow only a simple yes or no response. (In essence,
checklists, which were described above, are compact yes/no questionnaires.)
This type of questionnaire is suitable for only some purposes and not for
others (depending in part upon the format).
Some general advantages of yes/no questionnaires are:
• They are easy to administer.
• They yield uniform information from all informants for accurate
comparisons.
• They can be given to large numbers of informants and to informants in
different branches, countries, etc.
• The results of large response pools can be aggregated and analyzed easily.
Among their disadvantages, questionnaires of the yes/no variety:
• Are not appropriate for all types of situations or issues.
• Are not suited to gathering in-depth knowledge.
• Reduce the auditor’s chance to observe the respondent’s behavior and
environment.
Internal Control Questionnaires (ICQs)
An internal control questionnaire (ICQ) is a pre-constructed array of
questions used to elicit key information about internal controls, especially
when documenting initial responses to questions about these controls. Such
questionnaires are sometimes referred to as “pre-designed surveys.” They do
not allow for follow-up questions and observation of audit customer
behavioral cues. They do, however, allow for efficient gathering of
information from large numbers of respondents at one time, while
interviews would require a much longer time (or many more auditors).
ICQs start with a known or desired answer (a yes or no response) and then
seek specific comments. In this regard, the basic structure of an ICQ differs
from open-ended questionnaires, which are used to solicit only narrative
responses. ICQs may be used in different business areas to answer various
control-related questions. ICQs may be completed by the auditor, as
depicted below, or completed directly by the process owners.
Exhibit III-3 shows an excerpt of an ICQ for an accounts payable
disbursement process. The internal auditor or client records comments in the
space provided. Observing the event is better than asking the client (the
inquiry method) or having the client directly enter comments but not as
desirable as examining evidence obtained through testing.
Exhibit III-3: Sample ICQ for a Disbursement Process
ICQs are efficient and easy to administer. Essentially, they provide a
checklist to help with further evaluation after an initial risk assessment.
Good applications for ICQs are across multiple units with the same
processes, risks, and expected controls. Basic constraints in using ICQs are
that they are limited to questions with yes/no answers about procedures and
they do not provide for in-depth investigation.
An important assumption in using an ICQ is that the internal auditor knows
what the procedures should be. Ongoing review helps to ensure that the
questions asked and the information collected remain relevant. Many of
these questions may identify activities to be tested during the audit
fieldwork.
Topic C: Sampling and Statistical Analysis Techniques
(Level P)
It is generally not possible to review every item subject to audit, or it may
not be desirable to do so for cost reasons. Therefore, the internal auditor
must select a sample of items to review. In most cases, the goal of
sampling is to select a subset of items that provides a reasonably accurate
reflection of the whole population. (The total number of items is often
called the “population” and may also be called the “universe” or “field.”)
The use of computers to store and analyze information makes auditing
every item in a population increasingly practical for audit tests that can be
automated. Nevertheless, an auditor may still decide that sampling is the
more cost-effective and efficient approach.
Internal auditors use a variety of techniques for deciding how many and
which items may best yield reliable conclusions about the population as a
whole, and they use a variety of sampling techniques. The number of items
in a sample and the particular technique to use vary from organization to
organization.
Sampling techniques fall into two broad categories: nonstatistical (or
judgmental) and statistical. Sample selection is influenced by the audit
objective, the type of data, the nature of the population, and practical
considerations such as cost and available time. When choosing a sampling
method, the auditor should consider the purpose and desired conclusion and
clearly spell out the sampling technique to be used in the audit plan.
Nonstatistical (Judgmental) Sampling
All nonstatistical samples are referred to as judgmental. Instead of being
based upon careful randomization and explicit probability statements,
judgmental samples are selected according to the auditor’s informed
assessment of how many samples will be required to yield a reasonably
reliable result given the type of population and the audit objective.
Judgmental sampling may be carried out:
• Systematically (e.g., every nth item, beginning with number x).
• Unsystematically (e.g., pulling files from a file cabinet with no selection
criteria).
• According to the auditor’s judgment (e.g., picking large or unusual items
from a computer report).
(Note that the language used in describing nonstatistical sampling may be a
bit confusing. Although all nonstatistical approaches are called judgmental,
only the third type in this bulleted list is specifically known as “judgment
sampling.”)
In any of these cases, the scope statement should read “Judgmentally
selected __ items” and should then go on to describe how these items were
selected.
Unsystematic sampling is not the same as random sampling. Technically,
random sampling refers to the use of statistical techniques to ensure with
reasonable confidence that each item in the population has an equal chance
of being selected. Unsystematic, or “haphazard,” selection does not provide
reasonable certainty that the selection is truly random and is not able to
support a statement about level of confidence and margin of error.
Judgmental sampling is particularly appropriate when the population has no
variability. For example, if the auditor believes that a particular computer
program is either always right or always wrong, testing only one item may
suffice.
Advantages/Disadvantages of Nonstatistical Sampling
Nonstatistical sampling:
• Gives the auditor the flexibility to use professional judgment to select the
items that most need testing.
• Can be designed to achieve cost-effective, reasonably reliable results.
However, it:
• Cannot yield statistically valid results.
• Cannot state a measurable sampling risk.
• May lead to auditing too many, or too few, items.
• Depends upon the experience and insight of the auditor for its
effectiveness.
Statistical Sampling Basics
Statistical sampling methods provide an objective basis for determining
sample sizes and/or for randomly selecting sample items (units) for testing.
Statistical sampling also provides a methodology for projecting sample
results to the population (i.e., the likely number of errors or exceptions that
would be found if all items were tested). These projections are compared to
the thresholds that differentiate acceptable from unacceptable numbers of
exceptions or errors. While an auditor could describe the actual results of
testing a random sample of items, for example, as “We are 95% confident
that the error rate of the population is 6%, plus or minus (±) 3%,” this is a
two-sided precision range projection to the entire population (i.e., errors are
95% likely to be between 3% and 9%). However, auditors are typically not
concerned with errors or exceptions that are lower than expected. An
auditor generally will instead provide just a one-sided upper precision limit
(UPL) projection. For the same example, the auditor might instead say “We
are 95% confident that the maximum error rate (projected UPL) in the
population is 8%.” (Note that this isn’t the 9% high side of the range
because a one-sided precision range will be different from a two-sided
range, for reasons not discussed here.)
Auditors start this process by setting maximum limits on the projected UPL,
called the tolerable error rate (TER) or planned UPL, which in this example
is 8%. These rates may be based on assurance coverage testing.
Management may set standards for maximum error rates for their own
purposes, and this information can be used to help set these levels, but
internal auditors may perceive the level of risk from an assurance coverage
perspective as requiring a tighter standard (a lower TER/planned UPL) than
management has established.
Auditors will also estimate an expected error rate (EER) for the purposes of
determining the sample size to use, perhaps also based on prior assurance
coverage testing. For our running example, let’s assume that the auditor
estimates that there will be an EER of 5% after testing is done. This would
also be the auditor’s best guess of the error rate in the sample to be tested.
After the testing is performed, an actual number of errors or exceptions in
the sample will be found, which is then projected to the entire population
using statistical analysis, and this results in a projected UPL. Assume that
this was actually 5.5% for our example. The projected UPL is compared to
the TER/planned UPL (which was defined as 8% in our example). If the
projected UPL is less than or equal to the TER/planned UPL, as is the case
in this example, the auditor can conclude that the particular control being
evaluated was operating effectively over the analysis period based on the
results of sampling performed, assuming that qualitative analysis confirms
this result, as is discussed next.
While quantitative results (including error/exception projections) are
important, the qualitative aspects of each individual error or exception
identified during testing must also be thoroughly investigated, analyzed, and
assessed by internal auditors for their potential impact or significance. If no
significant issues are identified in the auditor’s follow-up on (and
quantitative analysis of) each error or exception that was detected and the
projected UPL is less than or equal to the planned UPL, then the auditor
can conclude that the control is operating effectively. If there are material
errors or exceptions in the qualitative analysis, or if the projected UPL is
greater than the planned UPL, the auditor cannot conclude that the control
is operating effectively without adequate additional testing and objective
follow-up support.
The terminology introduced above is defined and discussed in more detail
in the discussions of attributes and variables sampling later in this topic.
No Guarantees of Perfection
Notice that statistical sampling does not guarantee perfection. To take a
perhaps extreme example, consider weather forecasts, which are stated in
statistical shorthand. When the forecast is for a 30% chance of showers, for
example, there is an implicit plus or minus percentage. Similarly, when
political pollsters estimate that 46% of the voting population favors
candidate X, there is a plus or minus margin of error that may or may not
be stated. The pollster is actually estimating something more like 46% plus
or minus, let’s say, 3%. As we can see, neither the weather forecaster nor
the political pollster predicts with complete reliability. The pollster, for
example, may actually be implying that 95% of the time samples such as
the one taken will yield a reliable estimate for the total population. But
there is also a 5% chance that the sample results are not reliable. Candidate
X, in other words, may actually be supported by 50% of actual voters or
may be losing in a landslide, but the likelihood of either case is quite small.
Advantages/Disadvantages of Statistical Sampling
A statistically selected sample:
• Provides a more objective, consistent, and mathematically reliable and
supported means for determining sample sizes for assurance coverage
testing purposes.
• Yields quantified data.
• Includes a measure of sampling risk, a level of confidence, and precision.
• Is well adapted to computer testing.
• Provides more credible support for conclusions and recommendations.
However, statistical sampling:
• Can be expensive and time-consuming. (Sample sizes tend to be greater in
statistical sampling than in nonstatistical sampling.)
• May require staff training and software costs.
• May preclude insights about the population available from experienced
audit staff.
• Determines the sample size but not which items to select, so as with
nonstatistical methods, avoiding sample bias is dependent on auditor skill
and integrity.
Despite its quantitative precision, statistical sampling is not commonly used
in internal auditing. Statistical testing tends to be too time-consuming to be
cost-effective, for one thing. For another, audit testing often calls for
making judgments that can’t be meaningfully converted to quantities.
Statistical Sampling Techniques
Statistical sampling techniques include attributes and variables sampling, but
other variants are also discussed later in this topic.
Attributes Sampling
Attributes sampling, sometimes called just attribute sampling, allows internal
auditors to test a sample for an either/or state (sometimes called a binomial
distribution, which means that there can be only two states, such as on or
off for a light switch). Attributes sampling is called for when the audit
objective is to estimate the number of times a certain characteristic occurs
in a population without regard to the size of the characteristic (i.e., the
characteristic exists or does not exist; an error occurs or does not occur).
This is well adapted to compliance audits when the purpose is to determine
compliance or noncompliance with a control in terms of the number or
percentage of compliance failures—items that have not been inspected,
entries that have not been made, etc. Testing of the sample is used to draw
a conclusion as to the rate of occurrence in the population (the full set of
items, including those not tested).
The first thing to do when performing attributes sampling is to define the
control objective and determine which controls are used to achieve the
objective. The objective could be proper authorization of the release of
goods from a warehouse, and segregation of duties could be one control
that needs to be tested to provide assurance of the objective. Taking the
time to spell out this link reduces the risk that the audit procedure will be
testing for the wrong things.
Next, the internal auditor determines what constitutes a control violation
(i.e., error/exception). Evidence that an inventory transfer requisition in the
warehouse management system was generated and approved by the same
employee would be a control violation in this example. Taking the time to
do this step reduces the risk that a deviation from the expected control will
be not recognized.
Once these preliminary steps are done, the next thing to do is to determine
the sampling unit and the population size.
• Sampling unit. Before internal auditors can determine the size of the
population, it is important to determine what exactly constitutes one unit
of the population. In our example, this could be one transfer requisition.
• Population size. This is the total number of items from which the sample
is to be drawn. In general, the larger the population, the larger the
required sample. However, for very large populations, as population size
increases, the effect on the sample size will become less significant
(compared to other factors such as confidence level, tolerable error rate,
expected error rate, etc.). Tables are available from various sources with
population sizes for attributes sampling, or the audit activity may have or
could create spreadsheets with the proper formulas for determining sample
sizes.
The next step is to use audit judgment, discussions with management, and
resources such as information from prior audit reports to set values for the
confidence level, the tolerable error rate/planned upper precision limit, and
the expected error rate. (Note that these parameters need to be specified to
use audit sampling size tables or calculation spreadsheets.)
• Confidence level. The confidence level is the desired percentage of
probability that the sample results will provide a reasonable estimate of
the characteristics of the total population. It’s the 95% probability of
correctly assessing control risk in the preceding examples. While any
confidence level can be set, if sample size tables are to be used, the
confidence level will need to conform to one of the predetermined
confidence levels in the tables. This could be a 90% confidence level
(possibly stated as its inverse, or a 10% risk of assessing control risk
incorrectly) and a 95% confidence level (possibly stated as a 5% risk of
the same). The greater the degree of desired confidence, the larger the
sample needs to be. To achieve 100% confidence, the sample would have
to include 100% of the items (and even then mistakes in auditor testing
application are possible).
• TER/planned UPL. The tolerable error rate, also called the tolerable
deviation rate, is a cutoff value determined by the auditor beyond which
the audit test would not be considered valid assurance. A sampling error
rate is the likely amount of variation that a given sample size would
produce relative to the actual value that would be found if the entire
population were to be tested. The planned upper precision limit represents
the maximum error that an auditor could accept in the testing results when
projected to the population and still be able to conclude that controls are
effective, balances are fairly stated, and so on, as applicable. If the
projected UPL (a value found after testing) is greater than the planned
UPL, and the auditor objectively feels that test results may not be
indicative of actual population characteristics, he or she may need to
consider increasing the required sample size. (This would likely also
increase the time and cost of the procedure.) However, the auditor should
not just keep on increasing sample sizes and follow-up testing until he or
she arrives at a favorable testing outcome. Before deciding whether or not
to increase sample sizes and testing, the auditor should objectively
evaluate whether such a decision is warranted or appropriate based on the
relevant circumstances or factors.
• EER. The expected error rate, also called the expected population
deviation rate, is the auditor’s best estimate of the actual error or deviation
rate in the population. For an audit test to be cost-effective, the EER
should generally be lower than the TER/planned UPL. (The tables do not
provide audit sample sizes for when the EER is higher than the TER.)
Once the testing is complete, a projected error rate and a projected upper
precision limit are determined using statistical analysis.
For example, the auditor might specify:
• Unit type as a warehouse move requisition in the warehouse management
system.
• Total population of 3,000 move requisitions (between the applicable start
and end dates of the designated audit period).
• Confidence level of 95% (and therefore 5% risk of the sample results not
being reflective of the actual population’s characteristics).
• Tolerable error rate (or tolerable deviation rate)/planned upper precision
limit of 8%.
• Expected error rate of 5%.
Once these values are selected, the next step is to determine the appropriate
sample size using the attributes sampling tables. One simply looks up the
TER in the columns and the EER in the rows to find the sample size. (Note
that various tables could use synonyms for these terms.) The tables may
also list the number of errors the internal auditor should expect to find. If
no table is available for a particular set of specifications, formulas exist to
calculate sample size.
Sometimes internal auditors use nonstatistical methods to alter the EER
based on the results of testing, which will in turn alter the sample size. This
might be done in attributes sampling when there are numerous variables that
the statistical tables do not account for. The size of the sample required to
give a reasonable estimate of the total number of errors (or other attributes)
may vary with conditions. The auditor may look at the situation and decide
that the likelihood of error is very small, because the calculations are
simple, the procedures are carried out by reliable computer programs, the
inspectors are all well-trained and experienced, previous audits have found
few errors, and so on. Or, conversely, the auditor may suspect that new
employees, new methods, new products, new procedures, or new software
has introduced the possibility of numerous mistakes.
In the absence of other means of estimating the size of the potential
problem, the auditor might ask for an expert’s opinion or test a number of
items in a pilot sample.
The final steps are to randomly select the proper number of individual
sample items, audit the sample items, and count the number of exceptions
or errors from the control or other attribute being tested. The actual number
of errors is statistically projected to the entire population to determine the
projected UPL based on the results of testing. Tables exist for calculating
this projection as well, or, again, the audit activity may have or could
develop a spreadsheet with the proper formulas. Evaluations should also
include a qualitative review and analysis of any significant observations
from individual errors or exceptions. Finally, the internal auditor evaluates
the results, including describing conclusions in statistical terms (such as was
shown at the start of this discussion), and makes an audit decision, which
includes determining whether the projected actual maximum error rate
(projected UPL) is less than the TER (planned UPL). If this is true and
there were no significant observations from the qualitative review and
analysis, the auditor can provide positive quantitative and qualitative
assurance regarding the level of control effectiveness observed.
Variables Sampling
Variables sampling, also called classical variables sampling, is a statistical
sampling technique that requires calculating a normal distribution (i.e., a
bell curve) and the related standard deviation for a set of data. It uses fairly
complex calculations both to determine sample size and to evaluate sample
results.
Variables sampling applies to audits in which the size of the items matters,
not just occurrence or nonoccurrence. It is sometimes called dollar
estimation because of its obvious relevance to monetary value situations.
Consider, for example, an audit that aims to determine the dollar size of the
difference between account values as determined by the audit and the book
value of the accounts. Being off by U.S. $.01 is clearly less a problem than
being off by U.S. $1,000.00. But variables sampling can also apply to other
variables, such as measures of size, weight, length of time, or average
values. If the objective is to measure the on-time performance of airlines,
for example, the auditor needs to measure more than the number or
percentage of failures to arrive and depart on schedule. The amount of time
delay involved is also important.
For variables sampling, the auditor needs a sufficiently large sample to
serve as a basis for estimating the actual (as opposed to the recorded) size
of the population. If there are 2,000 receivables to be audited, for example,
and their book value is U.S. $200,000, how many receipts does the auditor
have to inspect to determine the actual total?
Variables sampling can be done nonstatistically in some cases. For instance,
the auditor may feel certain that errors are either very small or perhaps
nonexistent. In this case, a very small sample may be sufficient for testing.
On the other hand, the auditor may suspect that the total is off by a large
amount. He or she may use a small pilot sample to confirm that suspicion
and then recommend a review of all transactions.
If the auditor decides to select the sample statistically, there are several
ways to proceed, including formulas to determine the appropriate sample
size, software packages, and tables.
To use variables sampling, the auditor needs to specify population size,
confidence level, and TER/planned UPL. (This is similar to attributes
sampling, except that here the TER should also be considered in terms of
units, such as dollars, and not just error rates.) In contrast to attributes
sampling, the auditor here needs to specify the standard deviation. (The
standard deviation takes the place of the EER.) Higher standard deviations
equate to higher sample sizes.
The standard deviation is an estimate of the amount of difference among
items in the population. It is the relative amount of deviation from the mean
(i.e., average) of the population expressed as a number of units for one
standard deviation. (For example, one standard deviation from the mean
could be $50.) To refer to the discussion of confidence levels, two standard
deviations, or the number of units for one standard deviation times two, or
$100 in this example, results in a 95% confidence level. That is, the
deviation from the mean will be between ±$100 95% of the time. On a bell
curve, this would be the bulk of the curve, with the remaining parts of the
curve making up 2.5% on the low end (lower than minus $100 from the
mean) and 2.5% on the high end (higher than plus $100 from the mean). A
relatively uniform population will have a smaller standard deviation and
therefore generally requires a smaller sample to achieve a desired level of
confidence and precision. For example, take purchase orders for a raw
material ordered in a standard quantity on a regular basis from the same
supplier at market prices. There should be little variability in the cost if the
market price for the raw material is fairly stable, but there will be much
greater variability from the mean if the market price is quite volatile or the
order quantity is not standardized.
There are three common types of variables sampling: mean-per-unit
estimation, difference estimation, and ratio estimation. While the CIA exam
will likely only test for conceptual understanding of the differences between
these methods (needing to do actual calculations is unlikely), seeing
examples can help distinguish between the methods. We’ll use the following
example to show how each method can be applied:
An auditor selects a sample of 100 accounts from a population of 1,500 receivables
accounts with a total book value of U.S. $1 million. The 100 accounts have a book
value of U.S. $66,666.67. Review of the 100 sampled accounts determines their total
value to be U.S. $69,350.00.
Mean-per-Unit Estimation
To apply the mean-per-unit method, the auditor proceeds through these
steps.
• Step 1: Calculate the average audit value of the sampled accounts
(U.S. $69,350.00/100 accounts = U.S. $693.50 per account). This is the
mean-per-unit value.
• Step 2: Multiply the mean-per-unit value for the sample by the number of
accounts in the population to estimate the correct receivables balance
(U.S. $693.50 × 1,500 accounts = U.S. $1,040,250.00). The audit estimate
indicates that the organization has undercounted the amount of money its
customers owe it.
Difference Estimation
To apply the difference estimation method, the auditor must be able to find
the average difference between the audit value and the book value for the
sample. Difference estimation proceeds this way:
• Calculate the average difference between the audit value and the book
value for the sample (U.S. $69,350.00 – U.S. $66,666.67 =
U.S. $2,683.33) and divide by the number of sample units
(U.S. $2,683.33/100 = U.S. $26.83). This is the average difference
between the book value and the audit value of each account in the sample.
• Determine a total difference estimate by multiplying the average difference
in book and audit values by the number of units in the population
(U.S. $26.83 × 1,500 accounts = U.S. $40,245.00).
• Add the total difference estimate to the book value for the population to
estimate the actual value (U.S. $40,245.00 + U.S. $1,000,000 =
U.S. $1,040,245.00).
The difference estimation method is useful only if the population contains
enough errors to generate a reliable sample estimate and the differences are
not proportional to the book values.
Ratio Estimation
When differences between book and sample values are proportional to book
values, then ratio estimation is appropriate and difference estimation is not.
Otherwise, the two methods are similar in application. Applying the ratio
estimation method works like this:
• Calculate the audit value for the sample (U.S. $69,350.00 in this case).
• Determine the book value of the sample (U.S. $66,666.67).
• Determine the ratio of audit value to book value
(U.S. $69,350.00/U.S. $66,666.67 = 1.04).
• Estimate the actual value of the population by multiplying the ratio of
audit value to book value by the population book value (1.04 ×
U.S. $1,000,000 = U.S. $1,040,000.00).
• If the same ratio holds for the population as for the sample, then the
actual value will be U.S. $1,040,000 instead of the book value of
U.S. $1,000,000.
Other Variations of Sample Selection
Whether or not the auditor relies upon statistical procedures to determine
numerical confidence levels and precision, he or she can choose among
several methods of selecting a sample that includes enough items of the
right type to yield a credible result for the purposes of the audit. Some of
these methods are described here.
• Random sampling. In random sampling, audit items are chosen from the
population without attention to any systematic characteristics like sequence
or size.
Here is an example of random sampling.
An internal auditor is looking for variances between orders and confirmations.
The size of the usual orders varies greatly. Working with a pool of about 2,000
orders, the auditor determines that a sample of 40 confirmations will satisfy the
company’s level of acceptable risk, which is a 4% rate of fraud. The auditor then
selects 40 confirmations by using a computer program to generate a random list
of 40 numbers in the appropriate range of confirmation numbers. The auditor
pulls the records for those confirmations, comparing orders with confirmations,
and finds no variances. The auditor can report that the fraud rate is less than 2%.
This example illustrates the most reliable way to select a random sample
—with random numbers. The group of 2,000 transactions in our example
could be numbered from 0001 to 2000 and items could be selected using a
random number table, which is an array of numbers that are random along
the rows and down the columns. From any starting point, the person using
the table can proceed sideways or downward. Using the table to select
numbers from 0001 to 2000 can be done by proceeding from any point in
the table, four numbers at a time, to select the items for the sample.
Random number tables have been in use by statisticians for more than half
a century and are available from the RAND Corporation and elsewhere.
Common spreadsheet applications can also generate random numbers. For
example, entering the function “=RAND()*1000” in a Microsoft Excel
table will generate a random number between 0 and 1,000 in the selected
cell. Services for generating random number tables are available online as
well. (A well-chosen entry at google.com will produce a list of sites.)
Using random numbers to pick sample items becomes difficult or
impossible if the items in the population cannot be easily numbered. And,
in some cases, the sampling method can—randomly—miss large errors.
For example, the 40 transactions in the sample may accidentally turn out
to be for relatively small amounts. They may not reflect a number of
much larger orders in which variances may have occurred. So while the
sample may be statistically sound, it may not detect instances of fraud that
represent more risk than the company is willing to accept.
• Stratified random sampling. Sometimes a population contains such wide
variation that it must be subdivided into more coherent units before
selecting random samples. In stratified random sampling, a population is
divided into strata with distinct characteristics. Although software exists to
aid in stratification, the auditor’s judgment is usually a reliable guide. For
example, perhaps there is a category of spending that is within a
manager’s discretion, followed by one or more categories requiring a
higher level of authorization. Stratified random sampling could be used to
separately evaluate each category since each has different controls and
risks.
• Interval sampling. Instead of using random numbers, an auditor may
decide to choose items that are a certain interval apart on a list. For
instance, to select a sample of 50 items from a population of 1,000, the
auditor might select every 20th item starting at a randomly selected point
—numbers 10, 30, 50, 70, etc. (The starting point should be less than 20.)
This method of sampling may fail to generate random items if the list
contains some sort of bias. For some reason, the list may be clustered in
groups of 20 and every 20th item may have the same characteristic (one
supervisor followed by 19 employees, for example). To control for that
possibility, the auditor might take three samples of items 60 numbers
apart, starting from three randomly selected points in the list (below
number 60).
• Cluster sampling.Cluster sampling resembles stratified sampling in that a
larger population of items is separated into groups, or clusters. But in this
case the clusters already exist and the auditor does not select the
characteristics for grouping them. A cluster might be a file drawer, a box,
a room, or perhaps a building containing a portion of the records or other
items that make up the population. Instead of attempting to collect all the
items before sampling, the auditor selects clusters to test and then may
decide to sample the items in a cluster rather than test them all.
Why do cluster sampling? Because collecting all the items from all the
clusters—which may be scattered among offices around a region or even
around the world—would be too time-consuming and costly.
• Haphazard sampling. Its name indicates that this type of sampling lacks
credibility. Instead of using either judgment or random sampling to select
a sample, the auditor simply takes whatever items are convenient, without
concern for their representative possibilities. This is not random sampling,
although in common language it would be described as selecting items “at
random.” The items selected haphazardly may, in fact, not be random at
all but may represent some bias on the part of the auditor or may include
a number of unrepresentative items that skew the results. Haphazard
sampling occurs, in effect, when a surveyor sends questionnaires to a
selected group, such as readers of a particular magazine, and uses
responses from those questionnaires returned voluntarily to characterize the
entire group of magazine subscribers (or, worse yet, all readers, or even
worse, the larger community). Since the selected sample contains only
those interested enough to respond, it is in no sense random and is very
likely biased toward some shared characteristic of the respondents.
• Stop-and-go sampling. To avoid testing an unnecessarily large number of
items when doing attributes sampling, auditors have developed the stopand-go method. Stop-and-go sampling applies to situations in which the
auditor suspects the population to be relatively error-free. The auditor
begins with a small sample, and, if the sample demonstrates the
anticipated low error rate, the auditor may stop sampling. If, on the other
hand, the error rate turns out to be larger than expected, the auditor will
go ahead with further sampling. This might require one or a succession of
small samples that either confirm the original view of the population or
continue to indicate the existence of a higher-than-expected error rate. In
the latter instance, the auditor may consider reverting to full-scale
statistical sampling to determine the actual error rate at a desired
confidence level and precision.
Tables are available to help the auditor evaluate the results of small stopand-go samples. If the auditor uses a sample of 25 items from a total
population of 10,000, a table could provide information correlating various
numbers of errors in the sample with the likelihood of various error rates
in the population. For instance, the table might show that zero errors in
the sample correlates with a 95% chance that the population has no more
than 5% errors.
• Discovery sampling. Unlike other attributes sampling methods, discovery
sampling does not intend to characterize a population on the basis of a
sample. Instead, its objective is to uncover at least one instance of a
suspected serious problem, such as fraud or a substantial mistake or
compliance failure. This sampling approach is most appropriate to use
when the expected deviation rate (deviation from expected controls,
policies, or laws) is low and the internal auditor wants to design a sample
based on a specified probability of finding one occurrence. It is also
appropriate when management policy is for zero tolerance of
noncompliance or error in a given area. The audit decision is made once
the first error is observed.
One of the major challenges of using discovery sampling to confirm or
allay suspicions of a serious problem is to determine a sample size that is
both accurately predictive and reasonable. Statistical sampling allows the
internal auditor to demonstrate “reasonableness”—to be sure that the
sample is large enough to include at least one example of a suspected
serious problem or fraud (like overstated sales, stolen inventory, stolen
payments from customers, or the creation of a fictitious employee or
vendor account) but small enough to merit the cost of investigation.
Say, for example, that the auditor has reason to suspect that out of 5,000
sales transactions, 25 involve fraudulent claims of purchases by
nonexistent customers. Instead of searching the entire population of
invoices, the auditor may select a sample large enough to provide a
satisfactory chance of finding at least one instance of the suspected fraud.
Fortunately, tables are available to assist in that determination. If the
objective is to determine with 95% accuracy that a sample is large enough
to find at least one of the 25 fraudulent invoices, the table might specify a
sample size of, say, 300. If all 300 invoices are clean, then the auditor can
report to management that there is less than a 5% chance that the suspect
level of fraud is occurring in the audited area. Nevertheless, additional
sampling may be necessary because of the value of the potential fraud.
Two commonly used methods of discovery sampling are random sampling
and dollar-unit discovery sampling (discussed below). In both cases, the
size of the sample is influenced by statistical theory and the cost and/or
goals of the audit. Organizational goals may vary. Some organizations may
calculate that they will tolerate a given percentage or dollar amount of
fraud; the sample size selected for the audit will reflect this accepted risk
level. For example, a 1% loss due to warehouse theft may be acceptable.
On the other hand, the level of legal and public exposure associated with
allegations of financial statement fraud may lead an organization to lower
the percentage of acceptable error in making financial statements. Internal
auditors may use statistical tables to define their sample sizes once the
organization has defined its tolerance for risk.
• Dollar-unit discovery sampling. An alternative to random sampling that
is particularly useful when auditing monetary transactions is dollar-unit
discovery sampling (DUDS). DUDS adds a variable to the definition of
the sample by weighting items by their size. In the random sampling plan
described above, the sampling unit is the individual confirmation. In a
DUDS sampling plan for the same situation, the sampling unit would be
each dollar (or other monetary unit) of the order amounts during the
period. The auditor selects a random sample in the same way, using either
a table or a computer program to select 40 random numbers (in this case,
dollar amounts) between 1 and 500,000. The auditor then examines the
transaction that contains that number. For example:
• The first random number is 824, which occurs in confirmation #1 and
is for U.S. $1,000.
• The second number is 1,173, which occurs in confirmation #2 and is
for U.S. $876.
• The third number is 23, which occurs in confirmation #1 and is for
U.S. $1,000.
The probability of selecting a larger transaction is statistically greater
because of the larger statistical weight of higher-dollar transactions.
In cases where transactions may represent significant value, DUDS may be
combined with a prescreening of all transactions above a specific figure.
For example, in this case the auditor might screen the pool of
confirmations to select those exceeding U.S. $5,000. Let’s say that there
are 15 orders that meet this attribute. The auditor reviews each of these
confirmations and selects through DUDS another 25 numbers to complete
the statistical sample. In this way, the auditor can assure that a single
costly commission of fraud can be identified as well as multiple but
smaller instances of fraud, which may be equally costly when examined
cumulatively.
• Dollar-unit sampling. Dollar-unit sampling (DUS) or monetary-unit
sampling (MUS) combines features of the attributes and variables
sampling models—and for that reason is sometimes known as combined
attributes variables (CAV) sampling. It is also referred to as cumulative
monetary amount (CMA) sampling, probability proportionate to size (PPS),
and sampling proportionate to size (SPS).
In DUS, the larger the dollar value of an item, the more likely it is to be
selected for review, meaning that the probability of selection is
proportional to the relative size of the item (thus explaining the use of the
alternate term “probability proportionate to size” for this method).
Because it is a hybrid approach, dollar-unit sampling provides some of the
benefits of the two more common approaches. Like attributes sampling, it
can be applied to compliance audits. But like variables sampling, it
expresses results as variables, such as monetary units, rather than as
yes/no error rates. It can therefore also be used for substantive tests.
The unique feature of this method is expressed in the name “dollar-unit
sampling.” While traditional approaches divide the audit population into
physical units, such as receipts or invoices, DUS considers the audit unit
to be one dollar. That is, the auditor considers a stack of 1,000 receivables
worth (on the books) U.S. $10,000 as 10,000 units of U.S. $1 each instead
of as 1,000 units with variable dollar values. Since each unit has the same
value in this method, there is no variability; therefore, there is no need to
consider the standard deviation of the sample—a definite benefit. It also,
as we shall see, removes the need to stratify populations in which items
vary greatly in value.
Here’s how it works. Consider that file of 1,000 receivables varying in
actual value from, let’s say, U.S. $15 to U.S. $1,500. Traditional sampling,
working directly with the receivables, would require stratification of the
population to ensure that the sample does not skew the results by omitting
all large items. With dollar-unit sampling, such stratification is not
necessary. Instead of selecting a sample from the receipts, the auditor
selects dollars from the combined total of U.S. $10,000. Every dollar, no
matter which account it belongs to, has the same chance of being included
in the sample, and all the units are exactly the same size—one dollar.
Of course the auditor isn’t interested in testing dollar bills; tests are
performed on the account that includes the selected dollar. Whether it’s a
dollar from a U.S. $15 receivable or a U.S. $1,500 receivable, it’s the
receivable that’s subject to the audit. This means that a U.S. $1,500
account has 100 times more chance of being audited than a U.S. $15
dollar account. This not only provides a simpler alternative to stratification
but also reflects the fact that auditors are more concerned about highdollar items than the smaller items. A strictly random sampling of all
items in the population runs the risk of including too many small items
and too few larger ones.
The outcome of testing a DUS sample is an estimate of the maximum
amount of error in dollars at a specified confidence level.
The DUS method isn’t appropriate for all situations. It is generally
considered when auditing:
• Large populations (2,000 or more items—not dollars) in which the
expected error rate is low (less than 10%) and in which the amount of
an account error is less than the account value.
• Asset account balances, such as receivables, investments, inventory, etc.,
but not liabilities. The objective of DUS is to detect overstatements,
and that isn’t a concern when auditing liabilities. DUS is not sensitive
to understatements (the problem in liability audits), and it is unable to
detect unrecorded items.
• Judgment sampling. Judgment sampling relies upon the experience and
knowledge of the auditor to determine the size and nature of the sample.
While the low-tech nature of this process may seem to suggest weakness,
auditors have been successfully using judgment in sample selection for
about as long as auditing has existed.
If the population to be sampled is highly uniform, or the controls are
unusually impressive, for instance, spending a great deal of money to test
a large sample may be wasteful. A one-item sample can represent an
entire population if the auditor believes every item is either right or every
item is wrong.
If a population includes a subset of items that seem subject to weaker
controls or exhibit other danger signals, the auditor’s judgment will
indicate that these items should be the focus of testing while the rest of
the population can be ignored or tested very lightly. For example, the
auditor may be looking for cases of fraud and will choose to study only
those items that exhibit a suspicious characteristic. A sample of suspicious
items clearly does not represent the entire population.
Chapter 2: Analysis and Evaluation
Chapter Introduction
Data analysis and evaluation involve comparing information gathered during
an engagement to the expectations regarding that information. Internal
auditors develop expectations about what is typical or expected of a process
or function during audit planning. This provides a basis against which
reasonableness can be determined. Expectations are based in part on
evidence from prior audits or other sources and in part on forward-looking
considerations, such as organizational objectives, the objectives of the
process or function, significant risks, and the organization’s risk appetite.
When data does not conform to what should be reasonably expected, such
as an anomaly, a variance, or an unexpected correlation, the auditor
interprets this difference as evidence that there is an unaccounted-for
condition or force acting on the relationship in question. The auditor should
form his or her own conclusions as to the causes of these differences and
whether they are likely to be isolated events or trends. It is important to
perform this independent interpretation prior to seeking out the
interpretations of the audit client and other sources such as industry or
economic trends, as this can help guard against inheriting a bias from such
sources. If such sources provide corroboration, it becomes more evidence of
the reliability of the interpretation. If other explanations disagree, then
further analysis and interpretation may be needed.
With audit software and the advancement of technology, auditors may be
analyzing “big data” and can often test 100% of transactions using newer
systems and tools. Artificial intelligence advances may also help make audit
tests more efficient and effective.
The topics in this chapter address computerized audit tools and techniques,
sources of evidence, process mapping and analytical review techniques,
workpapers and documentation, and engagement conclusions.
Topic A: Computerized Audit Tools and Techniques
(Level P)
As soon as computer database software became reasonably priced,
reasonably reliable, and reasonably user-friendly—perhaps even a little
sooner than that—all the information in many organizations, large and
small, joined a mass migration from paper files into digital databases. Soon
the transformation was so complete that even public libraries had sent their
card catalog files to antique furniture stores.
In such a thoroughly digitized environment, what is an internal auditor to
do but become adept at using computer-assisted auditing techniques, or
CAATs, as they are commonly called? After all, it makes little sense to gain
access to audit information by asking the audit client to print out documents
that may never before have been committed to paper.
The beauty—and the challenge—of electronic auditing is the fact that files
stored in databases or other computer sources can be searched in
innumerable ways for any conceivable purpose—at least in theory—by audit
software. This assumes, of course, that databases have been properly
constructed, that the audit software is compatible with production and
storage software, and that auditors are well versed in the use of their
electronic audit tools.
Various programs are available for computer-based audits—ACL (Audit
Command Language) and IDEA (Interactive Data Extraction and Analysis),
which provide daily, weekly, or monthly system reporting on activities or
controls; SAS (Statistical Analysis System); and other data analysis
programs, including spreadsheets such as Excel, analytical programs such as
Crystal Reports, and others.
Embedded Audit Modules
Earlier in this section we looked at methods of selecting samples of data to
use in making reasonably reliable estimates of the attributes or variability of
larger populations. At that time we noted that computer programs, with their
increasing power and speed, may enable auditors to review each transaction
in the total population (or to let the computer do the job instead) rather than
working with samples.
Embedded audit modules—programs that run along with the software that
performs functions such as recording data—make it possible in some
instances to monitor transactions as they happen. Instead of looking for
errors, irregularities, and fraud weeks or months after they occur, managers
and auditors can choose to be alerted to potentially problematic events or
exceptions almost as they happen. If instantaneous review seems
unnecessary or impractical, monitoring may be scheduled to happen daily,
weekly, or monthly, depending on the type of activity involved.
This is a benefit for management as well as the auditor. Rather than having
to compensate for errors that have been accumulating since a prior audit,
managers can react in time to avoid the serious consequences that can occur
when problems are left unresolved. This responds to a growing demand
among all stakeholders in organizational processes for access to reliable,
timely data, and it responds to top management’s concerns that, since
passage of the U.S. Sarbanes-Oxley Act, they can be held personally
accountable for fraudulent financial statements, inadequate controls, or both.
The embedded program works by checking incoming data against specific
criteria to catch any discrepancies that would signal errors, fraud, or other
problems. These criteria may be related to key performance indicators as
well as to compliance-related matters. In addition to these internal
considerations, embedded programs such as firewalls check and flag
incoming viruses and other attacks and issue immediate warnings to users.
The IIA’s Practice Guide “Continuous Auditing: Coordinating Continuous
Auditing and Monitoring to Provide Continuous Assurance,” second edition
(previously GTAG 3), draws a distinction between continuous monitoring
and continuous auditing. The former is related to management’s
responsibility, under COSO, to monitor controls. The latter, continuous
auditing, relates to the auditor’s responsibility to analyze and assess the
effectiveness of controls that management puts in place to reduce identified
risks to acceptable levels.
Internal auditors may develop scripts for data extraction for use in CAATs,
and, in some cases, auditors might make those extraction scripts available to
management as a value-added service, that is, to help management identify
opportunities to turn data into meaningful business information. Similarly,
auditors may use data mining and extraction software in their audit work
and, if they identify new business insights, these may also be an
opportunity to add value. In some cases, management procures licenses to
use the software and has internal auditors train them on how to use it so
they can identify multiple cost-saving opportunities.
Embedded programs for continuous monitoring or auditing can be difficult
to install within or in conjunction with existing software. Their growing
popularity indicates, however, that they will eventually be the standard
rather than the exception. As programmers continue to develop and
organizations continue to install enterprise resource planning (ERP) systems
to integrate many or all of the applications managing end-to-end processes,
it seems likely that embedded audit modules will eventually be included to
monitor key events as they occur.
Generalized Audit Software
Much data extraction is done by generalized audit software (GAS).
Depending, of course, on the particular vendor, GAS can perform any or all
of the following tasks:
• Reading digital files. In fact, auditing software extracts “read only” files,
since the auditor’s task is not to change files or controls but to assess their
accuracy, their effectiveness in achieving the organization’s goals, and so
on. Changes in data and controls are made at management’s direction, and
one of their subordinates should make the changes. It is vital for internal
auditors to maintain their independence at all times, and, as such, they
should never be put in a position where they have access to or capability
to change organizational data.
• Examining particular records according to criteria the auditor
defines. In other words, GAS can perform targeted data extractions. GAS
programs can extract sensitive information, such as past due receivables.
GAS can also check accounts for specified exceptions, such as split
transactions—a procurement cardholder’s attempt to divide one transaction
into two or more parts that add up to more than the card’s limit. Along
the same lines, audit software can be directed to search through transaction
data for key words, such as “political campaign committee,” “toy,” or
“casino.” When procurement cardholders learn that their purchases are
subject to computer checks, those with a weakness for abusing their cards
may suddenly become much more “ethical.”
• Performing tests of calculations or making independent
calculations. Audit software can also prepare a trial balance to compare
with balances created in production software.
• Analyzing, summarizing, or re-sequencing data. Audit software can be
used to stratify samples and perform statistical analyses, among other
tasks.
• Testing the effectiveness of controls. Audit software can be used to test
controls as well as to perform substantive tests, such as assessing the
extent of account balance errors.
Generalized audit software offers:
• 100% scrutiny of suspect transactions rather than random verifications.
• The ability to target transactions of interest even when they are part of
huge populations.
• Reduction of audit durations to a fraction of the time required for
traditional manual methods, giving the auditor time for analysis and
interpretation.
• A uniform interface for all tasks regardless of the underlying formats and
systems involved, thus increasing ease and accuracy of auditing.
• Facilitation of reviews by other auditors and supervisors through
maintenance of test logs.
Audit software has not been adopted by all auditors for all internal audit
tasks. There are obstacles preventing an immediate rush into this new
territory, including:
• IT staff reluctance to adopt new systems, set up training, etc.
• IT bureaucracy or other issues causing difficulty in obtaining access to
sensitive and timely data. (This can occur in spite of the fact that auditors
should have complete access to records within their scope of work. Not
receiving access in a timely fashion could be a red flag, in some cases.)
• Access to production data blocked by management who fear that the audit
software will interfere with the production software. (The CAE should
discuss such concerns with management.)
• Issues with getting usable data (formatting, storage, extraction).
• Additional costs for training and new software.
Automated Workpapers
Audit workpapers now take many electronic forms in addition to their
traditional paper format. They may be created in software templates, stored
on servers or mainframes, and transmitted to various desktop and laptop
computers through electronic networks.
Audit workpapers are covered in more detail later in this chapter.
Topic B: Sources of Evidence (Level P)
Whether the occasion is a legal proceeding, a debate contest, or the delivery
of a final audit report, the most convincing case is built on a solid
foundation of facts (objective evidence). Not just any facts, of course, but
the right facts, properly arranged to support the case, the argument, or the
audit conclusions and recommendations.
This topic focuses on the types of evidence that can be used as source
material for audits, including how to evaluate the sufficiency, reliability,
relevance, and usefulness of that evidence.
Types of Evidence
The information gathered to support the conclusions and recommendations
in a final audit report—audit evidence—bears a family resemblance to legal
evidence, but the two types are not the same. (In fraud investigations, the
difference can be crucial, since evidence gathered to prove fraud may have
to stand up in court.)
We’ll look at both audit evidence and legal evidence.
Audit Evidence
Audit evidence—facts used to support audit opinions, conclusions, and
recommendations—can be physical, documentary, representational, or
analytical.
Physical Evidence
Physical evidence generally is considered more reliable than the testimony
of a person. It includes statements of observers, photographs, charts, maps,
graphs, or other pictures. Pictures, such as a photograph or videotape of an
act or condition, provide the most compelling evidence of physical facts,
but observed processes and objects also constitute physical evidence. Note
that simple observation by an internal auditor is no stronger evidence than
one individual’s testimony (weak evidence). Having two or more auditors or
an area supervisor also observe a situation is stronger evidence, while some
form of recording makes this form of evidence very strong. Even
documentary evidence (see below) can strengthen physical evidence. For
example, physically counting a fleet of vehicles can be backed up by
documenting each vehicle’s VIN number to ensure that no vehicle was
fraudulently swapped out.
Documentary Evidence
Documentary evidence is the most common type of audit evidence. It can
be recorded in media other than paper and includes, among other examples,
letters, memos, emails, invoices (external documents) and accounting
records, process flows (including flowcharts), program listings, internal
documents (purchase orders, time records, etc.), activity and control logs,
and systems development documentation.
Documents can be forged, of course, and other conditions bear upon the
reliability of documentary evidence. For example, a memo may provide
evidence that a person made a communication on a particular day, but it
could still be a false statement or the date could be falsified and so is no
more reliable than testimonial evidence. Similarly, records contained in IT
systems are only as reliable as the IT controls over such records. Someone
could have manipulated them, or they could contain processing errors.
Testing controls and audit trails can address such risks. Obtaining external
evidence is another way to corroborate internal documentation, especially
when the internal auditor initiates contact with the outside sources to
prevent situations such as a person supplying a false invoice from a false
vendor.
Representations (Testimonial Evidence)
Representations, which are also called testimonial evidence, include
statements or written responses to inquiries or interviews, questionnaires,
etc. Representations are considered to be the weakest form of audit
evidence, and therefore they should be backed up with documentary
support, such as system flowcharts or written policies and procedures. The
source of the representation also matters: A process owner will supply better
testimony about how a process is actually performed than his or her
supervisor, who may only be able to say how a process should be
performed. Even testimony from process owners is weak evidence, however,
because the person may be motivated to describe how a process ought to be
done rather than how he or she actually does it. The use of open-ended
questions that omit details about specific controls can help reveal what
controls are actually being performed. For example, instead of asking how
the transaction-processing duties are segregated, the internal auditor could
be more general and ask how the transactions are processed, leaving the
audit client to fill in the steps. Obtaining testimony from two or more
persons can also help make this form of evidence more reliable.
Analytical Evidence
Analytical evidence includes computations, comparisons of various types,
reasoning, analyses performed by the internal auditor, and so on. It can
show that a certain relationship, ratio, or discrepancy (variance) exists
between two sets of data at either a point in time or over the course of a
particular time period. Analytical evidence must be further assessed for its
relevance, significance, and root cause—why the relationship or discrepancy
(variance) exists.
Legal Evidence
Internal auditors need not be expert in gathering evidence for use in court,
but they may be called upon to gather data for use by a lawyer. If this
happens, the auditor needs to be familiar with the major types of legal
evidence:
• Best evidence or primary evidence is generally documentary.
• Secondary evidence is inferior to best evidence and may be a copy or
oral evidence of a document.
• Direct evidence proves a fact without requiring presumptions or inference.
Eyewitness testimony is direct evidence.
• Conclusive evidence leads to only one conclusion.
• Circumstantial evidence proves or supports an intermediate fact from
which a primary fact can be logically inferred.
• Opinions are generally not considered allowable testimony in court unless
offered by unbiased experts.
• Hearsay is secondhand evidence and is generally ruled inadmissible in
court: “Jones told me he saw the fraud committed” is hearsay, but “I saw
the fraud committed” is firsthand evidence and should be sought out.
Other Concerns
The type of evidence is not always the auditor’s sole concern when
deciding the best way to support conclusions and recommendations. The
auditor also needs to consider matters of source, availability, confidentiality,
and access.
• Source. The source of the audit evidence can add to, or detract from, its
persuasiveness. Corroborative evidence directly obtained from an
independent third party can be more reliable than audit evidence from the
organization being audited. Third-party evidence may be called “external
evidence” to distinguish it from “internal evidence” provided by the audit
customer.
The strength or weakness of the evidence depends on how persuasive it is.
A source is persuasive if it enables the internal auditor to formulate wellfounded conclusions and advice confidently. To be persuasive, the source
of evidence must be sufficient, reliable, relevant, and useful, as is
described later in this topic.
• Availability.The internal auditor should consider the time during which
evidence will be available for testing. This applies especially when the
evidence is electronically stored. Audit evidence processed by electronic
data interchange (EDI), document image processing (DIP), and dynamic
systems such as spreadsheets may not be retrievable after a specified
period of time if files are revised without being controlled or backed up.
Security and other recordings may be on a loop system, where data is
over-written with new information if not gathered in a timely fashion.
• Confidentiality. The internal auditor should always bear in mind the
mandatory injunction, in The IIA’s Code of Ethics, to honor the
confidentiality requirements of the owners of audited data. This may
become even more challenging as audit data increasingly resides on
servers connected throughout an organization and, quite likely, to the rest
of the wired world. When extracting data from computer databases by
means of computer-assisted audit techniques, the auditor needs to exercise
special care not to distribute sensitive information to unauthorized sources
or to corrupt data in the process of extracting it.
• Access. The other side of the coin is access to data. According to Standard
1110.A1:
The internal audit activity must be free from interference in determining the scope of
internal auditing, performing work, and communicating results. The chief audit
executive must disclose such interference to the board and discuss the implications.
Evaluating Sources of Evidence
According to Standard 2310, “Identifying Information,” “Internal auditors
must identify sufficient, reliable, relevant, and useful information to achieve
the engagement’s objectives.” (The word “competent” may appear in the
Implementation Guides with the same meaning as “reliable.” In this case,
consider the standard to be the higher authority.)
Audit evidence should be the best available to the auditor in terms of its
sufficiency, reliability, relevance, and usefulness. (Note that in the following
descriptions of these qualities, the text in italics is from the interpretation to
Standard 2310.)
• Sufficientinformation is factual, adequate, and convincing so that a
prudent, informed person would reach the same conclusions as the
auditor. Sufficient means that there should be enough evidence and that
different but related pieces of evidence should corroborate each other.
Sufficiency is therefore assessed for the body of evidence as a whole. The
sufficiency of evidence will necessarily be subject to interpretation by
different observers, depending on factors such as the extent of their
relevant knowledge and, perhaps, their biases. Statistical samples, for
example, may be considered by some observers to be insufficient evidence
of the characteristics of the population from which they were taken. Such
disputes about evidence may be initiated either by those with special
expertise, such as trained statisticians, or by those who have no
competence in the methodology and are simply skeptical. Sometimes
evidence must not only be sufficient but must be presented in such a way
that its sufficiency is obvious to the audience.
• Reliableinformation is the best attainable information through the use of
appropriate engagement techniques. Reliability implies that the evidence
must come from a credible source. This considers whether or not the
internal auditor directly obtained the evidence. The reliability (or
competence) of audit information also depends upon the type of evidence.
Sawyer points out that an original document is more reliable than a copy,
a corroborated oral statement is more reliable than a statement that stands
alone, and direct evidence is more reliable than hearsay testimony.
Sampling techniques, again, may be disputed as to their reliability as a
basis for conclusions about the larger population. Also, traditional
skepticism about the reliability of photographic evidence has become even
more pronounced with the advent of computer software such as Photoshop
in which photographs can be easily altered. Therefore, photographs are
considered to be hearsay and should be corroborated by competent
testimony. Audit conclusions should be supported by the most reliable
evidence that is available to the auditor.
• Relevantinformation supports engagement observations and
recommendations and is consistent with the objectives for the
engagement. Evidence may be reliable in itself, but it is of no use if it is
not relevant to the matter at hand. Relying on evidence that has little or
no pertinence increases audit risk—the risk of reaching invalid conclusions
and providing faulty advice. In Sawyer’s examples, a purchase order,
though properly approved and issued, is not relevant to proving that the
goods ordered were actually received, nor does a receiving memorandum
provide relevant evidence that the goods received met specifications.
Internal Auditing: Assurance and Consulting Services, by Reding et al.,
available from the IIA book store, provides the following situation to
illustrate relevance:
An internal auditor needs to determine if a particular vehicle included in an
organization’s fixed asset ledger exists. Locating the vehicle in the parking lot allows
the auditor to reasonably conclude that the vehicle exists. However, in order to
reasonably conclude that the organization owns that particular vehicle, the internal
auditor would need to inspect pertinent documentary evidence, such as title of
ownership.
• Usefulinformation helps the organization meet its goals. The usefulness of
information is a function of whether the information is germane to the
organization. The timeliness of information can make a difference in
whether it is useful or not (i.e., the information could influence decisions
currently being made).
There are no hard and fast rules regarding the sufficiency, reliability,
relevance, and usefulness of evidence. For example, while evidence obtained
from independent (external) third parties is generally considered more
reliable than evidence from internal personnel and corroborated evidence is
more sufficient than uncorroborated or contradictory evidence, it is rare that
the internal auditor would go directly to external sources without first
investigating internal evidence. The internal evidence creates a base case
upon which external corroboration is sought.
Gathering Audit Evidence
Audit evidence can be gathered in a number of ways, and the auditor
should select the procedure most appropriate for the evidence to be
gathered. Common methods include inquiry, observation, inspection,
confirmation, re-performance, vouching, tracing, and analytical procedures.
Collectively, these can be applied through the use of manual audit
techniques, computer-assisted audit techniques, or a combination of both.
Exhibit III-4 provides a summary of manual audit procedures as discussed
in Internal Auditing: Assurance and Consulting Services.
Exhibit III-4: Manual Audit Procedures
Procedure
Description
Examples
Inquiry
Asking questions of audit client
personnel or third parties and obtaining
their oral or written responses.
Produces indirect evidence, which by
itself is rarely persuasive.
• Circulate a questionnaire among
senior executives asking them to
identify the top ten risks threatening
the organization.
• Ask the organization’s outside legal
counsel to provide information about
any litigation, claims, and/or
assessments against the organization.
Observation
Watching people, procedures, or
processes. Generally considered more
persuasive than inquiry, as the internal
auditor is obtaining direct evidence.
• Tour the audit client’s facility to gain
a general understanding of day-today operations.
• Observe the year-end physical
inventory counts.
Inspection
Vouching
Tracing
Studying documents and records and
physically examining tangible
resources. Inspection of documents and
records provides direct evidence of
their contents. Physical examination of
tangible resources (i.e., a building or a
piece of equipment) provides the
internal auditor with direct personal
knowledge of the resource’s existence
and physical condition. Inspection
generally reflects internal auditors’
level of expertise—their capacity to
comprehend what they read and see.
• Review board of directors’ meeting
Tracking information backward from
one document or record to a previously
prepared document or record or to a
tangible resource. Performed
specifically to test the validity of
documented or recorded information.
• Vouch a sample of inventory items
Tracking information forward from one
document or record to a subsequently
prepared document or record or to a
tangible resource. Performed
specifically to test the completeness of
documented or recorded information.
• Trace internal auditor test counts of
minutes looking for authorization of
significant events (i.e., the acquisition
of another company).
• Inspect selected inventory items to
determine their condition and
saleability.
from the accounting records to the
warehouse to see that the inventory
items exist.
• Vouch a sample of sales invoices to
corresponding shipping documents to
verify that the shipments occurred.
inventory to the audit client’s
inventory compilation records to
verify that the counts are properly
included in the compilation.
• Trace checks dated within a period of
several days before and after yearend to the accounting records to
ensure that checks were recorded in
the proper year.
Reperformance
Analytical
procedures
Confirmation
Redoing control activities or other
procedures; re-performing calculations
to assess whether the audit client’s
calculations are correct. Provides direct
audit evidence regarding operating
effectiveness.
• Recalculate accumulated depreciation
Comparing information with
expectations identified or developed by
the internal auditor. Certain
relationships among different pieces of
information may be reasonably
expected to continue in the absence of
known conditions to the contrary.
• Analysis of common-sized financial
Obtaining direct written verification of
the accuracy of information (positive
or negative) from independent third
parties.
• Confirm the principal balance of a
and depreciation expense to verify
their accuracy.
• Independently estimate the allowance
for doubtful accounts to test the
reasonableness of the accounting
department’s estimate.
statements.
• External and internal benchmarking.
• Ratio and trend analysis.
• Regression analysis.
note payable, the maturity date, and
interest rate with the lender.
• Confirm cash account balances with
banks.
The most common technique is inquiry. Sometimes, however, the procedure
for gathering information will be dictated by the evidence. The best method
of gathering useful quantitative information may be by recalculating
transaction data to confirm that it is accurate. Physical evidence may be
more effectively presented in photographs rather than through verbal
description. In control self-assessments (which were described in more detail
in Section I, Chapter 2), the method of gathering and assessing evidence
usually combines a questionnaire and a facilitated workshop. In a CSA, a
range of significant stakeholders in the audited process contribute to the
inquiry.
Topic C: Process Mapping Techniques (Level P)
Before an internal audit engagement proceeds, the internal auditor needs a
thorough understanding of related operational processes. Without this
understanding, it is challenging—if not impossible—to do an accurate audit.
A tour or walkthrough of the area to be audited reveals the physical flow of
material and documents and promotes an understanding of the operation’s
processes and process control points. A walkthrough may be of a process or
a transaction and could be documented as a process map or as a narrative.
(Note that walkthroughs were discussed in Chapter 1 of this section.) A
process map can be any graphic or tabular representation of the steps in a
process. Most often this means a flowchart.
Flowcharts
Next to personal inspection, process documentation is most commonly
achieved through the use of flowcharts. A flowchart is a graphical
representation of the actual or ideal path followed by any service or
product. It provides a visual sequence of the steps in a process, illustrates
the relationships between parts, and identifies what the process does or
should do. Flowcharts range from simple to complex, depending on the
level of detail shown. They are effective because they are easy to
understand and therefore practical to review with the audit client. A
flowchart eliminates abstractions about how work flows through a system.
Flowcharts can be created in a variety of ways, from highly informal pencil
drawings on paper to technically sophisticated computer graphics. One
effective way to construct a flowchart is to gather all stakeholders in the
process together to identify the steps. Each step can be described on a Postit™ note, and the notes can be arranged and rearranged to create a map of
the sequence all can agree upon.
Flowcharting software ranges from Microsoft Word or Excel (which include
standard flowcharting symbols), through various graphics programs, to
specialized charting applications such as Microsoft Visio, SmartDraw,
Edraw, and numerous others. Common symbols used in flowcharts are
shown in Exhibit III-5.
Exhibit III-5: Standard Flowchart Symbols
The auditor can develop a flowchart of any process, from the process of the
audit itself to the processes to be audited. In any case, the benefits are the
same: Drawing out the steps of a process provides an easy-to-follow, startto-finish map. When each proposed or existing step has been placed on the
map, the auditor and other reviewers can more readily assess which steps
are crucial, which can be omitted, and which should be sequenced
differently—as well as where new steps should be added.
Flowcharts have multiple uses—developing, refining, and auditing processes
among them. Flowcharts may depict the current state and/or the desired
future state of a process. In the process of creating the flowchart,
participants may discover points of weakness in controls—lack of
supervision, assignment of responsibilities to the wrong level of the
organization, failure to segregate functions to avoid conflict of interest, and
so on. In other instances, participants may all agree that a flowchart
provides an accurate description of what happens in reality, but when
comparing it against field observations the auditor may find that in fact it
does not reflect what actually happens.
During the planning phase of an engagement, internal auditors may review
existing flowcharts or they may prepare new flowcharts. When reviewing an
existing flowchart, an internal auditor can make a preliminary assessment
about identification of risks or the adequacy of controls or if there are
unnecessary controls in the process. The internal auditor should also verify
that the flowchart is current and accurately reflects the process. The reality
is that processes change but flowcharts are not always updated.
Flowchart Formats
Flowcharts can be laid out either vertically or horizontally. While merely
practical considerations of space may determine the better format, more
significant considerations may influence this choice. For example, one
format may favor the flow of a process through particular units (or
functions) of the organization, while another may emphasize the process
flow with less reference to organizational units—or none at all.
In Exhibit III-6, you can see the same process laid out in three different
ways. The process charted includes the scheduling, pressing, assembly, and
inspection of components. Note that the inspections function appears at two
different points in the process, complicating the map.
Mapping this relatively simple process requires only two standard symbols
—the rectangle indicating a process and the diamond indicating a choice
point—plus the directional arrows. Note that at each of the two choice
points (at the inspection nodes), a “no” decision sends the process back a
step or two while a “yes” sends the process one step further toward
completion.
The horizontal chart emphasizes the horizontal flow of the steps in the
overall process, moving from left to right. It includes a reference to the
functions involved in the process (scheduling, pressing, assembly, and
inspection), but they are de-emphasized and moved to the far left side of
the diagram. The vertical flowchart emphasizing flow through departments
uses a combination of horizontal and vertical structures, placing the function
names in a more prominent position at the top. The third process map
places all the process steps in a vertical flow and eliminates any reference
to the functions. The process itself is most easily followed in this format,
but it does eliminate relevant information about functions.
Exhibit III-6: Cross-Functional Flowcharts
These are all useful approaches to mapping, depending upon the desired
emphasis. Much more complex processes can be traced in flowcharts, and
much more information can be either included in the chart or referenced in
footnotes. For example, the vertical chart emphasizing process flow leaves
considerable room outside the diagram for descriptions of the steps in the
process. But in any of the three diagrams, footnotes could direct the
reviewer to narratives describing the process steps or to any information
relative to the audit.
An operation should use the type of flowchart that best suits the process
being defined. This applies to internal auditors as well if they are creating a
flowchart and mapping a process. Care should be taken to accurately
document the actual process and avoid unnecessary complexity.
Flowcharts can yield much useful information for internal auditors as well
as the client. They provide a clear picture of how a process works by
illustrating the relationship of various steps and control points. They provide
a common reference point and standard language for talking about an
existing process or project during an audit engagement.
Other Methods
Not every organization uses flowcharts to document processes or activities,
and not every internal audit group uses flowcharts. The type of organization
and the nature of operations generally influence the method used. Narratives
and block diagrams are sometimes used in lieu of or in conjunction with
flowcharts. Other tools discussed here include spaghetti maps and RACI
charts.
Narratives
Narratives provide a step-by-step picture of a process in a single document
without the use of detailed symbols or keys. Similar to a flowchart, the
purpose of a narrative is to identify the key controls and cases of under- or
over-control and processing redundancy.
Narratives are the documentation methodology many organizations use to
describe simple processes and activities performed to achieve process
objectives. Compared to flowcharts, narratives can provide more detailed
information about the steps in a process, and they are therefore often used
to augment a flowchart and provide context and nuance. Narratives are
flexible and facilitate open-ended questioning. However, there is no inherent
discipline or standardization in how to prepare a narrative; several renditions
are possible. For some organizations, narratives work well and provide
meaningful information; for others, the format may not be complete enough
and the lack of standardization may increase the likelihood of missing key
issues or control weaknesses and may be difficult for an internal auditor to
follow.
Block Diagrams
A block diagram is a pictorial representation of a process or activity,
typically including a series of boxes and connecting lines to indicate
association and direction/order. Quick and simple to construct, they can be
used to show the flow of information and organizational arrangements.
Block diagrams are sometimes used in lieu of flowcharts because of their
simplicity. Exhibit III-7 is an example of a block diagram showing the
evaluation of job descriptions.
Exhibit III-7: Sample Block Diagram for Evaluating Job Descriptions
In some representations, other geometric shapes (e.g., circles) may be used.
Block diagrams are useful for high-level representations. This kind of
information would help in planning where internal audit activities would
need to be performed in order to validate the efficiency and effectiveness of
the operation and that the organizational objectives are being achieved.
Block diagrams are not appropriate for detailed analysis.
Spaghetti Maps
Spaghetti maps are Six Sigma diagrams limited in scope to a particular
area. They are used to track the flow of products, people, and/or paperwork.
Multiple flows can be documented using different line colors. One might
visually map, on graph paper, the actual steps an operator takes in one
instance of a normal operation (not the idealized process or unusual
situations).
First, the various stations are placed on the diagram using simple boxes.
The distance between stations represents the actual distances, for example,
one box on the graph paper could represent an actual square foot or square
meter. The operator’s path is added to the diagram, starting at the first step.
If the operator goes between the same places more than once, lines are
added, even if they overlap. This can reveal instances when exceptions to
an idealized flow are actually fairly common. The time the process is taking
is also recorded, including delays such as waiting for materials or tools. The
lines on the map are then used to calculate the actual distance traveled. The
internal auditor and area specialists discuss how the process could be
improved to reduce distance and time. Improvements are documented in text
and in a new diagram.
Exhibit III-8 shows a “before” version of a spaghetti map for screen door
manufacturing; Exhibit III-9 shows the “after” version. Improvements
include placing tools in an overhead crib, rearranging the layout, adding a
kanban board (an on-demand signaling system) to reduce wait times, and
replacing inventory areas with supermarkets (an on-demand inventory
system used with kanbans).
Exhibit III-8: Spaghetti Map, “Before” Version
Exhibit III-9: Spaghetti Map, “After” Version
RACI Charts
RACI stands for responsible, accountable, consulted, and informed. A RACI
chart lists various stakeholders of a process or area in rows and columns for
R, A, C, and I. An X or check mark is placed in the chart to indicate
whether a party has one or more of these designations. Responsible means
that the person will perform the activity or process, perhaps alongside
others listed in the chart. Accountable means that the person will be
answerable for the success or failure of the activity or deliverables in
question. This may be a supervisor, for example. Consulted means that the
stakeholder should be communicated with regarding the process or area and
has a say in various decisions that need to be made. This could be an
expert or a senior manager. Informed means that the stakeholder needs to be
kept up to date on the situation or relevant matters but does not have a say
in decisions being made.
Topic D: Analytical Review Techniques (Level B)
In the previous chapter, we looked at some techniques used in gathering
information, such as interviews, questionnaires, and observation.
Information, however, constitutes only an audit’s raw material. After
amassing complete sets of data or samples, the auditor is left with questions
such as:
• Does the information indicate that the organization is in good health,
operating efficiently, and possessed of adequate controls?
• Or does the information show potential deficiencies, legal liabilities, and
failure to perform according to management’s objectives?
Analytical reviews (also referred to as analytical auditing or analytical
procedures) examine relationships among information. In particular,
examining relationships among information that is often overlooked can
provide valuable insights.
Internal auditors apply a wide variety of techniques in the process of
analyzing information, as noted in two of the Standards.
• Standard 2300, “Performing the Engagement”: Internal auditors must
identify, analyze, evaluate, and record sufficient information to achieve the
engagement’s objectives.
• Standard 2320, “Analysis and Evaluation”: Internal auditors must base
conclusions and engagement results on appropriate analyses and
evaluations.
The information examined may be financial or nonfinancial, quantitative or
qualitative. Monetary amounts, inventory quantities, ratios, and percentages
are some of the typical measures internal auditors use during an analytical
review.
Analytical reviews are used during different phases of the audit engagement.
They are used at a high level during a preliminary survey to substantiate
the internal auditor’s engagement plan—to ensure that the audit scope is
appropriate—and to help identify conditions for further investigation during
fieldwork. Analytical reviews are then used during fieldwork testing to
evaluate the data or samples collected and to develop findings, conclusions,
and recommendations.
Many of the analytical techniques useful in auditing were developed in the
discipline of operations management for use in business planning. The
internal auditor, therefore, should be able to evaluate management’s use of
these techniques or to suggest their use if that would be appropriate to
achieving the objectives of the business.
Comparisons Used in Analyzing Information
Processing information into a finished product—the audit report with
findings and recommendations that answer the questions noted at the
beginning of this topic—requires drawing comparisons between what the
auditor has found and what the auditor expected to find. The internal
auditor’s expectations may be based upon any number of relevant
information sources, such as financial statements, budgets (Are actual
expenditures the same as budgeted amounts?), regulations, industry
standards, organization objectives, and past performance. The auditor should
maintain an objective perspective based only upon the evidence gathered
during the audit. Similarly, the auditor’s conclusions should be based only
upon the results of interviews and tests.
Analytical audit procedures may include:
• Comparison of information for the current period with similar information
for prior periods.
• Comparison of information for the current period with budgets or
forecasts.
• Study of the relationships of financial information with the appropriate
nonfinancial information (for example, recorded payroll expense compared
to changes in the average number of employees).
• Study of the relationships among elements of information (for example,
fluctuations in recorded interest expense compared to changes in related
debt balances).
• Comparison of information with similar information for other
organizational units.
• Comparison of information with similar information for the industry in
which the organization operates.
As you can see from this list (which is by no means exhaustive), the
common element in analytical audit procedures is comparison. Whether a
particular set of numbers has audit significance depends upon its
relationship with another set. That can mean comparing current information
with past information about the same element of the organization—current
and past store sales, for instance, or current and past profits. It can also
mean comparison of results in one organization with those of another
organization of the same type or with averages for all similar organizations.
And it can mean comparison of a financial item, such as cost of goods sold,
with the physical items purchased. In all these cases, the value of a piece of
information is revealed only by comparing it with another value.
The simple assumption underlying all analytical comparisons is that
relationships can be expected to remain constant unless there is some
identifiable condition that would lead the auditor to expect a change.
The goal of an analytical review is to identify discrepancies (variances) in
information. These may include unexpected differences (when a stable
relationship is expected to persist) and no differences (when changes are
expected). Unexpected deviations or the lack of expected changes may
result from any of the following factors, alone or in combination:
• Errors or omissions by the organization, operations, or the internal auditor
• Noncompliance
• Illegal acts (e.g., unethical business conduct or fraud)
• Unusual events or transactions
• Accounting method used
• Inefficient or ineffective processes
After finding an unexpected change, the auditor should do further work to
find the cause. And if the auditor expects a change but finds none, that, too,
indicates a need for further study. The research that follows discovery of an
unexpected relationship is generally not analytical and might include
substantive tests such as discussions with supervisors or review of
transactions. Auditors should also be alert to a lack of discrepancies in data
and determine whether the data would normally be this pristine; pristine
data can be a red flag for fraud.
Selecting Which and How Many Analytical
Procedures to Perform
When selecting an analytical procedure, internal auditors should consider:
• The significance of the area being examined.
• The assessment of risk and the effectiveness of risk management in the
area being examined.
• The availability and reliability of financial and nonfinancial information.
• The precision with which the results of analytical procedures can be
predicted.
• The availability and comparability of information regarding the industry in
which the organization operates.
• The extent to which other engagement procedures provide support for
engagement results.
Analytical procedures take time and money to complete but may
nevertheless be critical to audit testing. Therefore, additional procedures
should be added only to areas that represent significant risk, either because
they relate to achievement of objectives or because they have special
qualities such as complexity or vulnerability to failure. For example, a
bank’s portfolio of outstanding loans is clearly critical to its revenue
objectives, so spending more time on testing sample loans for
creditworthiness is appropriate. Auditing complex hedging strategies may
also be considered necessary because of the high potential for loss if
derivatives are used inappropriately or without sufficient understanding.
Internal auditors should add an analytical procedure only when it helps
prove a point relevant to the engagement objective. This requires internal
auditors to determine what the procedure can and cannot prove and compare
this to the engagement objective. If the internal auditor wants to know
whether a purchasing function has unauthorized expenses, he or she can
trace a sample of purchase requisitions forward to related documents such
as purchase orders, receiving documents, invoices, and payments.
Alternatively, the internal auditor can start from the end and vouch the final
payments backward toward the initial purchase requisitions. What would
each of these tests prove? The tracing process would prove that purchase
requisitions were properly paid for and received, but it would not prove
when items were purchased without requisitions. The vouching process
would prove that all outgoing payments were for properly requested items,
making it the correct test to apply.
Analytical Techniques
Whatever the type of measurement employed, the purpose of analytical
comparisons is always to assist the auditor in identifying conditions that
may warrant further inquiry. Specific analytic techniques include (but are
not limited to) those shown below.
• Reasonableness tests
• Variance analysis
• Trend analysis
• Ratio analysis
Reasonableness Tests
• Regression analysis
• Cause-and-effect diagrams
• Pareto analysis
A key tenet in analytical reviews is the concept of reasonableness.
Reasonableness is the wise application of the internal auditor’s auditing
experience and knowledge of the organization and industry to any audit test
result. Reasonableness is as much a criterion underlying the other tests
described in this topic as it is a specific type of test. When comparing one
set of data to another, that is, the auditor expects to find reasonable
relationships among pieces of information. If unexpected differences are
found when comparing information or, conversely, expected differences are
absent, the internal auditor needs to make a judgment as to whether the
change or consistency is reasonable. If the answer is no, the internal audit
activity should investigate the reason for this during the engagement.
For example, the auditor might be looking at the ratio of commissions to
sales revenues for two comparable periods, expecting to find little difference
based on evaluation of turnover in the staff, introduction of new products,
and so forth. Instead, the auditor finds that revenues have dropped off while
commissions have remained constant. Since this does not seem reasonable,
the auditor needs to conduct further inquiries or tests to discover the cause.
Similarly, the auditor might be looking at trends in collections and find that
the age of accounts receivable spiked unexpectedly upward in the current
period in comparison to the historical norm for the department. This, too,
would fail the reasonableness test and would require further inquiry. Or
perhaps the auditor conducts a comparison of entertainment expenses and
employee awards and finds that costs are out of proportion with industry
norms. If the disparity seems unreasonably large, the auditor will want to
conduct inquiries into the rationale behind those expenditures. Management
may be able to explain the relationship between those outlays and superior
performance.
Variance Analysis
Variance analysis is a fundamental type of analytical procedure that begins
with the recognition that one set of data differs from another set in an
unexpected way. For example, expenses for the current period may be
unexpectedly greater than expenses for the past period—or unexpectedly
lower—by a significant amount. In addition to recognizing unexpected
changes in data, variance analysis can be used for:
• Comparisons of the objectives for the activity or process being audited to
organizational objectives.
• Analysis of the factors that have caused a difference between a planned or
standard amount and the actual results.
• Comparison of the timeliness of how actual products and services are
delivered against area and organizational objectives.
• Comparison of similar financial data for a current period with one or more
past periods.
• IT analysis practices to compare data discrepancies between two databases
(such as names or addresses between human resources and payroll files).
Budgets are a common focus of variance analysis. Although budgets can be
scrutinized from many different perspectives (credibility, amount of slack,
efficiency, effectiveness, growth or diminishment, etc.), in each instance
variance analysis can be used to compare one set of budgetary numbers
with another set that provides a standard for evaluation. For example, the
projected month-by-month budget might be compared with actual
expenditures for each month to uncover any unreasonable discrepancies. The
current period expense budget might be compared to the same expenses for
a previous period or to an industry average.
Exhibit III-10 illustrates a simple variance analysis for an annual budget
when compared against actual expenses, first in table form and then as a
chart.
Exhibit III-10: Analysis of Variance Between Actual and Budgeted Amounts
Period
Projected
Marketing
Expense (USD)
Actual Marketing
Expense (USD)
January
$9,000
$8,500
February
10,000
8,500
March
10,500
8,500
April
10,500
9,500
May
10,500
10,500
June
10,000
12,000
July
10,000
16,000
August
12,500
13,000
September
13,000
11,000
October
12,500
9,000
November
9,500
8,000
December
9,000
12,500
127,000
127,000
Total
As you can see in the table, but even more clearly in the chart, there is
considerable variance between budget and actual numbers in several places,
but especially in July, when actual expenditures exceed the budget by
U.S. $6,000, which is 60% over budget, and in December, when actual
expenditures exceed the budget by U.S. $3,500, which is about 38% over
budget. The auditor would most likely want to conduct inquiries to find out
what explains these two significant variances and, perhaps, the under-budget
amounts from September through November. (One might suspect, on the
face of the matter, that the December spending spike occurred when the
manager of the area discovered that considerable money remained in the
budget for the year.)
Trend Analysis
The most common of the analytical audit techniques, trend analysis, traces
relationships in historical financial or operational data as they evolve over
time. For that reason, it is also known as horizontal analysis. Trend analysis
is often used to identify performance indicators, highlight significant
changes, and evaluate the present position. It may be categorized as longor short-range. Long-range trend analysis examines performance data over
an extended period of time with the intent of prioritizing improvement
actions on the basis of the performance trends. Short-range trend analysis
identifies areas of potential process or system improvements.
Trends subject to analysis are many and varied. Some examples of trends
commonly analyzed are revenues, expenses, store openings, production, and
profits. Trends can also be traced for ratios, either financial ratios—such as
the trend in the price/earnings ratio for common stock—or operating ratios,
such as production cycles or units of product per hours of labor.
Trend analysis is often used to review the changes in an account balance or
another series of historical data. Because of its focus on historical sequences
of data, trend analysis is more appropriately used in reviewing data from
income statements or expense accounts than from balance sheets, which
present financial information for a particular point in time.
Although its use is associated with scrutiny of financial accounts, trend
analysis can also be applied to operating information. It can be used in the
comparison of similar data from repetitive audits and in the comparison of
organizational activities to industry activities.
A specialized form of trend analysis is known as the learning curve, which
tracks the trend in productivity in relation to the increased efficiency of
workers as they become familiar with equipment, procedures, etc. Exhibit
III-11 illustrates the general shape of a learning curve, which is, in effect, a
trend analysis of the impact of experience upon efficiency.
Exhibit III-11: Learning Curve
The trend tracked by the learning curve generally moves upward relatively
rapidly at first as workers gain experience with the job in question and do it
more effectively. Eventually the impact of more experience becomes
negligible, and the curve levels off.
As is the case with learning curves, the purpose of trend analysis is to
evaluate the impact of one variable, such as experience, upon another
variable, such as efficiency or effectiveness. The shape of the curve can tell
an auditor whether the relationship between the variables is trending in the
right or wrong direction. If the curve in the preceding exhibit were charting
the relationship between, say, output on the vertical axis and investment on
the horizontal axis, you might conclude that any further increases in
investment would be wasted, since additional dollars were no longer
resulting in higher output.
Trend analysis might be used to track the impact of additional workers on
production output. A graph of that situation might show that added workers
increase output up to a point, after which output flattens as the added
workers outstrip the capacity of available equipment and become redundant.
A common question is whether it is effective to add more workers to a
project that is at risk of missing its deadline. A graph of the situation may
reveal that adding more workers makes sense only up to a certain point
because each worker needs time to get up to speed on the project. The
learning curve may reduce the incremental gains from adding more workers
until there is actually a net increase in total project time once too many
workers are added.
Ratio Analysis
Ratios are mathematical relationships among two or more numbers, often
stated in the form of percentages, times, or days. They compare
relationships at a specific point in time. Ratio analysis generally computes
and interprets these ratios; it can be employed in comparison of
relationships between similar divisions in the organization, perhaps in
different countries. Examples of commonly used ratios are provided in Part
3 of this learning system, in Section IV, Chapter 1.
Regression Analysis
Regression analysis is a statistical technique used to measure the amount of
change in one value in relation to a change in another value. In simpler
applications of the method, regression equations show the impact that one
variable has on one other. For example, regression analysis might be used
to track the relationship between interest expense and debt levels over time,
expenses and revenues, or any other values assumed to be related. The
method is not limited to financial variables or, indeed, to variables of the
same type. Regression analysis can also be used to examine the impact of a
nonfinancial variable on a financial variable (or vice versa), such as the
impact of increasing the size of a sales force on sales revenues. In fact, one
of the benefits of regression analysis is its adaptability to tracking
relationships among many different kinds of variables.
In a regression equation with two variables, the variable presumed to cause
a change in the other is called the independent variable; the variable subject
to change is called the dependent variable. For example, in an analysis of
the impact of the marketing budget on sales revenues, the marketing budget
is the independent variable and the sales revenues are the dependent
variables. Changes in sales revenues, that is, are assumed to be dependent
upon changes in the marketing budget.
The relationship between the two variables in a simple regression analysis
can be plotted on a graph called a scatter diagram, such as the one shown
in Exhibit III-12. The regression equation would chart the slope of the line
that shows the “best fit” with the dots representing the relative values of
expenditures and sales.
Exhibit III-12: Scatter Diagram
The dependent variable, sales revenues in this case, is plotted on the Y
(vertical) axis, while the independent variable is plotted on the X
(horizontal) axis. In the diagram, you can see the data points plotted for
various revenues achieved when marketing expenditures are at particular
levels. Having plotted those points on the diagram, it is sometimes possible
to draw a trend line, such as the one illustrated, that demonstrates a pattern
in the data (assuming that there is a pattern).
In this diagram, the scattered points clearly coalesce around the trend line.
If they did not do so but were truly scattered about the chart, then there
would be no correlation between the variables and no calculations would be
necessary. (Follow-up study, on the other hand, would be indicated to
determine why the expected relationship failed to occur.)
Having drawn the line, we can see intuitively that any data points at
unusually large distances above or below the line would seem to lack
reasonableness. This is the case in Exhibit III-13. Two “outliers” are
indicated by arrows. An auditor might well suspect that these two points
represent some sort of irregularity or error and would want to conduct
follow-up tests or inquiries to determine the cause.
Exhibit III-13: Scatter Diagram With Outliers and Regression Analysis Variables “a” and “b”
While the scatter diagram provides a fairly satisfying visual representation
of a pattern in the relationship between the two variables, solving a
regression equation based on the information about the two variables
provides a more accurate representation of any pattern. It also enables us to
calculate relationships for other values of the variables.
In a simple regression analysis of two variables, the equation is:
In the equation:
• Y = the dependent variable (the sales revenue in our example).
• X = the independent variable (the marketing budget in our example).
• a = a constant number that represents the value of Y when X is 0; in
other words, the point at which the line would intersect the Y axis.
• b = the increase in Y for each unit of increase in X, which is also the
slope of the line.
For example, we could calculate the expected sales revenues for a
marketing budget of U.S. $140,000. (Assume that the value of a is U.S.
$5,000,000 and the value of b is 500.)
The regression equation can be used by management to project future
budgets based on experience with past budgets or on other measures, and it
can be used by an auditor to judge the reasonableness of either the budget
or the actual results.
Like any mathematical model, a regression equation will be much neater
than reality. We wouldn’t expect the marketing budget of U.S. $140,000 to
correlate with sales revenues of exactly U.S. $75,000,000. As the scatter
diagram shows, real numbers only approximate the clean line drawn on the
graph (or calculated by the equation).
In addition to simple regression analyses that involve one independent
variable and one dependent variable presumed to be influenced by the
independent variable, it is also possible to perform multiple regression
analysis. This statistical technique is used to trace the effects of more than
one independent variable on one dependent variable.
Like any other statistical correlation technique, regression analysis
demonstrates, at best, only that one variable changes at the same time as
one or more other variables change. It doesn’t indicate the reasons for the
change. That remains for further analysis to demonstrate. Nor does
regression analysis guarantee that the independent variable is in fact the
reason for the change observed in the dependent variable. A consistent
pattern can be quite persuasive. It can also be wrong. Other variables may
be involved that cause the change in the observed variables. Some analysts
argue, for instance, that the persistent correlation between the number of
years in school and lifetime earnings is actually a function of the fact that
people who have the ability to earn high salaries, or the desire to work
hard, are also likely to finish more years of schooling. Whether or not that
particular analysis is persuasive, the fact that such an argument is possible
indicates the necessity to back up mere correlation with a convincing
description of the reasons behind the statistical relationship.
Cause-and-Effect Diagrams
A cause-and-effect diagram (also called a fishbone or Ishikawa diagram)
uses a visual to map out a list of factors that are thought to affect a
problem or a desired outcome. An audit team might use such a diagram to
determine the root cause of a process with many problem elements. An
example of a cause-and-effect diagram is shown in Exhibit III-14.
Exhibit III-14: Cause-and-Effect Diagram
Pareto Analysis
A Pareto analysis is based on the 80/20 rule, an observation that 80
percent of the problems, outputs, or rewards of a process tend to be caused
by just 20 percent of the total causes, inputs, or effort. Internal auditors can
use a Pareto analysis to prioritize recommendations on just those key
activities, controls or other changes that are likely to create the greatest
effect. For example, Exhibit III-15 shows the results of internal auditor
analysis of controls designed to minimize a department’s expenditures.
Exhibit III-15: Pareto Analysis of Department Expenditure Controls
Of these, two key controls appear to have the greatest effect, tying the
manager’s incentives to key performance indicators for the function and the
manager’s use of an automated expense dashboard for daily decision
making. Other secondary controls contribute to a lesser extent. The former
category is often called the vital few, while the latter is called the trivial
many. Pareto charts can help explain in a visual way the importance of
prioritization—such as identifying key controls and making them the audit
focus—and can also help convince decision makers to accept
recommendations—perhaps to eliminate an ineffective but expensive control.
Topic E: Workpapers (Level P)
Workpapers prepared by the internal auditor comprise all the
documentation of an audit engagement, from the planning stage through the
final report.
Standards in the International Professional Practices Framework that are
important for understanding workpapers include the following.
• Standard 2330, “Documenting Information”: “Internal auditors must
document sufficient, reliable, relevant, and useful information to support
the engagement results and conclusions.”
• Standard 2330.A1: “The chief audit executive must control access to
engagement records. The chief audit executive must obtain the approval of
senior management and/or legal counsel prior to releasing such records to
external parties, as appropriate.”
• Standard 2330.A2: “The chief audit executive must develop retention
requirements for engagement records, regardless of the medium in which
each record is stored. These retention requirements must be consistent with
the organization’s guidelines and any pertinent regulatory or other
requirements.”
• Standard 2330.C1: “The chief audit executive must develop policies
governing the custody and retention of consulting engagement records, as
well as their release to internal and external parties. These policies must
be consistent with the organization’s guidelines and any pertinent
regulatory or other requirements.”
Implementation Guide 2330 notes that “effective workpapers contain
information that is sufficient and relevant to the engagement objectives,
observations, conclusions, and recommendations, which makes the
information useful in helping the organization meet its goals.” Workpapers
generally:
• Aid in the planning, performance, and review of engagements.
• Provide the principal support for engagement results.
• Document whether engagement objectives were achieved.
• Support the accuracy and completeness of the work performed.
• Provide a basis for the internal audit activity’s quality assurance and
improvement program.
• Facilitate third-party reviews.
The workpapers clearly have multiple applications, not only for the
purposes of the audit but for other reasons:
• They provide the principal support for the audit report and, if they are
carefully written and organized, can make writing the report an easier task.
• As a tool for communicating with third-party reviewers, the workpapers
provide evidence for use by external auditors and, needless to say, should
project an image of professional care and competence. They also provide
background material for future internal audits. The next audit can be made
easier if the current audit is fully documented. Past workpapers create a
baseline for comparison and give successive internal audits a map to
follow.
• Not only do workpapers provide a basis for external reviewers to evaluate
the quality assurance and improvement program (which is mandated in
Standards 1300 to 1322); they provide a basis for quality self-assessments
and performance reviews of the internal auditor who is responsible for the
audit and its documentation. The workpapers also can demonstrate
compliance with the Standards. Standard 1321 states that “indicating that
the internal audit activity conforms with the International Standards for
the Professional Practice of Internal Auditing is appropriate only if
supported by the results of the quality assurance and improvement
program.”
• The workpapers provide support for discussions with operating personnel.
Thorough documentation that is easy to reference in discussions can
enhance the internal auditor’s credibility with the personnel who are
necessarily better versed in the matters under discussion. When delivering
the final report, the auditor can rely on the workpapers to defend the
report’s conclusions and recommendations. To facilitate that use of the
workpapers, the auditor should thoroughly annotate his or her own copy of
the final report with references to pages in the workpapers, so supporting
details are always quickly available. Workpapers may also be crossreferenced to client documents.
• Workpapers can help to document the organization’s compliance with
regulatory and legal requirements, such as the provisions of the U.S.
Foreign Corrupt Practices Act that mandate controls to prevent
surreptitious, illegal payments to foreign government officials. Workpapers
may document the evaluation of internal control processes using the
COSO framework to support Sarbanes-Oxley reporting—both quarterly
evaluations of internal controls and management’s annual sign-off on the
accuracy of financial reports.
• Special care should be taken with workpapers so that they can be relied
upon to fulfill all these functions at the highest standards required by
supervisors, managers, quality reviewers, external auditors, and legal
authorities.
Creating properly detailed workpapers and other documentation can be a
time-consuming task for internal auditors. However, it is a necessary task
that can help defend challenges to internal audit work, especially when the
results reveal problems that a manager for an audit client does not want to
acknowledge. Having the proper evidence at hand is vital for any
constructive criticism to be accepted. The internal audit activity also uses
reviews of workpapers as a management tool for the internal audit activity
itself. Audit supervisors use workpapers to determine that an engagement is
proceeding properly and can use interim results to change direction as
needed.
Because the workpapers contain data crucial to the success of the
engagement, and because some of the data may be sensitive, their
preparation, control, use, and retention are matters of great concern.
Preparing Workpapers
Implementation Guide 2330 states:
The content, organization, and format of workpapers generally vary by organization
and the nature of the engagement. However, it is important to achieve workpaper
consistency within the internal audit activity as much as possible, as it generally
helps facilitate sharing of engagement information and coordination of audit
activities.
This Implementation Guide further suggests that “the use of standardized,
yet flexible, workpaper formats or templates improves the efficiency and
consistency of the engagement process.”
Most organizations develop unique workpaper requirements for assurance
and consulting engagements. The requirements of the Standards and the best
practices of the Implementation Guides are used as guidance in the
development of the workpaper procedures for the organization.
It is important to note that consulting engagement assignments vary widely
in scope and objectives and differ in some fundamental respects from
assurance engagements. For example, the consulting client determines the
need for the consulting engagement rather than the internal audit activity
and, like any client, has considerable influence on the nature and scope of
the work. Documenting such engagements in workpapers, therefore, may
answer to somewhat different demands than are placed upon assurance
workpapers. Nevertheless, the internal auditor acting as consultant is still
fully responsible to The IIA’s Code of Ethics and Standards. And consulting
engagements, too, must be documented in workpapers.
Necessary Workpaper Contents
Workpapers, quite naturally, should contain the work done during the
engagement. That includes virtually everything committed to paper or
entered into a computer, from initial plans through the final report—
graphics and photos included—and other physical or electronic documents.
The test of the sufficiency of the workpapers is that they should document
the audit’s objectives and methods so thoroughly that a new auditor added
to the project at any point could fully comprehend the engagement from the
workpapers and bring the audit to a successful conclusion.
Workpaper Policies
According to Implementation Guide 2330, “Because the chief audit
executive (CAE) is responsible for coordination and for developing the
internal audit activity’s policies and procedures, it is logical for the CAE to
develop guidelines and procedures for completing workpapers for various
types of engagements.” Standardized engagement working papers, such as
questionnaires and audit programs, may improve the engagement’s
efficiency and facilitate the delegation of engagement work. Engagement
working papers may be categorized as permanent or carry-forward
engagement files that contain information of continuing importance.
(Records retention is discussed later in this topic as well as noted in
Standards 2330.A1, 2330.A2, and 2330.C1, which were described earlier in
this topic.)
Standards of Style and Order
According to Sawyer, workpapers should adhere to high standards of clarity
and order. Specifically, they should be:
• Understandable. Workpapers should need no supplements to be
understandable in terms of what the auditor intended to do, accomplished,
discovered, concluded, and decided not to do. Workpapers should balance
the need to be concise with the need to be clear. Transcripts of oral
statements, as well as recorded statements, should include the auditor’s
interpretation and the client’s confirmation of any paraphrase or
interpretation. Photographs should be accompanied by any necessary
explanations and corroborative evidence.
• Relevant. Workpapers should contain only that content that is relevant and
material to the engagement objectives. Material that is peripheral should
be omitted. Clear statements of purpose help auditors determine what is
and is not relevant.
• Uniform. Electronic and hard-copy workpapers should be the same format
or size. (Small printed workpapers should be attached to standard-size
sheets, etc.) Binders help maintain printed workpapers in good order, allow
easy modification, and prevent loss.
• Economical. Keeping workpapers relevant can also help with economy.
Workpapers should not attempt to be all-encompassing, especially when an
audit indicates satisfactory conditions. Duplication of effort should be
avoided whenever possible. Client records should be copied rather than
reproduced. As many tests as possible should be included in one
workpaper, and the same sample should be used for several tests when
feasible. Answer only as many questions as are necessary to establish a
fact. Use workpapers from previous audits as much as possible. For
example, permanent or carry-forward workpapers can contain information
of ongoing importance such as flowcharts, and work from recent
engagements such as risk control matrices might be adaptable to the
current engagement. The copied data can then be updated with new
information and dates, renumbered, re-referenced, and initialed.
• Complete. No relevant question should be left unanswered. If a question
cannot be answered, the reason for this should be stated. All crossreference fields must be filled out. A “to do” checklist should be created
and included in the workpapers. All questions raised during supervisor
reviews should be answered, and the answers should be presented to the
supervisor to sign and date. Previous audit findings should be followed up,
and the workpapers should include summaries of prior notes and findings
with current status.
• Simply written. Jargon and technical or arcane language should be
avoided. If there is a reason to include words and phrases the audience
may not understand, clear explanations should be included. Statements
should be kept short.
• Logically arranged. The workpapers should be arranged in segments
corresponding to the segments of the audit, for easy reference. Keeping
the audit process and the workpapers parallel in structure will make the
workpapers useful during and after the audit. A narrative explanation
should start each segment, beginning with a description of the objectives
of the area being audited, its organizational structure, and relevant
performance data and other background information. The purpose of the
segment should also be described in detail as needed, including a
description of the audit objective for that segment and the scope of work
(what will and will not be done and sample size and selection procedures).
Sources of data, tests done, and observations of fact follow. Finally, the
segment should present conclusions that support the overall opinion when
taken together with conclusions from other segments.
Workpaper Format
The following are typical workpaper format considerations:
• Each engagement workpaper should identify the engagement and describe
the paper’s contents or purpose using descriptive headings.
• Each workpaper should be signed or initialed and dated by the internal
auditor performing the work.
• Each workpaper should contain an index or a reference number.
• Audit verification symbols (tick marks) should be explained (and should
be kept uniform throughout the audit).
• Sources of data should be clearly identified.
• Supervisory review of the workpapers should also be evidenced.
Some internal audit departments develop departmental style guides and
standardized formats that help to streamline audit procedures and facilitate
consistent, high-quality work across engagements. Care and common sense
should be used so that workpaper formats are not overly rigid and do not
inhibit internal audit ingenuity and creativity.
Media
Workpapers are not restricted to paper files and, in fact, may include a
variety of media. Electronic files are common because they are easily
stored, retrieved, backed up, and distributed through networks and updated
throughout the engagement from different locations. In addition, workpapers
may include portable storage devices (removable memory, flash drives, CDs,
DVDs), emails, photographs, and videos. Electronic files often allow storage
of such media as attachments so that the information can be accessed easily
when an auditor needs to drill down to get more detail.
Software packages, available from multiple vendors, include templates for
workpapers and other audit communications. The IIA Australia has an
online guide to software at https://www.iia.org.au/technicalresources/software-directory. In other cases, organizations may rely on
generally available software, such as Microsoft Office products, instead of
buying specialized audit software. For example, in “Automated Workpapers
Made Easy,” Pamela Jerskey describes how the Boston College Internal
Audit Department set up their electronic workpapers in Word and Excel,
with additional flowchart software and a scanner to bring in text and images
from physical sources. However, the capabilities of newer software systems
can provide additional time-saving features such as easier team collaboration
and document control.
Backup copies should be considered for all media, taking into account their
vulnerability to deterioration, loss, and obsolescence.
Some of the benefits of automated workpapers include:
• Cost savings. The savings from elimination of physical processes and
records that consume time and storage space more than make up for the
costs related to setting up and using the automated systems.
• Convenience. With transmission possible through the web and other
networks, there is no need to transport documents by any physical means,
even by walking them across the hall. This frees up staff time for more
significant work.
• Efficient communication and easy reference. Electronic documents can
be transmitted simultaneously to multiple readers, or they can be posted
on secure websites for discussion during phone or online conferences.
Document links can facilitate moving from an assertion to its proof source
(or from any point to any other) as easy as clicking a word or image—
without having to riffle through multiple paper pages.
• Consistency (standardization). With templates, quality standards are
easier to meet and information is easier to record, find, and store.
• Ability to incorporate multimedia documentation. If a picture is worth a
thousand words, how many words can a PowerPoint slide show or a video
replace? Electronic workpapers can be illustrated by graphs, control charts,
diagrams, digital photos, videos, etc. And they can incorporate scanned
materials as well.
• Security. Although no storage medium is completely secure from physical
damage, tampering, or theft, electronic files can be backed up to multiple
servers in multiple locations. Password protection is a must to protect
files, however, and files on the web should be “read only” to protect them
from malicious or inadvertent revision.
• Reputation/professional pride. As the world relies more on pixels and
less on paper, no internal audit function will want to lag behind the
standards in the profession. When everyone else is carrying a laptop and a
smartphone into conferences, the auditor holding a briefcase stuffed with
paper will feel increasingly like a refugee from another time and place.
While automated workpapers offer an organization many benefits, getting up
and running requires preparation and training. Long-term storage, while
extremely space-efficient, also presents some special challenges.
• Getting trained. Although workpaper software may be fairly easy to
learn, training is required. Organizations need to plan for the necessary
training and schedule it at the right time so that staff is ready to use the
software when it’s available. A vendor may provide training; otherwise, it
may be advisable to bring in a training professional.
• Making a smooth transition. Software templates may be close to the
formats used on paper, in which case the transition can be relatively
seamless. On the other hand, if current procedures for paper documents
aren’t standardized, setting up the automated system will require creating
standardized templates. There may also be a need to customize the
software to match the needs of the organization. Not all available packages
will be right for every organization’s needs. It’s necessary, therefore, to
research available software in light of the organization’s existing systems,
networks, etc. Making too many modifications to a workpaper package
may void the warranty and make the software incompatible with future
upgrades.
• File deterioration and obsolescence. Although electronic storage reduces
the need for huge warehouses to contain documents and provides greater
protection from damage due to fire and natural disasters, it also requires
special care if long-term retention is a goal. Paper may seem vulnerable to
physical damage, but if carefully stored it can last for centuries—not that
many audit workpapers demand such lengthy preservation. Files stored on
CDs or DVDs, however, are much more vulnerable to deterioration over
time than paper. Moreover, as the software in which they were created
goes through upgrades—or disappears from the market—recovering the
files for use may become problematic. Storage of electronic data creates a
new set of demands to be solved by the science of data preservation.
Control of Workpapers
Control of workpapers deserves the careful attention of the internal auditor,
the CAE, senior management, and, sometimes, legal counsel, because:
• Workpapers are crucial to the engagement. Workpapers are crucial to
the success of the engagement, since they are the main support for
conclusions and recommendations; they may even be critical to the
engagement’s survival. If workpapers are lost, misplaced, or stolen, the
engagement stumbles to a halt and then returns to the starting line.
Consequently, the internal auditor should know at all times where the
workpapers are and should be equally certain that they are secure. As
noted before, workpapers should always be sufficiently up-to-date and in
good order so that a new internal auditor could use them to become
familiar with, and complete, the engagement. Electronic workpapers should
be “read only” for all users except the auditor, who should be the only
person to make changes in the documents.
• They may contain confidential or personal information. The
information in workpapers may include personal data about members of
the organization, proprietary information, or data with possible legal
ramifications. For these reasons, workpapers should be shown only to
those authorized to see them.
As noted earlier, Standard 2330.A1 states that the CAE must control
engagement record access and obtain approval for the release of records to
external parties. Requests for access to records by other members of the
organization or by external auditors should be subject to approval according
to the guidelines in Implementation Guide 2330.
Backup copies of workpapers should remain in the control of the audit
department.
The CAE should develop policies regarding post-audit retention of
workpapers and all engagement records. At the end of an audit engagement,
workpaper files should be cleared out so that they contain only the final
versions of the documents. Typically, audit departments retain workpapers
for a minimum of seven years unless those documents are part of an
ongoing investigation. Workpapers should be maintained in a secure location
and access restricted.
In regard to control and retention of records from consulting engagements,
internal auditors are encouraged to adopt appropriate policies and address
related issues, such as ownership of consulting engagement records, in order
to protect the organization adequately and to avoid potential
misunderstandings. Situations involving legal proceedings, regulatory
requirements, tax issues, and accounting matters may call for special
handling of certain consulting engagement records.
Topic F: Engagement Conclusions (Level P)
Internal auditors develop conclusions regarding controls by generating
observations that result from their data gathering, analytical procedures, and
other audit tests. Implementation Guide 2320 states:
Internal auditors apply their experience, logic, and professional skepticism to evaluate
the information discovered throughout the engagement and reach logical conclusions.
Internal auditors generally approach engagements with an objective and inquisitive
mind, searching strategically for information that could fulfill the engagement
objectives. At each step in the engagement process, they apply professional
experience and professional skepticism to evaluate whether evidence is sufficient and
appropriate to formulate conclusions and/or recommendations.
Observations are defined as pertinent statements supported by objective
evidence. The internal auditor communicates those observations necessary to
support or prevent misunderstanding of his or her conclusions and
recommendations. The internal auditor may communicate less significant
observations or recommendations informally.
Engagement observations and recommendations emerge by a process of
comparing criteria (the correct state) with condition (the current state).
Whether or not there is a difference, the internal auditor has a foundation
on which to build the report. When conditions meet the criteria,
communication of satisfactory performance may be appropriate.
This topic breaks down this process, looking at results, observations,
findings, conclusions, opinions, and recommendations, but we will start by
considering the rationale for providing control assurance.
Providing Control Assurance
Providing assurance to senior management and the board that the internal
control system is operating efficiently and effectively is one of the principal
functions of the internal audit activity. Standard 2130, “Control,” states that
“the internal audit activity must assist the organization in maintaining
effective controls by evaluating their effectiveness and efficiency and by
promoting continuous improvement.”
Standard 2130.A1 (Assurance Engagements) states that “the internal audit
activity must evaluate the adequacy and effectiveness of controls in
responding to the risks within the organization’s governance, operations, and
information systems.” Requiring control reviews in these three broad areas
reinforces that internal control involves much more than financial reporting
issues.
Language in Standard 2130.A1 specifies additional requirements for internal
control assurance evaluations. The evaluation must look at the adequacy and
effectiveness regarding:
•
•
•
•
•
Achievement of the organization’s strategic objectives.
Reliability and integrity of financial and operational information and reporting.
Effectiveness and efficiency of operations and programs.
Safeguarding of assets.
Compliance with laws, regulations, policies, procedures, and contracts.
Standard 2130.C1 (Consulting Engagements) requires a transfer of
knowledge gained in consulting activities related to controls: “Internal
auditors must incorporate knowledge of controls gained from consulting
engagements into evaluation of the organization’s control processes.”
How Internal Audit Supports Management in Compliance
Organizations must adhere to laws, regulations, contracts, and other
requirements. Management and the internal audit activity both have
important roles in ensuring that this happens. Management’s role is to
maintain extensive knowledge of the compliance requirements, implement
policies, and monitor compliance. Internal auditors have the responsibility of
staying fully educated about the intricacies of, implementation strategies for,
and compliance with all current regulations and legislation. Ignoring such
requirements and even inadvertent noncompliance can seriously derail an
organization.
When providing compliance assurance, internal auditors review compliance
requirements and ensure conformity to obligations. Specifically, internal
auditors:
• Review objectives and provide insight into the ramifications of
noncompliance for the organization.
• Inform senior management of indications of significant noncompliance.
The types and numbers of compliance issues an organization must deal with
are determined by organizational specifics (e.g., size, industry, location of
operations). And the reality is that compliance requirements are always
changing. New laws, new or revised products and services, new markets,
different types of investments, and many other factors are potential sources
for new compliance requirements.
Assessing Adequacy and Effectiveness of Control Processes
A key challenge for the internal audit activity is to evaluate the
effectiveness of the organization’s control processes based on the
aggregation of many individual assessments. These assessments are largely
gained from internal audit engagements, reviews of management’s selfassessments, and other assurance providers’ work. As the engagements
progress, internal auditors communicate their findings, on a timely basis, to
the appropriate levels of management so prompt action can be taken to
correct or mitigate the consequences of discovered control discrepancies or
weaknesses.
Implementation Guide 2310, “Control,” states that “the internal audit
activity provides varying degrees of assurance about the effectiveness of the
control processes in place.” The internal audit activity should have in place
an established process for auditing and reporting control issues.
A competent evaluation of the effectiveness of controls entails assessing the
controls in the context of risks to objectives at each of those levels. To
evaluate the efficiency and effectiveness of controls, the following should be
considered:
• Were significant discrepancies or weaknesses discovered from the audit
work performed and other assessment information gathered?
• If so, were corrections or improvements made after the discoveries?
• Do the discoveries and their consequences lead to the conclusion that a
pervasive condition exists resulting in an unacceptable level of business
risk?
Implementation Guide 2310 guidance further notes that the temporary
existence of a significant risk management and control discrepancy or
weakness does not necessarily lead to the judgment that it is pervasive and
poses an unacceptable residual risk. The pattern of discoveries, the degree
of intrusion, and the level of consequences and exposures are factors to be
considered in determining whether the effectiveness of the whole system of
controls is jeopardized and unacceptable risks exist.
The CAE communicates to senior management and the board on the state
of the risk management and control processes, usually once a year.
Implementation Guide 2310 recommends:
To promote continuous improvement in maintaining effective controls, the internal
audit activity typically provides the board and senior management with an overall
assessment or compiles the results of control evaluations accumulated from individual
audit engagements.
The report states the critical role played by the control processes in the
achievement of the organization’s objectives. It also describes the nature and
extent of the work performed by the internal audit activity and the nature
and extent of reliance on other assurance providers in formulating the
opinion.
Control Self-Assessment for Assessing Adequacy of Control
Processes
As more organizations incorporate and adopt various elements from
established control frameworks (e.g., COSO), internal auditors have found
control self-assessment (CSA) exercises useful in “scoring” the control
system and evaluating soft controls.
A CSA program augments the traditional role of the internal audit activity
by assisting management in fulfilling its responsibilities to establish and
maintain risk management and control processes and to evaluate the
adequacy of that system. Through a CSA program, the internal audit activity
and the business units and functions collaborate to produce better
information about how well the control processes are working and how
significant the residual risks are.
Implementation Standard 2210.A3
The language of Standard 2210.A3 (Assurance Engagements) applies to
evaluating control for the organization as a whole and also for individual
audit projects. The standard states:
Adequate criteria are needed to evaluate governance, risk management, and controls.
Internal auditors must ascertain the extent to which management and/or the board has
established adequate criteria to determine whether objectives and goals have been
accomplished. If adequate, internal auditors must use such criteria in their evaluation.
If inadequate, internal auditors must identify appropriate evaluation criteria through
discussion with management and/or the board.
The importance of the evaluation requirements outlined in Standard 2210.A3
is best summarized as making sure that the internal audit activity considers
accountability structures, including linkage to organizational goals and
objectives through appropriate established criteria, in any assurance
engagement related to governance, risk management, and/or internal control.
Accountability and responsibility for achieving objectives are essential in
any organization. They are absolutely critical elements in a system of
internal control.
Results
After the internal auditor has completed all the research for an assurance or
consulting engagement, with complete documentation in the workpapers, the
time has come to communicate the results of all the testing, sampling,
inquiring, recalculating, and other research efforts to the audit client in a
final report. The requirements for communicating with the client appear in
the 2400 series of the Standards, beginning with the succinct injunction in
Standard 2400, “Internal auditors must communicate the results of
engagements.”
The term “results” is open-ended, but Standard 2410, “Criteria for
Communicating,” specifies that those results must include “the engagement’s
objectives, scope, and results.” Implementation Guide 2410 states:
[The final engagement communication should include] applicable conclusions, as well
as applicable recommendations and/or action plans. Opinions at the engagement level
may include a rating, conclusion, or other description of the results and their
significance.
Standard 2410.A1 provides a bit more guidance in regard to conclusions
and opinions in assurance engagement reports:
An opinion must take into account the expectations of senior management, the board,
and other stakeholders and must be supported by sufficient, reliable, relevant, and
useful information.
Interpretation states:
Opinions at the engagement level may be ratings, conclusions, or other descriptions
of the results. Such an engagement may be in relation to controls around a specific
process, risk, or business unit. The formulation of such opinions requires
consideration of the engagement results and their significance.
And, finally, Standard 2410.C1 notes that the final reports in consulting
engagements may diverge from one another and from assurance reports:
Communication of the progress and results of consulting engagements will vary in
form and content depending upon the nature of the engagement and the needs of the
client.
Observations
We learned earlier that observations are pertinent statements of facts. The
need to gather, analyze, evaluate, and record factual information has been
the primary focus of this section of the course up to this point. Internal
auditors report observations after describing their audit tests and processes
but before presenting any findings or conclusions. In this way, facts are
kept logically separate from any interpretation that follows.
But final reports are not just about the facts. They are about the meaning of
the facts for the organization and the assurance or consulting client. The
final report builds, upon a substructure of well-ordered facts, the findings,
conclusions, opinions, and recommendations that reassure the client about
what is right, point out what is not right, and recommend, if necessary,
possible solutions to identified problems.
Findings
Findings organize the facts discovered during audit research. More precisely,
they organize the facts the auditor thinks the audit client should know about
and, most likely, act upon.
A finding is generally considered to have the following parts, commonly
referred to as the five Cs:
• Criteria. The criteria are the standards, measures, or expectations used in
making an evaluation and/or verification (the correct state). As we have
seen, analysis is essentially about comparison. The criteria are the
standards of comparison the auditor uses to decide whether or not the
situation under consideration needs to be changed. The criteria might be
established by laws and regulations, or, in consulting engagements
especially, they might be established by benchmarks, such as industry
norms or best-in-class performance. In any case, the criteria establish
“what ought to be.”
• Condition. The conditions are the objective evidence that the internal
auditor finds in the course of the examination (the current state). The
conditions establish “what is”—the facts established by evaluation and
testing. The question to be answered is “How well does ‘what is’ measure
up to ‘what ought to be’ ”?
• Cause. The cause is the reason for the difference between the expected
and actual conditions—the reason that “what is” doesn’t equal “what ought
to be.” The identification of the root cause can be a challenge in some
engagements. It means identifying what has to be corrected in order to
prevent the recurrence of the condition, not just recommending something
that would solve the current finding while leaving the underlying cause
unresolved. For example, sales results in an organization may be
substandard when measured against sales results in similar organizations
because of inadequate sales training, product defects leading to higherthan-average customer complaints, lack of incentives, etc.
• Consequence (i.e., effect). The consequence is the risk or exposure the
organization and/or others encounter because the condition is not
consistent with the criteria (the impact of the difference). In determining
the degree of risk or exposure, internal auditors consider the effect their
engagement observations and recommendations may have on the
organization’s operations, reporting, and compliance objectives.
• Corrective action. The corrective action component of a finding may
include recommendations and action plans.
The effect of the disparity between what is and what ought to be, in
Sawyer’s phrase, is the answer to the question “So what?” Is the disparity
significant enough for the organization to undertake actions—and spend
money—to make a change in the conditions? The answer to the “so what”
question might be “Because you’re headed for bankruptcy” or “Your CEO
is liable, under Sarbanes-Oxley, to wind up in jail” or some less dramatic,
but still significant, potential problem for the organization.
Conclusions
A conclusion represents the auditor’s professional judgment concerning the
activities reviewed in the engagement. It is a formal version of the natural
question “What did you think of our operation?” Conclusions and opinions
(discussed below) are the internal auditor’s evaluations of the effects of the
observations (the facts) and recommendations on the activities reviewed.
They usually put the observations and recommendations in perspective
based upon their overall implications.
Any conclusions that appear in the report should be clearly identified.
Conclusions may cover but are not limited to:
• Whether operating or program objectives and goals conform to those of
the organization.
• Whether the organization’s objectives and goals are being met.
• Whether the activity under review is functioning as intended.
Opinions
Internal audit activities are asked by the board, management, and other
stakeholders to provide opinions as part of each individual audit report and
on the overall adequacy of governance, risk management, and control in the
organization. These requests may be for an assurance or opinion at a broad
level for the organization as a whole (macro-level opinion) or on individual
components of the organization’s operations (micro-level opinion). As stated
in Implementation Guide 2450:
An overall opinion is the professional judgment of the CAE based on the results of a
number of individual engagements and other similar activities—such as reviews by
other assurance providers—for a specific time interval.
The Standards don’t require opinions at a macro level; however, Standard
2410.A1 states:
Final communication of engagement results must include applicable conclusions, as
well as applicable recommendations and/or action plans. Where appropriate, the
internal auditor’s opinion should be provided. An opinion must take into account the
expectations of senior management, the board, and other stakeholders and must be
supported by sufficient, reliable, relevant, and useful information.
Without such an opinion, there are potential gray areas for interpretation.
For example, two audit reports may each contain a similar number of
control weaknesses. From the wording, it may be difficult to tell which are
the most serious, when, in fact, one of the areas might be well controlled
and the other seriously out of control.
The Standards Glossary differentiates between engagement opinions and
overall opinions:
• An engagement opinion is a rating, conclusion, and/or other description of
results of an individual internal audit engagement, relating to those aspects
within the objectives and scope of the engagement.
• An overall opinion is a rating, conclusion, and/or other description of
results provided by the CAE addressing, at a broad level, governance, risk
management, and/or control processes of the organization. An overall
opinion is the professional judgment of the CAE based on the results of a
number of individual engagements and other activities for a specific time
interval.
Interpretation for Standard 2450, “Overall Opinions,” tells us:
The communication [of an overall opinion] will include:
•
•
•
The scope, including the time period to which the opinion pertains.
•
•
A summary of the information that supports the opinion.
•
The overall opinion, judgment, or conclusion reached.
Scope limitations.
Consideration of all related projects, including the reliance on other assurance
providers.
The risk or control framework or other criteria used as a basis for the overall
opinion.
The reasons for an unfavorable overall opinion must be stated.
If the CAE is asked to provide an opinion, the opinion should clearly
specify:
• The evaluation criteria and structure used (such as the COSO internal
control framework) and the scope to which the opinion applies.
• That management has responsibility for the establishment and maintenance
of internal controls.
In addition, there should be an explanation of the specific type of opinion
being expressed by the auditor and what it means regarding the strength of
internal controls, as terms may have different meanings in different
environments. For example:
• Positive assurance (reasonable assurance) provides the highest level of
assurance and is one of the strongest types of audit opinions; therefore, it
is most preferred. Different ratings may be used, such as that internal
controls are satisfactory or unsatisfactory, effective or ineffective, meet
expectations or don’t meet expectations, etc. Variations include the use of
grading systems.
• Negative assurance indicates that nothing came to the internal auditor’s
attention that would indicate inadequate internal controls. Such an opinion
is less valuable than positive assurance, as it provides limited assurance
that sufficient evidence was gathered to determine whether internal
controls were inadequate.
• In a qualified opinion, specific findings contradict (“qualify”) the overall
opinion. This type of opinion can be useful in situations where there is an
exception to the general opinion. For example, a qualified opinion may
indicate that controls were satisfactory, with the exception of accounts
payable controls, which require significant improvement.
The CAE should ensure that the opinion expression and scope are consistent
with the internal audit activity’s board-approved charter and supported by
sufficient audit evidence. The CAE should also be aware of the intended
audience and potential use for the opinion. For example, an opinion that is
to be used by the CEO and CFO to attest to the effectiveness of internal
controls in the annual report would have different requirements than an
opinion intended strictly for internal organizational use.
More information on expressing an opinion regarding internal control can
be found in the Practice Guide “Formulating and Expressing Internal Audit
Opinions.”
Recommendations
Sawyer notes that the internal auditor owes management a recommendation
for corrective action and should point the way to the continued efficacy of
that action.
In formulating recommendations, considerations include:
• The course of action that is most practical and economical in correcting
the disparity.
• The objectives that should be kept in mind when recommending corrective
action.
• The considerations for management in setting forth an improved course of
action.
• The open choices and how they measure up when compared with the
objectives.
• The best choice with the fewest unsatisfactory side effects.
• The mechanism that should be suggested to control the corrective action
after it is taken.
A Tone of Fairness
Audit findings and opinions need not be entirely negative. Standard
2410.A2, in fact, reads, “Internal auditors are encouraged to acknowledge
satisfactory performance in engagement reports.” This assumes, of course,
that there are legitimate reasons to compliment the client. If there are both
positive and negative conclusions to be drawn about the audited function,
including both emphasizes that internal audit is objective and not biased
against the client or simply prone to find fault. If the auditor sets a tone of
fairness and objectivity, the client is much more likely to be receptive to the
findings and recommendations.
Sawyer states that:
[Any audit conclusions] should be contained in a single sentence and should:
•
•
•
Answer the question raised by the audit’s purpose.
Include a statement of the subject of the report.
Include the writer’s opinion on what was found.
The CAE should ensure that the terminology used for reporting opinions is
clearly defined and consistently applied. The Practice Guide “Formulating
and Expressing Internal Audit Opinions” includes sample opinion rating
systems and definitions.
Chapter 3: Engagement Supervision
Chapter Introduction
To ensure that the internal audit function fulfills its charter, meets its key
performance indicators, and continually improves the quality of its work,
the CAE or designee must maintain oversight of the activity’s individual
engagements. This chapter describes what is involved in the process of
supervision, coordinating work for greater efficiency and effectiveness, and
ensuring quality results and continuous improvement.
Topic A: Key Activities in Supervising Engagements
(Level B)
Performance Standard 2340, “Engagement Supervision,” states that
“engagements must be properly supervised to ensure [that] objectives are
achieved, quality is assured, and staff is developed.” Interpretation tells us:
The extent of supervision required will depend on the proficiency and experience of
internal auditors and the complexity of the engagement. The chief audit executive has
overall responsibility for supervising the engagement, whether performed by or for
the internal audit activity, but may designate appropriately experienced members of
the internal audit activity to perform the review. Appropriate evidence of supervision
is documented and retained.
Implementation Guide 2340 elaborates on how supervision is accounted for
during engagement planning, emphasizing the CAE’s role in facilitating staff
work, reviewing or ensuring review of workpapers and reports, and
resolving differences in professional judgment.
Engagement supervision is a process that begins with engagement planning
and continues through the engagement. The engagement supervisor assumes
responsibility for:
• Ensuring that designated auditors collectively possess the required
knowledge, skills, and other competencies to perform the engagement.
• Providing appropriate instructions during the planning of the engagement
and appropriate levels of ongoing supervision throughout the engagement
and approving the engagement program.
• Ensuring that the approved engagement program is completed unless
changes are justified and authorized.
• Authorizing any changes to the work program.
• Determining whether engagement workpapers adequately support
engagement observations, conclusions, and recommendations.
• Ensuring that engagement communications are accurate, objective, clear,
concise, constructive, complete, and timely.
• Providing opportunities for developing internal auditors’ knowledge, skills,
and other competencies.
Supervision focuses on progress toward achieving the engagement
objectives. The audit manager can use an audit plan checklist to gauge this
progress. A sample checklist is shown in Exhibit III-16.
Exhibit III-16: Sample Audit Work Plan Checklist
Source: Archie R. Thomas, Essentials: An Internal Audit Operations Manual, 2008
The audit manager must also monitor the engagement’s progress against
time budgets and schedules. Time records submitted periodically by audit
staff can be aggregated and compared to budgets for the different phases of
the audit. Milestone deliverables or accomplishments can be compared
against the audit schedule to determine if the audit is on track to meet its
completion date.
On a tactical level, planning and supervision involve considerations such as
(but certainly not limited to) the following items:
• Travel arrangements. Are travel authorizations (e.g., passports, visas) or
other arrangements required for the internal auditors to go to and from the
client location?
• On-site logistics. On-site logistics encompasses whatever is required for
the internal auditors to perform their work. Meeting rooms, office space,
access to necessary locations, access to required equipment such as
computers or telephones, and interpreters (if the client’s and the auditor’s
native languages differ and require translation) are all examples.
• Assignments. In an engagement requiring multiple internal auditors, will
individuals work independently, in teams, or in paired assignments (e.g.,
pairing an experienced practitioner with a novice)?
• Team coordination and supervision. What level of team communication
is necessary before the engagement starts? Will the lead auditor hold
regular meetings during the engagement? Planning should address how
engagement workpapers will be reviewed during the audit to ensure that
they properly support the engagement communications and that all
necessary audit procedures have been performed. It should also be
determined if the internal auditors will have a team meeting to close the
engagement.
• Team development. Consideration should be given to how staff training
and development, employee performance evaluation, time and expense
control, and similar administrative items will be handled.
An engagement that is properly planned, including provisions for
supervision, provides a smooth transition to on-site performance and helps
to ensure that engagement objectives are fulfilled. Planning and supervision
also support appropriate assessment of risks, controls, and governance
processes associated with the activities under review.
This topic will now look more closely at several aspects of engagement
supervision: fostering relationships, coordinating work assignments, creating
schedules, reviewing workpapers, and appraising staff performance.
Fostering Cooperative Relationships
Part of the supervisory responsibility is to tend to the human factors in an
auditing project. This includes fostering cooperative relationships between
the audit team and the individuals in the audited area—both managerial and
nonmanagerial—as well as with individuals in other areas of the
organization or outside the organization who may be involved in gathering
evidence.
These individuals can support the work aimed at meeting the engagement’s
objectives, or they can obstruct it. They can restrict access to people who
should be interviewed. They can delay providing evidence because they are
“too busy.” The absence of good working relationships makes the audit less
effective and less efficient. In a consulting engagement, the quality of
relationships created during the audit can affect internal auditing’s ability to
secure repeat and similar engagements in other areas of the organization.
The problems and challenges of relationship building can grow in relation
to the physical distance between auditor and audit client.
The types of working relationships implied in the title above are usually
based on mutual interest and honest, respectful behavior.
During initial contact with audit clients, the audit manager should discuss
what each side—auditor and audit client—needs to succeed and how the
audit can serve the needs of each. Managers must understand how they can
benefit from the engagement—that the auditor is not there to find fault and
assign blame but to apply objectivity to finding the answers to problems the
manager is probably already aware of and to identifying issues that may
have escaped the manager’s attention. The outcome of the audit may be
ways in which operations can be more efficient and profitable, support in
avoiding time-consuming and expensive disputes with regulators, or finding
the root causes of problems with quality.
Honest and respectful behavior entails:
• Accurately estimating how much time and trouble a task will require from
the manager and nonmanagerial staff.
• Listening and showing interest in the auditee’s business.
• Keeping promises and following up on questions and concerns.
• Avoiding accusations and assumptions.
• Using clear and common language. (Auditor jargon may make audit clients
feel confused and threatened. A shared language underscores shared
perspectives and goals.)
Audit managers should monitor the performance of staff in this area. The
audit manager can ask audit clients informally about their reactions to the
audit process and the auditors themselves. Client surveys can include this
dimension. Post-audit and annual performance discussions can focus on
ways to develop relationship-building skills.
Coordinating Work Assignments
During initial planning meetings, the auditor-in-charge should communicate
specific area assignments and audit tests to audit team members to ensure
that all aspects of the work program are covered and that work is not
duplicated. Dependent tasks should be clearly identified and acknowledged,
and close communication and coordination between affected team members
should be encouraged. A primary contact or liaison with the audit client
should be assigned.
Coordination can be a particular challenge in a large and/or complex audit
in which many auditors with different expertise are working independently
on different parts of the audit. The challenge increases when engagements
involve multiple audit sites, some remote. It increases further when the
audit is occurring within a global organization where business and cultural
practices may vary.
The auditor-in-charge may consider using periodic meetings, in addition to a
kickoff meeting, to support communication among audit team members.
This can help avoid duplicating effort and can foster sharing of knowledge
and experience in solving problems that have arisen. Meetings can be
conducted virtually, either in web-assisted meeting platforms or
teleconferences.
The auditor-in-charge should examine ways to encourage the right amount
of communication among the right team members. While it is desirable to
share information, team members can find themselves wasting time on
emails that have little to do with their areas of responsibility. Part of the
initial team meeting should be devoted to defining what should be
communicated to whom, within what time frame, and in what manner.
Some issues may merit an email sent to the entire team, while others should
be more focused in distribution. Legal restrictions on communication should
also be reviewed so the team understands what communications must be
retained and when it may be better to communicate in person or over the
phone.
Secure pages on internal auditing intranet sites can be created so that team
members can easily and quickly view each other’s work or pose questions.
Resources (e.g., previous audit reports, guidelines, templates) can also be
placed on the intranet site.
The auditor-in-charge or delegate may issue regular progress memos or
team meeting minutes. These memos can document assignments, agreements
on procedures and approaches, commitments, and open issues that require
follow-up. By revisiting minutes at a subsequent meeting, the auditor-incharge can confirm that open issues have been resolved.
Creating Staff Schedules
A staff schedule is an important tool in using the activity’s resources
efficiently. Engagement schedules are built on six basic blocks of time:
• Initial planning (e.g., interviews with the audit client, research into risks)
• Creation or update of the audit program, including audit objectives and
tests
• Fieldwork (e.g., interviews, observations, process mapping, analyses, other
audit testing)
• Internal audit supervisor or manager reviews
• Report writing, which also includes review and editing
• Client review and discussions
The time required for each of these stages will vary, depending on the type
of engagement and the complexity of the engagement’s objectives. Staff
required at each stage should be indicated in the schedule. It may be
possible to allocate resources at certain points in the schedule and then
transfer those resources to another engagement. Events must also be
sequenced with an eye toward dependent tasks—tasks that must be
completed before another activity can begin. Efficiencies in scheduling can
be achieved by finding tasks that can be completed concurrently.
The Gantt chart (also known as a horizontal bar chart, a milestone chart, or
an activity chart) is a scheduling technique that divides a project into
sequential activities with estimated start and completion times. It allows the
decision maker to visually review a schematic presentation of the project
time budget and allows for subsequent comparison with actual times. It can
be a useful tool in identifying dependencies and concurrent activities and in
scheduling audit team members.
To create a Gantt chart, the project manager plots the steps of the project
and their sequence and duration. The list includes the earliest start date for
each task, the estimated length of time it will take, and whether it is
parallel or sequential. This forms the basis of the chart, shown in Exhibit
III-17.
Exhibit III-17: Gantt Chart
A Gantt chart’s simplicity allows for easy schedule modifications. Gantt
charts also:
• Help plan tasks that need to be completed.
• Provide a basis for scheduling when tasks will be executed.
• Help plan the allocation of resources necessary to complete the project.
• Help determine the critical path for a project that needs to be completed
by a specific date.
• Are appropriate for internal audit scheduling because the audit process
does not often require sequence revisions.
Reviewing Workpapers
Proper supervision of engagements includes reviewing and approving
workpapers. Beyond assuring that the engagement has been carried out in
accordance with high quality standards, workpaper review also allows the
CAE, or the delegated supervisor, to evaluate each internal auditor’s current
skills and future development opportunities.
Implementation Guide 2340 specifies that the reviewer can provide
acceptable evidence of workpaper review in one of several ways:
Evidence of conformance with Standard 2340 may include engagement workpapers,
either initialed and dated by the engagement supervisor (if documented manually) or
electronically approved (if documented within a workpaper software system).
Additional evidence may include a completed engagement workpaper review
checklist and/or a memorandum of review comments.
If the reviewer has questions about the workpapers, he or she may make a
written record—review notes—for the auditor to consider. The workpapers
should then include evidence that these questions have been resolved.
Completing Performance Appraisals of Engagement
Staff
One very important part of any quality program is the continuous
improvement of the people at all levels of an organization. Standard 1300,
“Quality Assurance and Improvement Program,” specifies that a such a
program “assesses the efficiency and effectiveness of the internal audit
activity and identifies opportunities for improvement.” Standard 1311
specifies that internal assessments should include both ongoing and periodic
reviews of the internal audit activity.
Performance appraisal is a process that measures the degree to which an
employee accomplishes the work requirements stated in the performance
standards and then communicates that information to the employee. It
focuses on several levels of performance:
• The entire internal audit activity is assessed on the basis of the annual
audit plan and aggregated accomplishments.
• An audit team for a specific engagement can be assessed by both the audit
client and the auditor-in-charge.
• Individual auditors can be assessed on the basis of their performance on a
specific audit. The post-audit appraisal can be delivered and discussed
immediately following the audit. It can also be aggregated into the basis
for discussion at the auditor’s annual performance review meeting.
The post-audit appraisals can enhance the quality of the audit activity by
helping auditors learn more effective techniques for carrying out their work.
The organization itself, through managers and supervisors, can assist in staff
development in a number of ways, such as off-site and on-the-job training,
job enhancement, continually more challenging assignments, opportunities
for promotion, and, central to all these efforts, periodic performance
reviews.
Gathering Evidence of Audit Performance Quality
Evidence of audit staff performance can be gathered from direct observation
and review of an auditor’s performance during an audit. Audit
managers/supervisors should note and record specific examples of the level
of work delivered by individual staff members. Examples of performance
may involve observed behaviors (e.g., the ability to conduct an effective
interview or meeting, knowledge of the audit area’s business, acuity in
developing appropriate audit tests) and reviews of workpapers.
Evidence may also be gathered from the audit client. An audit effectiveness
survey, like the one shown in Exhibit III-18, can be distributed
electronically to audit clients after exit meetings.
Exhibit III-18: Sample Audit Effectiveness Questionnaire
Performance Review Scheduling
The performance review provides the internal auditor with the opportunity
to define his or her professional objectives, cooperate with the manager in
designing an action plan to achieve those objectives, and periodically
discuss progress and problems related to the plan. This aspect of quality is
captured in Standard 1230, “Continuing Professional Development”:
“Internal auditors must enhance their knowledge, skills, and other
competencies through continuing professional development.”
Performance reviews might take place on a dual schedule. The first track
consists of reviews that take place after each audit in which the internal
auditor participates. The second track consists of the annual review by the
CAE.
Post-Audit Review
Post-audit reviews are conducted by the auditor-in-charge for the
engagement (along with supervisory or management oversight as deemed
appropriate). These reviews have both strong and weak points:
• On the plus side, the post-audit performance review takes place
immediately after the assignment, when the auditor’s performance is
relatively fresh in the mind of the auditor and the reviewer.
• On the negative side, post-audit reviews tend to exhibit wide variability
because engagements may be very different from one another and so may
the reviewers. Five reports on one auditor may seem to be referring to
five different people.
A post-audit review should not contain major surprises. Any significant
performance issues should have been addressed in a timely fashion during
the engagement, so the post-audit review will serve only to summarize or
reemphasize key points as deemed appropriate. Any progress noted on
performance issues that were brought up earlier should be consistent in the
post-audit review.
Since audit engagements can be so different from one another, the auditorin-charge may have to develop a unique rating form for each audit—or at
least may have to adapt standard forms used in the audit activity. Criteria
would seem to be fairly consistent from audit to audit, however:
• The quantity of work completed and the ability to stay on schedule and on
budget (and proactive communication of anticipated schedule/budget
variances, including rationale)
• The quality of the work, including such measures as:
• Computational accuracy
• Selection of appropriate tests and the right number of tests
• Thoroughness in completing all necessary fieldwork tasks
• Clarity, conciseness, logical organization, professional appearance, and
usefulness of workpapers in supporting the final report
• Quality of written documents and spoken presentations
• General grasp of audit procedures and specific understanding of the
requirements of the particular engagement
• “People skills,” demonstrated in interviews with the audit client’s staff and
management—and with the audit supervisor
• Special technical skills demonstrated on the job, such as ability to perform
statistical tests, work with computer-assisted audit programs, create graphic
illustrations in a spreadsheet, etc.
• General business knowledge evidenced in understanding of the particular
challenges faced by the audit client
• Critical thinking skills and insight
Post-audit appraisals should focus separately on job performance issues on
the one hand and traits on the other. Traits indicate qualities, such as
attention to detail, creativity in problem solving, resilience, leadership,
judgment, logical thinking, self-confidence, and ability to form and maintain
relationships.
Annual Review
The annual review should consider job competencies—the behaviors,
abilities, and attributes that can link individuals or teams to enhanced
performance and the critical success factors needed to perform the internal
auditing role in an organization.
Competencies are more than basic job knowledge, skills, and abilities. Job
competencies are behaviors that are usually developed over time and
represent the compilation of multiple abilities, traits, and knowledge
required for success. Competencies are personal to the employee and are
something the employee can take from one audit engagement and project to
another, from one position to another, and from employer to employer.
Specific job competencies vary from organization to organization. But many
organizations are focusing on the competencies that individuals or audit
departments need to perform jobs rather than on specific tasks, duties,
knowledge, skills, and responsibilities.
A task force composed of IIA volunteers, national institute education
officers, and former CFIA (Competency Framework for Internal Auditors,
1999) authors have developed a global competency framework for the
internal audit professional. Four competency categories have been associated
with specific staff levels and their designated competency levels for specific
areas. The categories include:
• Interpersonal skills.
• Tools and techniques.
• Internal audit standards, theory, and methodology.
• Knowledge areas.
Within these buckets are pertinent competencies, each with an assigned
proficiency level rating of:
1 = Awareness only.
2 = Basic competence and knowledge with support from others.
3 = Independently competent in routine situations.
4 = Independently competent in unique and complex situations.
The annual performance appraisal with the CAE should include
consideration of all the post-audit reviews performed for the internal auditor
during the year. The CAE can then provide a more unified viewpoint on the
auditor’s overall performance, creating continuity in the longer-term action
plan for professional development for that auditor.
The Face-to-Face Meeting
In addition to documenting the review on a rating form provided to the
internal auditor and the CAE, the auditor-in-charge should schedule a faceto-face conference to discuss the ratings with the internal auditor. While
these conferences can be approached in different ways, some tactics seem to
be generally effective and some seem almost universally ill-advised. The
following are some guidelines.
• Schedule the review in advance—don’t sneak up on the auditor—and be
specific about the time it will take and the agenda.
• Open the review in a manner suited to the personalities and relationship of
both parties. If a question or two about the auditor’s family or hobbies is
appropriate, for example, that may provide a comfortable start. It also may
turn out to be highly inappropriate if the reviewer and the auditor have no
previous personal connection. Fitting the introduction to the personalities
of both parties requires skill and practice.
• Preface the review with a brief outline of what will be covered, including
some indication of the overall ratings: “You did a very professional job on
the whole and there are a few areas in which I think I can suggest ways
you can enhance your performance.” Unfortunately, the performance of the
auditor doesn’t always lend itself to such a positive beginning. Negative
news should be delivered in as objective a manner as possible—without an
accusatory or scolding tone of voice. This, too, is a skill that auditors with
supervisory responsibilities should work to develop. Always begin with a
positive statement, assuming there is one.
• Some supervisors ask the person being reviewed to start the conversation
with a self-assessment. This should not be done without advance warning.
An easy way to provide that warning is to give the auditor a copy of the
ratings sheet to fill out before the conference. In essence, then, reviewer
and auditor compare notes during the meeting.
• Be straightforward during the discussion, whatever the format. The only
thing more unfair for the auditor than an hour of humiliating
condemnation is a falsely flattering review. The task of the reviewer is to
help the auditor develop, and that cannot happen unless reviewer and
auditor discuss areas of weakness (or opportunities for
improvement/growth) honestly. There is no kindness in allowing a person
to keep doing inferior work until he or she is unexpectedly fired.
• Summarize the review at the end—the positives and the negatives—and
gain a commitment from the auditor to take whatever actions have been
agreed upon.
Next Steps
You have completed Part 2, Section III, of The IIA’s CIA Learning System®.
Next, check your understanding by completing the online section-specific
test(s) to help you identify any content that needs additional study.
Once you have completed the section-specific test(s), a best practice is to
reread content in areas you feel you need to understand better. Then you
should advance to studying Section IV.
You may want to return to earlier section-specific tests periodically as you
progress through your studies; this practice will help you absorb the content
more effectively than taking a single test multiple times in a row.
Index
The numbers after each term are links to where the term is indexed and indicate
how many times the term is referenced.
active listening 1
analytical evidence 1
analytical review 1, 2
attributes sampling 1
audit documentation 1
audit evidence 1
audit work plan checklist 1
auditing
continuous 1
electronic 1
automated workpapers 1
block diagrams 1
budgets 1
CAATs (computer-assisted auditing techniques) 1
cause
in findings 1
root 1
cause-and-effect diagrams 1
checklists 1
closed questions 1
cluster sampling 1
compliance 1
computer-assisted auditing techniques 1
conclusions 1, 2
conditions, in findings 1
confidence levels 1
confidentiality of evidence 1
confirmation 1
consequence, in findings 1
continuous auditing 1
continuous monitoring 1
control self-assessment 1
controls
adequacy of 1
assessing 1
effectiveness of 1
corrective action 1
criteria
in findings 1
CSA (control self-assessment) 1
data
analysis 1
collection 1
evaluation 1
difference estimation 1
discovery sampling 1
documentary evidence 1
dollar-unit discovery sampling 1
dollar-unit sampling 1
DUDS (dollar-unit discovery sampling) 1
DUS (dollar-unit sampling) 1
EER (expected error rate) 1, 2
electronic auditing 1
embedded audit modules 1
conclusions 1, 2
opinions 1
recommendations 1
results 1
supervision of 1
evidence
access to 1
analytical 1
audit 1
availability of 1
confidentiality of 1
documentary 1
gathering 1
legal 1
physical 1
relevance of 1
reliability of 1
sources of 1, 2
sufficiency of 1
testimonial 1
usefulness of 1
expected error rate 1, 2
fairness, in audit findings/opinions 1
final reports 1
findings 1
five Cs of findings 1
flowcharts 1
Gantt charts 1
GAS (generalized audit software) 1
generalized audit software 1
haphazard sampling 1
ICQs (internal control questionnaires) 1
Implementation Guides
2310 1
2320 1
2330 1, 2, 3
2340 1, 2
2410 1
2450 1
information analysis 1
information gathering 1
inquiry 1
inspection 1
internal control questionnaires 1
International Standards for the Professional Practice of Internal Auditing
2130, “Control” 1
2130.A1 1
2130.C1 1
2210.A3 1
2330, “Documenting 1
2330.A1 1
2330.A2 1
2330.C1 1
2340, “Engagement 1, 2
2400 1
2410, “Criteria for 1
2410.A1 1, 2
2410.C1 1
2450, “Overall Opinions” 1
interval sampling 1
interviews 1
judgment sampling 1
judgmental sampling 1
learning curve analysis 1
legal evidence 1
listening, active 1
mean-per-unit estimation 1
narratives 1
negative assurance 1
nonstatistical sampling 1
observation 1
observations
as input to preliminary survey 1
in engagement conclusion 1, 2
open questions 1
opinions 1
overall opinions 1
Pareto analysis 1
performance appraisals 1
physical evidence 1
population size 1
positive assurance 1
post-audit reviews 1
preliminary surveys 1
process mapping 1
qualified opinion 1
questionnaires 1
questions, open/closed 1
RACI charts 1
random sampling 1, 2
rapport, establishing 1
ratio analysis 1
ratio estimation 1
re-performance 1
reasonableness tests 1
recommendations 1
regression analysis 1
relationships, cooperative 1
relevance of evidence 1
reliability of evidence 1
reports 1
representations 1
results 1
root cause 1
sampling 1
attributes 1
cluster 1
discovery 1
dollar-unit 1
dollar-unit discovery 1
haphazard 1
interval 1
judgment 1
judgmental 1
nonstatistical 1
random 1, 2
statistical 1
stop-and-go 1
stratified random 1
unit 1
unsystematic 1
variables 1
scatter diagrams 1
schedules 1
spaghetti maps 1
staff schedules 1
standard deviation 1
statistical sampling 1
stop-and-go sampling 1
stratified random sampling 1
sufficiency of evidence 1
supervision of engagements 1
TER (tolerable error rate) 1, 2
testimonial evidence 1
third-party information sources 1
tolerable error rate 1, 2
tracing 1
trend analysis 1
unsystematic sampling 1
UPL (upper precision limit) 1, 2
upper precision limit 1, 2
usefulness of evidence 1
variables sampling 1
variance analysis 1
vouching 1
walkthroughs 1, 2
work
assignments, coordinating 1
workpapers 1, 2, 3
yes/no questionnaires 1
Build 08/24/2018 15:40 p.m.
Contents
Section III: Performing the Engagement
Section Introduction
Chapter 1: Information Gathering
Topic A: Previous Audit Reports, Interviews, Walkthroughs, and
Observations (Level P)
Topic B: Checklists and Questionnaires (Level P)
Topic C: Sampling and Statistical Analysis Techniques (Level P)
Chapter 2: Analysis and Evaluation
Topic A: Computerized Audit Tools and Techniques (Level P)
Topic B: Sources of Evidence (Level P)
Topic C: Process Mapping Techniques (Level P)
Topic D: Analytical Review Techniques (Level B)
Topic E: Workpapers (Level P)
Topic F: Engagement Conclusions (Level P)
Chapter 3: Engagement Supervision
Topic A: Key Activities in Supervising Engagements (Level B)
Index
Section IV: Communicating
Engagement Results and Monitoring
Progress
This section is designed to help you:
•
•
Understand preliminary communication with engagement clients.
•
Demonstrate communication elements: objectives, scope, conclusions, recommendations,
and action plans.
•
•
•
•
•
Deliver interim reporting on engagement process.
•
•
•
Describe obtaining management’s response.
•
•
Assess engagement outcomes, including the management action plan.
Demonstrate qualities of communication: accurate, objective, clear, concise, constructive,
complete, and timely.
Formulate recommendations to enhance and protect organizational value.
Describe the audit engagement communication and reporting process.
Describe the exit conference.
Describe the development of the audit report, including draft, review, approval, and
distribution.
Describe the CAE’s responsibility for assessing residual risk.
Describe the process for communicating risk acceptance when management has accepted
a level of risk that may be unacceptable to the organization.
Manage monitoring and follow-up of the disposition of audit engagement results
communicated to management and the board.
The Certified Internal Auditor (CIA) exam questions based on content from this section make
up approximately 20% of the total number of questions for Part 2. A few of the topics are
covered at the “B—Basic” level, meaning that you are responsible for comprehension and recall
of information. (Note that this refers to the difficulty level of questions you may see on the
exam; the content in these areas may still be complex.) However, most topics are covered at the
“P—Proficient” level, meaning that you are responsible not only for comprehension and recall
of information but also for higher-level mastery, including application, analysis, synthesis, and
evaluation.
Section Introduction
This section of Part 2 focuses on what follows an audit engagement:
communicating the results and monitoring for management remediation of
audit findings. The results noted in the engagement report have the potential
to increase the effectiveness of controls and the management of risk
throughout the organization, but only if they are translated by management
into actions and if the success of implemented changes is monitored.
Internal auditing may facilitate this process of continual organizational
improvement by providing specific observations and recommendations,
conveying the potential benefits of remedial action, ensuring response to
significant risks, and conducting appropriate follow-up and monitoring
activities. These tasks can be supported by the development of a culture that
recognizes and addresses the obstacles to the remediation of audit findings,
the reduction of organizational risk, and the implementation of a system for
conducting follow-up. This section describes the four steps in this system,
from determining appropriate follow-up to communicating the results of
follow-up activities, and relates to the following standards:
• 2500—“Monitoring Progress”
• 2060—“Reporting to Senior Management and the Board”
• 2600—“Communicating the Acceptance of Risks”
Section IV is presented in two chapters:
• Chapter A, “Communicating Engagement Results and the Acceptance of
Risk,” looks at how to communicate effectively throughout the
engagement process with operational management, senior management, and
the board. This includes preliminary and interim communication and
preparation and distribution of the final engagement communication.
• Chapter B, “Monitoring Progress,” covers both planning—deciding what
will be monitored and how—and implementation of monitoring activities.
Actions include following up on management’s responses to audit
recommendations, escalating audit issues to senior management and the
board if necessary, and reporting on significant audit issues to senior
management and the board.
Chapter 1: Communicating Engagement Results and
the Acceptance of Risk
Chapter Introduction
Effective communication during an engagement—written and verbal, formal
and informal—has many dimensions and many benefits:
• Within the audit team. Good communication is needed among audit team
members to ensure that all aspects of the audit work program are covered
and not duplicated. Helpful information must be exchanged as it is learned
to improve the quality and efficiency of audit work.
• With the engagement client. The audit team and the client must share
expectations about the outcomes of the audit and the processes that will be
used. Communication can be used to build strong and cooperative
relationships with clients and to improve the overall efficiency of the
audit. Consistent communication with engagement client management
throughout the engagement will minimize the potential for “surprises” at
the conclusion of the audit.
This chapter focuses on the role of communication throughout the
engagement process: from initial meetings with clients, to interim or status
meetings, to the development and distribution of recommendations and
reports.
Topic A: Preliminary Communication with
Engagement Clients (Level P)
Communication is a critical aspect in engagement planning. Once the
internal auditor has a draft of the engagement plan, pertinent management
parties should be briefed about the upcoming audit.
Implementation Guide 2200, “Engagement Planning,” states:
To establish the engagement objectives, internal auditors generally identify data
required within the engagement scope and communicate the scope to management of
the area under review, giving management adequate lead time for preparation.
Internal auditors also communicate with management or other key personnel in the
area under review to ensure availability of key personnel early in the process.
Throughout the engagement planning process, internal auditors typically retain
documentation from discussions and conclusions reached during meetings and include
such documents in the engagement workpapers. During the engagement’s planning
stage, internal auditors will determine the level of formality and documentation
needed. The organization’s internal audit policy manual may specify the steps for a
formal process and include pertinent templates.
During engagement planning, internal auditors may start to develop the engagement
work program, giving consideration to budgets, logistics, and the final engagement
communication format. The chief audit executive typically determines how, when,
and to whom the engagement’s results will be communicated, as well as the level of
direct supervisory needs of the audit staff, specific to the engagement plan.
Communication topics of discussion may include:
• Planned engagement objectives and scope of work.
• The resources and timing of engagement work.
• Key factors affecting business conditions and operations of the areas being
reviewed, including recent changes in the internal and external
environment.
• Concerns or requests from management.
Initial Client Meeting
During the initial client meeting (also referred to commonly as the kickoff
meeting), practical considerations that may impact the engagement should
be addressed. A variety of logistics should be discussed and/or confirmed.
Items to cover may include but are not necessarily limited to:
• Identification of key contacts and their availability.
• Preferred communication methods (means and frequency).
• Documents and records needed.
• Complexity of operations to be examined.
• Access to necessary facilities and site locations (may require personal
protective equipment or safety certifications).
• Security clearances.
• Distance between site locations and travel time.
• Tours.
• Vacation, family leave, and other client schedule constraints.
If international travel is required, allowances for passports and/or visas may
need consideration. Language/translation issues should all be allowed for as
well.
The initial client meeting is a good time to consider these logistics, as any
downtime will need to be factored into the engagement schedule.
A significant point about the first meeting is that it often sets the tone for
the upcoming audit. If it is conducted well, it can mark the start of a
productive and cooperative relationship. Handled professionally, the
preliminary client contact can encourage positive, open communications for
the duration of the engagement. The internal auditor needs to manage the
expectations of the audit team and personnel in the area being audited in
order to reduce potential conflicts.
Typically, meeting arrangements should be made in advance. Unannounced
visits should be avoided unless warranted by the nature of the audit (e.g.,
cash or security audits or fraud investigation). While the formality and
duration of the initial meeting will be dictated by factors such as the
purpose of the audit, the audit scope, and internal auditor and client
participants, an agenda outlining the matters to be discussed is
recommended.
In addition to discussing the purpose and approach of the audit, the initial
meeting with the client provides an opportunity for the internal auditor to
gain insights about management in the area being audited. The meeting may
also serve as a forum for other tasks. The internal auditor may request
specific assistance desired from the client and/or discuss the role of internal
auditing in the organization.
Topic B: Communication Elements and Quality (Level
P)
This topic covers the elements of audit reports and the characteristics of
effective reports.
Report Elements
According to Implementation Guide 2410, the format and content of the
engagement’s final conclusions may vary with the type of organization and
engagement but nevertheless must include the engagement’s objectives,
scope, and results, including applicable conclusions and recommendations
and/or action plans.
• Objectives of the engagement. A precise statement of the objectives of
the engagement can provide coherence to the rest of the report and make
it easier to read and discuss. Presentation of the audit findings should
always be related to the audit’s objectives.
• Scope of the engagement. The scope statement identifies the activities
audited. It may also specify activities excluded from the audit, if the title
of the audit would naturally lead readers to expect to find coverage of
those activities. According to Sawyer, the scope statement may also
include the time period reviewed. The scope statement may be combined
with the objectives.
• Audit methods. This may or may not be a separate section. A separate
section is often merited if new methodologies or new technology is being
used or if the work of other bodies (internal or external) provides a
substantial basis for the work. In place of a separate section, the report
may include a section on relevant methodology in the discussion of each
observation.
• Results. The results section should include observations, conclusions,
opinions, recommendations, and action plans. Some complex reports may
be preceded by an observations summary, perhaps in a table format that
identifies and describes specific observations that will be discussed in the
body of the report. Minor observations may be put in a separate section.
The recommendations may also be in a separate section if they are general
and not tied to specific observations.
Where appropriate, the auditor’s opinion should be provided.
Final communications may also include other optional sections:
• Background information. Background information may describe the
organization and the activities to be reviewed along with the results and
status of previous audits of the same activities.
• Summaries. A summary can be a useful memorandum accompanying the
full report when it is provided to an executive in the organization.
Executives may want to know the overall results of each audit in their
area of concern but not have time to read a full report.
• Client accomplishments. The final communication may include
descriptions of improvements the client has made in response to a
previous audit.
• Client views. The report may include the client’s views on the
engagement’s conclusions and recommendations. Disagreements between
the client and the internal audit activity may require intervention from an
executive. The client’s written comments may be included in an appendix
or cover letter.
The final presentation to the client does not require a written document.
Many auditors present their conclusions and recommendations in a
PowerPoint presentation.
The report should be signed by the CAE or an audit activity team member
authorized to do so by the CAE. A signed version of the report should be
kept on file by the internal audit activity.
Quality Reports
Standard 2420, “Quality of Communications,” states that communications
must be accurate, objective, clear, concise, constructive, complete, and
timely. According to interpretation:
Accurate communications are free from errors and distortions and are faithful to the
underlying facts. Objective communications are fair, impartial, and unbiased and are
the result of a fair-minded and balanced assessment of all relevant facts and
circumstances. Clear communications are easily understood and logical, avoiding
unnecessary technical language and providing all significant and relevant
information. Concise communications are to the point and avoid unnecessary
elaboration, superfluous detail, redundancy, and wordiness. Constructive
communications are helpful to the engagement client and the organization and lead
to improvements where needed. Complete communications lack nothing that is
essential to the target audience and include all significant and relevant information
and observations to support recommendations and conclusions. Timely
communications are opportune and expedient, depending on the significance of the
issue, allowing management to take appropriate corrective action.
This interpretation emphasizes the characteristics of an effective report:
organization that is easily understood and followed, clarity, conciseness, a
constructive tone, and good mechanics (e.g., spelling, punctuation, grammar,
word choice).
Organization
The IIA seminar “Effective Writing for Auditors” lists five common logical
patterns that are used to organize complex content:
• Chronological. Observations are described in the order in which they were
recorded.
• Topical. Similar observations are grouped under headings—e.g., personnel
training, contracting terms and conditions.
• Comparative. Observations are compared to specific policies, such as a
requirement for management authorizations on certain financial
transactions.
• Cause and effect. Observations are grouped by similar causes or effects.
For example, lapses in physical security could be discussed with other
observations caused by a lack of management control over policies and
procedures.
• Spatial. This can reflect both geography and organizational structure. In an
audit covering multiple locations, observations can be grouped by unit or
office. Problems affecting only certain groups may merit their own
sections—such as subcontractor behavior or warehouse issues.
Organizational tools—such as headings, topic sentences, and bulleted lists—
can all help the reader understand the content more quickly and easily.
Clarity
The reading ability of the audience should be considered when drafting
report language, as should the audience’s familiarity with the business
processes being described. A reader’s ability to understand the “message” of
the report could be affected by sentence length, grammatical complexity,
and word choice. Common terms are preferable to jargon or specialized
language (and acronyms and abbreviations) that can be understood only by
experts in the field.
Conciseness
Summaries should be used as necessary, providing an understanding of the
main point first and then allowing the reader to pursue more details in the
following text. A more formatted organization, such as a table with
consistent headings, can be used to avoid repeating similar information.
Constructive Tone
The tone of the report should be objective—not overwhelmingly positive or
negative but balanced, not alarmist but focused on responses, not assigning
blame but focused on solutions. Tone is a hard thing to master, as witnessed
by the many misunderstandings and conflicts fueled by emails that were not
reread for tone before being sent. Some writers are good self-editors, but
most writers benefit from having someone else review and comment on
matters such as tone.
The presentation’s chances of success can be enhanced by taking care to
avoid creating an adversarial atmosphere. The following are a few “soft
skill” suggestions in that regard:
• Assume that auditor and client are on the same side as partners looking
for the best ways to achieve the organization’s objectives (or the
objectives established in a consulting contract). History is not necessarily
on your side; the traditional relationship of auditor and client has not
always been a happy one. Break with tradition.
• Always begin at a general level rather than launching immediately into
detailed findings. Show that you’ve grasped the nature of the operation, its
overall mission, and its special challenges.
• Put the most positive findings first.
• Insofar as possible, present negative findings as opportunities for
improvement, but don’t overdo it. Nothing is less persuasive than a falsely
positive presentation.
• Be sure to emphasize the “effects” aspect of findings—what consequences
loom down the road if changes aren’t made.
• Don’t simply stop talking; conclude. Summarize the results briefly with an
emphasis on action steps the client can take. End on a positive note, just
as you began on one.
Good Mechanics
Auditors are experts in auditing, not necessarily in the mechanics of writing.
It is desirable for auditors to improve their writing skills, and this can be
part of a personal development program. Until this happens, however, an
auditor assigned to writing a report should find someone in the activity to
review report drafts before they are shared outside the activity. Auditors can
also look to numerous websites that provide rules about grammar and the
use of punctuation. The credibility of a good report can be undermined by
poor mechanics that may convey to a reader carelessness and lack of
attention and value for detail.
Topic C: Interim Reporting on Engagement Progress
(Level P)
Some years ago, a major European manufacturer of luxury automobiles
aired a commercial about certain safety innovations it had made and, for
altruistic reasons, had not patented. The concluding line of the spot was
“Some things are too important not to share.” Those words accurately
identify a major reason for having interim progress reports during an audit
engagement. Some audit information is too important not to share
immediately rather than in the final report.
According to Implementation Guide 2410, “Criteria for Communicating,”
engagement communications are a critical component of how an internal
audit activity demonstrates value, and the format and content of such
communications may vary by organization or type of engagement. It states:
Communication with management is an ongoing process throughout the engagement.
The internal audit activity adds value by developing communications (both verbal
and written) that effect positive change in the organization.
Internal auditors should communicate with process owners and managers
during engagement planning, fieldwork, and testing. It is particularly
important to communicate if there are deviations to the planned scope,
objectives, or final results of the engagement.
Interim reports may be written or oral and may be transmitted formally or
informally. They may be used to communicate information that requires
immediate attention, to communicate a change in engagement scope for the
activity under review, or to keep management informed of engagement
progress when engagements extend over a long period of time.
This type of interim progress can be reported through a status meeting,
report, or email. The point is that ongoing communication should be
maintained throughout the audit engagement.
Sawyer adds another reason for interim communications: to inform
management of significant matters not related to the engagement.
Sawyer notes that communicating observations on an interim basis has
advantages:
• The engagement process becomes more efficient, as auditors can clarify
issues before unnecessary work is performed. Clients can save auditors
time by suggesting possible causes for problems uncovered. Important
issues can be uncovered and addressed early, reducing impact on
schedules.
• The engagement process is more effective since interim informal meetings
help ensure that relevant information is uncovered and understood before
evaluations are made and recommendations formulated. Clients also have
more time to develop responsive action plans.
• Auditor-client relationships are strengthened. Informal meetings can
increase client involvement in the audit process.
The use of interim reports does not diminish or eliminate the need for a
final engagement communication. Sawyer recommends interim written
reports as a path to higher-quality final reports and more client buy-in to
observations and recommendations. The amount of detail in the reports may
be increased, since the experience will be fresher in the writers’ memories,
and the time required to create a draft final report may be shortened, since
many sections will have already been reviewed by internal auditing
management and the audit client. Sawyer adds a qualifier to the effect that
the findings in the interim report may be excluded from the final report if
they have been properly resolved and are no longer of importance to the
client’s operations.
One especially significant piece of news too important not to share is a
strong suspicion that fraud has occurred. If a preliminary investigation has
established evidence that fraud may have occurred, the CAE should state
whether or not the internal auditor recommends a full fraud investigation.
The report should also summarize observations and recommendations
leading to the decision that an investigation is warranted.
Topic D: Recommendations to Enhance and Protect
Organizational Value (Level P)
If, as Sawyer characterizes them, audit findings are an answer to the
question “So what?” and if audit conclusions answer the question “What do
you think about our operation?” then recommendations answer the question
“How should we fix this problem?” Recommendations should address the
condition as well as the root cause.
Implementation Guide 2320 states that:
Internal auditors apply their experience, logic, and professional skepticism to evaluate
the information discovered throughout the engagement and reach logical conclusions.
Internal auditors generally approach engagements with an objective and inquisitive
mind, searching strategically for information that could fulfill the engagement
objectives. At each step in the engagement process, they apply professional
experience and professional skepticism to evaluate whether evidence is sufficient and
appropriate to formulate conclusions and/or recommendations.
What’s in a Recommendation?
Internal auditors must document information that logically supports the
engagement results and conclusions. Effective workpapers contain
information that is sufficient and relevant to the engagement objectives,
observations, conclusions, and recommendations, which makes the
information useful in helping the organization meet its goals.
The following is some specific guidance about recommendations:
• Recommendations are based on the internal auditor’s observations and
conclusions.
• Recommendations call for action to correct existing conditions or improve
operations and may suggest approaches to correcting or enhancing
performance as a guide for management in achieving desired results.
• Recommendations can be general or specific. For example, under some
circumstances, the internal auditor may recommend a general course of
action and specific suggestions for implementation. In other circumstances,
the internal auditor may suggest further investigation or study.
An effective approach to developing recommendations is to adhere to the
SMART principle shown in Exhibit IV-1.
Exhibit IV-1: SMART Model for Composing Recommendations
S
Specific
M
Measurable
A
Action-oriented
R
Relevant
T
Time-based
• Specific recommendations outline exactly what the organization should
aim to accomplish.
• Measurable recommendations can be evaluated to determine whether they
have been accomplished.
• Action-oriented recommendations specify the actions that the organization
will be able to take.
• Relevant recommendations relate to the nature of the organization, and
they are attainable.
• Time-based recommendations specify the time frame for accomplishing
the recommendations.
The internal auditor should go through a logical, thorough process and
analyze the recommendations before incorporating them into an engagement
communication.
Basic considerations include but are not limited to the following:
• Will the recommendations address the root cause?
• Are the costs realistic in terms of the expected benefits?
The audit client’s management won’t always agree with, or want to act
upon, the auditor’s recommendations. As part of the internal auditor’s
discussions with the engagement client, the internal auditor obtains
agreement on the results of the engagement and on any necessary plan of
action to improve operations. If the internal auditor and the engagement
client disagree about the results, the engagement communications should
state both positions and the reasons for the disagreement. The engagement
client’s written comments may be included as an appendix to the
engagement report, in the body of the report, or in a cover letter.
When making recommendations, the auditor should disclose any conflicts of
interest, such as having worked in the business function being audited
within the past year.
Audit, Don’t Manage
One of the difficulties involved in making recommendations is that doing so
threatens encroachment on territory that belongs to management, not to the
internal audit activity. Indeed, the internal auditor is specifically enjoined
not to take on responsibilities that rightly belong to management, because
doing so threatens the auditor’s objectivity. This is particularly true in
consulting engagements, since the auditor may have been called on
specifically to give advice based on research into a particular problem, such
as whether or not to accept a particular contract, how to develop a new
software system, whether or not a particular company is a good takeover
target, and so on.
Once a consulting auditor feels some sense of responsibility for the actions
based upon audit recommendations, the auditor’s objectivity for any future
assurance audit comes into doubt. Internal auditors should maintain their
objectivity when drawing conclusions and offering advice to management.
Any impairments that existed prior to or that develop during an engagement
should be disclosed to management immediately.
Give Managers Credit for Managing
Sawyer notes that recommendations are not commands, merely options, and
the auditor should not deliver a recommendation as if it were the only
possible course of action, because the manager generally has a broader view
of the possible consequences of acting upon a recommendation than the
auditor. It’s a good idea to bring recommendations to the manager for
discussion before the end of the audit. Working jointly with the manager to
come to agreement about a course of corrective action improves the
relationship. The manager will look better to superiors if the audit report
states that recommendations were developed after discussion with the
manager.
The internal auditor, in Sawyer’s view, should consider the relationship
between the cost of a recommended action and the benefit to the
organization. Some actions must be taken regardless of cost to bring the
organization into compliance with a law or regulation, but otherwise there
should be a balance between cost and risk.
Topic E: Audit Engagement Communication and
Reporting Process (Level B)
This topic covers three aspects of audit engagement communication: holding
the exit conference, developing the audit report, and obtaining
management’s response.
Holding the Exit Conference
The final act of an audit engagement is communicating its results—the
findings and recommendations. The discussion here focuses on written audit
reports, but the communication may be in other formats: verbal, web-based,
or a PowerPoint presentation.
Success in this phase is crucial, since audit reports are not meant to be filed
with other seldom-read reference works but are intended instead to stimulate
activity—or at the very least receive serious consideration. The CAE bears
responsibility for seeing that the final report is skillfully prepared, adroitly
presented, brought to the attention of the client’s decision makers, and kept
out of the hands of those who are not authorized to receive it. Chances for
a favorable reception of the audit report can be improved if client managers
have been consulted about the findings and recommendations before
communication of the final draft. It helps to have key people on your side
when you deliver the recommendations.
After creation of a rough draft of the final report, there should be a meeting
between the process owners (the audit clients) and the internal auditing
team. This is generally called the exit conference, exit interview, or postengagement meeting. People attending the exit conference are often the
same participants who attended the entrance conference, including people
familiar with operations details and people who can authorize the
implementation of corrective actions.
The objectives of the exit conference include:
• Discussing conclusions and recommendations.
• Resolving any misunderstandings or misinterpretations of fact by allowing
the client to ask for clarification of specific items and to express views on
observations, conclusions, and recommendations.
• Reaching agreement on possible solutions to problems identified in the
report. This can help management of the area to formulate its response to
the audit findings, which may require requesting budget or policy
making/changing by senior management.
• Expressing appreciation for cooperation during the performance of the
internal audit and for providing required information in a timely fashion.
The IIA seminar “Tools and Techniques for the Beginning Auditor”
recommends the following best practices for the logistics of exit
conferences:
• Ensure that the right people attend.
• Provide the necessary documents (i.e., the draft report) in advance.
• Set the agenda and manage the meeting.
• Explore and resolve as many issues as possible.
• Provide clear messages, even about difficult issues.
• Thank the audit customer for cooperation, both at the beginning and the
end of the conference, acknowledging the key contacts by name and
recognizing that audits can be disruptive to normal activities.
• Hold a post-meeting debriefing with the audit team.
Developing the Audit Report
The steps in developing the audit report include drafting, reviewing,
approving, and distributing it.
Drafting
When drafting the audit report, it is important to keep the needs of the
audience in mind. If there will be more than one audience, there may (or
may not) need to be more than one version of the report. If different
versions of the report are deemed beneficial, it is imperative that the overall
objective substance of the report (e.g., final results, opinions,
issues/recommendations, management responses) be unchanged between the
versions. Another thing to keep in mind is that report drafting needs to be a
timely endeavor, and a good way to ensure that this occurs is to create
deadlines for each draft and its review.
Few people can generate a document or other presentation that is easy to
read in one draft. For this reason, it is important to follow some basic
writing principles when drafting a report. This can include creating an
outline and then two or three drafts of the report.
An outline—a best practice that can also be a time saver—indicates what
you will discuss and in what order, perhaps listing the headings or slide
titles and then including brief summaries of what each will contain.
Outlining before writing can prevent the inclusion of things that could be
omitted.
The first draft will involve getting the information down—the hard work of
summarizing audit work and findings, formatting charts for presentation,
and so on. Once the first draft is written, it will be much easier to write the
executive summary, so perhaps save this for the last thing to write.
The second and third drafts can then focus on cleaning up the writing,
improving the organization of the materials for logical flow, and ensuring
that points are clear, concise, and provided in a constructive tone and that
good mechanics are used overall, as was discussed earlier in this chapter.
Reviewing
Having someone who did not write the outline and drafts read and comment
on the report at each stage of its development can help show areas that are
confusing, incomplete, have flaws in logic, and so on. A supervisor is a
good option; audit clients may also be appropriate. In selecting a reviewer,
internal auditors need to consider who should have access to the sensitive
contents of the report.
In addition to reviewing to improve grammar and style, reviewers should
look for whether the report is factually correct, whether it is complete, and
whether the conclusions are supported by evidence that is sufficient,
reliable, relevant, and useful.
A few areas of the report are especially important to review for proper tone.
One of these is the discussion of management’s action plans. It is important
to emphasize an expected completion time line and sufficient urgency, but it
is also important to ensure that the tone is constructive. Matters of lower
significance should not be made to seem more important than they actually
are, so that areas of high significance will stand out.
Approving
The CAE or a designee should review and approve the final engagement
communication before it is issued and should decide who should receive it.
The auditor-in-charge, supervisor, or lead auditor may be authorized by the
CAE to sign engagement reports on behalf of the CAE. In large
international organizations, requiring the CAE’s signature on all final
communications might cause delays.
However, Performance Standard 2440, “Disseminating Results,” states that
the CAE maintains responsibility for communicating the final results of an
engagement. Although it is not required in the standard, in many
organizations the CAE signs the final report before distribution as a sign of
commitment to the quality of audit work.
Distributing
The CAE bears responsibility for communicating final results to individuals
who can ensure that the results are given due consideration. The report
should go to those in a position to take corrective action—for example,
management of the audited area or operation, senior management, or
associated functions that may be affected by or can support recommended
action plans.
Communications may also go to external auditors, the board, and others
who are affected by or interested in the results.
If substantive corrections must be made to a report after it has been
distributed, the CAE should issue a new report that highlights the changes
and see that is distributed to all recipients of the original report.
Before releasing reports to parties outside the organization, the CAE should
assess potential risks to the organization and obtain approval of senior
management, legal counsel, or both. The CAE is responsible for controlling
the distribution of the report.
Release of consulting reports should be consistent with the organization’s
established practices. Because of the nature of the activities that internal
auditing helps to evaluate, many organizations allow only limited
distribution of consulting reports.
Obtaining Management’s Response
Having gone to the trouble of researching and assembling the findings and
recommendations, the audit activity would naturally like to see them acted
upon. Guidance on getting the findings reviewed and taken seriously is
spelled out in Implementation Guide 2440. It specifies, among other things,
appropriate times for discussion of the results as well as appropriate
recipients. For example, the Implementation Guide suggests that:
Before communicating the results, it can be advantageous for the CAE to review the
draft engagement communication(s)… It may be appropriate for results to be
delivered via a meeting with a presentation and an opportunity for discussion [before
the CAE issues the final engagement communications].
Why? Because once the final report has been completed and distributed, the
chance for serious dialogue on the findings may have passed.
Implementation Guide 2410 also suggests that when planning the final
engagement communication, internal auditors should consider any
discussions and interim communications they have had with the
management of the area under review. At this stage, the client can clear up
misunderstandings and react to the findings while there is time to
collaborate on revisions.
The participants in these discussions will generally be individuals who are
knowledgeable about the detailed operations and those who can authorize
the implementation of corrective action.
In other words, if you can get early agreement on the recommendations
from the people who can effect changes in the client’s operations, you have
a much better chance of getting the final report acted upon.
Topic F: CAE Responsibility for Assessing Residual
Risk (Level B)
We learned in Section I that residual risk is the level of risk that remains
after management executes its risk responses. It is sometimes also referred
to as net risk. Standard 2600, “Communicating the Acceptance of Risks,”
addresses the issues of accepting a level of residual risk that may be
unacceptable to the organization.
[When the CAE] concludes that management has accepted a level of risk that may
be unacceptable to the organization, [he or she] must discuss the matter with senior
management.
If, at that point, the CAE determines that the issue has not been resolved, it
is the CAE’s responsibility to communicate the matter to the board. Often,
it is helpful to identify the type of risk, such as financial, operational,
compliance, strategic, or a combination of several types.
The level of residual risk is either going to be acceptable or unacceptable.
If the residual risk is at a level less than the risk appetite of the
organization, the system of internal controls is considered to be operating at
a level that is deemed acceptable by the organization. However, if the
amount of residual risk exceeds that which the organization considers to be
within its risk appetite, the system of internal controls is considered
unacceptable. In that situation, the CAE must evaluate the system of
internal controls and determine whether additional cost-effective controls
should be implemented to further reduce the residual risk, so the residual
risk returns to an acceptable level. If this is not an option, management
must consider other options, such as transferring part of the uncontrolled
risk to a willing, independent third party through insurance or outsourcing.
This can also be accomplished by sharing the uncontrolled risk. If
transference of uncontrolled risk is not possible, management must
determine if they should accept a higher level of risk and adjust their risk
appetite to incorporate the higher level of risk. Alternately, management can
determine whether or not they want to continue the activity or if they want
to eliminate the activity altogether and thereby eliminate the residual risk.
Topic G: Communicating Risk Acceptance (Level B)
Management’s response to internal audit recommendations may at times be
considered inadequate by internal audit for benign reasons. For example, a
reorganization could bring a change of managers and the new manager may
not be aware of audit’s recommendations. However, an internal audit
observation that represents a significant risk to the organization requires a
prompt and effective management response. If the CAE believes that
management is not responding or is responding inadequately to the observed
risk and is unwilling to control the risk, then the CAE is professionally
obliged to escalate the matter to the next level of attention—to senior
management, if necessary, and to the board if senior management fails to
respond.
Performance Standard 2600, “Communicating the Acceptance of Risks,”
states:
When the chief audit executive concludes that management has accepted a level of
risk that may be unacceptable to the organization, the chief audit executive must
discuss the matter with senior management. If the chief audit executive determines
that the matter has not been resolved, the chief audit executive must communicate
the matter to the board.
Interpretation of Standard 2600 states:
The identification of risk accepted by management may be observed through an
assurance or consulting engagement, monitoring progress on actions taken by
management as a result of prior engagements, or other means. It is not the
responsibility of the chief audit executive to resolve the risk.
Chapter 2: Monitoring Progress
Chapter Introduction
Internal auditing assurance or consulting engagements are discrete projects
that use project management approaches during planning and execution. A
risk of project-based work is that no one will follow up on management
action plans documented in the final audit report because the team members
have moved on to the next project and have other responsibilities, deadlines,
and time constraints. However, if this natural tendency is allowed to occur
or persist, it can create a more significant risk for the internal audit activity.
The activity could be perceived as irrelevant and not value-added because
no real changes are being made and no improvements in results are
therefore being realized. The entire internal control framework could be
jeopardized if management falls into a routine of ignoring audit
recommendations and fails to properly execute its control-monitoring
responsibilities. Therefore, monitoring of engagement outcomes is not just a
minor afterthought but a critical control step that needs to be championed to
the board, senior management, and process owners; included in the annual
audit plan and personnel schedules; and assigned to specific personnel who
are held responsible and accountable. (Note that in a consulting project,
monitoring is performed only if it was an objective requested by
management.)
We learned in the previous chapter that at the end of the internal auditing
engagement, the internal audit team and the CAE prepare a report that
assesses conditions found against the criteria defined at the beginning of the
audit. This report identifies gaps in performance or potential risks and
prioritizes their correction according to a variety of factors, including their
financial effects (e.g., inefficiency, waste, damage or loss claims, fraud) and
ethical or legal impacts (e.g., violation of organizational codes of ethics,
laws, or regulations). The report also describes (if necessary) criteria to
correct shortcomings—an action plan that management of the area being
audited agrees to accomplish along with a time line for implementation.
This decision can be to implement all or some of the recommendations or
to accept the risk and do nothing. The decision should have been arrived at
in collaboration with internal auditing, so the CAE should be in support of
the method used. If not, the CAE should escalate the issue to the board or
senior management.
This chapter starts by looking at the assessment of engagement outcomes.
Internal auditors will typically decide on the most appropriate monitoring
method in consultation with the manager responsible for implementing the
action plan. Internal auditors document the process to be used in the final
engagement report. The remainder of the chapter shows how to execute this
follow-up process and report adequate or inadequate actions to the board or
management.
Topic A: Assessing Engagement Outcomes (Level P)
Follow-up is defined as a process by which internal auditors determine the
adequacy, effectiveness, and timeliness of actions taken by management on
reported engagement observations and recommendations, including those
made by external auditors and others.
The responsibility for follow-up is stated in the following standards:
• Performance Standard 2500, “Monitoring Progress”: “The chief audit
executive must establish and maintain a system to monitor the disposition
of results communicated to management.”
• Standard 2500.A1: “The chief audit executive must establish a follow-up
process to monitor and ensure that management actions have been
effectively implemented or that senior management has accepted the risk
of not taking action.”
• Standard 2500.C1: “The internal audit activity must monitor the disposition
of results of consulting engagements to the extent agreed upon with the
client.”
The responsibility of the internal audit activity to perform these activities
should be specifically expressed in the internal audit activity’s written
charter.
Implementation Guide 2500 tells us:
Monitoring processes can be sophisticated or rather simple, depending on a number
of factors, including the size and complexity of the audit organization and the
availability of exception tracking software. Whether sophisticated or simple, it is
important for the CAE to develop a process that captures the relevant observations,
agreed corrective action, and current status…
Often, the CAE will develop or purchase a tool, mechanism, or system to track,
monitor, and report on such information. Based on information provided to internal
audit by the responsible management, the status of the corrective actions is updated
in the system periodically and often directly by management using a shared
exception tracking system.
The method used to monitor management response and ensure that it is
addressing the identified risk (or opportunity) will vary according to the
change being implemented. Scheduling of follow-up should be based on the
risk and exposure involved as well as the difficulty and significance of
timing in implementing the corrective action.
Planning
As with the initial engagement project, planning is the key to success in
monitoring outcomes of engagements. The monitoring plan will depend on
factors like the significance of the recommendation and its complexity.
Basic issues of planning include the following:
• Who will monitor the outcomes of the recommendations. Specific
internal auditing personnel are assigned to monitoring responsibilities. If
the cooperation or support of other areas is required, those areas are
contacted and efforts are coordinated. For example, if a recommendation
involves the physical redesign of an area to improve employee safety, it
may be necessary to secure the services of an ergonomic workplace
designer and work with management of the physical operations area to
schedule remodeling.
• What will be monitored. The objectives of the recommendation must be
translated into measurable and observable criteria. For example, a
recommendation that the accounting department adopt new accounting
methods may require only acquisition of training and necessary materials
(e.g., computer applications) and adoption by a certain date. A
recommendation that all existing and future vendors demonstrate
compliance with tax and other regulatory requirements will require proof
of compliance. Assigned internal auditors will need to review all or a
sample of vendor documents to verify this proof.
It is important that management be aware of and agree with what internal
audit needs to conduct monitoring and verify achievement of the
recommendations so management actions will be accepted by internal
audit.
• How the monitoring will be accomplished. Monitoring can be conducted
on an ongoing basis. For example, internal auditing may recommend that
transactions above a certain amount be conducted only by personnel with
certain privileges. Internal auditing might access the computer system
remotely a certain number of times to confirm that the new practice is in
place and is being followed.
For an especially significant recommendation—e.g., to correct irregularities
in preparing a financial statement—internal auditing may schedule a
complete follow-up engagement. For less critical recommendations—e.g.,
improvement of physical security measures in an office—a follow-up
questionnaire or conversation may be adequate. Internal auditing should
specify in what manner management should confirm their response to the
recommendation—in writing, by email, or verbally.
• When or how often monitoring will be conducted. Management is
informed of the time frame in which its response is expected and of the
fact that follow-up will occur after that date. As discussed, the time frame
for follow-up depends on the significance of the recommendation. An
urgent matter should be resolved immediately. If a recommendation has
involved a particularly complex solution, internal auditing may decide to
schedule a series of follow-up activities keyed to specific stages. For
example, internal auditing may review a proposal by an information
systems vendor who will create the recommended application. Later,
internal auditing may check to see if the project is on schedule. Auditing
may be involved in a beta test of the system and in analysis of the test
results. Auditing may decide to confirm final results after full
implementation of the new system.
Two important elements developed during planning include an issuestracking system and specific monitoring procedures.
Issues-Tracking Systems
Since monitoring of outcomes will occur sporadically throughout the year
for various engagements, an important step is to develop a system for
tracking issues and the resolution status of action plans. To ensure that
follow-up is conducted on the agreed-upon schedule, the CAE may use
scheduling software to record follow-up periods for each engagement. These
could be:
• Particular dates or milestones.
• Monthly, quarterly, annually, or on some other regular basis.
• During the lead-up to the next audit of the area, such as in four years.
An automated system with automatic schedule reminders for specific
internal auditors can help reduce the risks that these tasks will be
overlooked. If process owners have committed to following up themselves,
these individuals should also be provided with automated schedule
reminders.
Monitoring Procedures
Internal audit activities can develop specific monitoring procedures and
predetermine a threshold for deciding whether a control weakness or other
audit recommendation has been resolved or not. The purpose of such
procedures is to give internal auditors clear decision criteria on how to
evaluate data collected during monitoring.
Monitoring procedures should be based on a monitoring framework.
According to COSO’s description of its Internal Control—Integrated
Framework and a 2009 supplemental COSO document, Guidance on
Monitoring Internal Control Systems, the monitoring component of the
COSO framework is developed by establishing a foundation and designing
procedures. The framework guides execution and assessment of results,
prioritization of observations, reporting results at the appropriate level, and
following up on corrective action. COSO makes monitoring the
responsibility of management; however, the internal auditing function
enables management to fulfill these responsibilities and so should adopt or
develop a monitoring framework for the internal audit function.
While some monitoring may require specialized or customized methods and
thresholds for acceptance, others can be standardized. A common area for
setting predetermined methods and acceptance thresholds is sampling. For
example, the procedure may be to use judgmental sampling for certain
transaction controls and may require testing of, say, 50 items. The
procedure could be as follows:
• If five or fewer fail, then pass the control.
• If more than five items fail, then test another sample of 50 items, and if
five or fewer fail, still pass the control.
• If more than five items fail in the first or second pass, fail the control.
A control that requires more reliability could set the allowed failure rate to
fewer noncompliant transactions, for example, just one failed item.
Topic B: Monitoring/Follow-Up of Disposition of
Engagement Results (Level P)
This topic looks at several aspects of audit engagement follow-up: CAE
responsibility for monitoring outcomes, follow-up procedures, determining
appropriate follow-up, securing action and conducting follow-up, and
documenting and reporting on progress.
Responsibility of CAE to Monitor Engagement
Outcomes
According to Implementation Guide 2500, “Monitoring Progress,” if the
audit report produces recommendations to management, it is the
responsibility of the CAE to establish procedures that ensure that
management responds to those recommendations.
For outstanding observations, the information tracked and captured typically
includes:
• The observations communicated to management and their relative risk
rating.
• The nature of the agreed corrective actions.
• The timing/deadlines/age of the corrective actions and changes in target
dates.
• The management/process owner responsible for each corrective action.
• The current status of corrective actions and whether internal audit has
confirmed the status.
Achieving adequate corrective action from management will be more likely
if internal audit ensures that its recommendations are made to individuals
who have the authority and the ability to make the changes that will
address the problems. Management may respond to auditing’s
recommendations orally or in written form. Management should be informed
about the level of detail that internal auditing needs in order to evaluate the
appropriateness of management’s corrective action and determine the most
appropriate follow-up. An adequate management remediation should
demonstrate a complete and accurate understanding of the risks that have
been identified and should establish clear objectives for the control that is
being implemented.
Types of Follow-Up Procedures
The audit charter may specify that a particular type of follow-up must be
used, or it may allow the CAE to determine the nature, timing, and extent
of follow-up. The latter method allows internal auditors to employ different
follow-up procedures depending on the significance of the recommendations
or other factors.
Three types of follow-up procedures are described here:
• Process owner is responsible for following up. Making the process
owner responsible for following up minimizes the required schedule time
and involvement of the internal auditor; it also provides the weakest form
of audit evidence of the three methods because the process owner may not
be objective or could be reporting results that have not actually been
accomplished. However, this method can be structured by specifying a
reporting frequency and by requiring written documentation on each action
item with key performance indicators that can be verified. This method
can also help with relationship building, as it empowers the process owner
and reinforces that the process owner is responsible for changes. The
drawback of empowerment is that the process owner could unilaterally
change the action plan or do nothing as the situation changes, believing
(perhaps incorrectly) that the new events justify the changes. The auditor
can decide whether he or she is satisfied with the corrective actions or if
additional clarification or follow-up is needed.
• Auditor conducts a targeted follow-up review. The internal auditor can
decide to be directly involved in follow-up but only target action items of
high priority related to significant risks. While the level of involvement
may be less overall, checking on the resolution of these significant risks
may still require multiple follow-ups on a regular basis. The review may
take the form of observation and discussions, selected audit tests, or some
balance between the two. There should be clear criteria for success, and
internal auditors must determine the level of reliance that is given to each
type of evidence. Determining what is significant enough to follow up on
and an appropriate frequency are also important. Formal and informal
communication procedures and a method of addressing when plans need to
change should also be determined. This type of audit can appear less
bureaucratic than a full-scale follow-up audit because it is clearly linked to
risk-based methodology. Auditor check-ins can help keep actions moving,
but process owners could come to rely on regular reminders rather than
being proactive. This method can produce risk-based, objective, and
verifiable results.
• Auditor conducts a follow-up audit. Scheduling a follow-up audit after
allowing sufficient time for correction is the most involved type of followup, and it provides the strongest evidence of whether or not all changes
have been implemented correctly. Scheduling the follow-up can be
difficult; process owners need sufficient time to finish changes but should
not be given so much time that critical changes can be delayed
unnecessarily. Even so, some changes may still be incomplete when the
audit begins. The level of reliance on various types of tests should be
determined during follow-up audit planning as well as a process for formal
or informal reporting.
It is important for internal auditors to determine whether changing
business conditions have created any changes in action plan
implementation. The process owner may not have consulted with the
internal auditors prior to making such changes, and the changes could fail
to address significant risks. An advantage of this method is that the
internal auditor may not only be in a position to determine that the plans
have been implemented but may be able to measure the effect the new
methods have had on the related risks or intended results. This evidence
could influence internal auditors’ recommendations for future audit areas.
While this type of follow-up can be risk-based, it may still involve
spending more time than needed on some less-critical items. Process
owners could also view the process as being bureaucratic.
Determining Appropriate Follow-Up
The CAE’s decision on the type of follow-up procedure to use and on its
specific scope may be affected by several considerations:
• Significance of the reported observation or recommendation.Significant
engagement observations are those conditions that, in the judgment of the
CAE, could adversely affect the organization in achieving its objectives.
They may include conditions dealing with irregularities, illegal acts, errors,
inefficiency, waste, ineffectiveness, conflicts of interest, and control
weaknesses.
For example, consider the internal audit activity in a pharmaceutical
company that uncovers a pattern of payments to investigators (usually
physicians responsible for independently gathering data that will be used
to obtain approval for new drug products). This is a serious violation of
federal law. It may have serious repercussions for the company, from
heavy fines to denial of product approval. It is reasonable for internal
audit to anticipate that management will promptly implement measures to
mitigate the potential risk associated with these improper actions. Internal
audit will also want to ensure the effectiveness of management’s corrective
action through periodic monitoring of accounting records.
In contrast, the internal audit department may find fairly minor instances
of inadequate documentation of travel expenses by the company’s
marketing executives. This audit result requires management’s corrective
action to mitigate the risk, but follow-up may require only a notation in
the internal auditor’s records to recheck this item during the next audit.
• Degree of effort and cost needed to correct the reported
condition. Will the cost of repair—in terms of both money and lost
productivity—compare favorably with its benefits? For example, say that
internal auditing in a manufacturing company finds that there is a risk of
injury to employees from a specific traffic pattern of equipment and
workers. After discussing the situation, internal audit and manufacturing
management agree that redesigning the physical layout of the facilities
would be extremely costly and would decrease productivity by making
access to materials more difficult. In addition, while risk exists, it is fairly
remote, as shown by an absence of accidents in the past two years.
Auditing and management might agree to explore alternative, more costeffective solutions.
• Impact that may result should the corrective action fail. The CAE must
consider whether management’s corrective action is likely to succeed in
correcting the problem and what an inadequate remediation will mean to
the organization. For example, say that local regulations require that data
sheets on all hazardous materials used in a workplace be placed in a
binder located in a specific place. Internal auditing discovers that the
required documentation is, in some cases, missing or outdated.
Management agrees to revise the data sheets to reflect the situation after
the coming peak work season. The CAE considers the fact that the
employees are thoroughly trained in the materials and that warning signs
are posted wherever the materials are used and decides that, for now, there
is probably little cost associated with any potential tardiness by
management in adding the necessary data sheets. The CAE accepts the
time frame offered by management.
Consider a contrasting example. An internal auditor uncovers evidence that
suggests an unusual level of familiarity between an employee and certain
vendors but is not able to identify specific acts of fraud. Internal audit
includes this potential risk in its report to management of the area.
Management responds by saying that, in the absence of specific evidence
of wrongdoing, it will not transfer this individual but will instead “keep an
eye on the situation.” The CAE may determine that management’s
corrective action is inadequate because of the size of the theft that could
occur.
• Time period involved. Sawyer recommends that the CAE assign different
time frames to management’s corrective actions for audit findings.
Significant findings of risk—as measured by “frequency, magnitude, and
exposure”—should elicit an immediate corrective action. Ideally, when a
finding involves a significant level of risk or loss, management should
begin to respond (and implement corrective action) during the audit itself,
so that the problem is at least partially corrected by the time the audit
report is released. For example, the CAE might expect that a security gap
in data exchange between the organization’s intranet and its external users
(e.g., customers) that could result in compromising the security of both the
organization’s and its customers’ financial data should be corrected as soon
as possible and at least within 60 days.
Less significant findings may have a longer time frame for corrective
action. For example, internal auditing may recommend that, for economic
and social reasons, the lighting in the organization’s offices should be
changed to more energy-efficient fixtures that produce less hazardous
waste upon disposal. It will be reasonable for management to implement
the recommendation over time, completing areas on a schedule over the
next four months.
Other findings may not call for a specific time frame of corrective action,
either because they are so critical that they have been implemented
immediately (e.g., the disciplining or discharge of an employee who is
harassing other employees) or so minor that they do not require specific
follow-up (e.g., a recommendation regarding off-site archiving of
documentation to save office space). Minor recommendations often
become items for potential further follow-up in the next regularly
scheduled engagement.
The CAE should ask management to identify a time frame for corrective
action and a target date for completion of the implementation. There should
be an agreement from management to report progress on a weekly or
monthly basis until the target date has been reached. At times, this
communication may require discussions between internal audit and
management as to the correctness of the activity to address the audit
findings and recommendations. This progress helps to assure that the right
activity is being done and that the risks identified will be mitigated by the
controls and changes to the process. Depending on the significance of the
findings, internal audit may want to do an evaluation of the success of the
new process implementation when it is completed. In some instances, an
additional audit may be performed.
Securing Action
In 1991 the U.S. General Accounting Office (now the U.S. Government
Accountability Office, or GAO) released a white paper that focused on the
issue of audit follow-up, “How to Get Action on Audit Recommendations.”
The GAO report recommends considering the following six questions:
• Is the recommendation still valid? Conditions may have changed that
have removed or lessened the risk or significantly changed the solution
requirements. For example, internal audit may report that few purchasing
employees in a multinational corporation are trained in local regulations
and pose a risk to compliance with local laws. The corporation decides
that instead of training the employees, it will rely on local independent
companies to accomplish purchasing. Since local purchasing is no longer
the responsibility of this department, the recommendation, as it stands, is
no longer relevant.
• Were the recommendations’ objectives met by an alternative
approach? For example, auditing may have recommended a physical log
to record access to a secure area. Management decides instead to install a
video surveillance camera system. The objective for increased security has
been achieved although the specific recommendation has not been
followed.
• Is there anything else that can be done to change management’s mind
about implementing the recommendation? Getting the desired
remediation from management may benefit from helping management
more fully understand the costs of the risk and the full benefits of
addressing the risk. It may help to provide management with a more
comprehensive analysis of the financial costs associated with not
implementing the recommendation or the ancillary benefits of the
recommendation (e.g., increased flexibility in using employee time).
• Should implementation of the recommendation be delayed? For
example, a CAE may know that a company needs a better transportation
tracking system, but the need may be more apparent after a pending
acquisition is completed. With a larger market and transportation fleet, the
need for greater coordination, risk management, and efficiency will be
more obvious.
• Is the recommendation key to resolving issues of control in this
area? This is an issue of deciding priorities. A CAE may decide that,
ultimately, one recommendation may be sacrificed in the interest of
gaining management agreement to implementing a more critical
recommendation.
• Can the recommendations be revised in any way to make them more
achievable and therefore more palatable to management? For example,
an area that feels understaffed and overworked may not welcome a
recommendation for a new and additional process. Could management
accomplish the recommendation by purchasing a service from a vendor?
The GAO’s suggestions underscore the need for thorough communication
and possibly negotiation between the CAE and management. Solutions
imposed by senior management without management’s understanding and
support may face a more challenging future than solutions mutually agreed
upon.
Conducting Follow-Up
During the follow-up activity, internal auditing will gather data to confirm
the status of its recommendations to management. Monitoring may directly
involve internal audit in the form of a follow-up engagement, but it may
also include receiving periodic progress reports from management or other
organizational units assigned responsibility for procedures of a follow-up or
corrective nature. As with the initial engagement, internal auditors in a
follow-up engagement should cultivate a cooperative, positive relationship
with the area. The engagement may involve data collection and analysis,
review of documents, observation of job performance, and interviews with
management and personnel. Both data collection and monitoring
observations may be enhanced through the use of computer tools. This can
be as simple as the use of a spreadsheet for tracking items that need followup on a quarterly or other basis. The spreadsheet can be kept current by
adding or removing items as various issues are corrected or still have
deficiencies.
Internal audit’s goal during the conduct of follow-up is to determine:
• Whether the recommendation has been acted on and accomplished or
whether a corrective action to the recommendation is in progress.
• If the change has focused on the root cause, which will prevent or limit a
future occurrence or recurrence of the observed problem.
• What benefits are accruing to the area and to the organization as a whole
from implementation of the recommendation.
• Whether the measured benefits and savings match those anticipated in the
recommendation and, if not, why.
In regard to the last point above, note that defining benefits in follow-up
reports will help establish the validity of recommendations and the value of
the internal auditing function. However, measurement of benefits or savings
may be difficult, as some benefits or savings accrue only over time and will
be difficult to assess in the short term after a recommendation has been
implemented. Other benefits may be difficult to quantify.
Documenting Progress
If progress is being made, it must be documented in some relevant fashion
—e.g., through auditing of documents or data, testing, or direct observation.
If progress is not being made, internal auditing must uncover and document
the reasons for this:
• Is management resisting implementation of the recommendation?
• Has management encountered unforeseen problems?
• Have conditions changed in such a way that the recommendation is no
longer necessary?
• Has implementation of the recommendation been delayed due to the
pressures of daily operations and/or crises?
• Is the recommendation itself now considered inadequate? In what ways?
It may be possible to resume progress on the recommendation by talking
through issues with management and personnel in the area and, if necessary,
developing alternative approaches for implementing the recommendation.
If progress is not being made, additional follow-up monitoring must be
scheduled.
Reporting Adequate Management Corrective Action
The nature of internal audit’s communication of the monitoring plan and
results depends on whether the CAE has determined that management’s
corrective action has been adequate or inadequate.
Performance Standard 2060, “Reporting to Senior Management and the
Board,” states:
[The CAE should report] on the internal audit activity’s purpose, authority,
responsibility, and performance relative to its plan and on conformance with the
Code of Ethics and the Standards. Reporting must also include significant risk and
control issues, including fraud risks, governance issues, and other matters that require
the attention of senior management and/or the board.
As part of this charge, the CAE submits periodic activity reports, including
regular reports of ongoing projects. These reports should reflect newly
initiated auditing engagements as well as the results of the monitoring
activities conducted to follow up on and confirm completed engagements. In
many organizations, this reporting process involves the CAE performing this
reporting in person at senior management meetings and periodic board
(audit committee) meetings.
Reports may highlight the need to continue monitoring (because a
recommendation has not been fully implemented, has been implemented
incorrectly, or has failed to resolve the observed problem). They may also
highlight the ability to cease monitoring and declare an issue resolved
(when a recommendation is successfully implemented, its objectives are
achieved in some other way, or the problem has disappeared). For example,
a market change may eliminate a specific business risk, or an unexpected
benefit of a new technology may be that it has addressed a control issue. It
is critical to remove monitoring engagement items from internal audit’s list
of ongoing projects when appropriate. The guidelines for properly removing
items from future ongoing monitoring requirements should be clear to all
monitoring report recipients.
The report should document the monitoring plan’s results specifically—
ideally using the criteria set out in the original recommendations—and
emphasize the benefits that have accrued to the organization because of
implementation of the recommendations. Results may be qualitative
(“Customer satisfaction with order delivery has improved”) or quantitative
(“Time from order placement to customer receipt has decreased from an
average of 8.5 days to 6 days”). The report should be brief—a summary of
objectives, monitoring activity, and results, with supporting evidence listed
in appendices as needed. However, if other issues have arisen that have
interfered with achieving the recommendations, necessitated changes in the
recommendations, or suggested new and related issues of risk or control,
these matters should be discussed fully in the report.
Next Steps
You have completed Part 2, Section IV, of The IIA’s CIA Learning
System®. Next, check your understanding by completing the online sectionspecific test(s) to help you identify any content that needs additional study.
Once you have completed the section-specific test(s), a best practice is to
reread content in areas you feel you need to understand better. Then you
should complete the Part 2 online post-test.
You may want to return to earlier section-specific tests periodically as you
progress through your studies; this practice will help you absorb the content
more effectively than taking a single test multiple times in a row.
Bibliography
The following references were used in the development of Part 2 of The
IIA’s CIA Learning System. Please note that all website references were
valid as of April 2018.
“The Audit Committee: Purpose, Process, Professionalism.” The Institute of
Internal Auditors, www.theiia.org/download.cfm?file=6676.
“Auditing External Business Relationships” (IPPF Practice Guide).
Altamonte Springs, Florida: The Institute of Internal Auditors, 2009.
Committee of Sponsoring Organizations of the Treadway Commission.
Enterprise Risk Management—Integrating with Strategy and
Performance. Jersey City, New Jersey: American Institute of Certified
Public Accountants, 2017.
Committee of Sponsoring Organizations of the Treadway Commission.
Guidance on Monitoring Internal Control Systems. Jersey City, New Jersey:
American Institute of Certified Public Accountants, 2009.
Committee of Sponsoring Organizations of the Treadway Commission.
Internal Control—Integrated Framework. Jersey City, New Jersey: American
Institute of Certified Public Accountants, 1994.
Committee of Sponsoring Organizations of the Treadway Commission.
Internal Control—Integrated Framework (2013). Jersey City, New Jersey:
American Institute of Certified Public Accountants, 2013.
Committee of Sponsoring Organizations of the Treadway Commission and
World Business Council for Sustainable Development. “Enterprise Risk
Management—Applying ERM to Environmental, Social and GovernanceRelated Risks.” COSO, www.coso.org/esg/Pages/default.aspx.
“Comparison of SOC 1, SOC 2 and SOC 3 Reports.” AICPA,
www.aicpa.org/content/dam/aicpa/interestareas/frc/assuranceadvisoryservices/downloadabledo
soc-1-3.pdf.
“Continuous Auditing: Coordinating Continuous Auditing and Monitoring to
Provide Continuous Assurance,” second ed. Altamonte Springs, Florida: The
Institute of Internal Auditors, 2009.
“Difference Between GDPR and ePrivacy Regulation.” PrivacyTrust,
www.privacytrust.com/guidance/gdpr-vs-eprivacy-regulation.html.
“Effective Writing for Auditors” seminar. Altamonte Springs, Florida: The
Institute of Internal Auditors.
“Formulating and Expressing Internal Audit Opinions” (IPPF Practice
Guide). Altamonte Springs, Florida: The Institute of Internal Auditors, 2009.
“Gramm-Leach-Bliley Act.” Federal Trade Commission, www.ftc.gov/tipsadvice/business-center/privacy-and-security/gramm-leach-bliley-act.
Hargraves, Kim, et al. Privacy: Assessing the Risk. Altamonte Springs,
Florida: Institute of Internal Auditors Research Foundation, 2003.
Hornreich, Jaike. “Understanding the New SSAE 18—What You Need to
Know.” Skoda Minotti, skodaminotti.com/blog/understanding-new-ssae-18need-know/, April 12, 2017.
Hubbard, Larry. Control Self-Assessment: A Practical Guide. Altamonte
Springs, Florida: The Institute of Internal Auditors, 2000.
Jerskey, Pamela. “Automated Workpapers Made Easy.”
Kennedy, Carrie. “SSAE 18 vs. SSAE 16: Key Differences in the New SOC
1 Standard.” Online Tech, resource.onlinetech.com/ssae-18-vs-ssae-16-keydifferences-in-the-new-soc-1-standard/, July 25, 2017.
Mautz, Robert K. Internal Control in U.S. Corporations: The State of the
Art. New York: Financial Executives Research Foundation, 1980.
“Measuring Internal Audit Effectiveness and Efficiency” (IPPF Practice
Guide). Altamonte Springs, Florida: The Institute of Internal Auditors, 2010.
Pham, Thu. “SOC 1, SOC 2 & SOC 3 Report Comparison.” Online Tech,
resource.onlinetech.com/soc-1-soc-2-soc-3-report-comparison/, August 19,
2011.
Reding, Kurt F., et al. Internal Auditing: Assurance and Consulting
Services. Altamonte Springs, Florida: The Institute of Internal Auditors
Research Foundation, 2009.
Sawyer, Lawrence B., Mortimer A. Dittenhofer, and James H. Scheiner.
Sawyer’s Internal Auditing, fifth edition. Altamonte Springs, Florida: The
Institute of Internal Auditors, 2005.
“Software Directory.” The Institute of Internal Auditors Australia,
www.iia.org.au/technical-resources/software-directory.
Thomas, Archie R. Essentials: An Internal Audit Operations
Manual. Altamonte Springs, Florida: The Institute of Internal Auditors,
Research Foundation, 2008.
“Tools and Techniques for the Beginning Auditor” seminar. Altamonte
Springs, Florida: The Institute of Internal Auditors.
U.S. General Accounting Office. “How to Get Action on Audit
Recommendations.” U.S. General Accounting Office,
www.gao.gov/special.pubs/p0921.pdf.
World Economic Forum. “The Global Risks Report 2018.” World Economic
Forum, www.weforum.org/reports/the-global-risks-report-2018.
Zhang, Charles. “The Art of Coordination.” Internal Auditor, April 1998,
Volume 55, Issue 2.
Index
The numbers after each term are links to where the term is indexed and indicate
how many times the term is referenced.
acceptance of risk 1
audit methods 1
client meetings 1
Committee of Sponsoring Organizations frameworks
Internal Control—Integrated Framework 1
communication 1
in audit team 1
See also reporting, reports 1
with engagement client 1
corrective action 1
COSO frameworks
Internal Control—Integrated Framework 1
follow-up 1, 2
objectives 1
outcomes, assessing 1, 2
scope 1
exit conferences 1
Implementation Guides
2200 1
2320 1
2410 1, 2
2440 1
2500 1
initial client meetings 1
interim reports 1
International Standards for the Professional Practice of Internal Auditing
2060, “Reporting to 1
2420, “Quality of 1
2440, “Disseminating 1
2500, “Monitoring 1
2500.A1 1
2500.C1 1
2600, “Communicating the 1, 2
issues-tracking systems 1
management response to audit reports 1
monitoring
procedures 1
progress 1, 2
progress
documenting 1
monitoring 1, 2
recommendations 1
reporting
See also communication, reports 1
reports 1, 2, 3
approval of 1
constructive tone in 1
distribution of 1
drafting 1
elements of 1
interim 1
management response to 1
organization of 1
quality 1
reviewing 1
See also communication, reporting 1
residual risk 1
results 1
risk
acceptance 1
residual 1
Build 08/24/2018 15:40 p.m.
Contents
Section IV: Communicating Engagement Results and Monitoring Progress
Section Introduction
Chapter 1: Communicating Engagement Results and the Acceptance of
Risk
Topic A: Preliminary Communication with Engagement Clients (Level
P)
Topic B: Communication Elements and Quality (Level P)
Topic C: Interim Reporting on Engagement Progress (Level P)
Topic D: Recommendations to Enhance and Protect Organizational
Value (Level P)
Topic E: Audit Engagement Communication and Reporting Process
(Level B)
Topic F: CAE Responsibility for Assessing Residual Risk (Level B)
Topic G: Communicating Risk Acceptance (Level B)
Chapter 2: Monitoring Progress
Topic A: Assessing Engagement Outcomes (Level P)
Topic B: Monitoring/Follow-Up of Disposition of Engagement Results
(Level P)
P2_Bibliography
Index