www.pwc.com
Quality Assurance and
Improvement Program
October 2015
Learning Objectives
Understanding Quality Assurance Review (QAR) Practices
Review of current standards and expectations for quality assurance and improvement
program
Leading practices and approaches for quality assurance and improvement
PwC
2
Understanding Quality Assurance Review Practices
The Standards
The International Standards for the Professional Practice of Internal
Auditing (ISPPA) represent principle focused standards intended to provide a
framework for performing and promoting internal auditing.
Standard 1312 – External Assessments must be conducted once every five years by a
qualified, independent assessor or assessment team from outside the organization.
Most internal audit
departments view IIA
standards as
mandatory.
PwC
3
Understanding Quality Assurance Review Practices
The Standards
IIA Standards
Internal Audit departments are assessed against 11 Standards developed by the IIA. Four
standards (1000-1300) address the attributes of Internal Audit (i.e., who or
what internal audit is); seven standards (2000-2600) address the performance
of Internal Audit (i.e., how internal audit conducts its work).
PwC
Standard Number
Summary of IIA Standards
1000
Purpose, authority, and responsibility
1100
Independence and objectivity
1200
Proficiency and due professional care
1300
Quality assurance & improvement program
2000
Managing the internal audit activity
2100
Nature of work
2200
Engagement planning
2300
Performing the engagement
2400
Communicating results
2500
Monitoring progress
2600
Communicating the acceptance of risk
4
Understanding Quality Assurance Review Practices
Types of External Strategic Assessments (ESA) services
Companies typically to perform an ESA for a variety of reasons, ranging from developing
a strategic plan to benchmarking to complying with the IIA standards. We can break
down ESAs into two types:
•
Type 1: Full ESA – This assessment provides the greatest value to companies as it
assesses 1) stakeholder expectations and opinions on Internal Audit’s current
performance and compares those opinions against Internal Audit’s current operating
practices; 2) Internal Audit’s operating practices against peer results; and 3) Internal
Audit’s operating practices against the IIA standards.
•
Type 2: IIA Standards Assessment – This is a subset of the full ESA, with more
limited insight as it evaluates only whether Internal Audit operating practices
conform with the IIA standards and how the departments operating practices
compare against peers.
PwC
5
External Strategic Assessments
Types of ESA services (continued)
The table below provides a summary of the objectives, deliverables and value for each
type of service:
Type 1: ESA
Objective Assess Internal Audit for the following:
• Stakeholder expectations and perception of IA’s performance against the eight attributes
of excellence
Type 2: IIA Standards Assessment

• Maturity of IA operating practices against the eight attributes of excellence

• IA operating practices against peer company operating practices


• Conformance to IIA Standards


Insights The following information is summarized to gain insight into the
Obtained IA department:
• Results of stakeholder assessment of Internal Audit (i.e., the stakeholder’s expectations vs
their perception of performance)
• Comparison of IA’s operating practice results against 1) stakeholder expectations; and 2)
stakeholder perception of IA’s performance


• Results of operating practices for each of the 8 attributes and overall

• Conformance to IIA Standards and actions warranted to achieve conformance,
as needed

Value
Strategic assessments allow departments to assess the value they deliver:
Delivered • Insight into where Internal Audit is not meeting the expectations of their stakeholders
PwC

• Benchmarking of operating practice results against peers


• Insight into whether that misalignment is a result of under-performing teams or a need to
enhance existing operating practices

• Understanding of IA’s operating capabilities compared against peers


• Roadmap of actions warranted to achieve conformance with the IIA Standards


• Achievement of requirements for IIA Standard 1312


6
External Strategic Assessments
Overview
The primary internal audit performance improvement service offered by PwC is an
External Strategic Assessment (ESA), performed using a proprietary approach and
technology known as Profiler™. Companies typically may require such a service if they
desire a perspective on how their internal audit group is performing relative to leading
practices and/or professional standards, or at the onset of developing a Strategic Plan.
Areas to be reviewed may encompass the entire spectrum of internal audit strategy and
operations or be very specific to a certain area. A full external strategic assessment
consists of:
•
•
•
•
PwC
Understanding internal audit stakeholders’ perspectives of
internal audit’s performance and value. Stakeholders typically
include: Audit Committee &/or Board members, Executives and
Senior leadership, other risk and compliance leaders, internal
audit staff and external auditors;
Evaluating internal audit working practices, including evaluation
of select audits, to understand the maturity of the department’s
current operating capabilities;
An assessment of conformance against each of the 11 Standards
within the Institute of Internal Auditors' ("IIA") International
Standards for the Professional Practice of Internal Auditing
("IIA Standards" or "the Standards"); and
Benchmarking of internal audit working practices against peer
companies from Profiler™.
Stakeholder
Value
Stakeholder
Expectations
& Alignment
Operational
Capability
Performance
Compliance with IIA standards
7
External Strategic Assessments (continued)
The ESA framework
Our ESA framework is built off of the Internal Audit Maturity scale across the internal
audit Eight Attributes of Excellence. This means that we assess Internal Audit’s operating
practices as well as stakeholder expectations and opinion of Internal Audit’s performance
against each of the Eight Attributes of Excellence.
The Maturity scale and Eight Attributes of Excellence are detailed below.
PwC’s Maturity Model
Providing value-added services and
proactive strategic advice to the business
well beyond the effectiveness and
efficient execution of the audit plan
Bringing analysis and perspective
on root causes of issues identified
in audit findings, to help business
units take corrective action
Trusted
Advisor
Business Value
Insight
Generator
Problem
Solver
Assurance
Provider
Problem
Finder
Minimum
Contributor
Immature
Taking a more proactive role in
suggesting meaningful
improvements and providing
assurance around risk
Delivering objective assurance
on the effectiveness of an
organization’s internal control
Core
Maturity of Internal Audit Practice
PwC
8
External Strategic Assessments
The ESA framework (continued)
PwC’s Eight Attributes of Excellence
Focuses on the development of quality standards,
performance of formal reviews against quality
standards and promotion of a culture that supports
and rewards innovation and improvement
Quality and
innovation
Focuses on providing professional services
to their stakeholders throughout the
organization in a flexible, responsive, and
professional manner
Focuses on Internal Audit’s use of
technology to assist in identifying risks
and business issues and to generate
efficiencies within the business and
audit process
Business
alignment
Focuses on the design of a dynamic
audit plan which addresses both
strategic and risk-based approach
Service
Culture
Risk focus
Internal
audit
Technology
Talent
Model
Cost
effectiveness
Focuses on the efficient delivery of internal audit
services through use of staffing models, productivity
analysis, audit process and audit infrastructure
PwC
Focuses on Internal Audit’s strategic planning,
communication of expections and the
measurement of progress towards the stated
mission and vision of
the department
Stakeholder
management
Focuses on the approximate mix of
core internal audit and subject
matter specialists to meet required
expectations. This model includes
the incorporation of performance
feedback for staff and department to
facilitate growth and development
Focuses on Internal Audit’s management of both
internal and stakeholder relationships including
stakeholder expectations, communication
strategies, delivery of value and incorporation of
feed back
9
Expectations for a Quality Assurance Program Review
Engagement Overview
The ESA and IIA Standards Assessment are typically performed in three phases of work
depicted in the picture below.
Project planning
Data collection
Analysis & reporting
PwC
10
Expectations for a Quality Assurance Program Review
Engagement Overview
Review internal audit operating practices, documentation and tools
The internal audit operating practices review will assess various components of the
Internal Audit function, spanning across the Eight Attributes of Excellence, to determine
what foundational components are in place to assist Internal Audit in effective
operations.
The assessment includes but is not limited to a review of Internal Audit’s charter, a
selection of work papers and audit reports, communications with
stakeholders, etc.
This portion of a Strategic Assessment is similar to other audit procedures in that there is
a client request list, workprogram and meetings with appropriate individuals to gain
evidence on each topic and determine conformance with the IIA standards.
PwC
11
Expectations for a Quality Assurance Program Review
Engagement Overview
Review internal audit operating practices, documentation and
tools (continued)
When assessing conformance with the IIA Standards, it is important to note that the
Standards also address 'implementation standards' which provide further
clarification of the 11 IIA Standards at a more granular level.
Companies should not only be assessed by the 11 IIA Standards but also the
implementation standards included in the International Standards for the Professional
Practice of Internal Auditing.
This will result in an assessment of an Internal Audit department's operating capabilities
as well as conformance with the IIA Standards.
PwC
12
Expectations for a Quality Assurance Program Review
Project planning
Understand the environment
Various factors need to be considered when gaining an understanding of the Internal
Audit department and the overall environment of the company.
•
Key stakeholders - consider Audit Committee members and executive leadership's
possible perceptions and past experiences with internal audit as well as expectations of
internal audit that have already been articulated.
•
Enterprise strategies and risks - review analyst reports and the CEO's letter in the
latest Annual Report to understand the company's current position, three to five year
strategy, and potential changes to major risks identified by Internal Audit or other
Risk Management functions.
•
Industry and regulatory issues - consider industry and regulatory changes that
may impact the company's risk environment.
•
Internal audit cost and size benchmarks - look for significant under or
overspend based on relevant data from the IIA's GAIN benchmarking reports.
•
Internal audit trends - consider recent and planned developments and trends
within the profession.
PwC
13
Expectations for a Quality Assurance Program Review
Data Collection
Understand the environment (continued)
Strong IA departments have the following:
•
Internal Audit Charter: Internal Audit’s charter to better understand the mission of
Internal Audit and further assess components of the charter that are required within
the IIA Standards.
•
Risk Assessment: Teams should obtain evidence of Internal Audit’s risk assessment
process as well as the steps taken to execute risk assessment(s) during the period
under review.
•
Final deliverables provided for a sample selection of audits: Upon selecting a sample
of Internal Audit projects during the testing period, teams should request final
deliverables and issues reported to auditees to better understand the reporting and
wrap up stages of Internal Audit engagements.
•
Audit Methodology: Teams should obtain Internal Audit’s methodology and other
policies and procedures and should take steps to better understand how these are
maintained and communicated to relevant Internal Audit practitioners.
PwC
14
Expectations for a Quality Assurance Program Review
Data Collection (continued)
Conduct stakeholder interviews and complete electronic survey
Interviews are the recommended technique for capturing and understanding the needs
and expectations of key internal audit stakeholders.
A typical engagement will likely require between 10 and 25 interviews of board
members and executives, depending on the size and scope of internal audit
activities and the stakeholder group.
PwC
15
Expectations for a Quality Assurance Program Review
Data Collection (continued)
Conduct stakeholder interviews and complete electronic survey (continued)
Companies also have the option to send an electronic survey directly to stakeholders.
Stakeholders can typically be grouped into two or three categories:
1. Top executives (C-suite, Audit Committee, CAE, etc.): These stakeholders can be
interviewed only or they can answer an electronic stakeholder survey and then
participate in an interview to discuss specific answers and comments from the survey.
2. Other stakeholders (Internal audit staff, compliance, other key mid-level finance or
operations management): These stakeholders can generally follow a similar method
for gaining knowledge as top executives, however more reliance on the electronic
stakeholder survey to obtain input could allow organizations to reach a broader group
of stakeholders.
PwC
16
Expectations for a Quality Assurance Program Review
Data Collection (continued)
Conduct stakeholder interviews and complete electronic survey (continued)
Stakeholder expectations
By assessing Internal Audit operating practices against stakeholder expectations,
engagement teams are able to identify where stakeholder expectations are not being met.
Additionally, teams are able to distinguish whether misalignment is due to underperformance or under-developed operating practices.
Typically, the greatest degree of misalignment is caused by under-developed operating
practices. By enhancing operating practices, Internal Audit should also see an increase in
the level of performance identified by stakeholders.
PwC
17
Expectations for a Quality Assurance Program Review
Data Collection (continued)
Peer benchmarking
A Peer Benchmark can be a valuable tool for assessing a Company’s quality assurance and
improvement program by evaluating benchmark scores specific to operating capabilities
of peers.
While this information does not help to achieve alignment of stakeholder expectations
with Internal Audit’s performance and operating capabilities, some company’s find
comparative data against peer companies to be insightful.
Additionally, recommendations based on quantitative benchmarking data from IIA GAIN
reports may be provided.
PwC
18
Expectations for a Quality Assurance Program Review
Analysis & Reporting
Develop final deliverable
Depending on the initial scoping and assessment level chosen, the content of the report
may differ.
The final report may include the following:
•
Executive summary
•
Stakeholder expectations/voice of the stakeholder
•
Areas or processes that do not align with stakeholder expectations and areas or
processes where incorporation of enhanced practices addressed in the Eight
Attributes of Excellence will result in improve performance on the Internal Audit
Maturity Scale
•
Results of the IIA Standards assessment
•
Profiler™ best practice analysis with recommended actionable solutions
PwC
19
Expectations for a Quality Assurance Program Review
Analysis & Reporting
Each standard area should be reviewed to determine where current performance does or
does not meet the Standards. Conformance with both the spirit and letter of the Standard
should be considered. The assessment should conclude for each standard area with one of
the following ratings:
• Generally conforms – the internal audit activity has policies, processes and practices
that are in accordance with the Standards. Opportunities for enhancements may exist.
• Partially conforms – deviations from the Standards exist, but did not preclude the
internal audit activity from performing its responsibilities in an acceptable manner.
• Does not conform – deficiencies in practice are so significant as to seriously impair or
preclude the internal audit activity from performing adequately in all or in significant
areas of its responsibilities.
PwC
20
Leading Practices and Approaches of High Performing
Quality Assurance & Improvement Programs
Common Pitfalls
In delivering numerous engagements, some common themes have emerged:
• Lack of documented and supported strategic direction and supporting initiatives
• Department charter/activities are misaligned with stakeholder expectations
• Inadequate sponsorship of the internal audit department
• Department structural issues, e.g. reporting lines
• Department structure not aligned with the business both in terms of skill set and
geographic coverage
• Inadequate risk assessment process and alignment with company Risk Management
activities
• Poor linkage between risk assessment and audit plan
• Too little input to the risk assessment from departments outside of internal audit
• Issues identified not aligned with the high risk areas of the company
• Lack of use of technology for workpapers, data analysis, and knowledge management
• Ineffective communication/reporting
PwC
21
© 2015 PwC. All rights reserved. PwC refers to the US member firm or one of its subsidiaries
or affiliates, and may sometimes refer to the PwC network. Each member firm is a separate
legal entity. Please see www.pwc.com/structure for further details.