Compliance Monitoring and
Enforcement Audit Program
Background Information
Introduction
This presentation is
intended as an
introduction to the
audit process for
employees of
entities being
audited by MACD.
Please refer to the
manual for more
detail
2
Why is MACD Performing Compliance
Audits?
• Electricity is critical to our society
• The industry has established standards to ensure
reliable operation of the power system
• Under the Electricity Act (Section 32.1) , it is the
IESOs role to monitor and enforce compliance
with these standards
• IESO delegates these responsibilities to MACD
3
Reliability Standards
• Reliability Standards are created to improve the
reliable operation of the bulk electricity system
• Minimum standards for operation in key areas
• Market Participants must continuously follow
all applicable Reliability Standards in effect in
Ontario
• Depending on your company role, different
standards may apply e.g. Generator Operator
Standards compared to Transmission Operator
standards
4
Reliability Standards
• NERC – North American Electric Reliability
Corporation establishes reliability standards.
http://www.nerc.com/page.php?cid=2%7C20
• NPCC –North East Power Coordinating Council
has delegated authority from NERC.
• Ontario is part of the NPCC, IESO participates in
the development and approval of standard.
Stakeholder involvement through the Reliability
Standards Sub Committee of the IESO
• The Ontario Energy Board has the right to reject
5
NERC reliability standards
Introduction to Compliance Audits
• Audits must provide reasonable assurance that
evidence is sufficient and appropriate to support
the auditors findings and conclusions.
– Sufficient quantity of evidence, spans the time period
being audited
– Evidence must be of sufficient quality – authenticated
– Typically involves multiple types of evidence related
to the same thing e.g.
• Approved internal documentation
• Physical examination/inspection
• Analytical procedures with adequate data
6
• Observations and inquiries
MACD Audit Program
• A ‘risk based’ audit program, not a pre set audit
cycle
• MACD considers industry trends, Ontario
specific inputs when developing the program
• MACD makes an assessment of the risk to
reliability for various standards and participants
• The Audit Program begins with an
implementation plan issued in the fall
7
MACD Implementation Plan
• Issued in the fall – after NERC issues their plan
• Provides a list of reliability standards which
MACD will focus on during the upcoming year
• The implementation plan may be revised during
the year depending on industry developments
• Individual audits will include standards from
the implementation plan but may also include
additional standards.
• Individual audits are initiated with an Audit
Notification, are not listed in the Implementation
8
Plan
Introduction to Compliance Audits
• Part Two of this presentation will deal with the
audit process once an Audit Notification has
been sent to a Market Participant.
–
–
–
–
–
–
Audit Notification
Pre Audit Meetings
Data Submission
The Audit
Exit Meeting
Audit Report
9
Compliance Monitoring and Enforcement Audit
Program Background Information
End Of Presentation
10
Compliance Monitoring and
Enforcement Audit Program The Audit Process
Introduction
This presentation is
intended as an
introduction to the
audit process for
employees of
entities being
audited by MACD.
Please refer to the
manual for more
detail
12
Introduction to Compliance Audits
•
•
•
•
•
•
Audit Notification
Pre Audit Meetings
Data Submission
The Audit
Exit Meeting
Audit Report
13
Introduction to Compliance Audits
• Audits are not “out to get you”
• “Trust but Verify” is the philosophy for the
process
• The Audit team must, in their professional
judgment, be satisfied that the evidence
demonstrates compliance
• If the Audit team cannot confirm compliance to
a standard, the report will indicate potential non
compliance which MAY trigger an investigation
14
Key Roles
Your company’s Audit Coordinator
• The Single Point of Contact for the audit
• Will do all coordination with MACD
• The person you should talk to with any concerns
or questions
• The only person who should identify concerns
or issues to the MACD Audit Team
• The person responsible for all logistics with the
MACD Audit Team
15
Key Roles
Subject Matter Expert
• The person your company has identified as
being able to discuss audit information with the
audit team
• Essential to your company’s ability to
demonstrate compliance with standards
• May be interviewed by the Audit Team to
discuss reliability standards
• Should discuss any questions about the audit in
advance with your Audit Coordinator
16
Key Roles
Audit Team Lead
• An employee of MACD
• Coordinates all audit efforts for MACD with
your Audit Coordinator
• Discusses any issues or concerns with your
Audit Coordinator
• The local decision maker for MACD
17
Audit Notification
• At least 90 days before your audit, you will
receive a notification
–
–
–
–
–
–
–
Audit Team
Scope
Off Site vs. On Site
Date of Audit
Pre Audit Survey
Initial Data Requests
Pre Audit Schedule of Activities
18
Notification - Audit Team
• Audit Team Members:
– are responsible for producing an opinion on
adherence to standards
– have experience in the industry and/or
auditing
– have completed NERC Audit Training
– have confidentiality agreements in place
– are independent, have not worked in your
company in the recent past
– issues? Discuss with MACD in advance
19
Notification - Audit Scope
Scope - Standards, Audit Period, Sample Size
• Standards:
– Responsible for all applicable Reliability Standards
– Audit will focus on a sub set chosen by MACD
– Subset reflects industry trends, impact on reliability
• Period is a rolling six years, audit team can
examine compliance during entire six years
• Data requested is usually for previous 12
months
20
Pre Audit Activities
Pre Audit Survey
• A tool for the audit team to learn more about
your organization.
• Generic, some questions may not apply to your
organization.
• Questions about assets, organization structure,
physical location and neighbours
• Some logistical information and contact
information
• Helps the audit progress more smoothly
21
Pre Audit Activities
RSAWs – Reliability Standard Audit Worksheet
• Tool to help you demonstrate your compliance
• Guides you through providing evidence to
demonstrate your compliance
• Assists you in organizing and packaging the
information
• Clearly identify appropriate sections in the files
for evidence, reference the standard
• Submited to MACD through the Portal
22
RSAWs
23
Pre Audit Activities
• It is important to
annotate documents
• If using the same
document for
multiple
requirements, put a
copy of the
document in each
folder!
24
Pre Audit Activities
• The IESO Portal is a secure system already in
use for providing this information to the audit
team
• Portal training material is available on the IESO
website
• http://www.ieso.ca/imoweb/marketplaceTrainin
g/showCourses.asp?id=15
25
Pre Audit Activities
• It is important to organize
information to show where
it ties to the standard (to
show compliance)
• Include the RSAW for each
standard in the top folder,
and then evidence
documentation with
annotation for each
requirement.
26
Pre Audit Meetings
• Open dialogue is important during the audit
process. Opportunities for 3 pre audit meetings.
• Pre submission meeting – date agreed to, allows
for general questions and discussion
• Pre Audit meeting, optional, done after MACD
has reviewed the data submission
• Pre Audit Logistics – At least one week prior to
audit to discuss the logistics of the audit week
27
On Site Logistics
Audit Team will require:
• Rooms to work in
• Lunch/coffee (working lunch delivered to room)
• MACD to be invoiced
• Access to projector, internet, printers, if possible
via our laptops – All evidence should be
electronic unless otherwise agreed by the MACD
audit team
• Access to site as needed, with appropriate
escorts
28
Audit Presentations
The Audit team will start the audit with a brief opening
presentation.
• Introducing the team
• Expectations for the process
It can be helpful if the MP makes a brief opening
presentation to
• Introduce the MP audit team
• Provide some background on the company and the
facility
• Explain any safety/security requirements
29
Interviews
• The Audit Team may want to talk to SMEs
(Subject Matter Experts) and other staff
• Your Audit Coordinator will arrange interviews
• The Audit Team may want to ask questions
about procedures, their implementation, data
30
Exit Meeting
At the end of the audit period, the team will give
an exit presentation. This presentation will give
an overall summary:
• Recap of the audit review
• The Audit Team observations, preliminary
findings and recommendations
• Schedule for remainder of process, including
when the final report will be issued
31
Final Report
• The audit report would identify if the team was
unable to confirm compliance with a reliability
standard
• The potential non compliance is turned over to
the investigation unit in MACD for follow up.
• There is no “penalty” from the audit itself
32
Potential Non Compliance Findings
• If the Audit Team identifies an immediate risk to
reliability, the Team Lead will immediately
communicate this to your Audit Coordinator
and then to MACD for investigation
• For other potential non compliance observations
that do not have an immediate impact on system
reliability, these will be reported at the
conclusion of the audit
33
Escalation Process
Any questions or concerns during the audit:
• Discuss with your company Audit Coordinator
• The Audit Coordinator from your company should
discuss with the Audit Team Lead as soon as possible.
• If not satisfied after discussing with the Team Lead, your
Audit Coordinator can escalate the issue to the MACD
supervisor/director
• If still not satisfied, your company can take issues to the
Dispute Resolution Panel (very formal process) – OEB
process
34
Summary
• A minimum of 90 days notice
• Initial Information Request - before the audit
begins, submit information to demonstrate
compliance
• Subsequent Information Requests may come
during the audit
• SMEs and other staff may be interviewed
• The Audit Team will provide an exit interview
with preliminary results
• The Audit Report will follow with details
35
Compliance Monitoring and Enforcement Audit
Program The Audit Process
End Of Presentation
36