Audit practices blog
Over the last few years I have been involved with audits on behalf of clients, as they are audited by
various organisations. During these audits I have observed the audit process from the auditee
perspective, and I must admit that I am less than impressed by the manner in which some auditors
behave.
On one occasion I witnessed an auditor make a finding against an organisation as they were having
safety meetings once every three months. The auditor stated that in his opinion they should be
monthly. I asked the auditor what his audit criteria was and he stated it was just his opinion. I also
asked if the auditor had read the organisations Safety manual, to which the auditor replied that he
had not. The organisations Safety Manual stated that they would have safety meetings once every
three months as a minimum. It is also noted that the CASA SMS guidance material also states that
once every three months is the minimum. The auditor should have found the organisation was in
conformance with its manual, not issue a non-conformance for what the auditor thought should
happen. In my opinion this auditor should not be auditing.
On another occasion I witnessed an auditor ask the question of a Safety Manager, how many SMS
reports they received every month. When the Safety Manager advised the auditor that they had
about 10 SMS reports per month the Auditor issued a non-conformance. Now this one really upsets
me. There is no set standard, nor should there be, on how many SMS reports you should have per
month. SMS reporting is a bit too complex to try and stick a number against. The amount of SMS
reports you get may vary with but not limited to the following key criteria;
•
•
•
•
•
•
Number of aircraft
Age of aircraft
Number of employees
Reporting culture
Type of operation
Number of hours flown per day/month/year
An auditor should never make a finding against the number of SMS reports received unless the
organisation has set a number in its manual/s (which I would not advise).
An aviation auditor’s role is fairly simple and is documented in the international standards AS/NZS
ISO 9001:2016 Quality management systems – Requirements, and, AS/NZS ISO 19011:2014
Guidelines for auditing management systems. In these standards an audit is defined as, ‘systematic,
independent and documented process for obtaining audit evidence and evaluating it objectively to
determine the extent to which the audit criteria are fulfilled’. (AS/NZS ISO 19011:2014., 3.1) This is
the auditor’s role.
Audit criteria is defined as, ‘set of policies, procedures or requirements used as a reference against
which audit evidence is compared’. AS/NZS ISO 19011:2014, 3.2
Audit evidence is defined as, ‘records, statements of fact or other information which are relevant to
the audit criteria and verifiable’. AS/NZS ISO 19011:2014, 3.3
Let’s take a closer look at what this means. Depending on the type of audit being conducted the
audit criteria, which must be determined prior to the audit starting, could be:
•
•
Legislation (e.g.: Civil Aviation Safety Regulations, etc.)
Operations Manual/s (e.g.: Volume 1, or Part A, etc.)
•
Contractual requirements (e.g.: contract specifies terms, such as adherence to IATA IOSA)
If the audit is based on legal or regulatory requirements, the terms ‘Compliant’, or, ‘Non-compliant’
are normally used in the audit finding. If the audit is not a legal or regulatory audit, and is based on
your operations manuals the terms ‘Conformance’, or, ’Non-Conformance’ are normally used in the
audit finding.
Audit evidence could include;
•
•
•
•
•
•
•
Interviews with employees
Observations of activities
Documents such as policies, objectives, plans, procedures, standards, instructions, licenses,
permits and the results of measurements
Data summaries, analyses and performance indicators
Report from other sources
Databases and websites
Simulation and modelling
Let’s put this into an example. Let’s assume your organisation is a Part 145 maintenance
organisation being audited by a third party auditor on behalf of one of your clients. This is called a
third party audit. The auditor should be asking for access to your manuals so that they may create a
checklist (audit criteria) based on your own manuals. This could include your;
•
•
•
•
•
•
Manual of Exposition (MOE)
Interface procedures manual (if you use one)
DAMP Manual
Emergency response manual
WHS Manual
Other manuals as applicable to your business
An auditor may also use other standards to establish the audit criteria, but, only where you have
agreed to this in a contract or services agreement. This could include;
•
•
•
BARS
Oil and Gas industry standards
IATA or ICAO standards as applicable
When conducting the audit, the auditor should be using the checklist created and then measure you
against that checklist by collecting evidence such as;
•
•
•
•
•
•
•
Interviewing employees
Observation of maintenance practices
Review of maintenance records, maintenance logs, maintenance release (or CRS)
Review of SMS database
Stores records
Quarantine records
Parts in storage
The list for a Part 145 organisation can be quite large.
My point is, an auditor must measure you against the audit criteria, and that criteria must be based
on something that you agree to, like your manuals, and not to an auditor’s opinion of how they think
it should be.
If it was me, I would be pushing back on these auditors, and make sure that you make your objection
known. I would not sign any non-conformance paperwork that did not measure you objectively
against your manuals, unless you have agreed to it.