Safety Audits INACA TRAINING FACILITATOR Objectives • To introduce the participants with concepts, contents, and processes involved in safety auditing • To practice resolving audit finding through a case study 22102021_ver0 INACA - CASO Training 2021 2 Flight plan Definition Purpose of audit Basic audit activity Audit finding Audit forms Resolving audit finding - practical exercise Root Cause Analysis 22102021_ver0 INACA - CASO Training 2021 3 Definitions • Safety auditing is a means of identifying potential problems before they have an impact on safety • Safety regulatory audit means a systematic and independent examination conducted by, or on behalf of, the civil aviation authority to determine whether complete safety-related arrangements or elements thereof, related to processes and their results, products or services, comply with required safety-related arrangements and whether they are implemented effectively and are suitable to achieve expected results 22102021_ver0 INACA - CASO Training 2021 4 The objectives of safety audit • To assess the degree of compliance with the applicable safety regulatory requirements and with the procedural provisions of a Safety Management System • To provide assurance of the safety management functions, including staffing, compliance with applicable regulations, levels of competency and training • To fulfill the safety performance monitoring requirements 22102021_ver0 INACA - CASO Training 2021 5 Safety audits are used to ensure that: • Organization's SMS has a sound structure and adequate staffing levels; • Approved procedures and instructions are complied with; • The required level of personnel competency and training to operate equipment and facilities, and to maintain their levels of performance, is achieved; • Equipment performance is adequate for the safety levels of the service provided; • Effective arrangements exist for promoting safety, monitoring safety performance and processing safety issues; • Adequate arrangements exist to handle foreseeable emergencies. 22102021_ver0 INACA - CASO Training 2021 6 Regulatory auditing • The auditing process should put the relationship between Regulator and Company into balance • It enforces the requirement that the company is responsible for safety • If the company manages safety properly the company will not have any problems in addressing a Regulator audit • The company will not hide information about incidents and its failures because the company’s safety management team will accept and address the problem in open • The company will also want to explain to Regulator what went wrong and how the problem was rectified without fear 22102021_ver0 INACA - CASO Training 2021 7 Who conduct the safety audit? • Carried out by a single individual or a team of people who are competent (adequately qualified, experienced and trained) • Have a satisfactory degree of independence from the audited organization or unit 22102021_ver0 INACA - CASO Training 2021 8 Regulatory Safety Audits (external auditing) • Surveillance and compliance - the authority needs to ensure that international, national and local standards are complied with prior to issuing any license or approval and continue to be complied with afterwards; • Areas and degree of risk - the audit should assess how risks are identified and how any necessary changes are made to ensure that all safety standards are met; • Competence - the audited organization should have adequately trained staff for all safety related positions • Safety management - ensure that the organization's SMS is based on sound principles and procedures, and that the organization is meeting its safety performance targets. 22102021_ver0 INACA - CASO Training 2021 9 Internal Safety Audits (self auditing) • To assess the level of compliance with the applicable regulatory framework and the organizational SMS processes and procedures, • To verify the effectiveness of such processes and procedures • To identify corrective measures if needed • Should consider the safety significance of the processes to be audited and the results of previous audits. • Must have an annual audit program 22102021_ver0 INACA - CASO Training 2021 10 Third Party Audits (external auditing) • External audits of the SMS may be conducted by relevant authorities responsible for acceptance of the service provider’s SMS • May be conducted by industry associations or other third parties selected by the service provider • Enhance the internal audit system as well as provide independent oversight 22102021_ver0 INACA - CASO Training 2021 11 Basic activity of an audit Pre-audit Physical audit Post audit Audit follow-up 22102021_ver0 INACA - CASO Training 2021 12 Audit Activity Breakdown Scope & Depth Entry Meeting Team Selection Physical Inspection Budget Functional Areas Team Meeting Preparations Findings Documents Daily Meetings Pre-Audit Audit Start 22102021_ver0 De-briefing Corrective Action Wrap-up Activities Follow-Up Audit Report Post-Audit Follow-Up Finish INACA - CASO Training 2021 13 Pre-Audit activities Audit planning Team assembly Pre-audit team meeting Final preparations including document review 22102021_ver0 INACA - CASO Training 2021 14 Audit Planning • Rule of Thumb • Small 1 - 5 days • Medium 5 - 15 days • Large 15+ days 22102021_ver0 INACA - CASO Training 2021 15 Scope • Company or Authority defines scope as the number of areas within company’s operation that will be audited 22102021_ver0 INACA - CASO Training 2021 16 Communication • Daily internal meetings with Team • External meeting with the Department or Company 22102021_ver0 INACA - CASO Training 2021 17 Depth The period over which a company’s operation will be audited, normally from the last audit until present day It could also be thought of as the extent or the size of audit 22102021_ver0 INACA - CASO Training 2021 18 Sampling basis • The audit procedures manual sets out a table for sampling to 95% confidence level • Confidence level 95% is identified in Authority Guideline • In practical terms it may not be realistic to review, for example, all aircraft in the fleet. Significant findings against one type of aircraft may require additional review against other type. It must be realistic when conducting the review and look at each audit situation independently 22102021_ver0 INACA - CASO Training 2021 19 Preparation & Documentation review Outstanding previous audits Company Manuals Surveillance reports since the previous audit 22102021_ver0 INACA - CASO Training 2021 20 Entry Meeting • Greetings • Explain the purpose of the meeting • Introduction of Team and Company Representative • Discuss the objective and conduct of audit • Discuss scope and depth • Communications between Team Audit & the Company, including Daily Meetings • Introduce forms that the company may see over the course of audit 22102021_ver0 INACA - CASO Training 2021 21 Steps in conducting Physical Review Review Standard Obtain Checklist Review and ‘test’ Does it meet standard Does not Prepare the paperwork Does Proceed with next task Write the audit finding(s) 22102021_ver0 INACA - CASO Training 2021 22 Checklist Use Ensure Write down Check 22102021_ver0 Use as directed Ensure that no area is overlooked Write down information Check against regulation requirements INACA - CASO Training 2021 23 Interviewing people • Why? • To determine if the system documented in the approved manual is the system in use • To assess the knowledge of the operator’s personnel. • To review their duties and responsibilities • To determine if any other areas need to be audited • To tell the people what is going on 22102021_ver0 INACA - CASO Training 2021 24 Physical inspection 22102021_ver0 INACA - CASO Training 2021 25 Area of inspection • Base/station inspections • Aircraft inspection • Pre-flight & In-flight inspections • Training program inspection • .. 22102021_ver0 INACA - CASO Training 2021 26 After Interview & Inspection Makes extra notes as necessary Inform/tell Team Leader if necessary 22102021_ver0 INACA - CASO Training 2021 27 Evidence • What is it? • Anything that supports or shows non-compliance or non-compliance or nonadherence 22102021_ver0 INACA - CASO Training 2021 28 Immediate threat to safety • Inform Team Leader immediately.. 22102021_ver0 INACA - CASO Training 2021 29 Key principles in audit report Consistency of observations and recommendations; Conclusions substantiated with references; Observations and recommendations stated clearly and concisely; Avoidance of generalities and vague observations; Objectivity of observations; Avoidance of criticism of individuals or positions. 22102021_ver0 INACA - CASO Training 2021 30 Audit Finding • Definition • ‘audit finding’ means the determination of noncompliance/non-conformance/non-adherence of a product, process, procedure, or practice or a characteristic thereof to a regulation or standard • This will be documented in the Audit Finding Form 22102021_ver0 INACA - CASO Training 2021 31 Nature of audit finding Non-compliance • A deficiency in characteristic, documentation, or procedure with respect to provisions of the Legislation or Regulation or a major safety risk Non-conformance • A deficiency in a characteristics, documentation, or procedures. which renders the quality or the safety of a product or service unacceptable or indeterminate, or not according to specified requirements Non-adherence • A deficiency in characteristic, documentation, or procedure with respect to a recommended practice, procedure, guideline or good aviation safety 22102021_ver0 INACA - CASO Training 2021 32 Audit de-briefing • ‘Exit-Meeting’ with the auditee’s team should provide an overview of the audit 22102021_ver0 INACA - CASO Training 2021 33 Corrective Action Plan The Company will then write down their corrective action plan 22102021_ver0 INACA - CASO Training 2021 The Regulator auditor must ensure that the corrective action plans are made in such a way that they prevent re-occurrence of the same in the future 34 Audit Follow-up • The Regulator inspector must continue to monitor the company shorts and long-term responses to the audit findings • The Regulator must regulate the company by maintaining constant surveillance 22102021_ver0 INACA - CASO Training 2021 35 Audit forms 22102021_ver0 INACA - CASO Training 2021 36 Audit forms 22102021_ver0 INACA - CASO Training 2021 37 Exercise 22102021_ver0 INACA - CASO Training 2021 38
