Welcome to your CQI and IRCA Certified ISO 9001:2015 Quality Management Systems Auditor/Lead Auditor course IRCLASS Systems and Solutions Pvt. Ltd. has been independently assessed and approved by the CQI and IRCA. This means they have the processes and systems in place to deliver certified courses to the highest standard. About the CQI and IRCA The CQI is the only chartered professional body dedicated entirely to quality. IRCA is its specialist division dedicated to management system auditors. Take the next step in your career and become a member. Join a unique global network of nearly 20,000 quality professionals and gain unrivalled professional recognition as an individual and in your career. Find out more about the CQI and IRCA at www.quality.org We hope you enjoy your course IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 1 of 48 Quality Management Systems Auditor/Lead Auditor Training Course Delegate Manual (CQI-IRCA APPROVAL # 1898) This material is for the personal use of a delegate attending a course presented by IRCLASS. No part of the materials may be reproduced, stored electronically, or transmitted in any form or by any means without the prior written consent of IRCLASS. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 2 of 48 Table of Contents Sl No. 1 2 3 Course Description Page No Welcome Note Course Introduction Additional information on Presentation Slides IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 4 5 13 © IRCLASS Systems and Solutions Private Limited 2019 Page 3 of 48 Pre-requisite to attend this course: It is recommended that the students attending this course shall have adequate knowledge of ISO 9001 Standards and some prior knowledge of QMS auditing. It is also conveyed that no prior knowledge as above may lead to unsuccessful completion of this course and the gaps in this knowledge may not be covered during this course. We wish you every success and look forward to see you on future courses. Schedule Please return to class on time after breaks or lunch. The agenda included in this delegate workbook outlines the course schedule. Personal Property Please do not leave valuables unattended. Keep them with you or make other arrangements for their safe custody. Facilities The tutor will inform you of the nearest restrooms and the location of public telephones for use during class breaks. Recording Please do not use recording devices since they tend to restrict free discussions. Safety Please familiarize yourself with any safety notices and the actions to be taken in the case of fire including the position of fire exits in the lecture and other rooms that you may occupy whilst at the course venue. Smoking Please do not smoke in the lecture or other rooms used for course work. Mobile phones, pagers and laptops Please do not have pagers, mobile phones or laptops switched on during class sessions. Special Needs Please inform the tutor of any special needs (dietary, physical, etc.) that you may have. Activity-o: Delegate introduction, IRClass introduction. Trainer/s will introduce them. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 4 of 48 Learning Objectives At the end of this course, the participants will be able to: Identify the elements of ISO 9001:2015 as criteria of Quality Management System. 1. Demonstrate the processes involved in the planning, preparation, performance, presentation of findings and post audit activities for conducting Management System Audits 2. Have the competency to plan, undertake and report results of audits of Quality Management Systems and to follow up any consequent corrective actions 3. Outline the skills in auditing Quality Management System components against criteria by use of a generic model for the audit process. 4. Identify personal attributes, ethics and values that are desirable in competent Quality auditors. 5. Identify the roles, legal and regulatory responsibilities, duty of care and due diligence required of persons conducting Quality audits along with the technical and regulatory environment in which Quality auditors operate. 6. Have the competency to pass examination required to satisfy the training criteria of the NABET prior to registration as Quality Auditors. Competency Assessment covers: 1. Document review of QMS 2. Process Approach 3. Preparation of Audit Plan 4. Audit Role Play 5. Audit Checklist 6. Mock Audit inclusive of Opening & Closing meeting 7. Preparation and Presentation of Team Report to Auditee, based on Case Study Material 8. Auditing Trail 9. Effectiveness of Corrective Actions Course Duration: This Lead Auditor Course for Quality Management System is a 40 hours course, which is equally divided into 5 days. Attendance during the full course is compulsory. Continuous Assessment: A daily grade shall be assigned for each student, reflecting the assessment of both instructors. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 5 of 48 Course instructors shall identify students who appear to be having difficulty in achieving the learning objectives or who are not performing adequately in course activities. Such students shall be informed privately and in a timely manner of the instructor’s observations and be given opportunity to improve. A student who fails the continual evaluation must satisfactorily complete another full training course before being eligible to receive a certificate of successful completion. Each delegate will be required to obtain minimum 70% marks in each of continuous assessment. Delegates who clear the continuous assessment but fail to clear the final examination are eligible to appear for the exam in the subsequent LA course. In case, the delegate fails to clear continuous assessment, he/she will have to attend a fresh QMS LA course. Exam duration Exam content areas: • Section 1: Concepts and principles of Management Standards and Systems • Section 2: Audit concepts and auditor responsibilities • Section 3: Planning the audit • Section 4: Conducting the audit • Section 5: Reporting and closing out the audit 1 hour 45 minutes (40 questions in total) Recommended time for each section: 10 minutes (6 questions) 10 minutes (6 questions) 10 minutes (6 questions) 45 minutes (14 questions) 30 minutes (8 questions) The exam questions will be a mix of: • Multiple choice (choose one correct answer) • Multiple response (choose several correct answers) • Fill in the blanks (drag and drop the correct words into the sentence) • Matching (drag and drop responses to correctly match items) • Sequencing (drag and drop responses to form the correct order) The last two sections of the exam will include scenario questions related to conducting, reporting on, and closing out audits. Learner will be asked to evaluate the scenario presented to learner and select the correct course(s) of action. All questions will be computer-marked. Once learner have answered a question and clicked Next, learner will not be able to navigate backwards to review or change your answer. This is a security measure to reduce the risk of cheating. The exams are open book, so learner can refer to his notes from his training course. However, learner are not permitted to use the internet to search for answers to questions. Learner will need access to a copy of the ISO management systems requirements standard associated with his course to answer some of the questions. The standards can be paper or electronic format. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 6 of 48 Seven Quality Management Principles (QMPs): Refer ISO 9000 2015 - Quality Management Systems —Fundamentals and Vocabulary Clause # 2.3 Why audit? What is Audit – ISO19011:2018 Guidelines for Auditing Management Systems? An audit is a method of gathering factual information based on an independent and unbiased assessment of objective evidence rather than subjective opinion. It is an information gathering exercise that allows the need for improvement or corrective action to be evaluated. It is not an inspection or a “witch hunt” nor is its purpose to lay blame. Due to this it is a very powerful tool, which can correct any preconceived ideas about the status of the organizations Quality Management System. A Quality Audit will assess whether businesses are complying with their documented Quality Program and requirements of ISO 9001:2015 Systems are an overall framework for performance of tasks, they provide for co-ordination of individuals and groups of individuals to enable the achievement of organizational objectives. Auditing enables the organization to establish if systems are being followed and if they are fully effective. This information enables the organization to proceed with confidence. Systems can also be considered as collections of inter-related activities or processes. Systems result in the achievement of objectives: processes (or activities) result in individual output that ultimately provide for the achievement of objectives. Information concerning systems operation and effectiveness can be achieved by undertaking audits of Systems, Processes and Process outputs. Audit will also establish if other organizations, such as suppliers or sub-contractors operate acceptable and effective management system. Principles of Auditing: Refer ISO19011 standard clause 4 Activity-1: Terms and definitions Refer ISO19011 standard clause 3 Activity-2 ……Audit Types ( 1’st, 2’nd and 3’rd party ) There are three types of audits: a) First Party Audit b) Second Party Audit c) Third Party Audit First Party Audit: IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 7 of 48 Auditing undertaken by an organization on itself is known as First Party Audit. It is a very powerful and important feedback mechanism, which provides both confidence to management and employees about the current performance, strengths and weaknesses and also identifies opportunities for improvement. Such audits may include: • • • • • • • • Auditing of Management System to verify implementation and effectiveness of Quality Management System. This is often called as Internal Audit, which is performed, by an organization or department upon its own systems, procedures and facilities. Generally people from within the organization are used. In some instances subcontractors are hired to perform this audit. Auditing of projects or programme of work to verify conformity with Terms of Reference, contracts, plans etc. Auditing of production processes to verify conformity with process specifications. Auditing of key business processes and procedures to verify conformity and adequacy of process descriptions and procedures. Auditing of key documents, or process outputs to verify adequacy of processes used. Auditing of products to establish confidence in production methods and quality control techniques employed. Auditing of products to verify conformance of product standards Auditing of service provision to verify conformance to new service standards. Second Party Audit: A Second Party Audit comprises of an audit undertaken by one organisation upon another organisation and is an integral part of a Supplier Management Programme. Audits of potential suppliers and contractors to establish confidence in their ability to meet the requirements of the purchasing organisation, and to assist in the process of supplier selection and determination of supplier control mechanisms. Before deciding to use a particular supplier it is often necessary to establish a level of confidence in the potential supplier’s capabilities. This may be undertaken by simply reviewing data concerning the potential supplier but will more often than not require formal visits by technical and quality staff as appropriate in order to verify in the potential supplier’s operation that the necessary technical competence is in evidence together with all appropriate managerial, health and hygiene controls. This will involve the formalities of assessment and audit with an audit base agreed between customer and supplier. Audits of existing suppliers to verify conformance with contract requirements. Audits of existing suppliers as a result of problems experienced and to determine likely causes & corrective action. Supplier management is often a vitally important part of an organisation’s approach to the management of processes in relation to the provision of products and services. Many organisations do not undertake the full process of production or service delivery themselves, but rely on a number of suppliers to either provide them with part processed materials or to undertake key processes as a contracted activity. In the case of companies providing processed foodstuffs, many of the ingredients and raw materials that they process must be purchased from reliable sources that are exercising the same requirements in relation to health and hygiene. There may also be a range of suppliers involved in the overall chain of supply, involving first, second and even third tier suppliers right back to the primary agricultural source of supply. Contracted services may include maintenance services, testing and laboratory services, hygiene services, pest control, training, packing, warehousing (of both raw materials and finished products), transport and distribution. A second party audit becomes more critical for customers where there is a potential risk to product, business and brand name, should a labeled product be mishandled or misused during the transport/ distribution/ resale. e.g. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 8 of 48 storage of products.The regulatory audits are often referred to as second party; although some argument exists that they are actually third party Third Party Audits: A third party audit is an audit of an organisation by an external independent body (the third party) whose main function is to verify compliance to the planned arrangements and does not include any advisory services. To avoid any potential conflict of interest, external auditors are not permitted to be a consultant and an auditor in the same organisation. Though third party audits are usually carried out to verify systems implementation, some audits may be related to process or products if found appropriate to the audit objective. The authorities carrying out third party audits can be broken down into: Accredited certification bodies undertaking audits to verify conformance to a Quality Management System Standard such as ISO 9001 and for the purpose of granting certification to that standard. The requirements for the certification body for carrying out third party audit are defined in ISO 17021:2006. Trade organisations specifically set up by members of a particular trade or industry group to undertake audits on behalf of the group, in order to assist purchasing decisions within the group or industry. The audit process usually followed by Accredited certification bodies is as follows: 1. The planning process to focus and direct the audit 2. An on-site assessment to gather relevant information 3. An evaluation process to analyze findings, determine compliance and decide follow-up actions. Although there may be variations in the way that individual organizations carry out such audits there are some generally accepted protocols and elements of good auditing practice that have been incorporated in ISO 19011 and is accepted as a desirable approach to audit. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 9 of 48 Activity-3 : Audit Process : IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 10 of 48 Activity-4 : Audit Methodology AUDIT ACTIVITIES 1) Initiate the Audit: 1’st party 2’nd party 3’rd party informal Formal Formal Everybody 2) Review the Documents: knows about the Less importance. Separate checklist … system 3) Develop Audit Plan: YES YES YES 4) Assign Work to Auditors per Plan YES YES YES Checklist Checklist Checklist YES YES YES May not May not YES 8) Review Documents and Communicate: YES YES YES 9) Carry out the Audit: YES YES YES 10) Generate Audit Findings YES YES YES Less Formal Score Card Formal Only Findings Formal YES YES 5) Prepare Working Papers: 6) Determine the Audit Sequence: 7) Conduct Opening Meeting: 11) Present Findings and Conclusions: 12) Formally Distribute Audit Report 13) Follow Up on Actions / Corrective Actions: IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 Only findings YES © IRCLASS Systems and Solutions Private Limited 2019 Page 11 of 48 Activity 5 : objectives, scope and criteria OBJECTIVE: When initiating the audit it should be remembered why the audit is taking place. Its important to identify what issues exist and what action is required to overcome these issues. SCOPE: The audit scope describes the extent and boundaries of the audit in terms of factors such as physical locations, organizational units, activities and processes to be audited as well as, where relevant, the time period covered by the audit. CRITERIA: The audit criteria may include applicable standards, regulations, legislation, procedures and other management system documents. Other reference documents may include sector guidance or environmental guidelines. The audit objectives define what is to be accomplished a) the extent of conformity of the management system with respect to ISO9001:2015; b) the capability of the management system to assist the organization in meeting relevant statutory and regulatory requirements and other requirements to which the organization is committed; c) the effectiveness of the management system in meeting its Quality Goals, with respect to Quality, Delivery, Cost and Service; d) identification of opportunities for potential improvement; e) the suitability and adequacy of the management system with respect to the context and strategic direction of the auditee; f) the capability of the management system to establish and achieve objectives and effectively address risks and opportunities, in a changing context, including the implementation of the related actions. The audit objectives, scope and criteria and any subsequent changes to these are agreed by the client, audit program management and as appropriate the auditee after consultation with the audit team leader prior to onset of audit. For any audit an initial contact will be used to: • • • • • • • Establish communication channels Confirm authority to conduct audit Provide information on proposed timing and audit team composition Request access to relevant documents Determine applicable safety rules Make general arrangements for the audit Agree on the attendance of observers and the need for audit guides. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 12 of 48 Usually the team leader or the person planning the audit makes the initial contact and ensures adequate communication with the auditee related to the audit. The factors that determine the feasibility of the audit should include aspects such as: • • Sufficient and appropriate information about the subject of the audit Adequate co-operation from the auditee Where the audit is not feasible, an alternative solution which is acceptable to the audit client should be proposed by the audit program management in consultation with the auditee. Establishing the audit team After determining feasibility of the audit, it is very important to select a competent audit team to perform the activity. When deciding size and composition of the audit team, consideration should be given to: • The objectives, scope, criteria, location and estimated duration (man days) • The overall competencies needed to achieve the audit objectives • The language of communication and understanding of the auditee’s social and cultural characteristics. • The need to maintain the audit team’s independence from the activities to be audited and to avoid conflict of interest • The ability of the audit team members to communicate effectively with the auditee and to work together to best use their collective competency to conduct an effective audit. • type and complexity of the processes to be audited • If the necessary competence is not covered by the auditors in the audit team, technical experts with additional competence should be made available to support the team. Knowledge and Skills Quality Auditors shall have knowledge and skills in the following areas: a) Audit principles, procedures and techniques to enable the Auditor to apply those appropriate to different scenarios to ensure that audits are conducted in a consistent and systematic manner. An auditor should be able: • To apply audit principles, procedures and techniques; • understand the types of risks and opportunities associated with auditing and the principles of the risk-based approach to auditing • To plan and organize the work effectively; • To conduct the audit within the agreed time schedule; • To prioritize and focus on matters of significance; • To collect information through effective interviewing, listening, observing and reviewing documents, records and data; • To understand the appropriateness and consequences of using sampling techniques for auditing; • To verify the accuracy of collected information; • To confirm the sufficiency and appropriateness of audit evidence to support audit findings and conclusions; • To assess those factors that can affect the reliability of the audit findings and conclusions; • To use work documents to record audit activities; • To prepare audit reports; • To maintain the confidentiality and security of information; • To communicate effectively, either through personal linguistic skills or through an interpreter. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 13 of 48 b) Management system and reference documents to enable the auditor to comprehend the scope of the audit and apply audit criteria. Knowledge and skills in this area should cover: • The application of QMS in organizations; • The interaction between the components of the QMS systems; • Recognizing differences between and priority of the reference documents; • Application of the reference documents to different audit situations; • Information technology for, authorization, security, distribution and control of documents, data and records; • Industry related management system standards (see Section 9) c) Organizational situations to enable the auditor to comprehend the organization's operational context. Knowledge and skills in this area should cover: ➢ Organizational size, structure, functions and relationships; ➢ General business processes and related terminology: ➢ Cultural and social customs of the Auditee. d) Applicable laws, regulations and other requirements relevant to the discipline to enable the Auditor to work within, and be aware of, the requirements that apply to the organization being audited. Knowledge and skills in this area should cover: • • • • • Local, regional and national codes, laws and regulations: Contracts and agreements, International treaties and conventions; Other requirements to which the organization subscribes. Generic knowledge and skills of audit team leaders. Audit team leaders should have additional knowledge and skills in leadership to facilitate the efficient and effective conducting of the audit. An audit team leader should be able: ➢ To plan the audit and make effective use of resources during the audit; ➢ discuss strategic issues with top management of the auditee to determine whether they have considered these issues when evaluating their risks and opportunities; ➢ To represent the audit team in communications with the audit client and Auditee; ➢ To organize and direct audit team members; ➢ To provide direction and guidance to auditors-in-training; ➢ To lead the audit team to reach the audit conclusions; ➢ To prevent and resolve conflicts and ➢ To prepare and complete the audit report. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 14 of 48 Activity 6 : Audit Roles: Responsibilities in relation to the Audit Process Audits are most effective when performed by qualified professionals who work together and are focused on clear objectives. Each audit can be viewed as a project, an activity with a start and finish. A team is formed to accomplish the project. Everyone who will participate in the project is part of the team. Whether an audit is to be undertaken by a team of several auditors, a Team Leader should be nominated to act in an overall managerial capacity to ensure that required objectives are met and the audit is undertaken in a professional and fully acceptable manner. The necessity for a team of auditors to undertake the onsite audit will vary depending upon the magnitude and complexity of the task, in some cases technical experts may also need to be included to work with and assist the auditors in their course of work The auditor, team leader, client and auditee each have key contributions to make in order to achieve a successful audit outcome. Some of these are listed below: a) Auditor Responsible for: • • • • • • • • • • • Complying with audit requirements Communicating and clarifying audit requirements Planning and carrying out assigned responsibilities effectively and efficiently Documenting the observations Reporting the audit results Verifying the effectiveness of corrective actions taken as a result of the audit (if requested by the client) Retaining and safeguarding documents pertaining to the audit Submitting documents as required Ensuring such documents remain confidential Treating privileged information with discretion Co-operating with and supporting the team leader Auditors should: • • • Remain within the audit scope Exercise objectivity Collect and analyze evidence that is relevant and sufficient to permit the drawing of conclusions regarding the audited quality system IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 15 of 48 • • Remain alert to any indications of evidence that can influence the audit results and possibly require more extensive auditing Act in an ethical manner at all times b) Team Leader Responsible for: • • • • • • All phases of the audit Making final decisions regarding the conduct of the audit and any audit observations Planning and assisting with the selection of other audit team members Preparing the audit plan Representing the audit team with the auditee’s management Submitting the audit report Auditors should: • • • • • • • Define the requirements of each audit assignment, including the required auditor qualifications Complying with applicable auditing requirements and other appropriate directives Planning the audit, preparing working documents and briefing the audit team Reviewing documentation on existing quality system activities to determine their adequacy Reporting critical non-conformities to the auditee immediately Reporting any major obstacles encountered in performing the audit Reporting on the audit results clearly, conclusively and without undue delay c) Client • • • • • Determines the need for and the purpose of the audit and initiates the process Determines the auditing organisation Determines the general scope of the audit, such as what management system or document is to be conducted against Receives the audit report Determines what follow- up action, if any is to be taken, and informs the department /function about it Auditee • • • • • • Informs relevant employees about the objectives and scope of the audit Appoints responsible members of staff to accompany members of the audit team Provides all resources needed for the audit team in order to ensure an effective and efficient audit process Provides access to the facilities and evidential material as requested by the auditors Cooperates with the auditors to permit the audit objectives to be achieved Determines and initiates corrective actions based on the audit report Other roles that may exist within an audit programme include: IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 16 of 48 • • • • • Observer Provisional Auditor / Team Leader Technical Expert Witness auditor Guide Observer: Simply watches the audit activities. Does not participate in the audit in any way. Provisional Auditor / Team Leader: Provisional auditor acts under the guidance of the team leader. Provisional team leader performs the audit under the scrutiny of a capable Lead auditor. Technical Expert : When required competence is not available in the team, a technical expert is used for complex and technical issues. This person may not necessarily be an auditor but provides to the audit team regarding the issues that may / may not pose a risk. Such a person may help to verify that the audit team has managed the risks associated with a particular organisation’s management system. Witness auditor: They verify the audit process on site and audit performance by the auditors. Their role is similar to that of observer. In some instances this is performed by accreditation bodies or certification bodies, or sometimes under regulatory situations. Guide: Guide is responsible for assisting the audit team and acts on request of the audit team leader. The guide ensures that rules concerning safety and security procedures are known and followed by the auditors on site. Auditors may make mistakes and timely query from a guide may avoid an embarrassing situation. Notwithstanding this, a guide should never offer an opinion and should avoid unnecessary interference. The team leader shall also explain the role of guides to the auditors and the auditee to ensure an effective audit. It should be clearly understood that the guides are not buffers between the auditors and auditee, but only mute spectators to the whole process. The guides should not cut across the auditors and auditee by asking or responding to audit questions. Allocation of duties to the audit team members The audit team leader, in consultation with the audit team, is responsible for assigning each team member, responsibility for auditing specific management system processes, functions, sites, areas or activities. Such assignments should take into account the need for auditor competence and efficient use of resources. Changes to the assignments may be made keeping in mind the achievement of audit objectives The audit team members should review all relevant information related to their audit assignments and prepare any work documents necessary for those assignments. Communication It is very necessary for the audit team leader to ensure a clear understanding at all times of the proposed audit arrangements between the audit team and the organisation to be audited. Particularly in relation to the audit schedule and key staff that will need to be available, support requirements (office facilities, guides, logistics arrangement, any security and safety related issues etc.) that will be made available during the audit. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 17 of 48 In relation to support requirements, the team leader should determine what will be required to support the audit process, such as office facilities etc. It is normal practice for the team leader to request the use of an area where the audit team may be based and where they may be able to meet for private discussions. As a part of the communication process it is advisable to telephone or fax the company a week before the audit, just to ensure that there have been no misunderstandings and that the company is fully prepared for the audit and has made all the necessary arrangements. It is the primary responsibility of the team leader to ensure adequacy of communication throughout the complete audit process. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 18 of 48 Activity-7: Audit Methods: The Audit Process The term audit is often used to describe an “assessment” or “evaluation” of an organization’s management system as specified in ISO 19011. As per ISO 17021 it is mandatory to carry out an audit in 2 stages: Stage I & Stage II. Stage I audit involves a review of the documentation provided by the organization to demonstrate that it has adequately addressed the requirements of ISO 9001: 2015 in its documentation system and verification of readiness and appropriateness / adequacy of the implemented elements of the Standard. Stage II audit specifically looks for evidence of compliance to the planned arrangements and elements of ISO 9001:2015. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 19 of 48 Stages in relation to the audit process Application made by the client Contract review Audit initiated at the request of client Stage I audit Document review, verifying appropriateness of implemented QMS onsite and preparedness of the Stage I Audit planning and selection of competent audit team Stage II audit Stage II Audit planning and selection of competent audit team In depth audit of a representative sample of the QMS system to verify compliance to the planned arrangements and ISO 9001 Audit Follow-up Audit report issued to the client (for verification for corrective Activity 8: Audit Plan Refer Clause 6.3.2 ISO19011:2018 Risk-based approach to planning : The audit team leader adopts a risk-based approach to planning the audit based on the information given by the certification body and Auditee. Audit planning considers risk related to audit activities and consensus from all the interested parties related to audit. Audit plan shall consider risk of not achieving the audit objectives. In planning the audit, the audit team leader should consider the following: ▪ the composition of the audit team and its overall competence; ▪ the appropriate sampling techniques ▪ the risks to achieving the audit objectives created by ineffective audit planning; ▪ the risks to the auditee created by performing the audit. ▪ opportunities to improve the effectiveness and efficiency of the audit activities; The audit team leader should prepare an audit schedule for the in-site audit activities. This schedule provides the necessary information to the audit team, auditee and the organization. It also facilitates scheduling and coordination of the audit activities. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 20 of 48 The extent of details provided in the audit schedule should be relative to the size and complexity of the audit. The details will vary with different stages of the audit. There are a number of considerations to be looked into while making an audit schedule e.g. Auditors with greater experience are likely to require less time than less experienced auditors to gain the same confidence in an organization. It also depends on the “sample” that is to be taken (activities to be audited and requirements that they will be audited against). More time will be required in larger companies, complex technologies or business processes involved. Two auditors can cover more ground and in greater depth that one and the involvement of more than one auditor allows for comparison of observations and active discussion on the direction that the audit should take, or trails that should be followed after the discovery of non-conformities. The other areas that form a significant part during finalizing the audit schedule are: • • • • • • • • • That appropriate auditors are allocated to activities commensurate with their skills and background Ensure that the processes and associated Quality elements are covered to an appropriate level. Ensure that all the clauses of the standard are covered. All locations/ sites and departments are adequately covered during the audit. Multi site approach may be applied in case of more than 2 sites where number of sites perform the same activity and have a centralized management system i.e common MRM, Internal audit. Ensure that adequate communications are allowed for within the schedule. Time should be included in the schedule. This will be particularly difficult when the audit team is at different sites. As a minimum, the team should be together for opening and closing meeting. An opportunity to review progress in some form should be made every day. Time should not be wasted in excessive traveling, meetings etc. The schedule should take account of process flows, information flows and material flows. The schedule should where possible sequence audit activities to follow these flows and facilitate development of effective audit trails. For the area(s) of the company that are part of the audit activity, the auditor must first develop a good understanding of what activities are undertaken, how and in what order. Once this has been done it is then a relatively straightforward task to identify which of the management criteria have some scope for application in relation to the different activities. A process approach audit concept includes: INPUTS OUTPUTS FUNCTION Activities involved in converting the input into output should be sequenced so that the audit process as a whole can be effective. E.g. receipt of iron sheets. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 21 of 48 (Input) Delivery of iron sheet Identify. Stamped sheet Put on the hangars Quantity verification Inspection Put them on the racks Testing (Output) Delivery to the customer /Records Nonconforming, return to supplier This kind of an interaction chart helps in determining which element of ISO 9001:2015 is applicable within this section of the company. The sequence and interaction of processes provides a very important base for identifying the audit trails as well the time that will be required for auditing each function / department. It is at this stage that the auditor needs to refer to the original audit objectives and to select an appropriate sample. This sample must be determined such that it will provide sufficient confidence that key criteria are indeed being implemented within the organisation. The main elements of the audit schedule include: • • • • • • Name of the organization, location and sites to be covered Dates of audit Audit objective and scope Audit criteria and reference documents Identification of the organizational and functional units, or processes, to be audited Expected time and duration for audit on-site activities, including meetings with the auditee’s management and audit team meetings • Identification of roles and responsibilities of the audit team members and any accompanying persons • Exact time of opening meeting and closing meeting The schedule should be reviewed and accepted by the client organization and presented to the auditee well in advance. Any objections to the audit team by the auditee should be resolved between the certifying body and client prior to the audit. The audit schedule should be sufficiently flexible to permit changes, such as any changes in emphasis which may become necessary as the on-site audit activities progress. Any revised audit schedule should be agreed between the parties concerned before continuing the audit. Preparing documented information for audit IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 22 of 48 Work documents used by the audit team for the purpose of reference and /or recording the proceedings of the audit can include: • • Audit procedures, checklists and sampling plans Forms for recording information, supporting evidence, records of meetings and audit findings The use of work documents, such as checklists and forms, should not restrict the extent of audit activities. Before starting any audit of a department or process the auditor should be clear on: • • • • • • • • Why audit this activity? What is the requirement against which the audit is to be conducted e.g. ISO 22000, supplier contract or product / service specification? Which parts of the audit standard are applicable to this activity? What does the audit standard expect from the activities that are going to be audited Who needs to be spoken with? Which procedures, instructions, quality plans etc. are applicable? Which activities need to be actually observed to affirm the practices in the organisation? What records will need to be reviewed? Preparation should include: • Answering each of the above question • Understanding what procedures, instructions, plans etc. are required • Developing method or strategy for the audit • Structuring the audit An aide memoire by definition is an assistant to memory. It should serve to help achieve the objective by being a documented plan for the audit activity. An aide memoire should provide prompts, which can be referred to and typically will include: • • • • • • • • • • Procedure and audit standard references Key processes Inputs to processes Sequence of activities Control points Control measures Responsibilities and authority Outputs Interfaces Records It may also include prompts for: • • • • Timings to support the audit schedule Anticipated findings Details of samples to be reviewed Audit trails Use of Aides Memoire IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 23 of 48 Preparing an aide memoire will contribute to the success of the audit, because it will enable the auditor to think through audit objectives and plan how to achieve them. Once prepared the aide memoire provides a reference point, when required. Distractions during an audit, such as interruptions, finding the unexpected and deficiencies that need investigation can all upset initial thoughts. If these events are not managed they may result in auditors being diverted from their plan and an ineffective audit will be the outcome. The aide memoire provides the point of reference to look back at. An aide memoire should not be a script for the audit, nor should it be followed blindly. Auditing frequently reveals the unexpected, and possibly deficient, finding that needs further investigation at the time. The auditor should not ignore something relevant to the audit just because it was not included in the aide memoire. Where it is appropriate and timely, follow through the point at the time. Once the point is resolved, be referring back to the aide memoire auditors can pick up where they had left off. Where the finding is better followed up later, typically where it would mean breaking off from the audit and moving to another person or area, an aide memoire that can facilitate note taking is a convenient place to record details of what was found. For example, recording the reference of an erroneous temperature indicator or a missing procedure that needs to be followed up. Looking back at aide memoire during the check back phase of the audit provides the reference for confirming that everything which was set out has been reviewed. The different types of aide memoire that can provide assistance during an audit include: • • • • Checklists Marked up procedures Flow charts Mind maps IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 24 of 48 CHECKLISTS A checklist is a list of issues that require reviewing during the audit process, however in many cases that is where the audit begins and ends. Whilst a checklist is useful for auditing compliance against a specific standard, code or regulation, often their use and limitations are not fully understood by auditors and organisations. A checklist is not a list of questions to ask the auditee. It is simply a “prompt” for aspects of the system, which require review. Checklists are generally constructed through transposition of the audit criteria as shown below, the prompt is based on action or “trigger” words e.g. shall, must, will, test procedures, records, specified requirements etc. Requirement Checklist prompts The carcasses to be received by Processing Who receives?, What temperature?, What records? Supervisor at 4˚C Deboning to be completed within 10 minutes for Record of times. Physical checks by supervisors each carcass Knives sterilization to be done frequently What temp.?, Any records of validation The carcass to be cut into pieces within next 10 Record of times minutes The pieces to be wrapped in plastic sheets and Record of times placed in blast freezer within next 15 minutes. Total time for meat processing should not exceed 1 Record of times hour Any problems with condition of goods to be What happens? reported to QA manager Responsibility for corrective action? Care must be taken to ensure that the checklist prompts review the existence and suitability of controls, acceptance ranges and usability of the documented system. In particular, ensure that these measures and targets reflect the system’s requirements. A checklist often fails as an effective audit guidance tool when it is used simply as a list of questions, which only conforms that an activity has been carried out. The auditor must always be prepared to follow up any other line of enquiry that could provide information or objective evidence. Care must be taken to ensure that checklists do not restrict enquiry.Several variations of the checklist are sometimes referred to and each has a particular application, e.g. criteria based, departmental based, ready-made and custom-built. Checklists, as far as possible, should be unique for every function or department, and should reflect the actual activities under review. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 25 of 48 Documented information created during the audit process involving confidential or proprietary information should be suitably safeguarded at all times by the audit team members. Activity 9: Opening Meeting An opening meeting is conducted to provide a clearly defined start point for the audit process. The opening meeting sets the scene and enables all the client’s representatives to meet the audit team as well as describing the audit process in more detail. During the opening meeting the client should be informed about the audit plan and how the audit activities will be undertaken. The opening meeting usually has a standard agenda which is used as a checklist by the lead auditor to control the meeting and ensure that all the necessary points are covered A standard opening meeting should include at least the following points: • • • • • • • • • • • • • Introduction of the participants, including the outline of their roles Confirmation of the audit objectives, scope and criteria Confirmation of the audit schedule and other relevant arrangements with the auditee, such as the time and date for the closing meeting, any interim meetings between the audit team and the auditee’s management and any changes Confirmation about methods and procedures to be used to conduct the audit, informing the auditee that the evidence collected during the audit will only be a sample of the information available and hence is only representative of the organisation’s implementation status. Confirmation of language and official communication links between the audit team and the auditee Information about the progress of audit will be provided to the client form time to time Confirmation that any resources and facilities needed by the audit team are available Confirmation of matters relating to confidentiality Confirmation of relevant work safety, emergency and security procedures for the audit team Confirmation of availability, roles and identity of any guides Method of reporting including any grading of non-conformities Information about accreditations, appeal system etc. Confirmation about conditions under which the audit may be terminated. At the end of the opening meeting opportunity should be given to the auditee to ask any questions pertaining to the audit process. The formality of the opening meeting may vary according to the circumstances. Where there is an established relationship e.g. during the renewal audit, the formalities may be lessened. Similarly the differences in company culture, say between a large multi-national organization and a small to medium sized enterprise will require the lead auditor to be able to tailor their approach to match the needs of both the auditor(s) and auditee(s). c) Sampling It is not possible to go through all the documents, records, observe all the processes and talk to all the people while conducting an audit. Hence consideration needs to be given to how an auditor takes adequate samples of information. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 26 of 48 Auditors may sample: • • • • • • If there is more than one department performing the same task Procedures / Work instructions (prioritizing on key processes) Equipment (e.g. checking use of and calibration of) People if they perform the same task (note: auditors should not audit a system based on one’s work) Products (on warehouse shelves as a basis for audit trailing) Records It is usually difficult to determine the sample size. Auditors sometimes use sampling plans where a large number of samples are needed to create the desired confidence level in the system; however, this will often incur considerable time. Auditors should ensure during the audit process, that the sample size is sufficient to establish confidence in the system. This may mean that the sample size will vary depending on the nature of the objective evidence sighted by the auditor and also the risk associated with the activity being audited. There is always a risk associated with the sampling of activities and this should always be highlighted to the client and the auditee. Considerations for sampling: Large volumes of data – Appear daunting but are not usually difficult to assess. Sampling for confidence – Do not just “hunt for non-conformance” and then move on Criticality of processes – Concentrate on key aspects and all the applicable clauses of ISO 9001:2015 Avoiding quoting percentages – Can give a false impression The audit evidence collected during an audit will inevitably be only a sample of the information available, since an audit is conducted during a finite period of time and with limited resources. Hence there is an element of uncertainty inherent in all audits, and attention of users of the audit conclusions should be drawn to this uncertainty. Risk based sampling: It is more appropriate to take samples that relate to activities presenting greater risk to an organization and its products than to those to which very little risk is attached. By this means the audit will provide better value to the auditee and the organization. In relation to processes there will be a range of measurements that may be taken in relation to the overall process, however some will relate to Quality Plan and it should be these that are the subject of audit sampling rather than less important general measurements (such as weight, colour etc.) Measurements that have a clearly important role in relation to the overall safety of the product would be the choice for audit sampling. Again, if the audits in relation to staff competence/training issues, then it is possibly more important to take training record samples that relate to staff involved in direct production and quality control rather than those that are effectively not associated with product handling. Records of competence of staff involved in making the Quality Plan should be verified as a choice. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 27 of 48 ACTIVITY 10 : Collecting evidence Audits involve the collection of evidence in order to verify that documented system is also evident in practice i.e. the practice is in line with the intent. Guidance contained in ISO 19011 suggests that evidence should be collected through interviews, examination of documents and observation of activities and conditions in the areas of concern The auditor’s aide memoire and notes will generally steer the audit process through range of activities aimed at searching out evidence to confirm the planned arrangements. The auditor always needs objective evidence to conform to requirements of the organisation or a deficiency. Throughout the audit a certain degree of flexibility needs to be maintained. Auditors will need to decide what they will examine in order to obtain the necessary objective evidence to be able to answer the questions on their aide memoire. Objective evidence can be found in documents / records which includes: • • • • • • • • • • • Procedures /Work instructions Process flow charts Non-conformity management Quality plans Test reports verification Laboratory test reports Monitoring and test data Specification and acceptance criteria Minutes of meeting Internal audit reports Supplier evaluation records Procedures may be examined for availability, application, interpretation, issue status and awareness among the employees. The auditor should be careful about not being too critical of the adequacy of a procedure unless there is evidence of things going wrong due to the lack of detail provided to employees in the procedures. The auditor should be capable of deciding which evidence he / she requires to meet the requirements of audit criteria and scope. Objective evidence can also be found in items, which include: Tools: Correct tools; Storage facilities; Operating instructions, Calibration status Materials: Appropriate raw materials, packaging material etc. Handling and storage facilities Identification, shelf life requirements Disposal facilities Instructions and safety warnings Products: Identification; Storage; Handling; Packaging Conformance status; Traceability; Test reports IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 28 of 48 People: Training / Qualification Attitude Physical attributes Suitable environment Adequate numbers Awareness of responsibilities, about procedures, process flow Activity 11 : Effective Interviewing Technique Interviewing skills are probably the most fundamental skill that an auditor needs to use. To some, interviewing is a familiar activity; however in an audit situation interviews can be quite different. Body language, testing, understanding and summarizing data may be very difficult. The auditor may be working in an awkward environment, due to noise, space, interruptions etc. The auditor may be unfamiliar with the terminology of the industry or organisation. Asking questions, reviewing documents and keeping accurate notes almost simultaneously is a difficult job. Effective questioning can result in saving time, receiving invaluable information and developing effective relationships. On the other hand poor questioning can invade privacy, cause animosity and result in resentment – thus leading to an ineffective audit. Auditors must have the capability to know how to get important information in a way that does not make the auditee feel interrogated or defensive. Use of open questions to establish general information and letting the auditee talk about the process / product/ organisation etc. An example might be: How does evaluation of a particular supplier help in ensuring conforming raw material in all deliveries? A good amount of information can be sought through this. When information related to specific fact is sought, closed questions may be asked. For example: Do you have a process for obtaining regular feedback from the HODs regarding the effectiveness of the trainings provided to their staff? This in particular requires only a yes or no answer. Reflective questions clarify and ensure understanding – for example, Are you saying that there are two processes available which can provide the same result? Questions are directed to • • • • • • • • • Check understanding Focus attention Establish reasons Discover the source Obtain feedback Stimulate reflective thinking Follow up Obtain evaluations Explore resources There are 2 main rules that need to be followed: • Ask straightforward questions • Wait for the answer and listen actively Rule 2 is very important to be a good auditor. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 29 of 48 There are also a number of guides that one can follow when asking questions. • • • • • • • • • • Avoid complex constructions Follow up question areas until it is clear what actually takes place. Auditee often may skim over an important area. Additional questions may have to be asked to determine what happens and when. It may be necessary to phrase a question so that a negative response can be made easily-such as “That step can often be omitted- can you give me an example of when you have to do this” Find out the “why” behind actions. Actions may have many different applications or reasons. The auditor may need to explore the reasons behind an action, step or process. Use comments in place of questions to reduce the questioning appearing more like an interrogation. For example “ I can imagine there are some really tough problems encountered during that process” Make use of contrasts and comparisons. Instead of saying “what is that process like?” ask “how does the method of calibration for monitoring devices on the freezers differ from hand held ones?” If an auditee representative uses a technical jargon which is not familiar, do not let it go for fear of appearing uninformed. Avoid expressing value judgments in questions. Start out with an open ended question and follow up more specifically Avoid collecting information not relevant to the scope of the audit. The core role is to gain information about specific processes or services. The criterion of relevance should apply. The auditor should not be concerned with accumulating facts, no matter how interesting, outside the scope of the audit. When asking the auditee questions, one must also be conscious of reading their behaviour from their responses and body language. If a situation is developing into a crisis and must be addressed, it needs to be approached calmly. One has to accomplish this by keeping emotions under control. Once the situation is under control, constructive actions can be taken and the problem can be resolved. Non-Conformance Reporting A non-conformance is non- fulfilment of planned arrangements. A non-conformance may be: • • • • • • • • • • • • Anomaly Incident Breach Problem Defect Subject of concern Discrepancy Deviation Negative finding Irregularity Concern / Issue Fault IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 30 of 48 There are many terms that can be classified a non-conformance – and in fact even the word “non-conformance” may not be used in some organizations. Many organizations prefer to maintain a positive focus in everything they say and do. It is ideal to use structured non-conformance reports for the purpose of reporting. Other terms used are: Corrective action requests, System /Process correction notes etc. Traditionally, this reporting is through the use of a template or form. Non-conformance statements are statements which can be substantiated, and upon which action can be taken by the auditee’s management. The structure of a non-conformance statement is critical to allow the organization to identify the issue and decide what action is needed. When recording the non-conformance identified, a separate statement should be written for each system problem or deficiency identified, unless they are of the same type or similar in nature. The three elements that comprise of a non-conformance statement are • Statement • Objective evidence • Reference • Explanation • Objective evidence: What did not satisfy the reference criteria? Example: It was observed that -----------------Reference: A reference to the specific section of the organization’s documentation and/ or reference criteria that states what should have been done Example: Refer to -----or “This is not in compliance with procedure/instruction/standard requirement/ etc. In many case there will be a number of references to the documented management system and /or the reference criteria. It is not necessary to list each possible reference, but to identify the most relevant one. When legislative/ regulatory requirements are being audited it is common to list all references applicable to the deficiency. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 31 of 48 Example of a Nonconformity Report (NCR) form Client: NCR # File No Site: Function/Area/Process: Std. and Clause No(s): Section 1- Details of non-conformity: Statement – Evidence – Requirement - Auditor: Auditee representative acknowledgement: Category: Date: Correction (fix now) with completion dates: Root Cause analysis (how/why did this happen?): Section 2- Auditee Proposed Corrective Action (Attach separate sheet if required) “Auditor” review and acceptance of Corrective Action Plan: Auditee representative: Date: Section 3- Details of “Auditor” verification of Auditee implementation of action plan Section 4- NCR closed out by “Auditor” on (date): IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 “Auditor” Team Leader name: © IRCLASS Systems and Solutions Private Limited 2019 Page 32 of 48 Evaluating Results Collected audit evidence should be evaluated against the audit criteria to generate the audit findings. An audit finding can indicate either conformity or non-conformity with the requirements. Audit findings are graded in accordance to its severity. As the audit proceeds, it can be revealed from the notes made by the auditor / (s) that the information can be transcribed into conformances, opportunities for improvement or non-conformance. The list may belong or short, but some findings may be more significant than others and some may be closely related or manifestations of the same problem. The audit team should undertake an evaluation of all the audit results combined to establish what the concerns are. The auditors need to bear in mind that the auditee is more knowledgeable about the processes and is looking at the audit team’s findings as a source of information and areas for improvement. At the conclusion of each day, the team leader should provide an overview of findings to the Company’s Management Representative. Conformities should be summarized to at least indicate locations, functions or requirements audited, where no non-conformities are observed. Non-conformities should be recorded and supported by audit evidence. Non-conformities should be reviewed with an appropriate auditee representative to obtain acknowledgement of the audit evidence. The auditee representative’s acknowledgement indicates that the audit is accurate and that the non-conformity is understood. Every attempt should be made to resolve any divergence of opinion concerning the audit evidence and any unresolved points should be brought to the notice of the team leader. Closing Meeting The purpose of the closing meeting is to continue the communication process with the audited company’s management team and provide them a feedback on the results of the audit together with the conclusions reached and the recommendation of the team. It is a forum to ascertain that the organization’s top management is fully aware of and understands the findings and associated implications and can take corrective actions in time. The audit team should confer prior to the closing meeting in order to: • • • • • • Review the audit findings and any other appropriate information collected during the audit. Prepare a list of audit findings Reach consensus on the audit conclusions Agree on roles and tasks for the closing meeting Prepare recommendations, if specified by the audit objectives Discuss subsequent audit follow-up, if appropriate In many instances a simplified approach may be taken for the audit team review and the subsequent closing meeting, depending on the audit objectives and scope. A closing meeting chaired by the audit team leader, should be held with the auditee’s management and those responsible for the functions audited. During the closing meeting, it is recommended that the objective and scope of the audit be re-stated for the benefit of members attending the closing meeting. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 33 of 48 The agenda for the closing meeting may be as follows: • • • • • • • • • • • • • • • Thank the management for their support and hospitality Brief review of audit coverage – document review and on site audit Sampling and the risk involved. The need for similar probing in Internal Quality Audits Briefing on Positive findings Areas for improvement or of concern if any without giving specific solutions. Brief note on categorization of non-conformities Summary of non-conformities and their closure status. Recommendations of the team Actions to be taken by the Organization before receiving the certificate Compliance to surveillance schedule Any clarifications Assurance on confidentiality. Advise the Organization to keep confidential the decision on recommendation for grant of Certification till the final receipt of certificate, which may take up to 3 weeks. To give opportunity to the members for any views including any appeals, disputes and complaints. Communicate procedure on appeals, disputes and complaints The non-conformances should be presented by the team leader and any queries on the same should be resolved immediately. All non-conformances should be signed by the auditee as an acceptance. Depending upon the nature of the non-conformances found there may be discussions on corrective actions. It is unlikely that the corrective action plan will be given by the auditee immediately during the closing meeting unless the non-conformities are too trivial. The team leader should specifically address the implications of unresolved non-conformities. The results should be presented first in a summarized forma by the Team Leader who should then give an opportunity to individual team members to explain the finding in their respective audit areas. Finally the team leader should present the summary and make the final conclusions. Presentation of the results may be assisted by the handing out of copies of audit reports. Audit Reporting Audit report preparation and content: The audit team leader is responsible for the preparation, accuracy and completeness of the Audit report. The audit report should provide an accurate record of the audit and should address the audit conclusions on issues within the audit objectives and scope. The issues include: • • • Extent of conformance of the management system to the audit criteria Effective implementation of the management system Ability of management review and internal audit process to ensure the continuing suitability and effectiveness of the management system The audit report should include or make reference to the following: • The identification of the organization audited IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 34 of 48 • • • • • • • • • • • • Audit objectives, scope and criteria including a list of reference documents against which the audit was conducted. The date and site / location of the audit Audit findings including non-conformities and supporting evidence. Identification of the auditee and the audit team members A summary of the audit process including any issues / hindrances Confidentiality of the report Conclusion whether the audit objectives have been accomplished within the audit scope in accordance with the audit schedule Any agreed corrective action plans Any unresolved diverging opinions between the audit team and the auditee. Recommendations for improvement, if included within the audit schedule The distribution list Date of next visit The audit report should be signed and dated by the team leader, and sent to the client / auditee. The audit report should be issued within the agreed time period. If this is not possible, the reason for the delay should be communicated to the auditee. As the report is confidential it should not be distributed outside the client organization without their consent. The audit report as a whole should provide the client a clear picture about their implementation status and areas, which need further efforts for the organization’s overall improvement of the system Audit Follow-up Corrective Action A corrective action is a change implemented to address a weakness identified in a management system. Normally corrective actions are implemented in response to a customer complaint, abnormal levels of internal nonconformity, nonconformities identified during an internal / external audit or adverse or unstable trends in product and process monitoring such as would be identified by various process control monitors. The process of determining a corrective action requires identification of actions that can be taken to prevent or mitigate the weakness. Effectiveness is generally thought to be improved by addressing the root cause(s) of the problem (in some cases the root cause of occurrence and non-detection are considered separately). Where possible an analysis is undertaken to identify other areas, products, processes or services which may be affected by the same problem and assess the feasibility of carrying the countermeasures across to those processes. Further, there may be systems in place to ensure that the problem is taken into account in future incidents where new products, processes or services are introduced, or existing products, processes or services are modified. Depending upon the nature of the non-conformity raised during the third party audit, it may be appropriate to discuss a timescale for the company to propose necessary corrective actions. The time frame required for the implementation of corrective action should be mutually agreed between the auditor and auditee. It is not the auditor’s role to undertake and determine the exact action required for a non-conformity, however, auditors are often in a position to add further value to the audit by suggesting possible immediate actions. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 35 of 48 This should be approached carefully, as the auditee should not see any suggestions as definitive solutions. It is usually not possible for an organisation to determine the corrective actions during the audit. Resolving non-conformity in itself is a process involving an effective root cause analysis and management needs time to undertake the necessary investigations. Hence the auditors should not impose on the auditee the requirement to give immediate corrective actions. The team leader should take a commitment from the management as to when the corrective actions proposed will be communicated to the certifying body and also what follow-up actions will be necessary. It is a normal practice for an auditor to provide a formal written report fully detailing the audit findings to the auditee within a reasonable time, and it is often necessary for the auditee to respond to this report by indicating the corrective action plan and target date for completion of the same. The auditee management will need to ensure that each non-conformity situation is analysed, where necessary gathering further relevant information initiating detailed investigations, and/ or internal audits to provide additional information to enable the root causes of the non-conformities to be determined. It is at this stage that management may employ various problems solving tools and techniques to arrive at suitable fact based conclusions. Corrective action process Non-conformance Understand and analyze Root cause determination Verification Implement Corrective action proposal Certifying Body The proposed corrective actions should be acceptable to the auditor since at this stage he/ she should be able to determine that the proposed corrective action will effectively handle the non-conformity and also prevent its recurrence. If the auditing organisation is satisfied with the proposed corrective action, they should indicate this to the company and make arrangements for verification audits to be performed at an appropriate time. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 36 of 48 The verification of corrective actions may be carried out by the certifying body’s auditors through a formal visit or based on documentation to ensure that the action is effective in overcoming the original non-conformance. This may or may not involve the original audit team. Once the certifying body is satisfied as to the effectiveness of corrective action taken, then this should be formally recorded (preferably on the original audit report form) and the audit “closed out”. It is necessary for the certifying body to check effectiveness of any corrective actions during subsequent audits. Effective root cause analysis Tools Events and Causal Factors charting: (Method) A complicated process that first identifies a sequence of events and aligns them with the conditions that caused them. These events and respective conditions are aligned in a timeline. Events and conditions that have evidence are shown in solid lines but evidence is not listed; all others are shown in dashed lines. After this representation of the problem is complete, an assessment is made by “walking” the chart and asking if the problem would be different if the events or conditions were changed. This leads to causal factors that would then be evaluated using a tree diagram Change Analysis: (Tool) A six-step process that describes the event or problem; then describes the same situation without the problem, compares the two situations and writes down all the differences; analyzes the differences and identifies the consequences of the differences. The results of the change analysis is the cause of the change and will frequently be tied to the passage of time and, therefore, easily fit into an Events and Causal Factors Chart, showing when and what existed before, during and after the change. Change analysis is nearly always used in conjunction with a root cause analysis method to provide a specific cause, not necessarily a root cause. Barrier Analysis: (Tool) An incident analysis that identifies barriers used to protect a target from harm and analyzes the event to see if the barriers held, failed, or was compromised in some way by tracing the path of the threat from the harmful action to the target. A simple example is a knife in a sheath. The knife is the threat, the sheath is the barrier, and the target is a human. If the sheath somehow fails and a human is injured, the barrier analysis would seek to find out why the barrier failed. The cause of this failure is then identified as the root cause. Why-Why Chart: (Method) One of many brainstorming methods also known as the “Five-Whys” method. This is the most simplistic root cause analysis process and involves repeatedly asking “why” at least five times or until you can no longer answer the question. Five is an arbitrary figure. The theory is that after asking “why” five times you will probably arrive at the root cause. The root cause has been identified when asking “why” doesn’t provide any more useful information. This method produces a linear set of causal relationships and uses the experience of the problem owner to determine the root cause and corresponding solutions. Pareto Analysis: (Tool) A statistical approach to problem solving that uses a database of problems to identify the number of pre-defined causal factors that have occurred to business or system. It is based on the Pareto principle, also known as the 80-20 rule, which presumes that 80% of problems are caused by 20% of the causes. It is intended to direct resources towards the most common causes. Often misused as a root cause analysis method, Pareto analysis is best used as a tool for determining where you should start analysis. Fault Tree Analysis: (Method) Fault Tree Analysis (FTA) is a quantitative causal diagram used to identify possible failures in a system. It is a common engineering tool used in the design stages of a project and works well to identify IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 37 of 48 possible causal relationships. It requires the use of specific data regarding known failure rates of components. Causal relationships can be identified with “and” and “or” relationships or various combinations thereof. Fault tree analysis does not function well as a root cause analysis method, but is often used to support a root cause analysis. Follow- up and surveillance visits It is usual upon completion of an audit to establish a formal activity to verify the implementation of corrective actions. This should be performed at an appropriate mutually agreed time following the audit and after receiving details of proposed corrective actions and timeframe for implementation. Minor non-conformances are resolved by proper addressable of the issue by the auditee through documented procedures / instructions or memos and may not necessitate a formal visit by the auditors to verify the same prior to granting the certification. However for major non-conformances it may be necessary to allow a greater period of time and undertake a limited re-audit. It is important to know that while reviewing the proposed corrective actions, the focus of attention should be establishing that the proposal shows clear signs of a thorough investigation having been carried out by the audited organization to determine the root cause of the problems revealed by the auditor(s). The auditor should be able to identify the fact that the auditee may take actions, which will not eliminate the root but correct the non-conformity in the present situation. When follow-up visits are made, the detail originally entered in the non-conformance report is a vital source of information for the verifying auditor and hence it is necessary for such information to be clear, complete and traceable. If the corrective action is found to be effective then the non-conformity is closed out and effectiveness of the corrective actions are also verified in subsequent audits. Periodic surveillance visits are the means by which a Certification Body continues to verify compliance with the standard. As per ISO 17021, the 1st surveillance audit is required to be carried out within 12 months from the last date of audit. The certifying body may not undertake a complete audit during the surveillance audit but select only a few functions / departments during each surveillance audit. It is necessary to audit all the functions / departments during the 3 year period of certification A complete re-audit is conducted every 3 years to evaluate the improvements and maturity of the system. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 38 of 48 Benefits of Quality Management System Benefits of QMS In "Good reasons for implementing a QMS”, a number of direct benefits of QMS are stated: • Improved customer satisfaction • Improved quality of products and services • Workers’ satisfaction and more commitment to the organization • Better management and a more effective organization • Improve relations with suppliers • Improved promotion of corporate image • Focus on the more important (“high‐risk”) processes and their outputs • Improved understanding, definition and integration of interdependent processes • Systematic management of planning, implementation, checks and improvement of • Processes and the management system as a whole • Better use of resources and increased accountability • More consistent achievement of the policies and objectives, intended results and overall performance • Process approach can facilitate the implementation of any management system • Enhanced customer satisfaction by meeting customer requirements • Enhanced confidence in the organization. Besides these direct benefits, there are also several indirect benefits to identify, which give opportunities to: • Review business goals, and assess how well the organization is meeting those goals • Identify processes that are unnecessary or inefficient, and then remove or improve them • Review the organizational structure, clarifying managerial responsibilities • Improve internal communication, and business and process interfaces • Improve staff morale by identifying the importance of their output to the business • Involving them in the review and improvement of their work IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 39 of 48 Activity-13: Terminology Refer ISO9000:2015 Plan-Do Check-Act PDCA is a tool that can be used to manage processes and systems. PDCA stands for: P Plan: set the objectives of the system and processes to deliver results (“What to do” and “how to do it”) D Do: implement and control what was planned C Check: monitor and measure processes and results against policies, objectives and requirements and report results A Act: take actions to improve the performance of processes PDCA operates as a cycle of continual improvement, with risk‐based thinking at each stage IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 40 of 48 Process Approach All organizations use processes to achieve their objectives. A process: Set of interrelated or interacting activities that use inputs to deliver an intended result. Inputs and outputs may be tangible (e.g. materials, components or equipment) or intangible (e.g. data, information or knowledge). The process approach includes establishing the organization’s processes to operate as an integrated and complete system. 1. The management system integrates processes and measures to meet objectives 2. Processes define interrelated activities and checks, to deliver intended outputs 3. Detailed planning and controls can be defined and documented as needed, depending on the organization’s context. Schematic representation of the elements of a single process Risk‐based thinking These three concepts together form an integral part of the ISO 9001:2015 standard. Risks that may impact on objectives and results must be addressed by the management system. Risk‐based thinking is used throughout the process approach to: 1. Decide how risk (positive or negative) is addressed in establishing the processes to improve process outputs and prevent undesirable results 2. Define the extent of process planning and controls needed (based on risk) 3. improve the effectiveness of the quality management system 4. maintain and manage a system that inherently addresses risk and meets objectives IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 41 of 48 Activity 14: Structure of ISO9001 2015 Activity 15 :Audit Trail Auditing technique At the time of audit, an auditor may use a combination of audit strategies and techniques in order to verify conformance to requirements. The audit trails should be followed as they arise. The auditor should be careful in following the audit trails since it may sometimes be a wasteful activity and may need the auditor to considerably deviate from the audit sample originally deemed necessary. ISO 19011 addresses the requirement of a process approach audit which essentially means that every function be treated as a process and be approached accordingly. This approach also helps in identifying the efficiency of interface channels and overall working of the system. The auditor should follow a logical sequence of audit progression through an activity. The auditor may start at the beginning of a process and observe the process as each process step is undertaken, asking questions at appropriate times or requesting to see various documents, examining materials, tools etc. There could also be another way wherein the auditor may start with the output of the process and work backwards to establish how the organization has arrived at the outputs that have been obtained. This all leads to a very systematic way of working e.g. Tracing Forward or Tracing Back. In these methods the auditor selects a particular product or product flow chart from commencement to completion. In these ways an audit trail can be followed. The trace forward method is most used when focusing on a particular process, e.g. manufacture of electrical cables. On the other hand the trace back method is particularly useful in tracing services e.g. retailer. Vertical and horizontal approach should also be practiced by randomly selecting and visiting the departments that are of interest to the audit objectives and scope. It is essential when using this method that the auditor is diligent in ensuring that relevant areas or processes are not missed. An experienced auditor who has a good knowledge of the organization’s management system typically performs an audit in this manner. It is important that regardless of which method is used the auditor avoids adopting what one may describe as tunnel vision- i.e. focusing closely on a process using particular senses and failing to observe other factors which may become apparent on the periphery through other senses- such as odours and sounds. Following up on seemingly peripheral issues can alert the auditor to the fact that there may be a problem. The auditor must be careful on these occasions to avoid being deliberately diverted up a false trail to ensure less time is available to be spent on the core task. Let us take an example of “Goods Receiving Process” The auditor decides to witness the goods receiving process in the receiving dock and observing the progress of the process through the stores onto the production area. The auditor would observe activities undertaken, asking questions to the staff as appropriate, examining work instructions, test reports, measuring and monitoring device calibration etc. An alternative approach is to work backwards, wherein the auditor picks a sample of the raw material from the production area and requests to the see the associated paperwork as well as storage conditions in the stores / IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 42 of 48 warehouse. Everyone associated with the handling of this raw material can be brought under the purview of this audit process to ascertain that system was followed as per the documented / established procedure / instructions. The auditor could verify that the instructions have been produced to reflect the requirements of associated product specifications, and have been correctly authorized. The auditor could also seek records of the supplier of the raw material to verify his approval and evaluation status with the organization. Thus whatever technique is applied during an audit, one should remember to remain focused during the audit and look for evidence that will ascertain conformance / non-conformance to the audit criteria, objective and scope. The auditor should not go with the approach to find non-conformities. A very positive approach to the audit is necessary wherein the auditors tries to seek objective evidence of conformance to the planned arrangements but in his / her pursuit to do so may come across some deviations or adverse conditions which requires to be highlighted to the management. f) Maintaining a good audit trail During the course of audit, an auditor uncovers something that is worthy of further investigation, however this deviates from the original plan of action and may involve progressing the audit into other areas of the organisation not originally identified in the audit schedule or areas that have been / are to be audited at some other time. Sometimes it is better to take notes and follow the trail at a more convenient time or when the audit moves to the area where the trail leads. It may even be more appropriate if time is limited to report the concern to the auditee in order that they may investigate. Though there should be flexibility during the audit, the auditor must adhere as much as possible to the audit schedule and the original samples selected to verify conformance. If a trail arises that is relevant to the overall objective of the audit, or relates to something of major concern, then consideration should be given to the same for investigation. A decision to follow an audit trail should be based on the auditor’s judgment and if necessary the same should be consulted with the team leader. It should always be remembered that the audit is being conducted to provide both the auditee and the organization, information with an aim for overall improvement. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 43 of 48 Activity 16: Documented information Documented information needed to be retained by the organization for the purpose of providing evidence of result achieved (Documents) and could be from; 1. Organization charts 2. Process maps, process flow charts and/or process descriptions 3. Procedures 4. Work and/or test instructions 5. Specifications 6. Documents containing internal communications 7. Production schedules 8. Approved supplier lists 9. Test and inspection plans 10. Quality plans 11. Quality manuals 12. Strategic plans 13. Forms Documented information needed to be retained by the organization for the purpose of providing evidence of result achieved (records) : 1. Documented information to the extent necessary to have confidence that the processes are being carried out as planned (clause 4.4). 2. Evidence of fitness for purpose of monitoring and measuring resources (clause 7.1.5.1). 3. Evidence of the basis used for calibration of the monitoring and measurement resources (when no international or national standards exist) (clause 7.1.5.2). 4. Evidence of competence of person(s) doing work under the control of the organization that affects the performance and effectiveness of the QMS (clause 7.2). 5. Results of the review and new requirements for the products and services (clause 8.2.3). 6. Records needed to demonstrate that design and development requirements have been met (clause 8.3.2) 7. Records on design and development inputs (clause 8.3.3). 8. Records of the activities of design and development controls (clause 8.3.4). 9. Records of design and development outputs (clause 8.3.5). 10. Design and development changes, including the results of the review and the authorization of the changes and necessary actions (clause 8.3.6). 11. Records of the evaluation, selection, monitoring of performance and re‐evaluation of external providers and any and actions arising from these activities (clause 8.4.1) 12. Evidence of the unique identification of the outputs when traceability is a requirement (clause 8.5.2). 13. Records of property of the customer or external provider that is lost, damaged or otherwise found to be unsuitable for use and of its communication to the owner (clause 8.5.3). 14. Results of the review of changes for production or service provision, the persons authorizing the change, and necessary actions taken (clause 8.5.6). 15. Records of the authorized release of products and services for delivery to the customer including acceptance criteria and traceability to the authorizing person(s) (clause 8.6). 16. Records of nonconformities, the actions taken, concessions obtained and the identification of the authority deciding the action in respect of the nonconformity (clause 8.7). 17. Results of the evaluation of the performance and the effectiveness of the QMS (clause 9.1.1) 18. Evidence of the implementation of the audit programme and the audit results (clause 9.2.2). 19. Evidence of the results of management reviews (clause 9.3.3). 20. Evidence of the nature of the nonconformities and any subsequent actions taken (clause 10.2.2). 21. Results of any corrective action (clause 10.2.2). IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 44 of 48 Activity 17: Initiating the Audit: Audit Client: Organization or person requesting an audit. In the case of internal audit, the audit client can also be the auditee or the person managing the audit programme. Requests for external audit can come from sources such as regulators, contracting parties or potential clients. Auditee management: the organization being audited Team Leader: Audit is initiated, the responsibility for conducting the audit remains with the assigned audit team leader until the audit is completed Activity 18: Conduct Document Review: Refer Metafora documents: Document review can give an indication of the effectiveness of document control within the auditee’s management system. Please ensure that the documented information in the documents provided is: — complete (all expected content is contained in the document) as per ISO9001 2015 requirements. — correct (the content conforms to other reliable sources such as standards and regulations); — consistent (the document is consistent in itself and with related documents); — current (the content is up to date); Activity 19: Prepare Audit Plan: Refer ISO19011:2018 You know how to prepare the audit plan please prepare and ready to present the plan to whole class. You are preparing audit plan for METAFORA ( a case study) Activity 20 : Work Documents Prepare a checklist for audit of TOP MANAGEMENT Covering the points, Quality Policy Quality Objectives Management Review Leadership & Committement. Activity 21 : Opening Meeting Refer ISO19011:2018 Activity 22 : Site tour. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 45 of 48 Activity 23 : Top Management Audit Prepare Checklist for audit of top management. Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Few question you may want to ask to top management… for examples Has top management committed to QMS? Has top management taken a accountability of QMS? How are resources provided? Review of resources! How riks based thinking promoted in the Orgnization? How do you motivate people towards QMS? How do you ensure continual improvements? How frequently QMS policy is reviwed? How do you derive Quality objectives from QMS Policy? How do you review objective? Are these objective “SMART”? Activity 24: Auditing Context Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Activity 24: Auditing Risk and Opportunities Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Activity 24 : Auditing “Communication, Calibration and Work Environment, Infrastructure” Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Activity 24 : Auditing Objectives, Competence and Awareness Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 46 of 48 Activity 24: Auditing Operations & Performance Evaluation for Clauses 8.1, 8.2, 8.3, 8.4, 8.5 & 8.6, 8.7 and Clause 9.1, 9.2, 7.1.3 and 7.1.4 Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Activity 24: Auditing Improvement, clause 10 Delegates will prepare audit findings based on the audit progress, Audit Conformance and +ve points shall be recorded in the auditor own audit notes. This information is vital for subsequent activities such as preparing audit report Activity 25 : Nonconformity: No addtional comments. Activity 26: Audit Conclusion and Closing Meeting. Refer ISO19011:2018 clause 6.4.9 Activity 27: Audit Report; Refer ISO19011:2018 clause 6.5 Activity 28: Audit Follow-up Activity 29: Specimen examination paper. IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 47 of 48 We hope you enjoyed your course You will be contacted by the CQI and IRCA for feedback on the course and your Approved Training Partner. Completing this short survey will help to ensure the continuing high standards of these courses. You can also record your certificate and receive information about the CQI and IRCA, auditing and quality news, ISO updates and much more. To record your certificate, visit www.quality.org/record-your-certificate IRCLASS/CQI-IRCA - TRG/QMS/LAC -DWBM/Rev 6 dtd: 01-NOIV-2022 © IRCLASS Systems and Solutions Private Limited 2019 Page 48 of 48