Registration Management Committee (RMC)
AUDITING CUSTOMER REQUIREMENTS
Chris Miller
EAGLE Registrations Inc.
Shannon Craddock
Perry Johnson Registrars, Inc.
Boston, MA
July 21-22, 2011
Company Confidential
Auditor Workshop
Boston, MA
July 21-22, 2011
1
Registration Management Committee (RMC)
AGENDA:
• Customer Specific Requirements (CSR) General and
Process Auditing with CSRs (10 minutes)
• Pre-Audit Planning Exercise (20 minutes)
• OEM Customer Specific Requirements, Links,
Documents and Items to Know (10 minutes)
• Case Study (55 minutes)
• Questions & Answers (10 minutes)
Boston, MA
July 21-22, 2011
Company Confidential
2
Registration Management Committee (RMC)
Customer Requirements
General and Process
Auditing with CSRs
Chris Miller
Boston, MA
July 21-22, 2011
Company Confidential
3
Registration Management Committee (RMC)
Objectives of AS9100:2009 Audits:
• On-time, on-quality performance
• See AS9100 paragraphs 5.2, 7.2.1 and 8.2.1
• CSRs mandate specifics targets
• Scorecards
• Aerospace auditors should be auditing for
PERFORMANCE by conducting risk-based,
process-based audits.
Boston, MA
July 21-22, 2011
Company Confidential
4
Registration Management Committee (RMC)
Typical Formats For Customer Specific Requirements:
• Purchase Order (PO), Long Term Agreement (LTA) and/or
Contract
• Standard Terms and Conditions with PO
• Supplier Manual, Management System Requirements Manual
• Engineering Drawings
• Specifications (product, special processes, material, BOMs)
• Scorecards or performance metrics
Boston, MA
July 21-22, 2011
Company Confidential
5
Registration Management Committee (RMC)
What is the Client’s process for getting their
arms around CSRs?
• Is this part of a process or defined procedure?
• Who is responsible?
• How do they know they have all relevant documents
and the latest revision level?
• The integrity of this procedure/process should be
tested by the auditor.
Boston, MA
July 21-22, 2011
Company Confidential
6
Registration Management Committee (RMC)
What is Meant by a Process-Based Audit?
• Process-based audits are planned in accordance
with the processes defined by the organization.
• Customer-specific requirements should be
integrated into the process-based audit.
• They should not be stand-alone items on the audit
plan.
Boston, MA
July 21-22, 2011
Company Confidential
7
Registration Management Committee (RMC)
What is Meant by a Risk-Based Approach?
• Customer performance data should be used to prioritize
audit trails.
• Internal and External Customers
• Consider the following tools:
• V-Cycle
• Check-Act-Plan-Do Approach (CAPDo)
Boston, MA
July 21-22, 2011
Company Confidential
8
Registration Management Committee (RMC)
V-Cycle - Covered in the July 30-31, 2009 RMC meeting
• Start and end your audit with top management
understanding the customer requirements & measurables
Top Management
Top Management
(Requirements,
Objectives, Policies
Resources)
(Improvement
Actions and Strategic
Decisions)
Process Owners
Process Owners
(Collect and
Analyze Information)
(Risk Analysis and
Planning)
Participants (Operations and Recording)
Boston, MA
July 21-22, 2011
Company Confidential
9
Registration Management Committee (RMC)
Check, Act, Plan, Do - CAPDo Approach
• Variation on the Plan-Do-Check-Act Cycle
– Check: Begin the audit of a given process by asking
questions about performance:
» What are the customer and other expectations for the
process? AS9100 paragraphs 5.2 & 8.2.1 (Measurable)
» What is the actual performance? (Measurables/Trends)
– Act: How is performance being improve (Corrective
Actions/Continual Improvements Plans)
Boston, MA
July 21-22, 2011
Company Confidential
10
Registration Management Committee (RMC)
CAPDo Approach
• Variation on the Plan-Do-Check-Act Cycle
(continued)
– Plan: How was the process planned? Does the
process design allow performance objectives to be
met? (Internal Management Resources & Goals)
– Do – Is the process being executed as designed?
– Do staff & operators understand the (customer)
expectations for the process? (Goals, Measurables &
Customer Satisfaction)
Boston, MA
July 21-22, 2011
Company Confidential
11
Registration Management Committee (RMC)
AS9101 Pre-Data Planning Exercise:
• List what customer specific pre-data will help in
planning your audit (10 minutes)
• List one audit trail for each piece of data
(10 minutes)
– Generate at least 2 examples
• Example:
– Customer-specific pre-data: Customer Scorecard
– Trail: Ask Top Management what is being done with these
data during management review?
Boston, MA
July 21-22, 2011
Company Confidential
12
Registration Management Committee (RMC)
Groups Report Out:
Boston, MA
July 21-22, 2011
Company Confidential
13
Registration Management Committee (RMC)
OEM Customer Specific
Requirements (CSRs), Links,
Documents and Items to Know
Shannon Craddock
Boston, MA
July 21-22, 2011
Company Confidential
14
Registration Management Committee (RMC)
Basics That Should Be Covered to Understand
CSRs:
• The purchasing document will detail which CSRs are applicable to
an order.
• Should request that the supplier access the portal or other
methods during the audit activity to verify CSRs and revision
status.
• Look at the scorecards in the portal or other methods in real time.
• Ask if they have any OASIS Feedback from their customers?
Verify in the OASIS system.
Boston, MA
July 21-22, 2011
Company Confidential
15
Registration Management Committee (RMC)
The Boeing Company
– Boeing’s D6-82479, Boeing Quality Management
System Requirements for Suppliers, is available at the
following link:
» http://www.boeingsuppliers.com/supplier/D6-82479.pdf
– The following public website has a BQMS FAQ
section and other helpful resources:
» http://www.boeingsuppliers.com/supQual.htm
Boston, MA
July 21-22, 2011
Company Confidential
16
Registration Management Committee (RMC)
Cessna
– Would expect the following items to be
considered for auditing based on a review of the
Cessna PO as part of the auditing of customer
requirements:
» Engineer Drawing Revision Level and Parts
List Revision Level
» OPSS Revision Level (Outside Production
Specification Sheet)
Boston, MA
July 21-22, 2011
Company Confidential
17
Registration Management Committee (RMC)
Cessna
– Review of quality notes: This review would, at a
minimum, lead the auditor to request a copy of
CQRS per quality note 11.
• Based on a review of CQRS additional audit
trails can be determined.
Boston, MA
July 21-22, 2011
Company Confidential
18
Registration Management Committee (RMC)
Cessna
» An auditor should pick up on the link to
www.supplier.cessna.com which provides
numerous audit trails including:
• Revision levels of engineering specifications
and standard drawings
• List of approved special processors and
required special process approvals
• Terms and conditions
• Description of Cessna’s supplier rating
process STARS
Boston, MA
July 21-22, 2011
Company Confidential
19
Registration Management Committee (RMC)
Embraer
http://www.embraer.com.br/ftp/pub/qualityrequirements
Click
Boston, MA
July 21-22, 2011
Company Confidential
20
Registration Management Committee (RMC)
Ball Aerospace & Technologies Corp.
– No on-line link for suppliers
– Standard Terms & Conditions for reimbursable
government contracts
Boston, MA
July 21-22, 2011
Company Confidential
21
Registration Management Committee (RMC)
GE Aviation Engines:
• GE - AVIATION QUALITY SYSTEM
REQUIREMENTS FOR SUPPLIERS
• SUPPLIER SOURCE SUBSTANTIATION
• GE-A SUPPLIER REQUIREMENTS FOR
CHARACTERISTIC ACCOUNTABILITY,
VERIFICATION, AND QUALITY PLANNING
Boston, MA
July 21-22, 2011
Company Confidential
22
Registration Management Committee (RMC)
Honeywell
•Honeywell Aerospace Standard Terms & Conditions and
Supplemental Purchase Order Conditions (SPOC)
Manual is accessible on the supplier portal website,
which is password-protected.
•Only want suppliers accessing the portal.
•Auditors should request that the supplier access the
portal when they are on-site auditing.
Boston, MA
July 21-22, 2011
Company Confidential
23
Registration Management Committee (RMC)
Lockheed Martin
– Lockheed Martin Space Systems cannot provide
documentation as it is considered proprietary.
– The existing and soon to be released Lockheed Martin
Aeronautics Supplier Quality Requirements are attached
here:
 Quality System Requirements by Commodity
 Quality System Requirements by Commodity Draft
Boston, MA
July 21-22, 2011
Company Confidential
24
Registration Management Committee (RMC)
United Technologies Corp.
•
ASQR-01 – Supplier Quality System Requirements
(invokes requirements in addition to AS9100 as well as
other the ASQR specifications)
•
ASQR-07.5 – Software Requirements
•
ASQR-09.1 – Flight Safety Hardware
•
ASQR-15.1 – Handling, Storage, Packaging and Delivery
Boston, MA
July 21-22, 2011
Company Confidential
25
Registration Management Committee (RMC)
Spirit Aerospace
•Two major procurement sites
– Wichita and Tulsa
– Each site has a unique set of requirements.
– Suppliers have these documents available to them
through the Spirit portal (SPN), which is password
controlled.
Boston, MA
July 21-22, 2011
Company Confidential
26
Registration Management Committee (RMC)
Spirit Aerospace
•Spirit Wichita
– Has a set of forms known as Procurement Forms (PF)
» PF-550 has a general T&C that go to every supplier and is
applicable in total.
» PF-515 has product-unique requirements that might apply
regardless of customer.
» PF-515 with a suffix would indicate notes unique to a
specific Spirit customer or program.
– The purchasing document will detail which specific
notes are applicable to an order.
Boston, MA
July 21-22, 2011
Company Confidential
27
Registration Management Committee (RMC)
Spirit Aerospace
•Spirit Tulsa
– Has 127 total Flysheets
– Work much like the PF note sheets out of Wichita, but
each one is much more limited in scope
•Attached is a listing of Flysheets relative to quality
requirements, and a specific example, Flysheet 100.
Boston, MA
July 21-22, 2011
Company Confidential
28
Registration Management Committee (RMC)
Spirit Aerospace - Expectations
– Key for an auditor to ask for a copy of the Spirit
purchase document and review the general
requirements, which would be called out at the top
of each of these documents.
– Supplier should have an understanding of either the
PF Notesheets or Flysheets, have ready access to
the applicable documents and have a documented
review process or procedure that would ensure all
customer requirements have been reviewed.
Boston, MA
July 21-22, 2011
Company Confidential
29
Registration Management Committee (RMC)
Case Study Exercise
Shannon Craddock
Boston, MA
July 21-22, 2011
Company Confidential
30
Registration Management Committee (RMC)
• Thinking of a process-based audit that stresses a riskbased approach to auditing use the information provided
in the case study to compile the following:
– Cite the higher-risk ABC Aviation CSRs that you would
sample during your audit. Be prepared to provide
justification for your choices by listing the most critical
audit trail.
Boston, MA
July 21-22, 2011
Company Confidential
31
Registration Management Committee (RMC)
• Time for this activity:
– Working in your groups (20 minutes)
– Report out to the other teams (35 minutes)
Boston, MA
July 21-22, 2011
Company Confidential
32
Registration Management Committee (RMC)
Groups Report Out:
Boston, MA
July 21-22, 2011
Company Confidential
33
Registration Management Committee (RMC)
Questions & Answers
Thank You
Craddock, Shannon SCraddock@pjr.com
Miller, Chris chris.miller@eagleregistrations.com
Boston, MA
July 21-22, 2011
Company Confidential
34