Registration Management Committee
July 22-23, 2015
A. Michael McRandall
NSF-ISR
Company Confidential

Registration Management Committee
Mike McRandall
Business Unit Manager, Aerospace
NSF-ISR
Over 30 years experience in the aerospace industry in both quality and operations
RMC Workshop Charleston, SC July 22-23, 2015
2

Registration Management Committee
• Process Identification ( can’t measure ‘em if you don’t know ‘em)
• Process Effectiveness
– What are we measuring?
– Does it get us where we need to be?
– Why aren’t OTD & Quality adequate measures for all?
• Options When Organization Is Process Challenged
• Case Studies/Examples
• Open Discussion
RMC Workshop Charleston, SC July 22-23, 2015
3

Registration Management Committee
Not Very…
• We have been touting a “process-based approach” to auditing for over 15 years and we still have :
− Auditees with no idea of what a process actually is
− Auditors in the same boat
− Industry “experts” just as confused and exacerbating the problem further
Our Methods are Not Overly Effective…WHY?
RMC Workshop Charleston, SC July 22-23, 2015
4

Registration Management Committee
– Are processes properly identified?
– Is there a difference between an activity and a process?
– Are processes and procedures the same thing?
– Is every standard clause/element a process?
– Who identifies these processes?
RMC Workshop Charleston, SC July 22-23, 2015
5

Registration Management Committee
• Defined as*: “a set of interrelated or interacting activities which transforms inputs into outputs.”
These activities require allocation of resources
(people, materials, equipment, etc.)
• A process must have defined:
– Objectives
– Inputs
– Outputs
– Activities
– Resources
If you can’t describe what you’re doing as a process, you don’t know what you’re doing…
RMC Workshop Charleston, SC July 22-23, 2015
6

Registration Management Committee
• An Activity is a unit of work to be completed within the execution of a process
• Activities can be performed in isolation - but a process requires a group of activities in conjunction with resources to actually be, a process
– 9001:2008 Section 0.2 confirms: “…An activity or set of activities using resources, and managed in order to enable the transformation of inputs into outputs, can be considered a process.”
RMC Workshop Charleston, SC July 22-23, 2015
7

Registration Management Committee
These are not interchangeable terms and do not mean the same thing
Procedure*: A specified way to carry out an activity or a process
Though not as common place as in the recent past this is still seen too often within an organization’s documentation.
RMC Workshop Charleston, SC July 22-23, 2015
8

Registration Management Committee
● The organization, and only the organization, identifies (determines/defines) their
processes (s ee 4.1a of the standard)
− It is not the place of the CB
− It is not the place of the auditor
● The organization must also describe the interaction between their processes - commonly
referred to as the IOP (4.2.2c of the standard)
RMC Workshop Charleston, SC July 22-23, 2015
9

Registration Management Committee
Simple Answer: No
Remember what a process is and what it takes to fit the definition.
Also remember that an organization is required to monitor, measure and analyze their processes. So if they choose to call each element of the standard a process, then they need to be able to demonstrate how each element qualifies as a process and how those processes are monitored, measured and analyzed.
RMC Workshop Charleston, SC July 22-23, 2015
10

Registration Management Committee
● What is a KPI (Key Performance Indicator)?
– Well, it is a metric, not a target
 The target would be the objective. The KPIs show us how we are performing toward our objective(s) or targets.
● The objective should be directly related to the process being assessed
– It can, and should, tie into a more global business objective (OTD, quality, customer satisfaction, etc.)
– The KPIs should allow stakeholders (management, owners, auditors, etc) to assess the effectiveness of the process.
RMC Workshop Charleston, SC July 22-23, 2015
11

Registration Management Committee
● If the targets are either misplaced or disjointed from the overall company goals and objectives, the results might be, less than satisfactory.
● It is important to be able to show the tie-in between the process objectives (along with the
KPIs used to track them) and the overall company objectives because…things may not always be as they seem.
Or as a wise man once queried: Do outputs really matter?
RMC Workshop Charleston, SC July 22-23, 2015
12

Registration Management Committee
Let’s try a sports analogy. We’ll use the 2014
Colorado Rockies of MLB) as a shining example
RMC Workshop Charleston, SC July 22-23, 2015
13

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
14

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
15

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
16

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
17

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
18

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
19

Registration Management Committee
– Fourth overall in MLB, on base percentage
– Third overall in MLB, runs scored
– Second overall in MLB, batting average
– First overall in MLB, slugging percentage
RMC Workshop Charleston, SC July 22-23, 2015
20

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
21

Registration Management Committee
RMC Workshop Charleston, SC July 22-23, 2015
22

Registration Management Committee
Best Starting Pitcher…72 nd in MLB/ERA
RMC Workshop Charleston, SC July 22-23, 2015
23

Registration Management Committee
What’s In It For Me??
RMC Workshop Charleston, SC July 22-23, 2015
24

Registration Management Committee
What’s In It For Us??
RMC Workshop Charleston, SC July 22-23, 2015
25

Registration Management Committee
● They are required to be measured and addressed
(5.2) – they must be good KPIs for every process, right?
– They are a means of having the organization maintain focus on the customer
– In most cases, they do not represent impactful data with regard to individual process performance
– KPIs point to symptoms, whether good or bad
– OTD and Quality are not symptoms – they are results
We’ll touch more on this during our open forum/case studies session
RMC Workshop Charleston, SC July 22-23, 2015
26

Registration Management Committee
The Rockies supply us, again, with ammunition for this battle
● Company Goal – Make Playoffs (10 teams total)
– Objective – 88 Wins (think OTD/95%)
– Actual – 66 Wins (think OTD/70%)
Now making “Wins” your OTD equivalent and using Offense as one of your product realization processes, how accurate and adequate would it be to use Wins as your KPI for that process
(Offense)?
RMC Workshop Charleston, SC July 22-23, 2015
27

Registration Management Committee
Remember, The Rockies were:
● 4 th in MLB for OBP
● 3 rd in MLB for runs scored
● 2 nd in MLB for batting average
● 1 st in MLB for slugging percentage
But their KPI for this process (Wins), says that they failed, big time. So…what is the CA for this process?
RMC Workshop Charleston, SC July 22-23, 2015
28

Registration Management Committee
● They showed me their IOP and it doesn’t correspond with what they do…
– Start at 4.1a and work out from there
√ Have they determined the processes? (Y/N)
√ Have they determined the sequence and interaction? (Y/N)
√ If yes to both, but still no match – 4.2.2c*
● They don’t have a process map or chart…
– First of all, is this a problem?˚
√ If a narrative exists but does not meet the intent of the requirements then a discussion needs to take place to determine what element of the standard the finding will be written against.
RMC Workshop Charleston, SC July 22-23, 2015
29

Registration Management Committee
You are auditing the purchasing process of Ellenem Industries and they have identified Customer OTD as their method for determining process effectiveness. Their target is 97% and their actual OTD was
75%. They also use OTD as the only KPI for this process. During the course of your process audit you find that they monitor and measure many things relative to purchasing such as: supplier quality, ppv, supplier delivery to contract requirements, time to receipt of quote, inventory turns and flow down escapes – all of which are performing to internal expectations. You found no nonconformances to the standard during your audit of this process.
• Seeing as how they missed their target of 97% (by a considerable margin), what type of actions would you expect to see they have taken to address this ineffective process?
• What actions, if any, would you as the auditor take?
RMC Workshop Charleston, SC July 22-23, 2015
30

Registration Management Committee
You are auditing the manufacturing process of Alacrity Amalgamated, a mid-size company that does short run jig grinding to customer specifications. They have identified product scrap as their effectiveness measure, with a target of < 2% of total volume. Their actual scrap rate is 4.2%. They have identified 2 KPIs for this process and they are: 1) the number of findings generated during internal audits (less than 3 / meeting target) and 2) the capacity scheduling of each work center (to be at 80% / meeting target). There were three findings against conformity to the standard identified during this audit:1) Failed to validate tools prior to release to production
(7.5.1.3), 2) Undocumented changes to production processes
(7.5.1.2), 3) No evidence that operations completed as planned
(7.5.1h). Their response to missing their scrap target was to more closely analyze MRB minutes.
As an auditor, what, if any, actions would you take?
RMC Workshop Charleston, SC July 22-23, 2015
31

Registration Management Committee
● Process Identification is Key
● KPIs Should Point to Symptoms
● OTD and Quality Are Not Great Individual Process KPIs
● Challenge Organizations to Understand Their
Processes And to Identify KPIs to Monitor The Process
Thank you all for your time and patience
RMC Workshop Charleston, SC July 22-23, 2015
32