ANALYSIS OF FDA MEDICAL DEVICE RECALLS TIME BETWEEN RECALL
INITATION DATE AND TERMINATION DATE
____________
A Thesis
Presented
to the Faculty of
California State University, Dominguez Hills
____________
In Partial Fulfillment
of the Requirements for the Degree
Master of Science
in
Quality Assurance
____________
by
Teonta Pace
Spring 2020
THESIS:
ANALYSIS OF FDA MEDICAL DEVICE RECALLS: TIME
BETWEEN RECALL INITATION DATE AND TERMINATION
DATE
AUTHOR:
TEONTA PACE
APPROVED:
______________________________
Bob Mehta, MSQA
Thesis Committee Chair
______________________________
Rochelle Cook, Ph.D., MS, PMP, CPHQ, ASQ
CMBB
Committee Member
______________________________
Mary McShane-Vaughn, Ph.D.
Committee Member
Copyright by
TEONTA PACE
2020
All Rights Reserved
Dedicated to the memory of my late mother Gail and my father Clay, who taught me to believe
there is no glass ceiling for those of us who want to reach the top of each peak. They taught me
that the resolution to any problem involves research, careful planning, and flawless execution.
Each moment in every library, museum, and educational campus we’ve visited has led me to this
moment. Thank you for encouraging my ambition, fostering my work ethic, and supporting
every moment of courage in my life. Without you, I would not be the force that I’ve become, and
many of my successful days would not have been realized. I owe everything that I am to your
leadership, may I take that with me as I push forward in the rest of life’s endeavors.
ACKNOWLEDGEMENTS
I would like to acknowledge the wonderful people that supported my studies and the
completion of this thesis. To my sister, Brittnee Booker, for her continuous support, expertise,
and advice in guiding me through the program and the thesis. Thank you to my executive and
senior leadership in quality and compliance for providing me with the knowledge, resources, and
expertise in my area of focus. To my colleagues who always provided me valuable industry
insight during my studies and encouraged and pushed me to go forward despite any challenges
that surfaced during my road to completion. Lastly, I would like to thank my academic mentors
and professors, with a special thanks to Bob Mehta, Dr. Rochelle Cook, Dr. Mary McShaneVaughn, and Dr. Jim Clauson for their support, encouragement, and leadership during my studies
and thesis completion.
v
TABLE OF CONTENTS
ACKNOWLEDGEMENTS ............................................................................................................V
TABLE OF CONTENTS .............................................................................................................. VI
LIST OF TABLES ......................................................................................................................VIII
LIST OF FIGURES ...................................................................................................................... IX
ABSTRACT....................................................................................................................................X
1. INTRODUCTION .......................................................................................................................1
Background ..........................................................................................................................1
Statement of the Problem ...................................................................................................10
Purpose of Study and Significance ....................................................................................10
Theoretical Bases of the Study ..........................................................................................12
Scope and Limitations........................................................................................................13
Definition of Terms............................................................................................................14
2. LITERATURE REVIEW ..........................................................................................................19
Roles and Responsibilities: Recall Regulation and Guidance ...........................................19
FDA Enforcement Actions for Software Defects and Recalls ...........................................30
Recall Timing and Effectiveness Studies ..........................................................................34
Plan-Do-Study-Act (PDSA)...............................................................................................37
Summary of Literature Review ..........................................................................................38
3. METHODOLOGY ....................................................................................................................40
Design of the Investigation ................................................................................................40
Population of Sample .........................................................................................................41
Data Collection ..................................................................................................................42
Data Analysis .....................................................................................................................47
Summary ...........................................................................................................................57
4. RESULTS AND DISCUSSION ................................................................................................58
Software Related Recall Results ........................................................................................58
Summary of Key Characteristics for Outliers ....................................................................63
Act Phase ...........................................................................................................................67
5. CONCLUSION ..........................................................................................................................71
vi
Summary .............................................................................................................................71
Conclusions .........................................................................................................................71
Recommendations ...............................................................................................................72
REFERENCES ..............................................................................................................................74
APPENDIX A: FLOWCHART OR RECALL MANAGEMENT PROCESS..............................81
APPENDIX B: CODED SOFTWARE RELATED RECALL DATA SET ..................................82
APPENDIX C: DETERMINTATION OF DISTRIBUTION FITTING FOR TIME TO
CLASSIFCATION ........................................................................................................................93
APPENDIX D: BOX-PLOT RESULTS: OUTLIERS FOR TIME TO CLASSIFICATION .......95
APPENDIX E: DETERMINATION OF DISTRIBUTION FITTING FOR TIME
TO RECALL ..................................................................................................................................96
APPENDIX F: BOX-LOT RESULTS: OUTLIERS FOR TIME TO RECALL ...........................98
APPENDIX G: CAUSE-AND-EFFECT, FISHBONE DIAGRAM FOR ROOT CAUSE
ANALYSIS OF FACTORS LEADING TO RECALL INITIATION FOR SOFTWARE
RELATED DEFECTS .................................................................................................................100
APPENDIX H: CAUSE AND EFFECT, FISHBONE DIAGRAM FOR ROOT CAUSE
ANALYSIS OF FACTORS LEADING PROLONGED TIME TO RECALL
DURATION.................................................................................................................................101
APPENDIX I: PROPOSED FMEA FOR CONFIRMED ROOT CAUSES CONTRIBUTING TO
TIME TO RECALL DELAY, INCLUDING RISK MITIGATION
RECOMMENDATIONS .............................................................................................................102
vii
LIST OF TABLES
1. Communication Responsibilities for Recall Notification and Corrective Action
Implementation for Responsible Parties Under E.U. MDR 2017/745...............................28
2. Goodness of Fit Test for Distribution of Data for Time to Classification .........................60
3. Goodness of Fit Test for Distribution of Data for Time to Recall .....................................62
viii
LIST OF FIGURES
1. FDA Triggers that Lead to Organization Product Recall ....................................................6
2. Primary and Secondary Effects of Organization Initiation of Product Recalls ...................9
3. Flow Diagram Representing an Overview of Roles and Responsibilities During Recall
Lifecycle for Organization and FDA .................................................................................22
4. Recall Identification and Initiation Pathways for Immediate Recalls Demonstrating Step 1
of Uniform Recall Procedures ...........................................................................................27
5. Summary of Basic Requirements of Recall Process Combining Regulatory Requirements
of US (FDA), Australia (TGA), ISO, Europe (EU), and Canada (HC) .............................30
6. The Model for Improvement ..............................................................................................38
7. Pareto Chart Demonstrating Frequency of Recall Defects by Category ...........................44
8. Fishbone Diagram Template ..............................................................................................45
9. Example FMEA Data Table...............................................................................................47
10. Pie Chart Representing Recall Initiation Modes ...............................................................51
11. Bar Chart of Recall Initiation Pattern by Region ...............................................................52
12. Pareto Chart Displaying Recall Event Defect Category ....................................................54
13. Bar Chart of Software Related Recalls by Recall Initiation Method .................................55
14. Histogram of Distribution Patterns for Software Related Recall Events ...........................56
15. Anderson-Darling Normality Test for Time to Classification ...........................................58
16. Anderson-Darkly Normality Test for Time to Recall ........................................................59
17. Identifying Outliers with Box-Plot Graph for Time to Classification ...............................61
18. Box Plot for Time to Recall ...............................................................................................63
ix
ABSTRACT
In recent years, the number of medical device recalls for software defects has climbed
steadily, with Class II recalls accounting for the majority of total voluntary recalls. Prolonging
the time it takes between recall initiation and recall termination could cause significant risk of
harm to patients. By implementing the Plan-Do-Study-Act framework to examine medical device
recall data, this study utilized risk management tools to analyze risks and root causes associated
with prolonged recall times for software related recalls. Through identification of root cause and
risks, mitigation strategies are discussed to reduce time to recall. The results of the study indicate
that recall events with prolonged recall durations are linked to organizations that struggle with
recall management through regulatory compliance. Organizations can benefit from analyzing
industry data to identify trending areas of risk within similar systems to mitigate impact of noncompliant policies that lead to prolonged time to recall.
1
CHAPTER 1
INTRODUCTION
Background
Medical devices are becoming more complex with the advancement of technology
leading to many revolutionary products for people navigating the treatment or cure of disease.
The Food, Drugs, and Cosmetics (FD&C) Act defines medical devices as follows:
(1) an instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent,
or other similar or related article, including a component part or accessory which is:
recognized in the official National Formulary, or the United States Pharmacopoeia, or
any supplement to them, (2) intended for use in the diagnosis of disease or other
conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other
animals. or (3) intended to affect the structure or any function of the body of man or other
animals, and which does not achieve its primary intended purposes through chemical
action within or on the body of man or other animals and which is not dependent upon
being metabolized for the achievement of any of its primary intended purposes. (FDA,
2018)
Medical devices range in style from band-aids and contact lenses to complex surgical systems
and pacemakers. The FDA’s subdivision Center for Devices and Radiological Health (CDRH)
maintains a database of over 6,000 diverse types of medical devices that have been classified by
the agency. As technology has become more incorporated into design, medical devices often
combine hardware, software, and cybersecurity components into devices to provide innovative
solution to all facets of healthcare. Software has become an important part of all products,
2
integrated widely into digital platforms that serve both medical and non-medical purposes (FDA,
2018). There are three types of software related to medical devices:
(1)
Software that is intended to be used for at least one medical purpose, that can
perform these purposes without being part of a hardware medical device (FDA,
2018).
(2)
Software incorporated into a medical device that is an integral part of the medical
device (FDA, 2018).
(3)
Software that is used in the maintenance or manufacture of a medical device
(FDA, 2018).
Industry will continue to see an increase in the use of the diverse types of software driven
medical devices due to its range of use and diversity of design. Even regulatory agencies are
preparing to take on new technological advancements in medical devices. According to a speech
from the FDA’s Commissioner of Food & Drugs (Gottlieb, 2018), even the FDA is making
changes to the regulatory landscape for inclusion of software driven devices and digital tools.
Digital tools would include wearable devices, telemedicine, telehealth, personalized medicine,
health information technology, and mobile health applications. The emerging technology of
digital health products has created a new lane of innovation in the medical device space
combining software connectivity and consumer technology. Gottlieb highlighted the FDA’s
current thoughts on the status of the agency’s perspective of digital health tools and their power
to diagnose and treat diseases. In 2018, the FDA unveiled a new initiative for digital technology
applications, including the following:
(1) A policy to streamline FDA’s oversight of regulated components multi-function
digital health products.
(2) Industry Guidance that explains the regulatory approach and policy of the agency
towards digital health products and provides clarification for software functions that
are within scope of the FDA’s authority and those functions which are not. Functions
that are within scope are treated as medical devices.
3
(3) Enhancements to the FDA’s Pre-Certification Pilot programs initiated in 2017. The
pilot program allows organization to register digital health tools for FDA’s oversight
to ensure the safety and effectiveness, and performance of health software. Digital
tools registered within the pilot will be treated as medical devices.
The FDA’s oversight of software also includes changing the focus to utilize digital health
technology as part of drug development recognizing the benefits and opportunities to improve
the effectiveness and safety of drug delivery systems. The agency’s goal is to create policies that
will ensure that software driven technologies can be validated once incorporated into drug
development systems (Gottlieb, 2018). The benefit of combining software with drug
development is related to increasing the effectiveness of patient care. Technology provides the
patient and provider an additional resource to help confirm that patients are taking medication,
tracking and monitoring side effects, and ensuring relevant data is included within the electronic
health record. The program updates from the agency highlights the necessity of ensuring
adequate evaluation and analysis of software related industry trends, such as recalls, to
understand the scope of digital expansion, and the potential risks exposed to patients when
medical device software fails during performance. Medical device recalls protect public health
by removing the defective and potentially harmful software or medical device from the market.
Current State of Industry Recalls
A recent analysis published by Stericycle in Q1 2018 reported that the number of medical
device recalls in United States increased 126% from the previous quarter and involved more than
208 million units (Stericycle, 2018). According to a review of Stericycle Recall Index, reports
conducted by the Association for the Advancement of Medical Instrumentation (AAMI),
software recalls topped the recall charts for eight consecutive quarters prior to the issuance of the
2018 analysis (AAMI, 2018). The same report form, Stericycle, in Quarter 2, 2019 showed
consistency, and reports that medical devices continue to lead as the top cause of industry recalls
4
at 49 recalls, followed by 32 for mislabeling, 31 for quality issues, and 18 for out of specification
(Stericycle Expert Solutions, 2019).
The Stericycle Index provides a quarterly industry analysis on recall trends according to
their published reports. Software has been listed as the top reason for medical device recalls
since 2016. Software defects in the medical devices are not rare and have the potential to
negatively influence medical care (Ronquillo & Zuckerman, 2017). An increase in software
related recalls can reveal that patients could be at a higher risk from medical errors caused by
software defects. The analysis additionally reported that Class I (serious risk of harm or death)
recalls are on the rise for medical devices, but Class II (moderate risk of harm) recalls remain
consistent as the most abundant classification of medical device recalls reported to FDA.
Medical device firms are responsible for ensuring that the organization is meeting
regulatory requirements for a developed recall process. Such a process considers the severity of
impact to patient safety when a defective product stays on-market. Products can be recalled when
it is found to be defective, either when it falls out of design specification or fails to perform as
intended.
FDA Regulation and Enforcement
The FDA governs medical device recalls under the Federal Code of Regulations, Title 21
Part 806 published in the Federal Register. Under 21 CFR Part 806 entitled medical devices;
Reports of Corrections and Removals, firms can follow the provisions to promptly report actions
pertaining to recalls of on-market products. The regulation includes provisions for corrective
actions, market withdrawals, routine servicing, and stock recovery as identified by the United
States (US) (FDA, 2018). Under Part B of the regulation, Section 806.10 provides the
requirements for reporting and records in detail. Each manufacturer or importer shall submit a
5
written report to FDA of any correction or removal of a device initiated by such manufacturer or
importer if the correction or removal was initiated (FDA, 2018). This action is completed for two
primary reasons 1) to alert the FDA of a potential safety hazard to consumers, and 2) to allow for
the firm to remedy the nonconformity that causes the health risk to the patient either through
correction or removal. The regulation describes the required information that should be included
within the report including:
•
Manufacturer information;
•
Product brand and classification;
•
Marketing status;
•
Unique Device Identifier;
•
Description of event;
•
Reporting of any known illnesses or adverse reactions related to device use.
Recalls can be initiated voluntarily or involuntarily and notified to the FDA depending on
the scope and severity of the recall and compliance to the regulations. Due to increased
enforcement by the FDA, many medical device recalls are initiated voluntarily by the
manufacturing organization. Overall, the FDA’s focus on industry correction and removal
compliance has contributed to a 50% increase in the annual number of voluntary recalls reported
since 2009 (CDRH, 2018). This can be attributed to a new FDA focus on identifying reporting
deficiencies for medical device under 21 CFR Part 806 during routine compliance inspections.
According to a recent enforcement report published by the CDRH in November 2018,
firms are 8 times more likely to report a recall in the year after inspection, than the entire average
for industry (CDRH, 2018). The increasing trend can serve as a safety net for patients and
consumers as additional security allows monitoring and enforcement more visibility to hazardous
6
defects. A recall due to safety hazard occurs because one of three main triggers (Wood, Wang,
Olesen, & Reiners, 2017). The first trigger happens when the firm detects a safety hazard after
the discovery of a product defect. The second trigger stems from the Customer Feedback loop
when patients or Health Care Providers inform the firm of a hazardous product. The third trigger
stems from patients, competitors, or other consumers report a potentially hazardous product to
the regulatory agency. Figure 1 illustrates these 3 FDA triggers that lead to organizational
product recall. This prompts investigation by the firm and the agency to investigate the cause of
the defect and initiate actions to eliminate the hazard or recover the product.
Customer
Feedback
Safety
Hazard
Report of
Hazard to
FDA
Recall Initiation
Figure 1. FDA triggers that lead to organizational product recall. Figure created by the author of
this thesis.
Types of Recalls
Recalls occur when the product is found to be defective causing harm to patient safety or
a deterioration to product quality and performance. The FDA uses the term recall when a
manufacturer takes a correction or a removal action to address a problem with a medical device
that violates FDA law. Recalls occur when a medical device is defective, when it could be a risk
7
to health, or when it is both defective and a risk to health (FDA, 2018). Organizations have the
option to perform a correction, by addressing the defect of the medical device, in the place the
device is being sold. In instances of correction, the medical device may be returned to use once
the devices have been assessed and fixed for known issues. The risks associated with use should
be determined by the organization and communicated to customers. An alternative option is
removing the product from the market where the medical device is sold. During removals, it is
important to ensure that the defective device is fully recovered from the market to prevent any
harm to patients by eliminating the opportunity for exposure.
The FDA classifies recalls based on the impact to patient health and safety, there are
three classifications for FDA recalls denoted as follows:
•
Class I (serious threat of harm)
•
Class II (moderate risk of harm)
•
Class III (unlikely risk of harm)
Class I recalls are categorized based on the risk of serious adverse health consequences or
even death upon exposure to the defective product. Class II recalls are initiated when a product
may cause a reversible or temporary adverse health event, or there is a slight chance that
exposure could cause serious injury or death of a patient. Class III recalls are initiated when the
defective product is not likely to cause any serious health concerns or death of a patient. Once
the FDA has decided on the classification of the recall, the agency monitors the recall progress to
ensure that the appropriate action plans are in place to ensure effectiveness of product correction
or removal (FDA, 2014).
8
Primary and Secondary Effects of Recall Impact
The primary and most important effect of recall is its immediate impact to patient safety.
Patient safety from the regulatory and firm perspective is to always protect the patient and reduce
the likelihood and exposure to hazard and harm. Design and manufacturing controls build in
hazard detection and harm reduction while product is still under the control of the organization.
Following distribution, recall protocols should be designed and executed to keep the patient from
using known defective products and cause an adverse reaction or health hazard. Recalls triggered
by safety concerns imply that the device poses a considerable risk to the patient. The recall
announcement can serve as a benefit to patient because it advises discontinuation of the recalled
device. For example, for an implantable device, if a recall is initiated it could mean that the
patient could be subject to removal or re-insertion which could require surgery. Delays in
recalling critical to care could cost a patient their life or cause significant injury. The primary
impact is a positive benefit for patients and regulators who become aware of a potentially
hazardous situation.
A secondary effect of recall follows the public announcement and notification to
consumers. Recall announcements can negatively impact patient and customer confidence and
brand image. In the age of social media, news can go viral within an instant, compromising the
public image of an organization. Recalls are becoming an increasingly common notification and
may reduce public confidence in the reputation of manufacturers (Wood et al., 2017).
Manufacturers of healthcare products must be in tune with how the effectiveness of the recall
program impacts public perception on the ability of an organization to ensure its products are
safe on-market. Figure 2 illustrates the relationship of primary and secondary recalls and their
impact on recall initiation. Previous research discusses the reasons behind why a product should
9
be recalled. However, the recent spate of product recalls has shifted attention from why products
are recalled to why is takes so long to recall a defective product that poses a potential safety
hazard (Hora, Bapuji, & Roth, 2011).
Recall Initiation Effects
Primary
Ensures patient
safety
Reduces exposure of
risk to harm
Secondary
Patient and
consumer
confidence
decreases
Figure 2. Primary and secondary effects of organization initiation of product recalls. Figure
created by the author of this thesis.
Time to Recall
One component of the recall process that has a significant impact to recall effectiveness
and patient safety is the time to recall. Time to recall occurs in three different reference points in
the recall process as follows:
•
Time between date of product launch, date first sold, and date of the recall
announcement.
•
Time between issuing the recall following notification of product hazards by patients.
•
Time between the announcement of recall to the recovery of recalled product from
on-market distributors and patients.
The time it takes for firms to recall their products depends on the following factors: 1)
their recall strategy (proactive vs. reactive); 2) type of defect (manufacturing defect vs. design
flaw); and 3) the recalling firm (manufacturer, distributor, retailer) (Wowak & Boone, 2015).
10
This data supports the thinking that more complex medical devices, complex distribution, and
supply chains will have a larger gap in recall timeliness.
Statement of the Problem
The medical device industry has seen a consistent increase in software related recalls in
recent years according to the Stericycle Recall Index (Stericycle, 2018). By the first quarter,
2018 software related recalls accounted for 23% of all recalls and have been the leading factor in
medical device recalls every quarter since 2016. According to Fu, Guo, Zhang, Jiang, and Sha
(2017), one out of every three software based medical devices has been recalled due to defects
and errors within the software itself. Software defects or coding errors could be difficult to detect
prior to distribution, and thus more likely to cause harm to patient when they occur. Software
defects can become critical for medical devices whose operational functionality relies on the
effectiveness and adequacy of the software component.
Once aware of a software defect, it becomes critical to patient safety to quickly cease
distribution and work to recover the product from the market. Prolonging the time, it takes
between recall initiation and recall termination and product recovery can lead to a higher risk of
harm to patient. The timeliness of recall recovery from initiation to termination will have a
significant impact to patient safety. When not adequately controlled, the following three factors
will have a negative impact on the timeliness of recall recovery potentially taking more time to
fully recover on-market products (1) product traceability, (2) recall protocols, and (3) distribution
pattern.
Purpose of Study and Significance
The purpose of this study is to examine industry data to support current literature and
industry thinking on the importance of recall effectiveness, timeliness, and its impact to patient
11
safety and product quality. This thesis will include an evaluation of the current industry state for
medical device recalls and provide recommendations and considerations for the implementation
of recall management systems based on these trends.
The study of recall management has implications for manufacturers and their supply
chain partners to ensure that they are following the regulatory requirements. A quality culture
that includes full transparency and traceability within the supply chain is important to ensuring
the success of multifaceted recalls even within the most complex systems. This thesis aims at
using industry data to benchmark best strategies supported by literature on reduction of time to
recall, and its success for an organization.
The focus on software related medical device recalls has positive benefits to data trending
and retrospective learning in industry. Many medical devices rely heavily on software systems as
a critical component of functionality. Additionally, there are software systems that standalone as
medical devices of their own without hardware systems. The tech-trends in industry have created
more options for technology driven data applications, wearable and wireless technology, and
smart medical devices. Quality professionals, senior leaders, and regulators must understand the
importance of recognizing areas of risks to better understand how to investigate industry trends
and recommend quality solutions to industry that address risk to device design, technology in
manufacturing and distribution, and patient safety.
Additionally, time to recall effectiveness has not been significantly studied for medical
devices. Considering the implications medical device hazards have on patient safety,
understanding the parameters around recall recovery is important to ensure adequate controls are
in place. Time to recall metrics could demonstrate the impact and the ability to retrieve products
from consumers to eliminate risk of harm. Organizations could use data for continuous
12
improvement opportunities in the recall process, but also within supply chain traceability
practices. This study’s focus highlights the potential for process improvements, industry
benchmarking, and reinforcing the continuous improvement lifecycle for software-related
medical devices. This study aims to resolve inquiries into reducing time to recall duration,
measuring effectiveness of recall strategy using data trends, and evaluate areas of risks and
mitigation strategies to incorporate into a standard recall process.
Theoretical Basis and Organization
This study will conduct a retrospective analysis of industry data to provide clear
recommendations to industry for strategies to reduce recall timeliness to reduce risk of harm to
patient. Analysis of data and information involves multiple phases of planning, execution,
evaluation, and conclusion to draw relevant perspective and encourage risk-based decision
making. The Plan-Do-Check-Act (PDSA) theoretical framework allows for a continuous cycle of
learning and improvement through multi-phase-based implementation. W. Edwards Deming
(1982), in the book Out of The Crisis, introduced industry to the concept of process
improvements occurring within a continuous cycle of learning.
PDSA is a four-phase theoretical framework towards quality improvement that is used to
identify a problem and plan a resolution, check the resolution in a small population, evaluate the
results, and implement the resolution into the system. For this thesis, PDSA represents a process
improvement framework that has been widely used to support organization learning from quality
improvement processes. The PDSA framework can be used to repeatedly build on opportunities
for improvement within a quality management system and will guide the identification and
analysis of data to present findings for areas of improvement within the recall process:
13
•
Plan phase. Identify gaps within the recall execution protocol that would identify
areas of improvement for reduction of time to recall duration for software related
recalls. Develop a plan for mitigation of risks associated with identified gaps.
•
Do phase. Carry out data analysis on recall data and develop the risk mitigation plan
for improvement for reduction of time to recall duration for software related recalls.
•
Study phase. Evaluate of the results by understanding what can be learned from the
data analysis. Understand how risk mitigation strategies are supported by the results
and literature.
•
Act phase. Decide if the results lead to adopting risk mitigation strategies and
updating protocols for recall execution if the results of the study phase are in
alignment with the goal of the plan phase. If the results are not successful, this phase
will consider additional cycling through the PDSA loop to gather new insights and
learning.
This thesis explores the use of the PDSA learning cycle in identifying risk mitigation
pathways to reduce the time between FDA recall initiation and product recovery by examining
potential risk factors associated with delays in total time to recall. Risk mitigation and process
mapping strategies are explored to take advantage of the FDA data sets to shorten the time
between (1) FDA recall initiation and (2) FDA recall termination and product recovery to reduce
opportunities for risk to patients. Additionally, this thesis will use the results of data analysis and
risk mitigation, supported by the literature to propose a protocol for recall execution, and data
trending based on the evaluation of recall triggers found using common cause codes within the
data set.
Scope and Limitations
For the purpose of this thesis, the study was limited to Class II medical device recalls
reported to the FDA between 2016 through the end of 2018. Additionally, the FDA enforcement
report data only includes recalls that have been determined by FDA as classified recalls and is
updated as recalls evaluated to meet requirements. The FDA publishes recalls as information is
classified and made available, but the number of recalls featured in these lists differ significantly
14
from the numbers shown in FDA's medical device recalls database, as some Class I recalls are
not included in the annual lists and some of the entries on the list account for more than one
database entry when the recall affects multiple models made by a company (Mezher, 2018). This
duplication or lack of addition of data will not account for 100% of the recall population within
the reporting period but will include all data that the FDA has made publicly available.
The data set will be even further restricted following categorical coding to identify and analyze
records specific to software related recalls. The study will additionally restrict the scope of
distribution patterns reported in the enforcement report in order to analyze larger subgroups such
as national and international to account for multiple variables that could have an impact on the
measurable metrics. Recalls without termination dates will not be analyzed within the context of
this study, as open recalls are out of the scope of time to recall calculations. Additionally,
following data analysis, the selection quality tools utilized, and risk mitigation strategies will
serve as a recommendation for recall management supported by current literature.
Definition of Terms
Association for the Advancement of Medical Instrumentation (AAMI): a non-profit organization,
devoted to the management, development, safety, and effectiveness of health technology. AAMI
is a source of medical device industry knowledge, and support for healthcare industry
professionals (AAMI, 2019).
Authorized Representatives: In the context of the European Medical Device Regulation (E.U.
MDR), any natural or legal person established within the Union who has received and accepted a
written mandate from a manufacturer, located outside the Union, to act on the manufacturer's
behalf in relation to specified tasks with regard to the latter's obligations under the Medical
Device Regulation 2017/745 (European Parliament, 2017).
15
Center for Devices and Radiological Health (CDRH): a subdivision of the FDA, responsible for
protecting public health, by facilitating medical device innovation through enhancements to
regulatory science. The CDRH provides industry with guidance for regulatory compliance to
ensure the safety and effectiveness of medical devices (FDA, 2019).
Class I Recall: a situation in which there is a reasonable probability that the use of or exposure to
a violative product will cause serious adverse health consequences or death (FDA, 2017).
Class II Recall: a situation in which use of or exposure to a violative product may cause
temporary or medically reversible adverse health consequences or where the probability of
serious adverse health consequences is remote (FDA, 2017).
Class III Recall: a situation in which use of or exposure to a violative product is not likely to
cause adverse health consequences (FDA, 2017).
Code of Federal Regulations (CFR): is an annual codification of the rules, both general and
permanent, published in the Federal Register by the agencies of the federal government and their
executive departments (National Archives and Records Administration, 2019).
Digital Health Tool: A scope of devices and technologies designed to better manage and track
health and wellness related activities to improve patient health (FDA, 2019).
Distributor: Organization responsible for distribution of medical devices from the manufacturer
to consumers and/or consumer markets.
Distributors: In the context of the E.U. MDR, any natural or legal person in the supply chain,
other than the manufacturer or the importer, that makes a device available on the market, up until
the point of putting into service (European Parliament, 2017).
Economic Operators: In the context of the E.U. MDR, means a manufacturer, an authorized
representative, an importer, a distributor (European Parliament, 2017).
16
Effectiveness Check: During the effectiveness check, the FDA works to evaluate if all reasonable
efforts that have been made to remove a product from the market. Once the recall is complete,
the FDA will also investigate the reason a product was found defective and the adequacy of
product destruction or recondition (FDA, 2010).
Enforcement Report: A library of medical device recalls monitored and classified by the FDA
Firm: used interchangeably with Manufacturer, an organization responsible for producing,
packing, and shipping medical devices.
Food and Drug Administration (FDA): the federal agency that has the authority to enforce
regulation for Medical Devices, Cosmetics, Food, Drugs, and Animal Products.
Harm: hurt or mischief, physical damage or mental damage caused to a person.
Hazard: a source of danger, or an effect of harm.
Health Canada (HC): Health Canada is the regulatory body that governs and enforces regulatory
requirements in Canada.
Importers: In the context of the E.U. MDR, any natural or legal person established within the
Union that places a device from a third country on the Union market (European Parliament,
2017).
International Organization for Standardization (ISO): is an international organization that serves
as a global network of national standard bodies. ISO is responsible for developing and publishing
international standards for a variety of industries including medical devices (ISO, 2019).
Outside of the United States (oUS): oUS is used within the context of this thesis to describe
geographical locations that are outside of U.S. mainland and individual territories such as Puerto
Rico, Guam, Virgin Islands, Hawaii, and any other U.S. territories. The acronym oUS
categorizes all international locations outside of the United States.
17
Recall Enterprise System (RES): An electronic system utilized by the FDA for the management
of recall data. Responsible personnel use the RES to submit and update information about recall
classification, and status.
Recall Initiation: A recall is initiated when the organization starts to notify the public or their
direct accounts of the recall (FDA, 2017).
Recall Termination: A recall is terminated when the FDA determines that all reasonable efforts
have been made to recall or correct the product to the degree of the hazard, and that all corrective
actions were deemed appropriate. Recall termination is requested by the organization with
objective evidence submitted to the FDA (FDA, 2017).
Serious Injury: means an injury or illness that: (1) is life threatening, (2) results in permanent
impairment of a body function or permanent damage to a body structure, or (3) necessitates
medical or surgical intervention to preclude permanent impairment of a body function or
permanent damage to a body structure. Permanent means irreversible impairment or damage to a
body or to a body structure or function, excluding trivial impairment or damage (FDA, 2018).
Therapeutic Good: The Therapeutic Good Administration defines a Therapeutic Good as a broad
range of products for use in humans in connection with: (1) preventing, diagnosing, curing or
alleviating a disease, ailment, defect or injury; (2) influencing inhibiting or modifying a
physiological process; (3) testing the susceptibility of persons to a disease or ailment
influencing, controlling or preventing conception; (4) testing for pregnancy (TGA, 2017).
Therapeutic Goods Administration (TGA): TGA is the regulatory body that governs therapeutic
goods in Australia.
18
Unique Device Identification (UDI): a coding system used to ensure traceability and adequate
identification of medical devices. It includes the assignment of a Unique Device Identifier to
each device in order to improve safety and post market surveillance (FDA, 2019).
Wearable Technology: Wearable technology are multi-function consumer devices designed to
track and provide data related to health and fitness. Wearable technology can include mobile
application as a component of usability. The devices may contain motion sensors, monitors, and
GPS.
19
CHAPTER 2
LITERATURE REVIEW
The literature review presented the opportunity to develop a clear and concise
understanding of applicable regulatory requirements, guidelines for industry, and industry
expectations of the roles and responsibility of the organization during the recall process. Current
regulatory requirements and policies are used for the identification of key elements of a
compliant recall process and assist with general understanding of a procedural baseline. The
review of FDA enforcement actions provided context in connection to understanding
vulnerabilities that exist in software driven medical devices and their impact to risk of harm.
Sources reviewed included well-established and emerging regulations as well as guidance
documents, international standards, and enforcement actions that signify the importance of recall
controls and the need continuous improvements.
Roles and Responsibilities: Recall Regulation
and Guidance
Ensuring the safety and effectiveness of medical devices is vital to human health, and the
medical devices must be managed by strict regulations according to their assigned levels of risk.
Current industry perspectives have changed the regulatory landscape of the organizations
develop and distribute medical devices around the globe and medical device regulatory reform
has been promoted by global harmonization and streamlining (Chen et al., 2018). Relevant
international organizations have been established such as the International Medical Device
Regulators Forum (IMDRF). Member countries of the IMDRF include Australia, Brazil, Canada,
China, the European Union, Japan, Russia, Singapore, and the US.
20
Currently the IMDRF has discussed several topics for Harmonization including UDI
implementation for traceability, and guidance for medical devices with software (Chen et al.,
2018). Global harmonization aims to reduce regulatory differences worldwide, eliminate
immoderate or country-specific requirements, and build up a consistent and transparent
international regulatory management system (Chen et al., 2018). Given the global context of the
supply chain, many organizations distribute medical devices on a global scale, impacting the
scope of regulatory responsibility for recalls. This literature review provides a comprehensive
overview of regulatory requirements within key global regions to provide insights on the
similarities and differences between regulatory enforcement impacting the recall process for
global organizations.
The quality system regulation that governs and outlines industry responsibilities for
recalls is managed under 21 CFR Part 7 Enforcement Policy (FDA, 2019). The regulation is
divided into five subparts labeled A-E; subpart A covering the scope of the regulation and
relevant definitions, subpart B and D, which are reserved for text, and subpart C covering recalls,
subpart E covering criminal violations. Subpart C provides detailed requirements for recall
policy, health hazard evaluation and recall classification, recall strategy, recall communications,
notifications, status reports, and termination of recall (FDA, 2019). Subpart C also includes
provisions for roles and responsibilities of the FDA and the organization during the recall
process from initiation to termination. Industry guidance has been written and published by the
FDA to provide additional support to organizational understanding of the FDA’s current
expectations and interpretation of 21 CFR Part 7.
According to the current regulation, the organization, either medical device manufacturer
or distributor may initiate a voluntary recall at any time by issuing a communication about the
21
voluntary recall directly to consumers or to the public. When an organization initiates a recall,
they notify the FDA and submit all information as required by regulation. The organization may
notify the of a recall FDA by phone, letter, email, or a combination of communication practices
when the recall is voluntary (FDA, 2019). Following initial notice to the FDA, the organization
is responsible for the following elements of the recall as show in Figure 3.
1) Create a recall strategy detailing the course of action that needs to be taken to
properly conduct a recall including the scope of the recall, issuance of public
notification, and measuring effectiveness checks during the recall. The strategy also
includes a risk assessment of the health hazard. Recall root cause analysis must be
conducted and communicated to FDA.
2) Supply information to the users identify the product and include information for users
to minimize the consequences of exposure.
3) Ensuring that establishes producers for implementing corrective actions are in place
and effective to aid leaders and responsible parties in ensuring that corrective and
preventive measures are in place.
4) Submit status reports to the FDA District office on the progress of the progress of the
recall. Once actions and implementation are complete, the organization must submit a
recommendation for termination of recall. Status reports will discontinue at FDA
recall termination.
22
Organization
Process begins.
FDA
Notifies
consumers
and direct
accounts
Organization notifies FDA of voluntary recall
Initiate
Recall in
RES
Creates
Recall
Strategy
Organization submits recall strategy to FDA for review
Review
Recall
Strategy
T1
Time to Recall
Corrective
Action &
status
reports
Continuous monitoring cycle
Organization requests recall termination
Monitoring
& Audit of
Recall
Termination
T2
Process ends.
Figure 3. Flow diagram representing an overview of roles and responsibilities during recall
lifecycle for organization and FDA. Figure created by the author of this thesis.
Once the FDA receives notice of recall initiation, the FDA is responsible for the
following various aspects of the recall process as defined in 21 CFR Part 7 Subpart C. FDA roles
and responsibilities displayed in Figure 3 are defined as followed:
1) Initiation of the recall to the Recall Enterprise System within 24 hours of notification.
2) Determine if the action is a recall by determining if the action is in alignment with the
definition of a recall per regulation.
3) Review the organization’s recall strategy, classifies recall, and assesses the health
hazard posed by the product defect or non-conformity.
23
4) Following review of the firm’s recall strategy, the FDA issues a public announcement
of the recall and notifies the firm of the classification of the recall (i.e., Class I, II, or
III).
5) The FDA continuously monitors the recall and develops a recall audit strategy to
evaluate the effectiveness of the recall strategy.
6) Recall termination is assigned when the FDA determines that all reasonable efforts
were made to correct or remove the defective product according to the recall strategy
and that all defective product has been removed with appropriate disposition.
The regulation set by the FDA CFR provides a basic template for the required elements
of a recall process that meets the requirements of the FDA regulation. Given the global context
of product movement and traceability in today’s current context, many organizations develop
recall programs that encompass traceability exercises that extend outside of the FDA’s
jurisdiction. It is important that global organizations consider international standards and
regulations to develop a comprehensive recall program that can meet the requirements of several
regulatory authorities. Additional review into the requirements of other regulatory agencies
provide similarities to the FDA requirements of recall program management.
The International Organization for Standardization (ISO) published the latest revision of
standards quality management systems (QMS) for medical devices in 2016. ISO 13485:2016 all
requirements for the management of the QMS including requirements for management,
measurement and analysis, product conformity, traceability, product realization, and
nonconformances. As part of its comprehensive structure, ISO 13485:2016 covers requirements
for recalls under the definition of advisory notices and adverse events (ISO, 2016). Per the
standard definition, advisory notices are issued by the organization after the delivery of the
medical device and includes the return of medical devices to the organization or the destruction
of the medical device.
24
Within the standard, clauses 7.2.3 and 8.2.3 outline that the organization should have a
plan and documented procedure to notify the agency of adverse events and communicate
advisory notices to regulatory authorities. Additionally, clause 8.3.3 requires an organization to
implement a procedure for reporting adverse events and issuance of advisory notices at any time
as an action in response to nonconforming products. In summary, ISO provides standards for
requiring organizations to establish a procedure for the notification and communication of
adverse events to regulatory authorities, maintenance of records, traceability, and corrective and
preventive actions.
The Canadian regulatory authority Health Canada (HC) provides regulation to industry
for the establishment of a recall process under the Canadian Medical Device Regulation
(CMDR), specifically SOR/98-282 Food and Drugs Act (Government of Canada, 2019). The
regulation includes general requirements for all quality system activities, but specifics pertaining
to recalls includes provisions for distribution records and establishing a recall procedure
outlining steps for how to recall product. Under the Canadian regulation, section 63 to 65.1 cover
the specifics for the recall policy, requiring manufactures and importers are required to report
recalls to the regulatory authority.
According to the guidance document published by HC to assist in the interpretation of
SOR/98-282, the recall process for HC is divided into five stages, recall initiation, recall strategy,
notification and correction, follow-up, review and recall closure. Similar to the FDA’s
requirements, following initiation of a product recall HC requires an evaluation of risk, timelines
of action plan execution, recall communications, effectiveness checks, and notification to the
authority of recall completion and request to closure (HC, 2016).
25
Therapeutic Goods Administration, the regulatory authority in Australia, develops and
regulates policies for quality system management under the Therapeutic Goods Act 1989. The
regulation covers provisions for medical device recalls under chapter 4, parts 4-9. When a
product does not comply with requirements and cannot be lawfully supplied, the Secretary can
request the initiation to recall. The regulation details provisions for recall initiation, public
notification, product recovery, notification to the regulatory authority, and criminal penalties for
violations to the act. In order to support industry understanding of the regulation, TGA published
a Uniform Recall Procedure (URPTG), to provide a consistent approach for undertaking recall
actions of therapeutic goods that have been made available to market through import into or
export from Australia (TGA, 2019). The URPTG outlines the action steps to complete a recall
outlining the requirements for immediate versus all other recalls.
For immediate recalls of medical devices, Figure 4 represents the high-level process flow
of recall initiation for immediate recalls under TGA authority. Immediate recalls are listed as
Step 1 of the recall process because it is important to contact the Australian Recall Coordinator
and customers immediately. The flow chart represents the order in which all concerned parties
need to be contacted. The process starts at the highest level by identifying the type recall,
denoted by the blue fields represented in Figure 4. Following identification of recall type, the
process continues within Step 1 notifying customers and contact the Australian Recall
coordinator of the recall event. Both forms of communication may occur simultaneously or in
any order. Once appropriate contacts are notified, the recall strategy must be implemented and
agreed to by relevant stakeholders. Following completion of this step, identified in the URPTG
as Step 1, the recall process continues with Step 2 to Step 11. Step 2 is the initiation point for all
26
other recalls and flows through the closure of the recall process. The TGA URPTG lists the
remainder of the recall process as follows:
•
Obtaining stock status and distribution status.
•
Conducting a risk analysis.
•
Deciding the type, class, and level of recall.
•
Developing a recall strategy.
•
Drafting a communication strategy.
•
Submitting recall information to TGA.
•
TGA Assessment of the recall, to provide advice and assistance in relation to letters,
consumer recall notices, and recall strategy.
•
Implementing the recall.
•
Reporting on the recall via progress and status reports, including root cause analysis
and corrective and preventive actions.
•
TGA will review the recall for verification of completion comparing action plans to
objective evidence, determining satisfaction of recall implementation, and assesses
the effectiveness of the recall.
27
Figure 4. Recall identification and initiation pathways for immediate recalls demonstrating Step
1 of Uniform Recall Procedure. Adapted from TGA. (2019, Feb 7). Uniform recall procedure for
therapeutic goods (URPTG). Retrieved from Therapeutic Goods Act:
https://www.tga.gov.au/publication/uniform-recall-procedure-therapeutic-goods-urptg
The European Union recently published a new revision to the old Medical Device
Directive in late 2017, now moving medical device regulation from directive to law under the
new Medical Device Regulation (MDR) 2017/745. The massive overhaul of the former
requirements has new implication for importers, distributors, and economic operators for roles
and responsibilities to carry out a medical device recall. Under MDR 2017/745, a recall is any
measure undertaken with the aim to achieve the return of a device that has already been made
available to the end user (European Parliament, 2017).
Medical device recall regulation is covered in Articles 94 and 95 of the regulation.
Manufacturers of the medical device hold the responsibility to inform distributors, authorized
representatives, and importers of a product recall once the product is found out of conformity. In
situations where the product poses serious risk to health, manufacturers must immediately inform
the competent authority of the non-compliance and any corrective action that is to be taken.
28
Importers are responsible for maintaining a list of product recalls that have been initiated, and
providing manufacturers, distributors, and authorized representatives of any product information
requested of them.
Table 1 illustrates the responsibilities for each concerned party, either manufacturer,
distributor, or importer holds the responsibly of notifying the competent authority of recall
initiation if a nonconformity poses health risks to patients. Additionally, each party is responsible
for contributing to the execution and closure of corrective actions associated with medical device
recalls.
Table 1
Communication responsibilities for recall notification and corrective action implementation for
responsible parties under E.U. MDR 2017/745
Action in Recall Process
Manufacturer
Initiate recall
X
Notification to Competent
Authority of hazard product
X
Competent
Authority
Importer
Distributor
X
X
X
X
X
X
X
X
Notification to manufacturer
Notification to Authorized
Representative
X
Notification to Importer
X
Notification to Distributor
X
Notification to notified body
X
X
Corrective Action planning
and implementation
X
X
Restrict or withdraw product
from market
Maintain register of all
recalls
Note. Table created bt the author of this thesis.
X
X
X
X
X
29
There are key similarities between global regulations and standards in terms of the
requirements and elements of a basic recall process. As many organizations are operating on a
global scale, it is necessary to develop a truly comprehensive recall process that broadens the
reach to include critical components of the regulatory authorities governing the markets of their
distribution channels. It becomes clear that the regulators see the recall process as an opportunity
to set-standards in resolving nonconforming product defects. Notification to the regulatory is a
key step in ensuring public safety and having support in adequately addressing recall strategy
execution.
Figure 5 provides a summary of expectations for a basic recall process based on the
compilation of regulatory requirements from the US, Australia, Canada, European Union, and
ISO. The basic understanding of the recall process will be relevant to this study to provide an
understanding of the baseline for recalls. Using this baseline, this study provides analysis for
opportunities for improvements where gaps in standard understanding based on basic
requirements exist.
30
•Issue a notification to direct accounts, consumers, and regulatory authorities once it has been
identified that product is hazardous
•Notify
distributors, importers, manufacturers as applicable
Initiation and
Notification •Evaluate product for hazard risk and risk of harm to patient.
Strategy and
Corrective
Actions
•Develop a recall strategy including distribution status of product, and stock recovery plan.
•Review strategy with regulatory authority for feedback and guidance. Ensure that controls
include root cause analysis, and corrective actions to address the root cause.
•Define preventive action plan to reduce reoccurrence of root cause
•Implement recall stragey and product recovery plan.
•Submit periodic updates to regulatory authorities including feedback of all responsible
parties (distributors, importers, and manuacturers as applicable).
•Request verification from regulatory authority upon completion of corrective actions,
Effectiveness preventive actions, and product recovery.
and Closure •Recall Closure
Figure 5. Summary of basic requirements of recall process combining regulatory requirements of
the United States (FDA), Australia (TGA), ISO, Europe (EU), and Canada (HC). Figure created
by the author of this thesis.
FDA Enforcement Actions for Software
Defects and Recalls
In 2018, Phillips Medical Systems (PMS), a medical device manufacturer, received a
Notice of Inspectional Observations FDA Form 483 following an FDA inspection in July 2017 at
the manufacturing facility in Cleveland, Ohio. The results of the inspection revealed that that the
organization lacked adequate controls for handling and processing thousands of customer
complaints, in addition to various other nonconformities, that lead to more than 20 different
product recalls. The month-long investigation exposed that over 129,000 complaints had been
closed based solely on pre-defined hazard codes without requiring further investigation. During
the investigation, the FDA also found that a corrective action plan (CAPA) was opened in
January 2014 for software anomalies was closed without further investigation even though 59
customer complaints were related to the same issue noted in the nonconformance (FDA, 2017).
The Form 483 reports that between February 2014 to the time of inspection, PMS had issued 30
31
Class II recalls related to software defects where the cause was contributed to software design
controls. Many of those software related recalls were linked to a single supplier that was
identified as on probation between 2014 and 2016. The company was additionally cited for
failure to initiate any corrective action to identify the problem with software design controls to
ensure that testing is adequate to ensure defects are detected prior to market release. Further
inquiry revealed that the citation was a repeat observation from the FDA, and that every version
of their software has been recalled. As of 2018, Phillips is still in communication with the FDA
on the status of their progress towards addressing software errors as a result of the inspectional
findings (Mulero, 2018).
Warning letters are issued by the FDA following the issuance of the 483 listing
inspection findings. Every organization is allowed a period of time to respond to the FDA with
their intended corrective action plans, and periodic status updates in 30 intervals. Once the
progress updates are submitted, the FDA reviews the response from the organization for
adequacy and effectiveness in addressing the citation. If the FDA finds that the actions are
ineffective or fail to address the issue, the FDA issues a warning letter to the organization.
Warning letters serve as a final notice from the FDA to correct the nonconformance before the
issuance of a consent decree.
In 2015, Merge Healthcare received a warning letter from the FDA following an
inspection at their facility in Wisconsin. Merge Healthcare is a manufacturer of software used in
clinical settings to manage patient data for medical images and patient vitals. The organization
received a 483 following the inspection noting violations in software validation and management
of software related recalls. For software validation, the citation was issued because the
organization’s process allowed for the shipment of devices prior to full completion of software
32
validation activities (Public Health Service, 2015). Furthermore, the organization was issued a
citation for failure to notify the FDA of two Class II software related recalls as per the regulatory
requirements of 21 CFR 806.10 Corrections and Removals. According to the warning letter, the
organization has not provided the FDA with a plan on how to address the determination of the
defects risk to health and the need to report the removal or correction to the agency.
Additionally, no timeline was provided for the completion of recall activities. Following
response to the warning letter from the organization, the FDA issued a close out letter confirming
receipt of responses from the organization and confirming that the organization has addressed the
concerns raised in the warning letter.
Between 2018 and 2019, the FDA issued public notices for several software related
Class I (high risk) recalls. In August 2018, Akron anesthesia delivery systems recalled their
delivery system devices due to a software defect that caused the device to go into a failed state
while the machine was in use (FDA, 2018). During the failed state, the device stops the
mechanical ventilation system from functioning resulting in an increased risk of harm to patient.
In September 2018, Medtronic recalled their Synergy Cranial Software and Stealth Cranial
Station S7 software due to inaccuracies displayed during surgical procedures. The software is
intended to be used in surgery to produced three-dimensional (3D) images of the patient’s brain
to allow surgeons to effectively navigate surgical tools and implants during surgery (FDA,
2019). The software was recalled due to reports to the organization of incorrect information
appearing on the display during biopsy procedures. The defect was noted to cause serious to lifethreatening risks of harm to the patient.
In April 2019, Brain Lab AG recalled their Brain Spine and Trauma 3D Navigation
Software due to the potential for incorrect information to appear on the display during surgery
33
(FDA, 2019). The defect would prevent the surgeon from being able to accurately navigate
surgical tools inside the patient potentially resulting in brain damage, or serious to lifethreatening injury to the patient. The Class I examples show the dangers and severity of
undetected software defects and their impacts on the risk of harm to patient. The public
notifications provide consumers, hospitals, and health care practitioners with relevant product
information, information of the defect, potential impacts to patient safety, and instructions on
what to do if the device in their possession is affected by the recall.
Summary of Enforcement Actions
Circumstances leading to the issuance of warning letters, inspection citations, and public
recall notices provide direct evidence of the necessity to maintain controls for the recall process
to ensure that patient safety is always at the forefront of the organizations focus once defective
product is known on the market. The combination of these various modes of enforcement
provide insights to the vast scope of issues that can trigger recall, but also highlight inadequacies
in the recall process managed by an organization. Customer complaints trends, established
protocols and procedures, software validation during design are all aspects of the product, and
quality system lifecycle that have the potential to impact the product once made available to the
market. It becomes critical to regulators to understand the scope of their responsibility in keeping
industry on track with compliance. It is equally important for organizations to understand the
varying components within their systems that may impact the likelihood of recall, and the
effectiveness of recall execution.
Recall Timing and Effectiveness Studies
In 2011, Hora et al. conducted an empirical study to identify and test three key factors
that can be associated with the time to recall a product from the date the product is sold until
34
recall initiation. The primary focus of the study was to use those key factors to explain why it
takes so long to recall a product that poses a safety hazard. The study focused its in depth
analysis on the relationship between time to recall against three critical characteristics (1) recall
strategies, (2) source of the defect, and (3) supply chain position of the recalling firm. Recall
strategy was involves the initial decision to voluntarily or involuntarily recall a product, and
whether the strategy to execute the recall is preventive or reactive.
Hora et al. (2011) discussed the impacts of recall strategy for preventive vs. reactive
approached to recalls and their impact in recall timelines. In a preventive recall strategy, the
organization elects to notify customers of initiation of recall based on the organizations ability to
identify a defect. Signaling to customers that the organization is diligent and proactive about
quality issues. Reactive strategy, incidents ,and injury have already been publicly reported and
the organization is responding by initiating a recall. Source of defect relates to the point in the
process the defect surfaces either in manufacturing and production, design flaws, or inadequate
instructions and warnings. The study evaluates which sources of defect would cause increases in
time to recall duration for the toy industry. Lastly, the final characteristic is supply chain position
relating to whether the product return is handled by the manufacturer or distributor and where in
the supply chain the product might be between the manufacturers and the end user. Supply chain
position evaluates the logistical aspect of product movement and estimates implications on its
impact to time to recall duration.
Time to recall was used as the dependent variable in the study while the critical
characteristics (1) recall strategies, (2) source of the defect, and (3) supply chain position of the
recalling firm represented the independent variables. Using statistical analysis, the researchers
conducted the experiment using correlations between all variables. The study results in a
35
confirmation of the original hypothesis for each characteristic. Time to recall was longer for
product recalls that utilized preventive strategy over reactive. Time to recall was longer for
product design flaws than manufacturing defects. Time to recall was longer for supply chain
players that had closer proximity to the end customer than those that do not. Although this study
was conducted on the toy industry, which has an established set of regulations that are vastly
different in comparison to medical devices, the study has strong implications for the context of
this research. This study is unique as its available research on time to recall evaluation is limited
in scope across many industries, including medical devices.
Similarities exist between the selection of data set, Hora et al. (2011) conducted this
evaluation using available industry data on toy recalls and considered recall strategy as an aspect
that will impact time to recall duration. Differences exist not only for recall context, but for
additional characteristics as well. The explored medical device recalls, between a shorter period,
and the time to recall parameter is set from recall initiation to recall termination. The Hora et al.
(2011) study does not take recall termination into account in the study, limiting recall analysis
from product availability date or date first sold to recall initiation.
Food recalls are used as an effective and preventive tool to limit exposure to illness by
removing contaminated product from the market. Yu et al. (2018), conducted a research study to
evaluate recall effectiveness indicators in the US meat and poultry industry in order to offer a
framework of relationships between (1) discovery time, (2) completion time, and (3) recovery
rate. Using three indicators, the research aimed to verify the effectiveness of the recall in relation
to recall timing and product recovery.
Discovery time represents the number of days from earliest production until the defect is
discovered and recall is initiated. Completion time represents the time between opening and
36
closing of the recall. Recovery rate is the proportion of recalled product recovered by the
organization (Yu & Hooker, 2019). The underlying assumption of the study is that these three
indicators are clear measures of recall effectiveness. The researchers hypothesized the following:
•
Recalls with shorter discovery times have higher recovery rates than those with
longer discovery times.
•
Recalls with longer completion times have higher recovery rates in contrast to those
with shorter completion times.
•
Recalls with higher recovery rates have longer completion times than those with
longer recovery rates.
Yu et al. (2018) utilizes data made publicly available, and conducts the analysis using
descriptive statistics for continuous vs binary variables. The research results in confirmation of
the hypothesis supporting that recalls with shorter discovery times and longer completion times
have higher recovery rates (Yu & Hooker, 2019). This encourages the theory that recall
effectiveness is directly associated with both completion and recovery times. Additional
implications were made during the discussion highlighting that the results could be attributed to
having better technological resources for product recall and recovery allowing for more effective
recall management. The research study is similar to the scope of this thesis as both will use
publicly available information for data analysis to evaluate recall effectiveness and product
recovery. The timing parameters vary as this thesis will evaluate time to recall between initiation
and termination of recall, against distribution pattern, recall protocols, and product traceability.
Plan-Do-Study-Act (PDSA)
The PDSA learning cycle provides a theoretical framework for iterative testing to
implement, evaluate, and measure changes within a quality system structure (Taylor et al., 2014).
The PDSA learning cycle has been used industry wide in a variety of applications from
manufacturing to healthcare improvements. In suggesting improvements to processes based on
37
established regulations, it is important to understand the context of historical data, and the impact
the data has had on the quality of the medical device and the safety of the patient. The PDSA
learning cycle is utilized within this thesis, to drive improvements to the recall process based on
a risk analysis of historical recall data in terms of time to recall duration and effectiveness.
According to Deming (1982), the reason to study the results of change is to try and learn how to
improve tomorrow’s product.
The PDSA learning cycle requires careful planning and the development of predictions
based on assumptions, testing the plan, studying the results, and evaluating the impact of the
change. Studying the results involves evaluating what we learned from the change. Did the
changes lead to improvements in the quality system, the process, or the product? In the case of
this thesis, does the change lead to lower risk of harm to patients. The PDSA learning cycle
allows for an adoption or a rejection of change based on the results, when results are favorable,
the PDSA learning cycle can be repeated under different conditions to learn whether favorable
results are false or are valid over a range of different conditions.
Taylor et al. (2014) adapted a theoretical framework for the PDSA learning cycle
highlighted by Figure 6 representing the four stages of the PDSA learning cycle, and key critical
considerations at each phase. Limited research has been conducted utilizing PDSA for quality
improvements within quality system functions, specifically recall analysis, however, PDSA has
been utilized to support research in quality improvements associated with patient care practices.
Furthermore, Wowak et al. (2015) noted that although the literature on product recalls is
informative, our understanding of recall events is still in its infancy.
38
Figure 6. The model for improvement. Adapted from Taylor, M. J., Mcnicholas, C., Nicolay, C.,
Darzi, A., Bell, D., & Reed, J. E. (2014). Systematic review of the application of the plan–do
study–act method to improve quality in healthcare. BMJ Quality & Safety, 290-298.
Summary of Literature Review
The literature review highlights the need for ensure compliance to global regulation while
considering areas of weakness within the recall communication chain. To identify the underlying
factors that contribute to an organization’s inability to reduce time to recall, it’s important to
understand the entire recall process. The current execution of this process is based on what is
required versus areas of flexibility between the perspective of the organization and regulatory
authorities. Industry enforcement data creates a communication channel between regulators and
organizations that identifies quality system areas of noncompliance that suggest opportunities for
organizational learning and improvement related to recall management. Review of the regulatory
39
requirements against enforcement actions when systems do not meet regulation demonstrates
how regulatory compliance gaps influence prolonged time to recall.
40
CHAPTER 3
METHODOLOGY
Design of the Investigation
This study utilized regulatory requirements, industry guidance, and international
standards to establish the structure for process improvement as the scope of inquiry. The PDSA
learning cycle was developed as the theoretical framework that served as the basis of
investigation. Using the PDSA learning cycle, this thesis explored four phases of investigation
execution to identify areas of risks discovered from historical data analysis, and to provide an
improvement strategy for implementation, evaluation, and discovery. The PDSA learning cycle
was executed as follows:
•
Plan Phase. Identified key characteristics of a basic recall process and predict areas
where improvements might be possible based on literature. Created a flow-chart
highlighting key characteristics and identifying proposed areas of improvements as
demonstrated in Appendix A. Plan phase was included in Data Analysis Procedures.
•
Do Phase. Conducted data analysis on recall data set employing the use of quality
tools such as Pareto charts, Fishbone diagrams, and box plots using coded data sets.
The Pareto chart was developed to identify display all defect categories and highlight
software defects, verifying software as one of the top causes of recalls. Developed
Fishbone diagram to represent root cause analysis to identify factors contributing to
prolonged time to recall duration as demonstrated in Appendix G and H. The
Anderson-Darling normality test, and the boxplots were presented to map the data
results for software related recalls to confirm which recall events were found to have
prolonged time to recall duration. Statistical plots were created based on analysis of
software related recalls time to recall and time to classification data points, means,
standard deviations, and control limits were calculated using Minitab®. Additionally,
Failure Modes and Effects Analysis (FMEA) was developed to identify failure modes
stemming from potential root causes that contribute to an increase in time to recall
duration and a decrease in recall effectiveness. Do Phase was included in Data
Analysis Procedures.
•
Study Phase. Presentation and discussion of results discovered during the Do Phase.
Reported results of root cause identification and discussed confirmation of gaps from
Plan Phase. The FMEA tool developed within the Do Phase, was updated to reflect
41
risk mitigation strategies for confirmed root causes and their failure modes ad
demonstrated in Appendix I. Study phase was reported under Chapter 4, Results and
Discussion.
•
Act Phase. Referenced literature to support the proposed implementation of risk
mitigation strategies for process and product failure modes discussed during the
Study Phase. Provided recommendations to industry to consider implementation of
risk management strategies following implementation of proposed tool for recall data
trending and risk mitigation for recall execution. Act phase was discussed under
Chapter 4, Results and Discussion.
Population or Sample
The data utilized for analysis within this study included publicly available data that has
previously been published within the FDA medical device recall database. Recall data collection
was performed via request of recall data from the FDA using the Freedom of Information Act
(FOIA) request from on the FDA website. Request response was provided by the Division of
Freedom of Information in the format of an excel spreadsheet including a compilation of FDA
reported recalls for Class I, II, and III medical device recalls initiated between 2016 through the
end of 2018. The data set includes data related to the duration of recall, location of recall,
distribution pattern, organization name, and address, product description, and reason for recall.
All data was previously published for public notification and does not include any information
related to human subjects.
This data set encompassed all recalls to allow for application of Treatment procedures to
the data set to identify software related recall events. Using stratified random sampling
procedures, the initial data set of 9,647 records was truncated after treatment to represent 1,605
specific recall events. Those 1,605 recall events were reduced by defect category identify
software related recalls, this treatment resulted in 282 recall events, detailed in Data Analysis
Procedures. The final sample size is represented by the 282 software related recall events which
42
were used to develop the quality tools referenced under data analysis. Results were reported in
Chapter 4.
Data Collection
Treatment
This thesis provided an in-depth analysis of Class II medical device recalls filtered from
the data provided by the FDA FOIA request. Of the Class II medical devices, this study analyzed
software related recalls that have both initiation and termination dates. Recalls without
termination dates were not analyzed through statistical analysis, as open recalls are out of the
scope of time to recall calculations. Distribution patterns were coded according to global and
national recalls as indicated within the dataset. Microsoft® ExcelTM and Minitab® software
applications were utilized to perform statistical and graphical data analysis on the sample data.
It is important to analyze data for identifiable characteristics and remove sensitive
organizational information. The data set was coded to categorize key characteristics for statistical
analysis within the control chart. The state of descriptive text within the data set does not allow
for a clean review of data and coding will provide increased flexibility and provide a valuable
benefit in facilitating the data analysis. Data coding by categorizing quantitative and qualitative
responses converts the data for suitable investigation with the assistance of statistical software.
The following columns of the FOIA data set provided by the FDA were modified using a
numerical coding system:
1) Recall Event IDs were recoded from assigned FDA numbers to sequential numerical
codes labeled beginning with the number one, ending at the last row of the data set.
Duplicate entries were removed using Microsoft® ExcelTM data tools to code multiple
entries for a single event.
2) Voluntary/Mandated responses were recoded numerically. Responses marked
Voluntary: Firm Initiated were coded using the number one. Responses marked FDA
Mandated will be coded using the number two.
43
3) Country responses were recoded initially by global region (i.e., U.S., Europe, Oceana,
North America, East Asia, South America, and the Middle East) based on country
location. Regions were recoded numerically starting with the number one, ending at
the last row of the data set.
4) Recall Categories were reviewed manually for 100% of the dataset. Reason for recall
responses will be recoded categorically by type of defect based on the description
within the response. Categories for coding include the following: Labeling, Software,
Component, Product, Process, Registration, and Unknown.
5) Distribution Pattern responses were reviewed manually for 100% of the dataset.
Distribution pattern responses were recoded categorically by the geographical region
identified. Categories for coding include the following: Domestic—US Only,
International—outside US (oUS) only, Domestic & International, and Unknown.
Regions will be recoded numerically starting with the number one, ending at the last
row of the data set.
Quality Tool Implementation Methods
Several quality tools were used within the implementation of the PDSA learning cycle for
data analysis. The first tool introduced within the study is a Pareto chart represented as a bar
graph that represents frequency of recall defects within the data set. Figure 7 demonstrates an
example Pareto chart for visual representation displaying six categories of defects, denoted
numerically. The purpose of the Pareto chart was to visually demonstrate significant recall defect
categories to highlight the frequency of software related recalls in comparison to other recall
related defect by category within the same period. For this study, the resulting Pareto chart was
constructed using recoded values for the reason for recall by category. The Pareto chart was
configured using following application of data treatment method using Minitab® software
application.
44
20
18
16
14
12
10
8
6
4
2
0
87%
96%
100%
76%
63%
39%
120%
100%
80%
60%
40%
20%
0%
Category Category Category Category Category Category
5
3
1
4
2
6
Figure 7. Pareto chart demonstrating frequency of recall defects by category. Adapted from
ASQ. (2019, September 17). THE 7 BASIC QUALITY TOOLS FOR PROCESS
IMPROVEMENT. Retrieved from American Society for Quality: https://asq.org/quality
resources/seven-basic-quality-tools
The second tool introduced within the data analysis and PDSA implementation is the
cause-and-effect diagram, otherwise known as the fishbone diagram. Figure 8 is provided as an
example fishbone diagram to illustrate the identification of specific issues that contribute to
software recalls. This quality tool was utilized to assist in identifying potential causes for
software related recalls based on common themes that are noted as responses within the recall
event data set. Additionally, the diagram assisted in identifying causes related to increased
duration in time to recall following the initial analysis. The fishbone diagram was created
manually within Microsoft® PowerPointTM.
45
Figure 8. Fishbone diagram template. Adapted from ASQ. (2019, September 17). THE 7 BASIC
QUALITY TOOLS FOR PROCESS IMPROVEMENT. Retrieved from American Society for
Quality: https://asq.org/quality-resources/seven-basic-quality-tools
Several tools were used to identify trends and patterns within the software recall data set
in order to understand characteristic variations in time to recall duration. This study utilized
mathematical calculations to compute the time between initial time and final time based the
following parameters.
1) Recall duration measured the timing between recall initiation to recall termination,
measured in calendar days, as reported within the data set.
2) Recalls with multiple recall event codes were condensed into one recall record for
count of total medical device recalls by organization.
3) Classification duration measured the timing between recall initiation to recall
classification, measured in calendar days, as reported within the data set.
The results of this calculation provided the numbers for the variables within the final data
set used for statistical analysis. Statistical analysis of time to recall duration and time to
classification duration from the 282 software related recall events should reveal inconsistencies
46
and deviations from the average that require further investigation for risk mitigation using
FMEA. This study employed the Anderson-Darling normality test to determine if the data for
time to recall and time to classification followed a normal distribution. Following the normality
test, each variable will be analyzed for outliers using boxplots generated within Minitab®. Once
the outliers are identified, this study will analyze each outlier for reason for recall and root cause
of recall and recommend mitigation strategies using FMEA analysis.
Finally, this study utilizes tools available for risk analysis using FMEA. FMEA is a risk
approach that is utilized to identify failures within the recall process that contribute to delays in
time to recall duration, and the initiation of software related recalls. Failure modes represent
errors or gaps within the process that need improvement or monitoring in the context of this
study. Effects analysis evaluates the impact of those failures to the process. Failures were
categorized based on their risk which includes frequency, severity, and the ability to detect the
issue. Figure 10 illustrates the basics included in a traditional FMEA.
In terms of software related recalls, the failure modes were classified based on common
themes found within the data set. The purpose of using the FMEA in this study to recommend
actions to eliminate or reduce identifiable failures to increase their impact on time-to-recall
duration, and software related recall initiation. FMEA developed the risk analysis strategies
listed below:
1) Process Failure Mode and Effects Analysis (pFMEA) was utilized to identify known
factors that contribute to the initiation of a Class II medical device recall. pFMEA
was utilized to identify known factors that contribute to an increase of time to recall
duration of a Class II medical device recall.
2) Product FMEA was utilized to identify known and predicted factors of software
related medical device components that contribute to the initiation of a Class II
medical device recall.
47
Figure 9. Example FMEA data table. Adapted from ASQ. (2019, September 17). THE 7 BASIC
QUALITY TOOLS FOR PROCESS IMPROVEMENT. Retrieved from American Society for
Quality: https://asq.org/quality-resources/seven-basic-quality-tools
Data Analysis Procedures
Plan Phase. Recall Process Proposal
The literature review of regulatory requirements in Chapter 2 provided a general outline
for the basic elements of a recall process. The key takeaways within the context of FDA recalls is
to think globally as many organizations have adopted a global supply chain including third party
manufacturers and distributors around the world. Combining the regulations led to the
identification of three high-level phases of the recall process (1) initiation and notification, (2)
recall strategy and corrective actions, and (3) effectiveness and closure. Each phase contains
elements of the recall process that have been highlighted by at least one of the global agencies
and combined to propose an outline that is inclusive of strict global requirements. Appendix A
provides a detailed flow chart to explain the following elements of a recall process.
Initiation and Notification begins with first identification of the hazardous medical
device, and an immediate notification to direct accounts, consumers, and regulatory authorities
as required (FDA, 2019). Additionally, the organization is responsible for notifying distributors,
importers, and manufacturers taking into consideration third party manufacturers per the
48
applicable requirements under the medical device regulation in the EU (European Parliament,
2017). Once notifications have been issued, it is the responsibility of the organization to begin a
hazard analysis for risk of harm to patient. This may be performed through investigation of
previously identified hazards from the design controls process or working to identify new
hazards and update the design FMEA.
Recall strategy represents a critical area of focus for this thesis, as it could directly impact
time to recall duration. The organization is responsible for the development of the recall strategy,
and the regulatory authorities are responsible for reviewing and advising prior to
implementation. Developing a recall strategy should include the following steps as included in
Appendix A.
1) Data collection for status of device in stock in inventory, and collection of
distribution history.
2) Development of a stock recovery plan based on the size of distribution and the scope
of geographic location of distribution channels.
3) Identification of root cause that led to hazard and development of a corrective action
plan to address root cause of occurrence. A preventive action plan should also be
developed and implemented to prevent recurrence of the issue.
4) Recall strategy execution.
The final phase of the recall process involves monitoring the recall through effectiveness
checks, review with regulatory authorities through status updates about the recall recovery plan,
and incorporation of feedback from all parties. It is important during the monitoring phase for the
organization to hold status updates and reviews with distributors, importers, manufacturers to
understand the progress of the investigation and ensure each element of the strategy is fully
implemented. Upon completion of all activities within the plan, and relevant milestones,
including records of recall completion are maintained, the organization must request final review
49
and closure with the regulatory authority. This step is vital to ensure that the recalled product has
been fully recovered and corrective action plans have addressed the root cause. Once all
activities are verified as complete, the recall is led to closure.
The purpose of this study was to incorporate retrospective review of recall data for
considerations in minimizing the risk of recall initiation, and to benchmark categorical defects to
minimize risk of hazard and time to recall duration. This part of the proposal is not currently a
regulatory requirement; however, it is listed within Appendix A as a proposed tool for inclusion
within Phase 0 of the recall process. By identifying defects from recalls of similar devices,
organizations can benefit in two ways:
1) Mitigate risks of known industry hazards by implementing new hazards into design
FMEAs or product improvement FMEAs based on industry data. This practice can be
supported using the guidance under ISO 14971—Risk Management of Medical
Devices.
2) Gain better control of identification of root cause following recall initiation to
decrease time to recall duration and minimize risk of harm to patient from prolonged
product exposure.
Review of the flow chart allowed for analysis of opportunities for areas of improvement
with the recall process. Potential gaps could exist in three key areas (1) recall strategy errors,
potentially suggesting that the organization could not adequate plan and execute an efficient
recall strategy, (2) Delays in monitoring the effectiveness of the recall during strategy execution
because this is the responsibility of the FDA suggesting process delays in reviewing milestones
within the recall plan, and (3) Root cause identification delay, which could lead to classification
delays and potentially slow down the speed of time to recall. Chapter 4 includes an overview of
the results confirming the identification of process gaps.
50
Do Phase: Application of Treatments to Class II FDA Recalls
Data provided by the FDA Division of Freedom of Information was condensed via
manual coding to reduce the entire dataset from all medical device recalls to software related
recalls within the two-year timeframe. The initial data set provided by FDA contained 9,647
records related to medical device recalls encompassing all recall events from 2016 to 2019.
Implementation of treatments for data coding resulted in a remainder 1,605 recall events, this is
the data that will be used to identify the specific number of software related recall events for data
analysis. For the identification of software related recalls, and relevant parameters for
measurement, treatments were fully implemented to reduce the data set. The analysis continued
with applying the treatments discussed in Chapter 3 to the entire working data set. The following
procedural steps were explained in detail to demonstrate the pathway for the identification of
software related recalls.
Recall initiation methods were analyzed for comparative analysis; voluntary/mandated
responses were recorded numerically and represented graphically in Figure 11 to demonstrate the
number of firm initiated versus FDA mandated recalls for Class II medical devices. According
pie chart, 1,593 recall events of all Class II recall events were initiated by the organization
voluntarily and 12 recall events were mandated by the FDA. Number one represents voluntary,
firm-initiated recalls, and number two represents FDA mandated recalls in the figure below.
51
Figure 10. Pie Chart representing Recall Initiation Modes. Figure created by the author of this
thesis.
Recall initiation location was marked by Country responses within the initial data set
representing the geographical location of the organization responsible for recall initiation to the
FDA. It is not clear if the organizations represent manufacturers or distributors but both parties
may be responsible for recall initiation. Treatment was applied to all responses, with initial
coding marking individual countries by their regions (i.e., United States., Europe, Oceana, North
America, East Asia, South America, and the Middle East). Additionally, regions were recorded
numerically starting with the number one and ending at the last row of the dataset. Figure 12
represents a pie chart to visually display the geographical regional distribution of organizations
initiating recalls within the two-year time frame. Categories are represented numerically as
assigned during data coding as follows:
•
Category 1: 1455 recall events were initiated within the US.
•
Category 2: 110 recall events were initiated in Europe.
•
Category 3: 4 recall events were initiated in Oceana (i.e., Australia, New Zealand).
•
Category 4: 17 recall events were initiated in North America (i.e., Mexico, Canada).
52
•
Category 5: 14 recall events were initiated in East Asia.
•
Category 6: No recalls were reported in South America.
•
Category 7: 5 recall events were initiated in the Middle East.
Figure 11. Bar chart of recall initiation pattern by region. Figure created by the author of this
thesis.
Recall categories were assigned to the 1,605 recall events listed within the data following
a 100% manual review of Reason for Recall responses reported to the FDA. Categories for
reason for recall were listed by type of defect based on the wording reported to the FDA. Issues
related to labeling (i.e., incorrect labeling, misinformation on labels, IFUs) were recoded as
Labeling. Issues related to packaging (i.e., unit carton issues, breakage of seal, incorrect coding
on outer packaging) were coded as Packaging. Issues related to product defect related to
components or functionality issues were coded as Product or Component as applicable. Issues
related to process (i.e., service, maintenance, manufacturing errors) we coded as Process. Issues
related to design registration or misclassification based on regulation were coded as Registration.
53
Issues that have not been identified or undetermined by FDA, or not yet determined by the
organization were coded as Unknown/other.
Issues related to software were coded as Software as applicable. Figure 13 is a Pareto
chart that visually represents the result of the manual coding of reason for recall by defect
category. The Pareto graph displays the two largest defect categories as Component and
Software, with recall events at 521 and 282 respectively. Identification of the leading categories
of recall defects supports the necessity to focus efforts on software related recalls as they have
one of the highest impacts to industry recall initiations. A summary of all categorized events was
provided as follows:
1) 521 Component related recall events.
2) 282 Software related recall events.
3) 207 Labeling related recall events.
4) 207 Product related recall events.
5) 168 Packaging related recall events
6) 131 Design related recall events.
7) 69 Process related recall events.
8) 11 Registration related recall events.
9) 9 Unknown/other causes for recall.
54
Figure 12. Pareto chart displaying recall event defect category. Figure created by the author of
this thesis.
For software related recall events, the resulting sample size for data analysis includes 282
recall events identifiable software characteristics that led to software errors. This represents the
second highest category of defects for all recalls initiated from 2016 through 2018. Relevant
keywords used to identify software recalls include factors such as coding, algorithm, display
errors, error messages, improper calculations, user interface, software errors, software versions,
software updates, digital systems, data, modules, modalities, electronic systems, or workstations.
Software recalls accounted for 17.57% of all recall events within the time frame between the
beginning of 2016 to the end of 2018.
Data Analysis of Software Recalls
Software recalls were further analyzed to identify recall events that had prolonged
durations from time to recall and time to classification, and the factors that potentially influenced
55
the delay. The resulting data consisted of tabulated, coded data for software recalls including
region of initiation, distribution pattern, time to classification duration, and time to recall
duration. Of the 282 software recall events, analysis found that in terms of recall initiation
method 275 recall events or 97.5% were initiated by the organization voluntarily while 7 recall
events or 2.5% were FDA mandated. Figure 14 provides a visual representation of this
comparison for software recall events.
Figure 13. Bar chart of software related recalls by recall initiation method. Figure created by the
author of this thesis.
A review of region initiation patterns show consistency between software related versus
all recalls in terms of origination location of the recalling organization. Most recalls initiated
were within the United States and its territories with a total of 252 recall events initiated. Europe
leads second at 24, followed by East Asia with four recall events, and North America (i.e.,
Canada and Mexico) with two recall events. Distribution patterns were analyzed as shown in
Figure 15, and the following patterns resulted: (1) 141 recalls were distributed domestically
within the United States only, (2) 131 recalls were distributed in both the United States and
56
internationally, (3) Nine software related recalls were distributed internationally only, and (4)
One recall distribution patterns were unknown at the time of recall termination.
Figure 14. Histogram of Distribution Patterns for Software related recall events. Figure created
by the author of this thesis.
Statistical Analysis Parameters
Time to recall duration and recall classification duration times were calculated using
standard formulas. Time to recall is measured as the time between recall initiation to recall
termination date, and reported in total calendar days. Recall classification duration times were
calculated between recall initiation to recall classification date, and reported in total calendar
days. For the sample of 282 recall events, the average for time to recall duration was 429 days
with a standard deviation of 244 days. For a sample of 282 recall events, the average time to
classification duration was 78 days with a standard deviation of 86 days. The results of this data
were used to conduct statistical analysis using Anderson-Darling normality test, Goodness of Fit
and box plots to test for normality, find the relevant distribution pattern, and identify outliers
within the data set.
57
Summary
The PDSA learning cycle was used to explore the identification, analysis, evaluation, and
recommendations for risk mitigation of software related recalls and increases in time to recall
duration. Using quality tools such as pareto charts, fishbone diagrams, and FMEA reveals
important aspects of the key factors that contribute to the issuance of recalls and prolonging time
to recall related to software defects. The evaluation of data yields new insights for understanding
the recall process, and the contributing characteristics to ensure that recalls are effective and
managed in a timely and effective way to reduce risk of harm to patient from exposure to
defective product. The results of the analysis led to recommendations to industry organizations
and the FDA on how to mitigate risks to better manage recall requirements based on regulatory
and retrospective data analysis.
58
CHAPTER 4
RESULTS AND DISCUSSION
Software Related Recall Results
Test for Normality of Distribution
Software related recall events accounted for 282 recall events identified from the data set,
and this is the final sample size used for the results of this study. In order to determine normality
of distribution, Minitab® software application was used to conduct an Anderson-Darling
Normality Test. Under the Anderson-Darling test, H0 and H1 represent two phases of normality
where H0 is normal, and H1 is not normal. If the resulting p-value for the test is less than 0.05,
then H0 is rejected and the data distributions is classified as non-normal. Figures 15 and 16
represent the Anderson-Darling results for Time to Recall and Time to Classification Variables.
Figure 15. Anderson-Darling Normality Test for Time to Classification. Figure created in
Minitab®.
59
Figure 16. Anderson-Darling Normality Test for Time to Recall. Figure created in Minitab®.
The resulting Anderson-Darling normality test concluded that time to classification and
time to recall data followed a non-normal distribution. For time to classification, Minitab®
calculated the mean of data at 77.73 days with a standard deviation of 86.41 days. Additionally,
the p-value is less than 0.005, confirming that the distribution is not normal. For time to recall,
Minitab® calculated the mean of data at 429.5 with a standard deviation of 243.8 days.
Additionally, the p-value for time to recall is less than 0.005, confirming the distribution is not
normal.
Since non-normality for distribution of data has been established for both variables, the
data for time to recall and time to classification were evaluated using a distribution fitting
procedure to determine which distribution each variable follows. Distribution fitting was utilized
to select the which statistical distribution bet fits the data in order to determine the probability of
the data producing results that fall outside of the expectation.
60
Time to Classification Identification of Outliers
Distribution fitting for Time to Classification resulted in four statistical plots,
representing fifteen different tests for goodness of fit as shown in Appendix C. According to the
p-values calculated by Minitab®, time to classification did not produce any distribution or
transformation that could be utilized to normalize the data for analysis. Table 2 represents
complied results from the analysis for distribution fitting, and the associated p-values that were
calculated. From the results, there were no p-values listed as high enough to be significant. Time
to Classification data was further evaluated for outliers following non-normal distribution.
Table 2
Goodness of Fit Test for distribution of data for Time to Classification
Distribution
AD
P
LRT P
Normal
Box-Cox Transformation
29.984
4.203
<0.005
<0.005
Lognormal
3-Parameter Lognormal
4.203
4.097
<0.005
*
0.716
Exponential
10.803
<0.003
2-Parameter Exponential
Weibull
9.846
10.773
<0.010 0.000
<0.010
3-Parameter Weibull
Smallest Extreme Value
9.801
42.628
<0.005 0.000
<0.010
Largest Extreme Value
18.539
<0.010
Gamma
11.098
<0.005
3-Parameter Gamma
10.189
*
Logistic
Loglogistic
22.837
3.612
<0.005
<0.005
3-Parameter Loglogistic
3.247
*
0.005
0.114
In order to find outliers for time to classification data, a box plot was utilized to identify
points within the data set that were beyond the threshold of acceptability. For the box plot shown
in Figure 18, the threshold is represented by 1.5 times the Interquartile range (IQR), calculated in
Minitab®. The IQR is represented in Figure 18 as whiskers, any points beyond the edge of that
61
line were identified as outliers, resulting in the recall events that are out of range for time to
classification.
Figure 17. Identifying outliers with Box-Plot graph for Time to Classification. Figure Created in
Minitab®.
Using descriptive statistics calculated in Minitab®, an outlier rule was created to identify
the points within the data set outside of 1.5 times the IQR, where IQR was calculated to be 58.25
days. The outlier rule applies to any recall event where time to classification was found to be
greater than or equal to 87.375 days. A total of 69 of 282 recall events were identified as outliers
for time to classification based on the outlier rule using Box-Plot. A summary of the associated
recall events is listed in Appendix D.
Time to Recall Identification of Outliers
Distribution fitting for Time to Recall resulted in four statistical plots, representing
fifteen different tests for goodness of fit as shown in Appendix E. According to the p-values
calculated by Minitab®, time to recall data was identified for Gamma distribution, according to
62
the p-values calculated during goodness for fit testing. Table X represents a summary of the pvalues calculated within Minitab®. Gamma distribution resulted in the most significant with a pvalue of greater than 0.25. Minitab® calculated the shape and scale of Gamma distribution to be
2.9939. and 143.4546 respectively.
Table 3
Goodness of Fit Test for distribution of data for Time to Recall
Distribution
AD
P
Normal
4.991
<0.005
LRT P
Box-Cox
Transformation
Lognormal
0.455
0.267
1.137
0.006
3-Parameter Lognormal 0.567
Exponential
23.594
*
<0.003
0.002
2-Parameter Exponential 16.883
<0.010
0.000
Weibull
1.004
0.012
3-Parameter Weibull
0.540
0.176
Smallest Extreme Value 12.054
Largest Extreme Value 1.006
<0.010
0.012
Gamma
0.433
>0.250
3-Parameter Gamma
0.385
*
Logistic
3.908
<0.005
Loglogistic
1.070
3-Parameter Loglogistic 0.892
<0.005
*
Johnson Transformation 0.278
0.649
0.014
0.526
0.174
In order to find outliers for time to classification data, a box plot was utilized to identify
points within the data set that were beyond the threshold of acceptability. For the box plot shown
in Figure 18, the threshold is represented by 1.5 times the Interquartile range (IQR), calculated in
Minitab®. The IQR is represented in Figure 18 as whiskers any points beyond the edge of that
line were identified as outliers resulting in the recall events that are out of range for time to
classification.
63
Figure 18. Box Plot for Time to Recall. Figure created in Minitab®.
Using descriptive statistics calculated in Minitab®, an outlier rule was created to identify
the points within the data set outside of 1.5 times the IQR, where IQR was calculated to be 319.8
days. The outlier rule applies to any recall event where time to recall was found to be greater
than or equal to 479.70 days. A total of 96 of 282 recall events were identified as outliers for
time to classification based on the outlier rule using Box-Plot. A summary of the associated
recall events is listed in Appendix F. The IQR is represented as the end of the whiskers within
the boxplot in Figure 18.
Summary of Key Characteristics for Outliers
FDA Recall Monitoring
The results of study suggest that FDA overview process for recall termination
additionally impacts time to recall duration. According to a literature search, a report by the
Government Accountability Office (GAO) reported in 2011 that the FDA should review recall
termination requests within three months of submission by the organization (GAO, 2011). FDA
termination timeline could not be determined from the data set provided by the Division of
64
Freedom of Information or the recall records made available online, this proved to be a
noteworthy limitation of the data analysis for publicly available information.
Currently, there are no target timelines from the FDA to suggest what the time to recall
duration should be in an ideal scenario, and additionally does not suggest that the FDA will
provide an immediate review for termination. It is probable that the FDA requires a longer
review period for organization’s that have been found out of compliance with key protocols that
impact time to recall. Kramer, Tan, Sato, and Kesselheim (2014) stated that Class II recalls tend
to involve strict FDA oversight, including follow-up and auditing of communications provided to
consumer and key stakeholders, and recall documentation and reporting. Monitoring of recall
implementation strategy and effectiveness could take longer time for review to ensure that
actions are effective and consistent for issues as serious and high-risk as medical device recalls.
This is outside of the organization’s control.
Enforcement and Inspection History
The enforcement history data for the organizations that had the highest number of time to
recall duration confirms were reviewed to identify pattern similarities between enforcement data
and recall initiation. Specifically, procedures and policies dealing with the handling of
nonconforming product and addressing known issues through corrective and preventive action.
This was identified initially through root cause analysis and confirmed through review of special
events found in Table 2. This finding is significant and supported a study conducted by Kramer
et al. (2014) suggesting that protocols for corrective and preventive actions collect reports of
problems and implementing solutions that include the initiation of recalls. Notification of
correction or removal protocols also ensure that regulatory requirements are met for informing
the FDA of a recall including correction of the defect or removal of the medical device.
65
Additionally, recalls initiated within the US have implication for other global markets. In Japan,
if a recall is initiated elsewhere, it is likely to be recalled there signaling an organization’s need
to have protocols in place to ensure all marketed territories are considered (Kramer et al., 2014).
In the case of Company ABC, all of the recalls found in this study were dated within the
year after an inspection at the facility which resulted in the issuance of an Inspection Citation
FDA Form 483. Company ABC was initially cited for lack of adequate implementation of
procedures related to Corrective and Preventive Action, Nonconforming Product, and
Corrections and Removals. Company ABC failed to adequately address the root causes and
implement effective solutions, which led to the issuance of the warning letter. The organization
was previously cited against 21 CFR 820 Quality System Regulation and 21 CFR 806
Corrections and Removals from inspections in 2009, 2010, and 2011. Three citations were issued
for corrective and preventive action activities, one citation was related to failure to establish
procedures for nonconforming product, and one was for failure to report device malfunction.
These citations reveal insights as to how the lack of control an organization has over their
systems via the implementation of protocols can lead to recall initiation and prolong time to
recall.
An additional review of citations from the FDA Inspectional Citation Database website
for citations issued between 2009 and 2019 led to the identification of several recurring citations
for Company DEF. An overview of inspection citations between 2012 and 2018 revealed that
Company DEF has been consistently cited on eight separate inspections for failure to report risk
of health or risk of malfunction to the FDA as required by regulation. Additionally, the company
was cited six times for failure to establish adequate procedures for corrective and preventive
action between 2012 and 2018. A warning letter was also issued for the citations related to the
66
inspection in 2017 detailing failure to address observations, especially corrective and preventive
actions. Company DEF received one citation in 2017 for failure to validate software as used
within the quality system. A review of Company BCD product history on the FDA’s recall
website listed several recalls from 2014 to 2018 for the medical devices citing software related
defects in several of the recalls initiated within that period. Additional review of citations
revealed that Companies CDE, EFG, FGH, GHI, and HIJ have also been cited for inadequacies
in their process for controlling non-conforming material.
Regulatory enforcement and inspection history highlight characteristics of the
organization in terms of responsibility and ability to comply to global regulations. Nearly every
Company identified in Table 2 was cited for inadequacies in addressing nonconforming product.
Since recall strategy is a large part of the recall process, this study suggests that if an
organization is struggling to maintain key quality system processes in a normal scenario, the
organization will struggle with rapidly responding to issues identified within the market.
Available data shows that companies cited for these inadequacies most recently had longer time
to recall durations. This could suggest that software related defects that triggered recalls were
identified during the inspection, or during resolution of inspection findings.
Root Cause Identification and Risk Mitigation
Strategy
Risk mitigation must begin and end with identifying the true root causes that contribute to
time to recall duration. In the initial assessment of cause-and-effect analysis, four root causes
were identified for prolonged time to recall duration. These included software defects, FDA
overview, organization protocols, and organization resources. After studying the results, the
confirmed root causes for time to recall duration remain at organization resources and FDA
67
overview. Software defects is identified as a contributing factor for recall initiation and should be
explored as part of Phase 0 of the proposed recall process.
A manual review to recall events as listed in Appendices C and E, lead to the creation and
development of two fishbone diagrams for root cause analysis. Key themes emerged in
reviewing the reason for recall and total product lifecycle data provided on the FDA website as
part of the public recall record. The root cause diagram identified three primary root causes for
software defects that lead to software related recalls, and an additional root cause assessment of
potential factors contributing to prolonged time to recall duration. Each branch added factors that
were found during the manual review and assumed to contribute to prolonged time to recall. The
fishbone diagrams are shown in Appendices G and H.
Software defects can be mitigated using the FMEA process to trend recalls from industry
for defects that could be applicable to all or similar medical devices. The new FMEA suggests
risk mitigation strategies for three of the four originally proposed root causes. Appendix I
provides an updated FMEA for confirmed root causes with suggested risk mitigation strategies to
ensure process issues are addressed to reduce time to recall duration.
Act Phase
The organizational learning process can be developed by preparing a review of the recall
requirements against industry data using the PDSA framework to propose a recall process that
will mitigate risks associated with longer recall times. The decision to adopt the proposed
process for risk and root cause mitigation of the recall process to decrease time to recall duration
has important implications for the prevention and management of software related medical
device recalls. Using quality tools, this study identified factors associated with the initiation of
the recall such as the software defect, and process gaps that contribute to prolonged time to
68
recall. Through root cause analysis and FMEA, steps to mitigate prolonged time to recall and
address software defects were proposed for implementation by an organization. By addressing
the software defects through continuous feedback evaluation and risk mitigation, the
organization and take a proactive position on reducing recall initiation. By addressing the process
failure modes through continuous risk analysis based on industry data trends, the organization
can develop and implement plans and protocols in place to ensure that recalls are managed at
rapidly and effectively, reducing time to recall duration. The decision to adopt is supported by
research on the product recall process from various industries.
According Kumar and Schmitz (2011), performing a risk assessment prior to the recall is
a proactive measure, and it puts the organization in a better position to react quickly and
effectively during a recall.The study conducted a recall analysis to identify failure points that
required risk mitigation from both a process and product perspective. Similar to the results in this
thesis, Kumar and Schmitz (2011) concluded that organizations need to understand their risks
and be prepared to defend their decision of whether or not to initiate a recall. This not only
includes the method of communication about the recall, but also includes selecting the
appropriate recall strategy.
The study suggests that organizations need to adopt a proactive approach for preventing
recalls. Proactive activities include four basic types of learning listening, testing, studying, and
tracking information. These activities encompass all aspects of the product lifecycle from design
to customer feedback incorporating external data back into the system for quality improvements.
Kumar and Schmitz (2011) states that organizations that participate in the global supply chain
need to work on improving their product recall process. Additionally, the research proposed that
improvements can be achieved through the use of technology tracking data throughout the entire
69
supply chain and include more diligent analysis of failure modes that result in a product recall.
This study confirms the relevance of the purpose of this thesis by connecting the use of risk
management tools into process improvements for the recall process to track and trend relevant
data to support a more effective and timely recall.
Furthermore, GAO reported in the 2011 analysis, which was the second the GAO had
produced since 1998, that there were several gaps in the medical device recall process and
oversight from the FDA. The gaps limited the organization’s and the FDA’s ability to ensure that
the highest-risk recalls were implemented in a timely manner (GAO, 2011). The GAO’s study
states that the FDA does not routinely analyze recall data to identify whether or not there are
systemic problems affecting recalls. Instead the FDA is trending recall data to support their
inspection resources, and compliance and enforcement actions. The GAO’s study confirms the
necessity of the analysis conducted within this thesis. By examining recall data for software
related defects, organizations can learn to identify systemic trends affecting industry recalls that
might be applicable to their product type.
Additionally, discovering how the FDA is using the recall data is important for
organization understanding and risk management. The FDA has not analyzed the data to educate
or inform organization learning, yet instead uses the data to enforce and inspect organizations.
This observation can be used to confirm that the trend observed within recall initiation from
organizations that had been recently inspected holds validity. By using the data to support risk
management activities for recall initiation and time to recall duration, organizations can also
maintain compliance and reduce regulatory risk while maintaining patient safety.
Simultaneously, the FDA can rely on the organization’s risk management strategy to manage
recalls.
70
A study conducted by Fu et al. (2017) concluded with a presentation of the procedure to
collect software related medical devices from the FDA to share important data about software
recalls. The analysis revealed important insights about limitations of data made available by the
FDA. The study notes that industry needs to collect more information about software-related
recalls to confirm detailed root causes that contribute to software failure modes in medical
devices. This learning would allow industry to adequately identify and address potential safety
hazards and risk mitigation strategies to prevent recurrence of similar defects in current and
future product designs. By addressing software defect root causes through industry data trending,
organizations can take a proactive approach at planning for defects to (1) reduce the likelihood of
recall initiation, and (2) reduce time to recall by rapidly identifying and mitigating software
defects with a pre-planned risk mitigation strategy.
All three studies provided information about the use of a recall management tool that
encompasses consideration of risk management prior to and during the recall process supporting
the purpose and results of this thesis. The proposed process improvements including recall data
benchmarking and risk analysis will not only provide learning within the organization but best
practices for industry. The results of this study provides evidence to support that using data
analysis and risk mitigation strategy an organization can (1) plan to address software defects as
they are identified, (2) incorporate systematic trends in similar devices into the design FMEA for
future considerations, and (3) reduce time to recall duration by planning ahead for hazardous
situations and having appropriate processes in control for management of nonconforming
product to increase patient safety by reducing risk of harm.
71
CHAPTER 5
CONCLUSION
Summary
The medical device industry has seen an increase in software related recalls in recent
years, and the consistent trend has raised some concerns for organizational awareness of recall
management. Prolonging the time to recall duration negatively impacts patients’ risk of harm by
prolonging exposure to hazardous software defects that could pose risk of serious injury to end
user. This thesis examined the recall process, including relevant regulatory considerations, to
understand the factors that contribute to prolonged time to recall duration. Initial hypothesis
suggested that product traceability, recall protocols, and distribution pattern were causes for
prolonged time to recall duration that needed to be addressed. Using the PDSA framework this
thesis produced an analysis of FDA recall data to evaluate the impact these factors had on
software related recalls. Comparison of these factors was made against similar metrics for all
recalls to determine if any differences exist between the management of software related defects
versus all other defects that lead to initiation of the product recall. Quality tools were utilized to
uncover root cause analysis for prolonged time to recall and confirm gaps within the process that
can be attributed to timing delays.
Conclusions
The results of this study refuted the hypothesis for the original factors suggested to
contribute to prolonged time to recall. This study provided a connection between literature and
data to support that lack of compliance to regulation and ensuring processes for recall
management were not adequately implemented, disabling an organization’s ability to efficiently
72
recall defective software embedded medical devices. Additionally, the FDA recall overview
process contributes to prolonged time to recalls suggesting communication gaps between the
FDA and organization on timelines and roles and responsibilities of timely recall management.
Analysis of enforcement data confirmed that prolonged time to recalls existed for organization’s
who consistently struggled to maintain full compliance for procedures related to corrective and
preventive actions, control of nonconforming product, and notification of product action. This
suggests that the organization’s recall strategy would not ensure a reduced time to recall since
proper controls were not already in place.
Recommendations
The following recommendations were made to industry, and the FDA for using risk
mitigation and root cause analysis to address gaps within the software related recall process. In
order to reduce software defects leading to the initiation of recall, the organization and the FDA
must trend previous recall data to identify trends in recall defects. Using this data, the
organization should propose risk identification through FMEA and explore risk mitigation
strategies for defects that are applicable to their device type to reduce recall likelihood. Data
trending should occur on a semi-annual to annual basis.
Using data from this thesis, the organization should consider addressing gaps with the
recall management process by evaluating relevant procedures for notification of adverse event or
device malfunction, handling nonconforming on-market product, and developing a process for
recall management. The process improvement recommendation is to include Phase 0 for data
trending to understand time to recall delays in conjunction with analysis of inspection
observations for processes related to recall management.
73
The FDA should also make changes to their data monitoring and trending process to
address limitations discovered within data analysis to allow for better use of public data. The
FDA should trend root cause of recall by relevant quality system category to highlight the
nonconformity and additionally, include subcategories for deeper review. The recall database
should include figures on total product recovery percentage to alert industry and the public on
the status of remaining recall inventory on market. Within the recall database, there should be a
field to include when the organization requested termination versus when the FDA determines
termination to understand the time delay between request and response. For distribution patter,
the FDA should include additional field for region, and the number of sites impacted to make
meaningful correlations to product location impact throughout the recall process. With changes
made per these recommendations, future research can be explored to understand additional
underlying factors for prolonged time to recall duration.
74
REFERENCES
AAMI. (2018, May 22). Medical Device Recalls Surge in 2018 with Software to Blame.
Retrieved from Advancing Safety in Health Technology:
https://www.aami.org/newsviews/newsdetail.aspx?ItemNumber=6475
AAMI. (2019, Aug 20). About AAMI. Retrieved from Association for the Advancement of
Medical Instrumentation:
https://www.aami.org/membershipcommunity/content.aspx?ItemNumber=1292&navItem
Number=4603
ASQ. (2019, September 17). THE 7 BASIC QUALITY TOOLS FOR PROCESS
IMPROVEMENT. Retrieved from American Society for Quality: https://asq.org/qualityresources/seven-basic-quality-tools
CDRH. (2018). Medical Device Enforcement and Quality Report. U.S. Food and Drug
Administration.
Chen, Y.-J., Chiou, C.-M., Huang, Y.-W., Tu, P.-W., Lee, Y.-C., & Chien, C.-H. (2018). A
Comparative Study of Medical Device Regulations: US, Europe, Canada, and Taiwan.
Therapeutic Innovation & Regulatory Science, 62-69.
Deming, S. N. (2016). Statistics in the Laboratory: Control Charts, Part 3. American Laboratory,
12-15.
Deming, W. E. (1982). Out of The Crisis. Cambridge: Masschusetts Institute of Technology,
Center for Advanced Educational Services.
Deming, W. E. (1994). The New Economics. Cambridge: Massachusetts Institure of Technology,
Center for Advanced Educational Services.
75
European Parliament. (2017, April 5). REGULATION (EU) 2017/745 OF THE EUROPEAN
PARLIAMENT AND OF THE COUNCIL. Official Journal of the European Union . The
European Parliament and the Council of the European Union.
FDA. (2010, May 18). FDA 101: Product Recalls. Retrieved from U.S. Food and Drug
Administration: https://www.fda.gov/consumers/consumer-updates/fda-101-productrecalls
FDA. (2014, July 31). Recall Background and Definitions. Retrieved from Food and Drug
Administration: https://www.fda.gov/safety/industry-guidance-recalls/recallsbackground-and-definitions
FDA. (2017, July 25). Enforcement Report Information and Definitions. Retrieved from U.S.
Food and Drug Administration: https://www.fda.gov/safety/enforcementreports/enforcement-report-information-and-definitions
FDA. (2017, August 18). Philips Medical Systems (Cleveland) Inc, Cleveland, OH, 483.
Retrieved from Food and Drug Administraion:
https://www.fda.gov/media/111401/download
FDA. (2018, August 10). Arkon Anesthesia Delivery System recalled due to unexpected failed
state while in use or idle. Retrieved from Food and Drug Administration:
https://www.fda.gov/medical-devices/medical-device-recalls/arkon-anesthesia-deliverysystem-recalled-due-unexpected-failed-state-while-use-or-idle
FDA. (2018, April 1). CFR - Code of Federal Regulations Title 21 Part 803 Medical Device
Reporting. Retrieved from Food and Drug Administration:
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=803.3
76
FDA. (2018, September 14). Medical Device Overview. Retrieved from
https://www.fda.gov/industry/regulated-products/medical-device-overview:
https://www.fda.gov/industry/regulated-products/medical-device-overview
FDA. (2018, April 1). Medical Devices; Reports of Corrections and Removals. Retrieved from
Code of Federal Regulations Titles 21 Part 806:
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=806
FDA. (2018, August 31). Software as a Medical Device (SaMD). Retrieved from U.S. Food and
Drug Administration: https://www.fda.gov/medical-devices/digital-health/softwaremedical-device-samd
FDA. (2018, 09 26). What is a Medical Device Recall. Retrieved from U.S. Food & Drug
Administration:
https://www.fda.gov/MedicalDevices/Safety/ListofRecalls/ucm329946.htm
FDA. (2019, Sep 04). 21 CFR Part 7 Enforcement Policy. Retrieved from Food and Drug
Administration:
https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=7
FDA. (2019, April 15). Brainlab AG Recalls Spine & Trauma 3D Navigation Due to Inaccurate
Display That May Result in User Misinterpretation. Retrieved from Food and Drug
Administration: https://www.fda.gov/medical-devices/medical-device-recalls/brainlabag-recalls-spine-trauma-3d-navigation-due-inaccurate-display-may-result-user
FDA. (2019, July 12). Center for Devices and Radiological Health. Retrieved from U.S. Food
and Drug Administration: https://www.fda.gov/about-fda/office-medical-products-andtobacco/center-devices-and-radiological-health
77
FDA. (2019, July 19). Digital Health. Retrieved from Medical Devices:
https://www.fda.gov/medical-devices/digital-health
FDA. (2019, Jan 08). Medtronic Recalls Synergy Cranial Software and Stealth Station S7
Cranial Software Due to Inaccuracies Displayed During Surgical Procedures. Retrieved
from Food and Drug Administration: https://www.fda.gov/medical-devices/medicaldevice-recalls/medtronic-recalls-synergy-cranial-software-and-stealth-station-s7-cranialsoftware-due-inaccuracies
FDA. (2019, June 05). Unique Device Identification System (UDI System). Retrieved from U.S.
Food and Drug Administrarion: https://www.fda.gov/medical-devices/device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system
Fu, Z., Guo, C., Zhang, Z., Jiang, Y., & Sha, L. (2017). Study of Software-Related Causes in the
FDA Medical Device Recalls. International Conference on Engineering of Complex
Computer Systems, 60-69.
GAO. (2011). Medical Devices FDA Should Enhance Its Oversight of Recalls. United States
Government Accountability Office.
Gottlieb, S. (2018, April 26). Transforming FDA's Approach to Digital Health. Retrieved from
Food and Drug Administration: https://www.fda.gov/news-events/speeches-fdaofficials/transforming-fdas-approach-digital-health-04262018
Government of Canada. (2019, June 17). Medical Devices Regulations (SOR/98-282). Retrieved
from Justice Laws Website: https://laws-lois.justice.gc.ca/eng/regulations/sor-98282/page-1.html
Health Canada. (2016, November 3). Medical devices recall guide. GUI-0054. Canada.
78
Hora, M., Bapuji, H., & Roth, A. V. (2011). Safety hazard and time to recall: The role of recall
strategy, product defect type, and supply chain player in the U.S. toy industry. Journal of
Operations Management, 766-777.
ISO. (2016, March 01). ISO 13485. Medical devices — Quality management systems Requirements for regulatory purposes. Geneva, Switzerland: International Organization
for Standardization.
ISO. (2019, 16 Sep). About Us. Retrieved from International Organization for Standarization.:
https://www.iso.org/about-us.html
Kramer, D. B., Tan, Y. T., Sato, C., & Kesselheim, A. (2014). Ensuring Medical Device
Effectiveness and Safety: A Cross - National Comparison of Approaches to Regulation.
Food and Drug Law Journal, 69(1), 1-i.
Kumar, S., & Schmitz, S. (2011). Managing recalls in a consumer product supply chain - root
cause analysis and measures to mitigate risks. International Journal of Production
Research, 235-253.
Mezher, M. (2018, Jan 10). Device Recalls in 2017: Making Sense of the Numbers. Retrieved
from Regulatory Affairs Professionals Society: Device Recalls in 2017: Making Sense of
the Numbers
Mulero, A. (2018, March 08). Philips Medical Systems Draws Lengthy FDA 483 Over Issues
with Thousands of Complaints. Retrieved from Regulatory Affairs Professionals Society:
https://www.raps.org/news-and-articles/news-articles/2018/3/philips-medical-systemsdraws-lengthy-fda-483-over
National Archives and Records Administration. (2019, July 22). Federal Register. Retrieved
from National Archives: https://www.archives.gov/federal-register/cfr/subjects.html
79
Public Health Service. (2015, September 30). Warning Letter Merge Healthcare, Inc. Retrieved
from Food & Drug Administration: https://www.fda.gov/inspections-complianceenforcement-and-criminal-investigations/warning-letters/merge-healthcare-inc-09302015
Ronquillo, j. G., & Zuckerman, D. M. (2017). Software‐Related Recalls of Health Information
Technology and Other Medical Devices: Implications for FDA Regulation of Digital
Health. The Milbank Quarterly, 535-553.
Stericycle. (2018). Stericycle Recall Index Q1 2018. Stericycle.
Stericycle Expert Solutions. (2019). Recall Index Q2 2019. Stericycle Expert Solutions.
Taylor, M. J., Mcnicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2014). Systematic
review of the application of the plan–do–study–act method to improve quality in
healthcare. BMJ Quality & Safety, 290-298.
TGA. (2017, June 9). Is my product a therapeutic good? Retrieved from Therapeutic Good
Administration: https://www.tga.gov.au/sme-assist/my-product-therapeutic-good#1
TGA. (2019, Feb 7). Uniform recall procedure for therapeutic goods (URPTG). Retrieved from
Therapeutic Goods Act: https://www.tga.gov.au/publication/uniform-recall-proceduretherapeutic-goods-urptg
Wood, L. C., Wang, J. X., Olesen, K., & Reiners, T. (2017). The effect of slack, diversification,
and time to recall on stock market reaction to toy recalls. . International Journal of
Productions Economics, 244-258.
Wowak, K. A., & Boone, C. A. (2015). So many recalls, so little research: A review of the
literature and road map for future research. Journal of Supply Chain Management, 54-72.
80
Yu, J., & Hooker, N. H. (2019). Exploring relationships among recall effectiveness indicators in
the US meat and poultry industry. International Food and Agribusiness Management
Review, 97-106.
81
APPENDIX A: FLOWCHART OF RECALL
MANAGEMENT PROCESS
82
APPENDIX B: CODED SOFTWARE RELATED
RECALL DATA SET
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
11
1
1
18
1
1
26
1
1
28
1
1
30
1
1
55
1
1
68
1
1
75
1
1
77
1
1
83
1
1
92
1
1
96
1
1
112
1
1
118
1
1
124
1
1
125
1
1
127
5
1
132
1
1
133
4
1
137
1
1
138
1
1
140
1
1
141
1
1
145
1
1
Distribution
Pattern
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
25
271
Software
67
407
Software
32
653
Software
61
543
Software
36
427
Software
49
494
Software
32
474
Software
21
384
Software
51
127
Software
13
635
Software
63
878
Software
63
180
Software
13
241
Software
46
693
Software
44
408
Software
65
348
Software
24
409
Software
56
87
Software
28
922
Software
140
429
Software
80
563
Software
52
886
Software
28
358
Software
31
580
83
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
160
1
1
165
2
1
169
1
1
172
1
1
174
1
1
179
1
1
180
1
1
185
1
1
190
1
1
191
1
1
207
1
1
217
1
1
218
1
1
226
1
1
229
5
1
230
1
1
235
1
1
238
1
1
240
1
1
242
1
1
254
1
1
256
1
1
258
1
1
265
1
1
266
1
1
267
1
1
Distribution
Pattern
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
International oUS Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
International oUS Only
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
24
352
Software
104
439
Software
41
819
Software
48
286
Software
23
169
Software
41
395
Software
20
616
Software
30
384
Software
41
827
Software
18
266
Software
43
596
Software
76
309
Software
49
771
Software
32
98
Software
34
244
Software
17
335
Software
138
830
Software
42
850
Software
34
543
Software
51
59
Software
21
381
Software
24
285
Software
10
37
Software
27
541
Software
69
354
Software
61
786
84
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
269
1
1
279
1
1
280
1
1
285
1
1
318
1
1
321
1
1
324
1
1
334
1
1
366
1
1
372
1
1
378
1
1
380
1
1
385
1
1
392
1
1
395
1
1
396
1
1
400
1
1
404
1
2
414
1
1
415
1
1
417
1
1
419
1
1
420
1
1
421
1
1
423
1
1
426
1
1
Distribution
Pattern
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
20
340
Software
53
815
Software
38
221
Software
30
617
Software
103
794
Software
47
956
Software
167
770
Software
60
381
Software
56
742
Software
40
154
Software
42
1016
Software
51
315
Software
107
803
Software
24
225
Software
90
816
Software
34
318
Software
36
182
Software
145
426
Software
46
809
Software
162
361
Software
40
179
Software
57
259
Software
160
370
Software
36
168
Software
82
102
Software
50
890
85
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
432
1
1
447
1
1
449
1
1
453
2
1
458
2
1
464
1
1
472
1
1
474
1
1
481
2
1
485
1
1
489
1
1
493
1
1
497
1
1
499
2
1
508
1
1
518
1
1
522
1
1
523
1
1
526
1
1
527
1
1
528
1
2
530
1
1
537
1
2
539
1
1
540
1
1
541
2
1
Distribution
Pattern
Domestic &
International
Domestic &
International
Domestic - US
Only
International oUS Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
International oUS Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Defect
Category
Time to
Classification
Time
to
Recall
Software
34
325
Software
18
792
Software
47
174
Software
56
447
Software
14
451
Software
49
146
Software
10
298
Software
29
159
Software
36
62
Software
22
173
Software
15
136
Software
27
928
Software
39
190
Software
136
480
Software
185
263
Software
32
150
Software
36
712
Software
35
314
Software
41
687
Software
52
217
Software
129
135
Software
41
195
Software
164
444
Software
12
470
Software
113
247
Software
23
266
86
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
545
1
1
549
1
1
551
1
1
553
1
1
556
1
1
564
1
1
570
1
1
584
1
1
593
1
1
596
1
1
598
1
1
605
1
1
610
1
1
612
1
1
618
1
1
619
1
1
624
1
1
630
1
1
632
1
1
634
1
1
635
1
1
636
1
1
637
1
1
638
2
1
643
1
1
647
1
1
Distribution
Pattern
Domestic &
International
Domestic - US
Only
Domestic &
International
International oUS Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
International oUS Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
101
768
Software
24
154
Software
47
827
Software
25
264
Software
205
376
Software
13
194
Software
53
76
Software
25
158
Software
205
562
Software
235
685
Software
50
638
Software
50
238
Software
32
200
Software
2
699
Software
59
653
Software
16
366
Software
43
230
Software
47
382
Software
59
557
Software
40
677
Software
34
149
Software
45
646
Software
21
244
Software
33
1013
Software
27
888
Software
31
613
87
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
661
1
1
662
1
1
670
1
1
672
1
1
685
1
1
691
1
1
692
1
1
697
5
2
717
1
1
721
1
1
728
1
1
729
1
1
730
1
1
735
2
1
744
1
1
747
1
1
752
1
1
753
1
1
754
1
1
761
2
1
769
1
1
773
1
1
781
1
1
782
2
1
783
1
1
788
1
1
Distribution
Pattern
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
63
265
Software
33
258
Software
127
777
Software
19
232
Software
303
1145
Software
43
297
Software
38
244
Software
114
491
Software
21
208
Software
15
167
Software
25
207
Software
14
241
Software
24
201
Software
33
247
Software
29
322
Software
51
188
Software
43
222
Software
41
825
Software
15
232
Software
57
973
Software
48
594
Software
15
371
Software
30
371
Software
28
944
Software
26
371
Software
19
595
88
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
789
1
1
793
1
1
801
1
1
802
1
1
804
1
1
823
1
1
841
1
1
843
1
1
844
1
1
855
1
1
857
1
1
860
1
1
881
1
1
899
1
1
911
1
1
917
1
1
926
1
1
927
1
1
929
1
1
931
2
1
940
1
1
945
2
1
946
1
1
954
1
1
955
1
1
959
1
1
Distribution
Pattern
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Defect
Category
Time to
Classification
Time
to
Recall
Software
23
204
Software
27
391
Software
27
528
Software
30
378
Software
22
151
Software
20
181
Software
78
555
Software
296
1106
Software
71
72
Software
32
852
Software
39
161
Software
26
563
Software
48
118
Software
37
259
Software
33
112
Software
59
106
Software
311
1099
Software
20
322
Software
20
309
Software
34
229
Software
28
574
Software
84
875
Software
25
189
Software
62
483
Software
51
301
Software
47
307
89
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
960
1
1
961
1
1
963
1
1
964
1
1
968
1
1
972
1
1
980
1
1
981
1
1
985
1
1
986
1
1
991
4
1
993
1
1
1001
1
1
1009
1
1
1018
1
1
1023
1
1
1025
1
1
1026
1
1
1027
1
1
1029
1
1
1035
1
1
1037
1
1
1038
1
1
1041
1
1
1057
2
1
1058
1
1
Distribution
Pattern
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
International oUS Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
54
510
Software
81
559
Software
350
350
Software
74
806
Software
87
93
Software
34
262
Software
408
812
Software
327
332
Software
44
517
Software
49
505
Software
90
357
Software
48
208
Software
335
343
Software
48
572
Software
38
489
Software
30
168
Software
29
445
Software
33
239
Software
25
733
Software
28
359
Software
21
98
Software
34
307
Software
28
244
Software
28
202
Software
27
547
Software
32
367
90
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
1070
1
1
1071
1
1
1073
1
1
1074
1
1
1079
1
1
1090
1
1
1091
1
1
1095
2
1
1097
1
1
1118
1
1
1129
1
1
1137
1
1
1145
1
1
1147
1
2
1171
1
1
1173
1
1
1181
1
2
1184
1
1
1188
1
2
1202
1
1
1206
1
1
1208
1
1
1209
1
1
1214
1
1
1215
2
1
1216
2
1
Distribution
Pattern
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
372
1053
Software
13
538
Software
17
132
Software
48
308
Software
13
119
Software
31
126
Software
39
423
Software
36
312
Software
25
721
Software
21
501
Software
404
460
Software
30
393
Software
36
280
Software
118
490
Software
45
148
Software
434
1062
Software
29
425
Software
51
463
Software
24
436
Software
183
406
Software
38
378
Software
24
421
Software
6
385
Software
601
831
Software
411
996
Software
203
850
91
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
1218
1
1
1219
1
1
1229
1
1
1236
1
1
1237
1
1
1262
1
1
1267
1
1
1284
2
1
1290
1
1
1291
1
1
1301
1
1
1302
1
1
1303
1
1
1321
2
1
1325
1
1
1326
1
1
1331
1
1
1334
1
1
1335
1
1
1339
1
1
1349
1
1
1354
1
1
1364
1
1
1372
1
1
1373
1
1
1377
1
1
Distribution
Pattern
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
International oUS Only
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
245
463
Software
181
424
Software
93
275
Software
47
602
Software
22
632
Software
9
538
Software
44
583
Software
87
121
Software
166
404
Software
253
470
Software
197
456
Software
211
664
Software
189
251
Software
183
408
Software
182
363
Software
68
247
Software
176
364
Software
19
316
Software
252
414
Software
144
549
Software
166
377
Software
200
605
Software
152
349
Software
231
642
Software
221
288
Software
50
494
92
Recall Event
ID
Region of
Origin
Voluntary/
Mandated
1378
1
1
1379
1
1
1391
1
1
1411
1
1
1417
1
1
1424
1
1
1440
2
1
1444
2
1
1445
5
1
1448
1
1
1453
1
1
1474
2
1
1486
1
1
1495
1
1
1515
1
1
1521
1
1
1524
1
1
1525
1
1
1540
1
1
1542
2
1
1562
2
1
1586
1
1
1591
1
1
1602
2
1
Distribution
Pattern
Domestic &
International
Domestic - US
Only
Domestic - US
Only
International oUS Only
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
Domestic - US
Only
UNKNOWN
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic &
International
Domestic - US
Only
Domestic &
International
Domestic - US
Only
Domestic &
International
Defect
Category
Time to
Classification
Time
to
Recall
Software
177
345
Software
152
370
Software
182
396
Software
244
466
Software
42
431
Software
162
534
Software
63
386
Software
133
471
Software
144
489
Software
217
469
Software
77
265
Software
86
414
Software
62
456
Software
124
437
Software
96
259
Software
219
224
Software
92
233
Software
166
300
Software
57
380
Software
56
153
Software
50
302
Software
54
160
Software
48
112
Software
39
155
93
APPENDIX C: DETERMINATION OF DISTRIBUTION
FITTING FOR TIME TO CLASSIFICATION
94
95
APPENDIX D: BOX-PLOT RESULTS: OUTLIERS FOR
TIME TO CLASSIFICATION
Recall Event ID
Time to Classification
137
165
235
140
104
138
Recall Event ID
1214
1215
1216
Time to Classification
601
411
203
318
324
385
395
404
415
103
167
107
90
145
162
1218
1219
1229
1290
1291
1301
245
181
93
166
253
197
420
499
508
528
537
160
136
185
129
164
1302
1303
1321
1325
1331
211
189
183
182
176
540
545
556
593
596
113
101
205
205
235
1335
1339
1349
1354
1364
252
144
166
200
152
670
685
697
843
926
963
127
303
114
296
311
350
1372
1373
1378
1379
1391
1411
231
221
177
152
182
244
980
981
991
1001
1070
408
327
90
335
372
1424
1444
1445
1448
1495
162
133
144
217
124
1129
1147
1173
1202
404
118
434
183
1515
1521
1524
1525
96
219
92
166
96
APPENDIX E: DETERMINATION OF DISTRIBUTION
FITTING FOR TIME TO RECALL
97
98
APPENDIX F: BOX-PLOT RESULTS: OUTLIERS
FOR TIME TO RECALL
Recall
Event ID
Time to
Recall
Recall
Event ID
Time to
Recall
Recall
Event ID
Time to
Recall
26
28
55
653
543
494
618
632
634
653
557
677
1216
1236
1237
850
602
632
83
92
118
133
138
635
878
693
922
563
636
638
643
647
670
646
1013
888
613
777
1262
1267
1302
1339
1354
538
583
664
549
605
140
145
169
180
190
886
580
819
616
827
685
697
753
761
769
1145
491
825
973
594
1372
1377
1424
1445
642
494
534
489
207
218
235
238
240
265
596
771
830
850
543
541
782
788
801
841
843
855
944
595
528
555
1106
852
267
279
285
318
321
786
815
617
794
956
860
926
940
945
954
563
1099
574
875
483
324
366
378
385
395
770
742
1016
803
816
960
961
964
980
985
510
559
806
812
517
414
426
447
493
499
522
809
890
792
928
480
712
986
1009
1018
1027
1057
1070
505
572
489
733
547
1053
526
545
687
768
1071
1097
538
721
99
Recall
Event ID
551
Time to
Recall
827
593
596
598
612
562
685
638
699
Recall
Event ID
1118
1147
1173
1214
1215
Time to
Recall
Recall
Event ID
501
490
1062
831
996
Time to
Recall
100
APPENDIX G: CAUSE-AND-EFFECT, FISHBONE
DIAGRAM FOR ROOT CAUSE ANALYSIS OF
FACTORS LEADING TO RECALL INITIATION
FOR SOFTWARE RELATED DEFECTS
101
APPENDIX H: CAUSE AND EFFECT, FISHBONE DIAGRAM
FOR ROOT CAUSE ANALYSIS OF FACTORS LEADING
PROLONGED TIME TO RECALL DURATION
102
APPENDIX I: PROPOSED FMEA FOR CONFIRMED ROOT
CAUSES CONTRIBUTING TO TIME TO RECALL DELAY,
INCLUDING RISK MITIGATION RECOMMENDATIONS.
Revision
: 02
Purpose: To identify failure modes for root cause categories identified through cause-and-effect analysis to
mitigate patient risk of prolonged time to recall duration.
Instructions: Use the following template in conjunction with cause-and-effect analysis to identify failure modes
for root cause. Recommend risk mitigation based on failure mode and calculate risk priority numbers prior to risk
mitigation and after to determine if resulting conditions were implemented and effectiveness.
Use this FMEA as a combined tool for product FMEA to understand software defects that lead to recall initiation
and time to recall duration and develop FMEA to identify vulnerabilities in organization or FDA oversight of the
recall process to recommend and adapt continuous process improvements.
Notes: Risk Priority columns have been removed.
Process Name: Time to Recall for Software Related Recalls
Product FMEA
FM
EA
Step
#
Root
Cause
Potential
Failure or
Defect
Mode
Report
Errors
1
Potential
Causes
of
Failure(s
)
Design
or
validatio
n error.
User
error.
Current
Conditio
ns
Potential End
(Patient/
User) Effect
Patient/User
Current
Controls
and
Assumpt
ions
Resulting
Conditions
Recomme
nded
Mitigation
Error not
identified
during
design
review.
Reevaluate
software
design
files and
customer
complaints
to identify
source of
the issue.
Correct
with
software
upgrade.
Review
design file
and
perform
software
validation
to verify.
Review
design file
and
Software
Defects
Trends
Freeze
Mode
Design
or
validatio
n error.
User
error.
User
Error not
identified
during
design
review.
Interface
Malfunction
Algorith
m error.
User/Patient
Error not
found
during
Verificat
ion or
Propose
d
Mitigati
on
Respons
ible
Depart
ment
Verify
report
errors
with inhouse
testing or
on-site
maintena
nce.
Design
Verify
report
errors
with inhouse
testing or
on-site
maintena
nce.
Monitor
complain
ts for
Design
Design
103
Revision
: 02
Process Name: Time to Recall for Software Related Recalls
validatio
n or end
user trial
testing.
Unable to
view images
Design
or
validatio
n error.
User
error.
iOS and
Android
application
error
Lack of
validatio
n
controls
during
design.
Incorrect
results
Algorith
m error
for data
calculatio
n.
User
Error not
identified
during
design
review.
Patient/User
Error not
found
during
validatio
n or end
user trial
testing.
Patient/User
Error not
found
during
validatio
n or end
user trial
testing.
perform
software
validation
to verify.
Review
design file
and
perform
software
validation
to verify.
Update
software
code and
interface.
Perform
mobile
application
testing,
validate
new
version
and issue
new
software.
Reevaluate
software
design
files and
customer
complaints
to identify
source of
the issue.
Correct
with
software
upgrade.
recurring
issues.
Monitor
complain
ts for
recurring
issues.
Design
Monitor
complain
ts for
recurring
issues.
Design
Verify
report
errors
with inhouse
testing or
on-site
maintena
nce.
Design
Process FMEA
FM
EA
Step
#
2
Root
Cause
Potential
Failure or
Defect
Mode
Potential
Causes
of
Failure(s
)
Potential End
(Patient/
Organization
) Effect
Current
Conditio
ns
Current
Controls
and
Assumpt
ions
Resulting
Conditions
Recomme
nded
Mitigation
Verificat
ion or
Propose
d
Mitigati
on
Respons
ible
Depart
ment
Quality
104
Revision
: 02
Process Name: Time to Recall for Software Related Recalls
Inadequate
Corrective
Action
procedures
Organiza
tion
Protocol
Inadequate
complaints
procedure
Inadequate
process for
reporting of
adverse
events
Inability
to
adequatel
y address
root
cause
identifica
tion and
remediati
on.
Inability
to
address
customer
feedback
for
customer
resolutio
n into
design
for
patient
safety.
Inability
to meet
regulator
y
requirem
ents.
Correctiv
e action
procedur
e may be
in place
but does
not
address
all
regulator
y
requirem
ents.
External
complianc
e
assessment
of
procedure
against
CFR.
Patient/Organi
zation
Complai
nts
procedur
e in place
to
receive,
but not
fully
investigat
e
reported
defects.
Review
customer
complaints
process
with
customer
service,
quality,
and design
for gaps.
Ensure
process
aligns to
regulatory
expectatio
ns.
Patient/Organi
zation
Adverse
Events
procedur
e
inadequat
e to
ensure
reports
are
submitte
d timely.
External
complianc
e
assessment
of
procedure
against
CFR.
Patient/Organi
zation
Monitor
CAPA
reports
for
effective
ness.
Internal
audit
CAPA
process
at one
year
from
complian
ce
assessme
nt.
Create
metrics
for
quarterly
review of
complain
ts data to
ensure
relevant
issues are
identified
and
escalated
to design
and
recall
team for
trending
and
review.
Monitor
adverse
events
reports
for
timelines
s and
correct
notificati
on.
Internal
audit this
process
at one
year
from
complian
Quality
and
Custome
r Service
Quality
105
Revision
: 02
Process Name: Time to Recall for Software Related Recalls
Lack of
process for
notification
of product
action
Inability
to notify
relevant
stakehold
ers of
product
action.
Failure to
follow
procedure/pr
ocess to
meet
regulatory
requirement
s
Inability
to
maintain
effective
quality
system to
ensure
patient
safety
and
reduction
of risk to
harm.
Patient/Organi
zation
Current
process
failing to
notify
regulator
y
authoritie
s and key
stakehold
ers.
External
complianc
e
assessment
of
procedure
against
CFR.
Establish
new
procedure
including
all
stakeholde
rs in recall.
Patient/Organi
zation
Failure to
comply
to
regulatio
n could
lead to
regulator
y
enforcem
ent
action.
Quality
system
not fully
effective.
External
complianc
e
assessment
of quality
system.
Identify
relevant
gaps and
use quality
plans and
CAPA
process or
improvem
ents.
ce
assessme
nt.
Monitor
recall
notices
for
timelines
s and
correct
notificati
on.
Internal
audit this
process
at one
year
from
complian
ce
assessme
nt.
Track
status
updates
quarterly
through
managem
ent
review.
Quality
and
Regulato
ry
Senior
Leadersh
ip