Leveraging Statistical Process Control for Continuous Improvement of the Manufacturing
Process
by
Stephen Patrick Fuller
B.S. Operations Research, United States Military Academy, 2005
Submitted to the MIT Sloan School of Management and the Engineering Systems Division in Partial
Fulfillment of the Requirements for the Degrees of
In conjunction with the Leaders for Global Operations Program at the
Massachusetts Institute of Technology
June 2015
I
z
-
Master of Business Administration
and
Master of Science in Engineering Systems
0:
Cn
0 2015 Stephen Patrick Fuller. All rights reserved.
The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electronic
copies of this thesis document in whole or in part in any medium now known or hereafter created.
Signature of Author
______Signature redacted_
Engineering Systems Division,
Certified by
IIT Sloan School of Management
May 8, 2015
_________Signature redacted_
Bruce Cameron, Thesis Supervisor
Director, System Architecture Lab
Lecturer, Engineering Systems Division
Certified by
Signature -redacted_
Tauhid Zaman, Thesis Supervisor
KDD Career Development Professor in Communications and Technology
Assistant Professor of Operations Management
Accepted
Signature redacted
Munther A. Dahleh, William A. Coolidge Professor of Electrical Engineering and Computer Science
Chair, Engineering Systems Division Education Committee
Accepted by
Signature redacted_
MauraIgrion, Director of MIT Sloan MBA Program
MIT Sloan School of Management
This page intentionally left blank.
2
Leveraging Statistical Process Control for Continuous Improvement of the
Manufacturing Process
by
Stephen Patrick Fuller
Submitted to the MIT Sloan School of Management and the Engineering Systems Division on
May 8, 2015 in Partial Fulfillment of the Requirements for the Degrees of Master of Business
Administration and Master of Science in Engineering Systems
Abstract
Statistical Process Control (SPC) has been applied to manufacturing processes for several
decades as a means of ensuring product quality and has become a primary tool for the application
of continuous improvement efforts. Continued Process Verification (CPV) is a Food and Drug
Administration requirement that requires biopharmaceutical companies, such as Amgen, Inc., to
demonstrate control of commercial manufacturing processes. Furthermore, the Food and Drug
Administration's guidance on CPV specifically calls for the use of SPC. This thesis suggests
including the use of the Akaike information criteria (AIC), a recognized statistical model
selection criterion, for objective model selection for the purpose of establishing the most
representative control limits in the application of SPC. The most representative control limits are
instrumental in eliminating unnecessary use of resources in the evaluation of manufacturing data.
Thus, the use of AIC is one way to reduce waste in the entire process of monitoring the
manufacturing process, evaluating data, and making improvements to the manufacturing process.
In addition, this thesis forms several key concepts for effective use of SPC and continuous
improvement efforts when working with contract manufacturing organizations (CMOs). Finally,
this thesis will discuss the applicability of the work done related to SPC as the foundation for
effectively monitoring, evaluating and improving the manufacturing process.
Thesis Supervisor: Bruce Cameron
Title: Lecturer, Engineering Systems Division
Thesis Supervisor: Tauhid Zaman
Title: Assistant Professor of Operations Management, MIT Sloan School of Management
3
This page intentionally left blank.
4
Acknowledgements
I would like to thank Amgen for the opportunity to work on a meaningful project and to continue
to develop as a leader. I am especially grateful to my project supervisors, Dave Stadolnik and
Stephen Arboleda, and my project sponsors, Bill Rich and Shabbir Anik. Their guidance and
support throughout my project were critical to its success. I would also like to thank Mark
DiMartino and Fuat Doymaz for their insight and recommendations for critical aspects of the
project. Moreover, I would like to thank Onyx Pharmaceuticals for welcoming me and
empowering me to drive the project.
In particular, I owe a special thanks to the cross-functional, cross-site CPV Working Team
composed of both Amgen and Onyx personnel - Stephen Arboleda, Amanda Marshall, Dave
Cunningham, Kim Mears, David Han, Liang Liao, and Demei Leung. My experience working
with this team was refreshing and is a source of pride for me.
Additionally, I would like to thank MIT and the Leaders for Global Operations (LGO) program.
The LGO program is a world-class program with world-class staff that truly enables the students
to be successful. A special thanks to my MIT advisors, Bruce Cameron and Tauhid Zaman, for
providing insight and theoretical knowledge to solve real-world problems.
Finally, I would like to thank God, my beautiful wife, Stephanie, and my amazing daughters,
Olivia and Emily. I am so grateful to my amazing wife for allowing me to pursue a life goal. I
am forever indebted and will cherish our memories during this adventure always.
5
This page intentionally left blank.
6
Table of Contents
Abstract ...........................................................................................................................................
3
Acknowledgem ents.........................................................................................................................
5
List of Figures .................................................................................................................................
9
1 I Introduction ...........................................................................................................................
10
2
1.1
Project Context...............................................................................................................
11
1.2
Problem Statem ent .........................................................................................................
12
1.3
Thesis Overview and Hypothesis...............................................................................
13
Background............................................................................................................................
2.1
Industry Overview ......................................................................................................
19
2.2
Am gen, Inc.....................................................................................................................
20
2.2.1
2.3
2.3.1
3
4
15
Am gen M onitoring Structure ...............................................................................
Onyx Pharm aceuticals.................................................................................................
Onyx M onitoring Structure.................................................................................
Literature Review ..................................................................................................................
21
23
23
25
3.1
Statistical Process Control...........................................................................................
25
3.2
M axim um Likelihood and Score Function.....................................................................
32
3.3
Continuous Improvem ent and Contract M anufacturing.............................................
35
Application of Statistical Process Control..........................................................................
4.1
Data M odeling ................................................................................................................
7
38
38
4 .2
C on tro l L imits ................................................................................................................
4.2.1
Amgen Example: Effectiveness of Control Limits.............................................
42
4.2.2
Onyx Hypothetical Example: Effectiveness of Control Limits ...........................
45
4.2.3 Control Limit Considerations ...................................................................................
4.3
5
6
41
Application of Maximum Likelihood and Score Function ....................
47
48
4.3.1
Reprocessed Lot Background .............................................................................
48
4.3.2
Intermediate 3 Reprocessed Lot Discussion........................................................
51
Management of Continuous Improvement with Contract Manufacturing Organizations..... 59
5.1
Flexible C ontracts .......................................................................................................
59
5.2
T im eliness of D ata .....................................................................................................
62
5.3
Location of C ontrol Lim its...........................................................................................
65
Conclusions and Recommendations.................................................................................
66
6.1
Key Findings and Conclusions of the Application of Statistical Process Control......... 66
6.2
Key Findings and Conclusions for Management of Continuous Improvement with
C ontract M anufactures ..............................................................................................................
67
6 .3
N ext S tep s ......................................................................................................................
68
B ib lio grap h y .................................................................................................................................
70
8
List of Figures
Figure 1: Process for Effective Application of Statistical Process Control...............................
10
Figure 2: Relationship between Control Limits and Action or Rejection Limits for Process
P aram eters .....................................................................................................................................
22
Figure 3: Normal Quantile Plot, Box Plot and Histogram for Sample Data.............................
28
Figure 4: Control Chart for Future Data ...................................................................................
29
Figure 5: Control Chart for Modified Future Data ...................................................................
30
Figure 6: Quantile Plot and Shapiro-Wilk Test Result .............................................................
39
Figure 7: Normal Quantile Plot, Box Plot and Histogram for Example Uncertain Normallyd istrib uted D ata .............................................................................................................................
40
Figure 8: Yield Parameter Control Chart with Control Limit Violations..................................
43
Figure 9: Reject Rate Parameter Control Chart with Control Limit Violations .......................
44
Figure 10: Control Chart for Onyx Hypothetical Example Data...............................................
46
Figure 11: Drug Substance Intermediate Process for Intermediate 3 ........................................
49
Figure 12: Drug Substance Intermediate Process for Drug Substance and Drug Product........ 50
Figure 13: Production Stages for Small Molecule Drug Substance Intermediates and Drug
S ub stan ce ......................................................................................................................................
51
Figure 14: Drug Substance Intermediate Flow of Impurities in Relation to Reaction Water
C on ten t..........................................................................................................................................
52
Figure 15: Normal Quantile Plot for Reaction Water Content Parameter Data .......................
53
Figure 16: Normal Quantile Plot, Box Plot and Histogram for Reaction Water Content Parameter
Data ...............................................................................................................................................
54
Figure 17: Timeline for the Flow of Information from the CMO to Onyx for Drug Substance
Interm ediates 1, 2, and 3 .......................................................................................................
9
. 63
1
Introduction
The purpose of this thesis is to suggest the use of the Akaike information criteria (AIC)
in selecting an appropriate model for manufacturing data and to form key concepts for effective
use of Statistical Process Control (SPC) when working with contract manufacturers. The
effectiveness of SPC methods for ensuring quality and as a tool for continuous improvement is
dependent on the entire process of collecting data, evaluating that data, and making
improvements based on the evaluation as illustrated in Figure 1. There are opportunities within
this entire process to reduce waste in order to make the use of SPC more effective, and the
selection of appropriate data models is one such opportunity. In addition, the intricacies involved
with this entire process in the context of contract manufacturing require the application of key
concepts in order to ensure the process' effectiveness. The content of this thesis is based on
research conducted during an internship with Amgen Inc. working in conjunction with Onyx
Pharmaceuticals as part of the Leaders for Global Operations program at the Massachusetts
Institute of Technology.
Statistical Process Control
Figure 1: Process for Effective Application of Statistical Process Control
10
1.1
Project Context
Amgen completed its acquisition of Onyx Pharmaceuticals as an independent subsidiary
on October 1, 2013. As part of the integration with Amgen's quality systems, Onyx was required
to implement Continued Process Verification (CPV). CPV is Stage 3 of Process Validation, an
industry term for collecting and evaluating data beginning in the manufacturing process design
stage and continuing through the commercial manufacture stage for human drug and biological
products. CPV is defined as an ongoing program that provides continual assurance that the
manufacturing process "remains in a state of control (the validated state) during commercial
manufacture."'
Regarding the integration of acquired companies into Amgen's quality systems in the
context of CPV, the process is difficult and can take a long time. Additionally, outcomes most
likely vary depending on the acquisition's age, products and manufacturing strategy. Age of the
acquired company may indicate the maturity of its quality systems and data collection
capabilities. Age could also indicate whether the acquired company already has a CPV program
established. The acquired company's products' maturity indicates how much data is available.
The manufacturing strategy involves the use of in-house manufacturing or contract
manufacturing and for what stages of the manufacturing process either or both are used. As a
brief example, Amgen acquired BioVex in 2011. BioVex was founded in 2006, so it was a
relatively young company, and used contract manufacturing 2 . As of mid-2014, Amgen continued
to integrate BioVex data and its quality systems.
"Guidance for Industry; Process Validation: General Principles and Practices," January 2011.
http://www.fda.gov/downloads/Drugs/Guidances/UCM070336.pdf.
2 "Excerpt from S-i/A SEC Filing, Biovex Group, Inc.," September 21, 2006. http://sec.edgar-online.com/biovexgroup-inc/s-i a-securities-registration-statement/2006/09/2 I/section2 1.aspx.
1
11
Onyx's second commercial drug, Kyprolis@ (carfilzomib), is a treatment for multiple
myeloma, a cancer of plasma cells that produces multiple bone tumors. Under Food and Drug
Administration (FDA) and European Medicine Agency (EMA) requirements, as well as
integration requirements with Amgen, CPV is required for Kyprolis. Due to the expedited
approval of Kyprolis and the drug's relatively early life as a commercially manufactured drug,
CPV had not been established for Kyprolis. The ability to detect unwanted variability is inherent
to ensuring that the manufacturing process remains in a state of control, and SPC provides the
ability to detect unwanted variability.
1.2
Problem Statement
Onyx employs contract manufacturing to produce Kyprolis and already collects data from
its contract manufactures to ensure parameters remain within predetermined specification limits.
However, Onyx has not applied SPC methods to detect unwanted variability that ensure control
of the manufacturing process. Furthermore, while Onyx does have a process to collect data,
Onyx does not have a process to evaluate that data and translate the results into insight and
improvements to the manufacturing process. In other words, the process that enables continuous
improvement is not present. Therefore, the goal is to implement SPC methods that allow Onyx to
detect unwanted variability and propose a process for evaluating data that provides insight about
improvements to the manufacturing process. However, the implementation of SPC methods and
the proposal for establishing continuous improvement must consider the following.
1. Onyx must follow Amgen quality requirements.
2. Onyx can utilize any Amgen resources as needed.
3. Onyx's infrastructure is not aligned with that of Amgen for executing CPV.
12
A key assumption of this research is that the parameters Onyx already monitors are the
correct parameters. In other words, this research assumes that the currently monitored parameters
indicate the quality of the final product and that there are no additional parameters that require
monitoring to ensure the quality of the final product.
1.3
Thesis Overview and Hypothesis
The content of this thesis is broken up into six chapters.
Chapter one provides the purpose of the thesis, the context of the material and data
presented, and the statement of the problem addressed.
Chapter two gives company and industry background. Additionally, this chapter provides
a description of the entire monitor, evaluate and improve processes at Amgen and Onyx.
Chapter three offers a literature review focusing on SPC, maximum likelihood and score
functions, and continuous improvement related to contract manufacturing.
Chapter four discusses the application of SPC and maximum likelihood and a score
function using data from the manufacture of Kyprolis. This chapter focuses specifically on
choosing the most representative model for a data set in order to set control limits for evaluating
whether a process is in a state of control.
Chapter five discusses continuous improvement in the context of contract manufacturing.
This chapter focuses on the restrictions of quality agreements with contract manufacturing
organizations (CMOs), the importance of timely data, and the effects of the proximity of control
limits to the manufacturing floor.
Chapter six concludes the thesis with key findings and recommendations for next steps.
13
The hypothesis that we test is subjective data modeling contributes to non-representative
control limits. It is important to determine the best statistical model for a given set of data to
ensure control limits are as representative as possible. Non-representative control limits can
contribute to false signals, which are a source of unnecessary use of resources, namely, the time
and effort used to investigate false signals. Non-representative control limits can also contribute
to missed signals, leading to downstream quality issues. This is a simple, non-directional and
associative hypothesis with the independent variable, statistical models, and the dependent
variable, representative control limits.
This thesis will also include an evaluation of the relationship with contract manufacturers
in the context of continuous improvement efforts. It is important to determine the effect of
contract manufacturing on continuous improvement efforts, not to discredit the financial
justification for contract manufacturing, but to develop key concepts for managing improvement
efforts in the context of contract manufacturing in order to ensure quality and supply to the
market. This is an associative evaluation of the relationship between the hiring firm and contract
manufacturing organizations (CMOs) related to the implementation of CPV.
14
2
Background
Amgen Inc. is a biopharmaceutical manufacturer in the biotechnology industry. The
global biotechnology industry was estimated at $288.7 billion in 2014 and is expected to grow to
$444.9 billion in 20193. Founded in 1980, Amgen is considered a veteran company, despite its
young age relative to other international companies, due to the proliferation of startups and
acquisitions in the industry. Additionally, Amgen is a major player in the industry with an
estimated $20.2 billion in revenue in 20144. Amgen has historically focused on developing
human therapeutics in the areas of supportive cancer care, inflammatory disease and kidney
function, or nephrology 5 . Amgen has historically acquired companies in order to complement
internal capabilities and programs. However, the focus of recent and future acquisitions involves
a strategy of ensuring a pipeline of new products6 as patents for some products expire and
expanding into the cancer therapeutics market as demonstrated by the acquisition of Onyx
Pharmaceuticals. The biopharmaceutical manufacturer conducts business internationally with
headquarters in Thousand Oaks, California and international sites in Puerto Rico, Ireland,
Netherlands, and Singapore. U.S. sites include Rhode Island, California (San Francisco), and
Massachusetts.
Onyx Pharmaceuticals was founded in 1992 and focuses on the discovery and
development of cancer therapies 7. Onyx is headquartered in South San Francisco, California.
Prior to its acquisition, Onyx Pharmaceuticals had one blockbuster drug, Nexavar® (sorafenib).
However, Bayer co-markets Nexavar with Onyx and owns rights to the kidney and liver cancer
3 Phillips,
Jocelyn. Global Biotechnology. IBIS World Industry Report. Melbourne, Australia: IBISWorld Services,
January 2015. http://clients 1 .ibisworld.com/reports/gl/industry/default.aspx?entid=2010.
4 Ibid.
5Ibid.
6 "Amgen Outlines Long-Term Strategy." Amgen News Release, February 7, 2013.
7 "Onyx Pharmaceuticals, Inc. SWOT Analysis." MarketLine, November 2013.
15
treatment, leaving Onyx with regorafenib (Stivarga@), palbociclib and carfilzomib (Kyprolis@).
Stivarga is a colon and rectal cancer treatment and is also co-marketed with Bayer. Pfizer
developed palbociclib and markets it as IBRANCE@. Upon regulatory approval, Onyx will
receive royalties for IBRANCE, a breast cancer therapeutic. Kyprolis, potentially a blockbuster
drug8 , was developed by Onyx and received fast-track status from the FDA. Fast-track status is a
designation that includes expedited review for drugs that meet an unmet medical need. Kyprolis
is currently approved as a treatment for patients that have had at least two other prior therapies.
Additionally, Kyprolis is currently manufactured by contract manufacturing organizations
(CMOs).
Many industries utilize contract manufacturing for multiple reasons including a
preference to avoid heavy capital investment in manufacturing facilities and equipment. Onyx
may have avoided developing its own manufacturing facilities for this reason or because of its
early joint ventures with companies like Bayer that already had manufacturing facilities.
Additionally, contract manufacturing spreads out risk for the hiring company 9 in that the hiring
company can access manufacturing capability in multiple geographies without the risk of losing
capital investment due to a natural disaster. However, there are some disadvantages, especially
related to the contract between the hiring company and the contract manufacturer, such as easy
access to manufacturing data.
Contract manufacturing is not uncommon in the biopharmaceutical industry. At the end
of 2009, the pharmaceutical contract manufacturing market was estimated at $22.5 billion with
pharmaceutical companies contracting out 30% of manufacturing and keeping 70% of
Guha, Malini. "Amgen Swallows Onyx Whole." Nature Biotechnology 31, no. 10 (October 2013): 859-60.
doi:10.1038/nbtl013-859.
9 Knodell, Jenny. "The Benefits and Disadvantages of Outsource Manufacturing." Accessed January 21, 2015.
http://blog.iqsdirectory.com/general-industry-articles/outsource-manufacturing/.
8
16
manufacturing in-house'. Additionally, executives predict a reverse of this trend to 70% contract
manufacturing". Furthermore, contract manufacturing could play a larger role in the
biopharmaceutical industry moving forward as patents expire and as the threat of biosimilars
grows. Some research indicates that the contract manufacturing industry could grow heavily in
Asian countries due to the capacity, capital and cost concerns of biopharmaceutical companies
based in the U.S. and Europe. However, intellectual property and quality issues would be a
concern for U.S. and European firms that consider using Asian CMOs. Consideration for use of a
CMO in the biopharmaceutical industry includes the presence of a Good Manufacturing Practice
(GMP) facility, the capacity of that facility, and the location or locations of facilities.
While Amgen utilizes both in-house and contract manufacturing for its products, Onyx
currently uses only CMOs to manufacture Kyprolis. For Onyx, the contract can be a constraint
because it requires a very lengthy and drawn out process to revise. Additionally, the contract is
the source of a dynamic between Onyx and the contract manufacturer involving ownership of
continuous improvement efforts. In one sense, the contract manufacturer has incentive for
continuous improvement in order to reduce costs. However, Onyx has an obligation to
continuous improvement of the manufacturing process based on the FDA's intended outcome of
Continued Process Verification (CPV) indicated in its guidance including the requirement to use
Statistical Process Control (SPC) to demonstrate control of the manufacturing process. In this
relationship, the contract manufacturer produces the data from the manufacturing process and, in
10 Han, Chaodong, Tobin Porterfield, and Xiaolin Li. "Impact of Industry Competition on Contract Manufacturing:
An Empirical Study of U.S. Manufacturers." InternationalJournal ofProductionEconomics 138, no. 1 (July 2012):
159-69. doi: 10.1016/j.ijpe.2012.03.015.
" Auerbach, Mike. "Contract Manufacturing Trends This Year & Beyond." PharmaceuticalProcessing26, no. 1
(January 2011): 8-12.
12 Chun, Soo Jin. "Brokering Strategic Partnerships between Asian and Western Biopharmaceutical Companies in
the Global Biologics Market: Assessment of Capabilities of Asian Participants in the Biologics Contract
Manufacturing Organization Marketplace." Thesis, Massachusetts Institute of Technology, 2009.
http://dspace.mit.edu/handle/1721.1/63014.
17
some cases, the data from product testing. The CMO then compiles this data and sends it to
Onyx for verification, analysis, filing and decision making for release of products. Onyx can use
the data to gain insight about improvements to the product or the manufacturing process.
However, if an improvement requires a change to the manufacturing process, the change must be
tested and approved by both Onyx and the contract manufacturer. Additionally, the contract may
need to be revised. Furthermore, if Onyx requires anything not specified in the contract such as
monitoring of a new parameter in the manufacturing process, the contract may need revision.
This could be a significant impediment for any continuous improvement efforts that result from
Onyx's analysis of the data. Essentially, incentives are misaligned. Onyx must demonstrate
control of the manufacturing process, but the contract limits its ability to do so, and the CMO
could realize cost savings from continuous improvement but the contract limits its ability to do
so. A more flexible contract would allow Onyx access to data that it needs to demonstrate control
of the manufacturing process, thus meeting the requirement of CPV. Likewise, a flexible contract
would allow the CMO more freedom in implementing continuous improvement of the
manufacturing process.
An important distinction in the pharmaceutical industry is between small molecule and
large molecule drugs. We will focus on the distinction in the manufacturing processes for each
type. Small molecule drugs are typically made through a series of chemical reactions that result
in an active pharmaceutical ingredient (API), which is then processed into pills. Large molecule
drugs, also known as biologics and biopharmaceuticals, are made using biotechnology, meaning
the manufacturing process involves using live organisms, often to produce a protein for injection
into patients. The research conducted for this thesis includes data for Kyprolis, which is a small
molecule drug, though, it is administered to patients through injection. Whether small or large
18
molecule, the manufacturing process follows the same general steps. Amgen uses the term, drug
substance, to describe the first step of the manufacturing process which produces the API.
Amgen uses the term, drug product, to describe the second step of the process which takes the
API and combines it with other substances to make it suitable for administering to patients. An
ancillary step that will not be discussed in this thesis is fill-finish in which the drug product is
filled into vials or another delivery device, labeled, and packed for shipment to distributors.
2.1
Industry Overview
Brand name pharmaceutical manufacturing in the U.S. is currently experiencing a large
wave of drug patent expirations. In response to expiring patents, many firms in the industry enter
into competition with biosimilar manufacturers and subsequently enter into licensing agreements
with the competitors. Another recent trend in the industry involves brand name manufacturers
3
.
beginning to include biopharmaceuticals, or biologics, in their product portfolios
The U.S. biotechnology industry is a very diverse industry. Human health technologies,
which include biopharmaceuticals, make up 57% of industry revenue. The U.S. biotechnology
industry is characterized by consolidation and mergers and acquisitions activity. Amgen holds a
14.8% market share of this industry' 5 and is susceptible to the effects of patent expirations that
also characterize the industry. Amgen is typical of the industry in that it has completed several
acquisitions over the past four years including the aforementioned acquisition of Onyx.
The global biotechnology industry is similar to the U.S. industry in that it is characterized
by diverse activities. Additionally, the industry is characterized by growth and varied operational
13 Turk, Sarah. "Brand Name Pharmaceutical Manufacturing in the US." IBISWorld 32541A (n.d.).
http://clients 1.ibisworld.com/reports/us/industry/default.aspx?entid=487.
" Ibid.
15 Phillips, Jocelyn. "Biotechnology in the US." IBISWorld NNOOI (n.d.).
http://clients 1 .ibisworld.com/reports/us/industry/default.aspx?entid=200 1.
19
models with the majority of revenue coming from Europe and the U.S' 6 . Within this industry,
Amgen is typical in that it receives about 77.5% of its revenue from its U.S. operations. This
indicates one reason for the firm's international expansion in recent years. Moreover, the
biopharmaceutical company looks to international markets for additional revenue as patents
expire and the threat of biosimilars increases.
2.2
Amgen, Inc.
Amgen is a major company in the global biotechnology industry with 7.0% of market
share' 7 . As Amgen continues international expansion efforts, the biopharmaceutical company is
implementing strategies to increase product offerings and taking steps to reduce costs and boost
profitability. Related to increasing product offerings and in an effort to expand its oncology
portfolio, Amgen completed its acquisition of Onyx Pharmaceuticals, a producer of cancer
therapies. Related to reducing costs, Amgen has initiated many efforts to improve operational
efficiencies.
The Onyx acquisition is a source of reduced profit for Amgen due to costs associated
with the acquisition and amortization of rights for Onyx technologies. In the short-term, the
acquisition provides an increase in product offerings but is also a source of increased costs. The
effects of the acquisition indicate that improvements in operational efficiencies play a key role in
assuring the profitability of the company.
The work done with Statistical Process Control (SPC) to implement Continued Process
Verification (CPV) and decrease the unnecessary use of resources for quality issues and
16 Phillips, Jocelyn. "Global Biotechnology." IBISWorld L6724-GL (n.d.).
http://clients 1.ibisworld.com/reports/gl/industry/default.aspx?entid=20 10.
17Ibid.
20
investigation of false signals contributes to the effort of improving operational efficiencies.
Furthermore, in an industry experiencing a trend in acquisitions, the work done can contribute to
more efficient and less costly future acquisitions. More importantly, the work done with CPV
ensures quality products for patients with serious illnesses and adherence to regulatory
requirements. Consequently, the work done allows Amgen to increase its product offerings.
2.2.1
Amgen Monitoring Structure
Amgen has a monitoring structure for manufacturing processes and product quality based
on several documents, IT infrastructure and organizational structure. The two broadest categories
of parameters are in-process (or process) parameters and product parameters. Process parameters
involve measurement of a step during the manufacturing process such as percent of reaction
completion. The Process Development group at Amgen manages data and information for
process parameters for products manufactured by contract manufacturers under the Process
Monitoring Program. Product parameters involve measurement of a parameter after manufacture
is complete and are therefore post-manufacture parameters. For example, methods of
chromatography, such as high pressure liquid chromatography (HPLC), measure impurities in
active pharmaceutical ingredient (API). Data for product parameters result from testing samples
of completed product. The Quality group at Amgen manages data and information for product
parameters under the Product Data Monitoring Program.
Process parameters are further designated as either operational or performance
parameters. Operational parameters are independent of the product itself. For example, an
operational parameter could be the amount of time taken to add a chemical to a solution.
Performance parameters are dependent on operational parameters and the process itself. For
example, a performance parameter could be percent of reaction completion.
21
The Process Development group at Amgen uses several documents to track and maintain
information about the different types of process parameters along with associated limits. Action
limits reflect process validation acceptance criteria and rejection limits are based on product
safety information or regulatory expectations. Control limits are statistically calculated using
SPC techniques. Violation of action or rejection limits is stricter in the sense that a violation of
these limits will lead to an immediate stop in the process and investigation to determine the
cause. The result of a control limit violation is less strict in the sense that the process will not be
stopped, but an investigation will be initiated that may prevent the release of the product. Figure
2 intends to capture the relationship between control limits and action limits.
There are no issues with data that
falls within the control limits.
Data points that violate the control
limits but fall within action/rejection
limits initiate an investigation.
Data points that violate the
action/rejection limits signal an
immediate stop in the process and an
investigation.
Figure 2: Relationship between Control Limits and Action or Rejection Limits for Process Parameters
Product parameters may be referred to as analytical or release test results. The Quality
group at Amgen tracks and maintains product parameter information in a separate document.
This document provides the specification limits for all product parameters. However, the
22
statistically calculated control limits for product parameters are not contained in this document
but are managed by a group within Quality at Amgen.
One other group involved in the monitoring programs at Amgen is the Quality
Engineering (QE) group. QE plays an instrumental role in the application of SPC, especially
calculating control limits. QE personnel are the subject matter experts at Amgen for the
application of SPC.
2.3
Onyx Pharmaceuticals
Onyx is a leading cancer therapeutic developer and was a prime acquisition target in the
industry because of its status as a midcap commercial-stage oncology company. It was a
particularly good fit for Amgen because Amgen sought access to a growing cancer therapeutics
market and the acquisition would provide Amgen an opportunity for intermediate and long-term
growth during a period of expiring patents. For Onyx, it was a period when they would have had
to invest heavily in trials demonstrating the efficacy of Kyprolis 8. Kyprolis is relatively young
in its product lifecycle, having received accelerated approval from the FDA in July, 2012.
Additionally, as previously mentioned, Onyx uses only contract manufacturing to produce
Kyprolis.
2.3.1
Onyx Monitoring Structure
There is a subtle difference between Onyx's organizational structure and Amgen's
organizational structure due to Onyx's sole use of contract manufacturing, Amgen's primary use
of its own manufacturing capability, and Amgen's well developed processes for applying SPC.
The monitoring structure reflects this discrepancy. Similar to Amgen, Onyx has a Quality group
18 Guha, Malini. "Amgen Swallows Onyx Whole." Nature Biotechnology 31, no. 10 (October 2013): 859-60.
doi: 10.1038/nbtl013-859.
23
that ensures products meet specifications to ensure product quality. Somewhat similar to
Amgen's Process Development group, Onyx's Technical Services group has oversight of the
manufacturing process. However, at Onyx, the division between drug product and drug
substance is more noticeable. At Onyx the Quality group has individuals that focus on drug
substance and others that focus on drug product. Similarly, the Technical Services group has
individuals that focus on drug substance and others that focus on drug product. This is likely the
result of Onyx's use of contract manufacturing. Because certain contract manufacturers are
responsible for drug substance and others are responsible for drug product, it makes sense that
Onyx's organizational structure would reflect this specialization of CMOs. In other words, it
makes intuitive sense to assign individuals to work with one CMO or only the CMOs responsible
for drug substance or drug product rather than multiple CMOs and both stages of production.
Onyx distinguishes less between process monitoring and product data monitoring than between
drug substance and drug product because the monitoring programs are less developed at Onyx.
Because Onyx requires the application of SPC and the implementation of the use of
trending data against control limits to identify unwanted variability, Onyx's monitoring structure
is less complicated with fewer documents to manage and less IT involvement. Regarding process
parameters, Onyx does designate them as either operational or performance parameters as does
Amgen. Process parameters have associated action limits that Onyx defined in process validation
studies and maintains in quality documents. Product parameters are included in a product
specification document. Additionally, Onyx does not have a group that serves as statisticians or
subject matter experts in the application of SPC as does Amgen. This is most likely due, again, to
the absence of the application of SPC.
24
3
Literature Review
The research for this thesis includes applications of Statistical Process Control (SPC) in
order to present issues related to modeling data prior to applying SPC methods. The research for
this thesis also includes maximum likelihood and score functions to discuss approaches to
objectively comparing data models to address the issues related to uncertainty when applying
SPC methods. Finally, the research for this thesis explores general process improvement
methodologies to develop background for process improvement efforts that allow Onyx to
monitor, evaluate and improve its manufacturing process in the context of contract
manufacturing.
3.1
Statistical Process Control
SPC has interesting roots including Bell Laboratories, World War II, and the rise of
Japan as a leader in quality and productivity. W.A. Shewhart introduced the control chart in a
Bell Laboratories memorandum in 1924. In 1940, prior to the U.S. entrance into World War II,
the U.S. War Department published a guide for using control charts to analyze process data. And
in 1946, the U.S. War Department invited W. E. Deming to Japan to assist in rebuilding Japanese
industry, beginning the education of Japanese industrial managers in SPC methods' 9 . Deming
contributed greatly to the industrial accomplishments of Japan post-World War
1120.
Jersey: Wiley, c2013., 2013.
Petersen, Peter B. "The Contribution of W. Edwards Deming to Japanese Management Theory and Practice."
Academy of Management Best PapersProceedings, 133-37. Academy of Management, 1987.
'9 Montgomery, Douglas C. Introductionto StatisticalQuality Control. Hoboken, New
20
doi: 10.5465/AMBPP. 1987.17534034.
25
Montgomery defines SPC as "achieving process stability and improving capability
through the reduction of variability." 2 ' Montgomery lists seven tools of SPC that allow for the
determination of whether a process is stable or repeatable.
1. Histogram or stem-and-leaf plot
2. Check sheet
3. Pareto chart
4. Cause-and-effect diagram
5. Defect concentration diagram
6. Scatter diagram
7. Control chart
Shewhart's control chart is included as the most technically sophisticated of the seven tools.
The development and analysis of control charts for normally-distributed data are
relatively straightforward. Given a normally-distributed data set, the control chart is developed
by using the mean, p, of the data as the centerline, adding three standard deviations, 3a, to the
centerline for the upper control limit and subtracting three standard deviations, 3a, from the
centerline for the lower control limit. The upper control limit and lower control limit are given
by the equation
i
3a.
These are referred to as "3u" control limits. The following sample data in Table 1 set is taken
from Montgomery and represents compressive strength of parts manufactured by an injection
molding process.
2
Montgomery, Douglas C. Introduction to StatisticalQuality Control. Hoboken, New Jersey: Wiley, c2013., 2013.
26
Cor
ressive Strength Sam le Data Points
81.2
75.8
78.4
78.0
86.2
81.5
81.3
82.1
78.2
77.8
75.7
75.2
78.2
73.4
84.1
73.8
82.0
73.4
78.2
84.4
88.6
80.8
75.3
79.2
75.7
80.6
79.2
78.8
84.5
84.5
78.8
82.5
87.3
81.5
71.1
79.3
78.6
80.2
83.5
78.6
84.2
75.7
81.8
74.5
74.3
81.7
75.3
80.8
79.2
80.6
Table 1: Normally-distributed Sample Data for Compressive Strength2
Based on the analysis in Figure 3, the data seems to be normally-distributed. Table 2
shows the centerline and the 3a- control limits.
22
Montgomery, Douglas C. Introduction to StatisticalQuality Control. Hoboken, New Jersey: Wiley, c2013., 2013.
27
0&
0
Q-
-
0.94
-0.9
.00
C
M
:3
-0.82
0
-0.65
1L64-
z
-0.45
-- 33-
-0.25
r-0.14
-0.08
0.04
-0.01
70
75
80
85
90
Figure 3: Normal Quantile Plot, Box Plot and Histogram for Sample Data
Mean/Centerine
79.533
Standard Deviation
3.8571
UCL
91.1043V
LCL
67.9617
Table 2: Mean, Standard Deviation and 3a Control Limits for Sample Data
The data in Table 1 is used to calculate the control limits. Future data will be trended
against these control limits to determine whether the process is in control. Figure 4 shows the
23
control chart for the next 75 data points for the same process trended against the control limits
Montgomery, Douglas C. Introduction to Statistical Quality Control. Hoboken, New Jersey: Wiley, c2013., 2013.
28
L
90-
9
1
85
U)
80L
~b
E
A
--Ii--
~
/ /
LJJ
jj
I
II
&
75-
~LL
V
705
10
20
30
40
50
60
70
80
Figure 4: Control Chart for Future Data
In this example, all data points fall within the upper and lower control limits indicated by
the uppermost and lowermost horizontal dotted lines, demonstrating that there is no unwanted or
special cause variation in the process, though there may be high variability. A point outside the
control limits would have indicated unwanted or assignable cause variation. In that case, the data
point would require investigation to determine the root cause of the variation. The use of 3cr
control limits with normally-distributed data means that 99.73% of data points should fall within
these statistically calculated limits. This also means that only 0.27% of data points will fall
outside of the control limits. In other words, there is a 0.27% chance of type I error, or a false
signal, meaning that the data point outside the control limit is due to the ordinary variation in the
process and therefore, does not require investigation. Consequently, assuming that the normal
29
distribution is a good model for the data, it is very likely that a data point outside the upper or
lower control limit represents unwanted variation in the process. In Figure 5 below, the data from
Figure 4 has been modified to illustrate violation of control limits. There are several points that
violate the control limits, indicating unwanted variability in the manufacturing process and the
manufacturing process is out of control, or indicating the manufacturing process has changed.
100
9590-
I
M85-
802
1
70656010
20
30
40
50
60
70
80
Figure 5: Control Chart for Modified Future Data
The control chart is the basic tool for further analysis. Use of the control chart in
combination with process capability, Cpk, provides better understanding of the manufacturing
30
process through an understanding of the uniformity of a process
.
Cpk is calculated using the
following equation.
Cpk
=
min(CPU, CPi)
where Cp, is the process capability of the upper specification limit (USL) and C, 1 is the process
capability of the lower specification limit (LSL) given by the following equations.
USL - i
3oy - LSL
3uThe control chart is also used in the application of run rules, or rules that allow for the
detection of smaller shifts in process variability. The Nelson rules and Western Electric rules are
examples of run rules. The Western Electric Statistical Quality Control Handbook refers to the
run rules as tests for instability25 . Table 3 references the four Western Electric tests.
Descri tion
Test
Test 2
Two out of three consecutive points outside 2a
Test 4
Eight consecutive points. on one side of the centerline
Table 3: Western Electric Run Rules
Due to the primary objective of establishing the ability to detect special cause variation as
a requirement for Continued Process Verification (CPV), the material in this paper will focus on
the implementation of control charts due to their foundational value for further analysis. The
24
25
Montgomery, Douglas C. Introductionto StatisticalQuality Control. Hoboken, New Jersey: Wiley, c2013., 2013.
Western Electric Co., Inc. Western ElectricStatistical Quality Control Handbook. First Edition 1956, Second
Edition 1958. Easton, Pennsylvania: The Mack Printing Company, n.d.
31
implementation of SPC and the use of control charts to identify special cause variation is not
only effective as means to control a process but also as means for supervisors to manage more
effectively, often leading to cost savings2 6 . In addition to the seven SPC tools, Montgomery
emphasizes the role of management and its ability to create an environment of continuous
improvement for the successful implementation of SPC. Work done at Boeing 27, Alcoa
Shanghai 28 and many other companies has emphasized the important role of management in
implementing SPC as the foundation for continuous improvement and quality programs.
Management plays an important role in effective implementation of SPC because SPC is a
problem-solving process 29 and not just a way to perform process or product monitoring.
Additionally, the results of effective implementation of SPC methods may provide management
with leverage to make a case for introducing more advanced methods of process monitoring and
process control, such as Multivariate Statistical Process Control (MSPC).
3.2
Maximum Likelihood and Score Function
The maximum likelihood method involves estimating parameters for a given probability
density function that maximize the likelihood that an observed set of data came from the
probability density function. In this way, maximum likelihood provides a way to determine the
most likely distribution for a given set of data. More specifically, given N random variables that
26
Noskievidovi, D., and R. Kucharczyk. "Effective Application of Statistical Process Control (SPC) on
the
Lengthwise Tonsure Rolled Plates Process." Djelotvorna PrimjenaStatistijke Kontrole Procesa (SKP) Rezanja
Uzduino Namotanih Limova Na 9karama. 51, no. 1 (January 2012): 137-40.
27 Stec, David J. (David Joseph). "Performance Measures for Lean Manufacturing." Thesis, Massachusetts Institute
of Technology, 1998. http://dspace.mit.edu/handle/ 1721.1/9887.
28 Yin, Shing. "Bringing Quality into a Chinese Organization." Thesis, Massachusetts Institute of Technology, 2000.
http://dspace.mit.edu/handle/1721.1/86558.
29
Noskievidovi, D., and R. Kucharczyk. "Effective Application of Statistical Process Control (SPC) on the
Lengthwise Tonsure Rolled Plates Process." DjelotvornaPrimjena Statistieke Kontrole Procesa (SKP) Rezanja
Uzdufno Namotanih Limova Na $karama. 51, no. 1 (January 2012): 137-40.
30 Eliason, Scott R. Maximum Likelihood Estimation: Logic and Practice.Sage University Papers Series.
Quantitative Applications in the Social Sciences: No. 07-096. Newbury Park, California: Sage Publications, c1993.,
1993.
32
are independently and identically distributed (iid), the column vector of observed data, x =
[xj,...xN] is a random sample from an unknown population. If we allowf(lO) to denote the
probability density function (PDF) that represents the probability of observing x given the
parameter vector, 0 = (01, . .
Ok)
with k parameters, then we define the likelihood function by
reversing the roles of the data vector x and the parameter vector 0. The likelihood function is
defined as
L(O|x) = f(x|&)
where L(OIx) represents the likelihood of parameter 0 given the observed data in x3 . The value of
the vector 0 that maximizes the likelihood function is known as the maximum likelihood
estimator (MLE)32 . The MLE is obtained by maximizing the natural log of the likelihood
function,
ln L(O Ix)
which is known as the log-likelihood function 33 . The natural log is taken for computational
convenience34 and, because of monotonicity of the logarithm, we obtain the same solution as if
we were to maximize L(OIx) 35 . In order to find the 0 that makes the PDF most likely given x, we
take the first derivative of the log-likelihood function and set it equal to 0 as in
31
Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journalof MathematicalPsychology 47, no.
1
(February 2003): 90-100. doi:10.1016/S0022-2496(02)00028-7.
Eliason, Scott R. Maximum Likelihood Estimation: Logic and Practice. Sage University Papers Series.
Quantitative Applications in the Social Sciences: No. 07-096. Newbury Park, Calif. : Sage Publications, c1993.,
32
1993.
3
Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journalof MathematicalPsychology 47, no.
(February 2003): 90-100. doi: 10.1016/S0022-2496(02)00028-7.
14 Ibid.
Eliason, Scott R. Maximum Likelihood Estimation: Logic and Practice. Sage University Papers Series.
Quantitative Applications in the Social Sciences: No. 07-096. Newbury Park, Calif. : Sage Publications, c1993.,
3
1993.
33
1
a ln L(e|x)
aoi
Note that an explicit solution is not always possible and non-linear optimization techniques must
.
be used. The Newton-Raphson method is typically used to address the local maxima problem3 6
Note that maximum likelihood assumes that a probability distribution has already been selected.
The method described above provides the probability distribution that makes the observed data
most likely 37 with a log-likelihood value for the distribution. In other words, given a set of data,
maximum likelihood provides the best distribution for a given probability distribution. However,
in a situation where there are several candidate distributions, a way to score the functions is
necessary in order to choose the best distribution for a given set of data. The Akaike information
criteria (AIC) is a model selection criterion that uses the log-likelihood value to rank candidate
distributions. AIC is given by
AIC = 2k - 2(maximum log - likelihood)
where k is the number of parameters in the model. The number of parameters, k, are included in
the expression as a penalty for more complex models. With enough parameters, a model can be
made to fit a set of data well, so AIC penalizes models for this tendency known as over-fitting.
AICc is AIC with correction for finite sample sizes 38 and is given by
AICC = 2k - 2(maximum log - likelihood) +
2k (k + 1)
(k +1)
n - (k + 1)
Konishi, Sadanori, and Genshiro Kitagawa. Information Criteriaand StatisticalModeling. Springer Series in
Statistics, n.d.
37 Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journalof MathematicalPsychology 47, no.
(February 2003): 90-100. doi: 10.10 16/S0022-2496(02)00028-7.
3' Burnham, Kenneth P., and Anderson, David R. Model Selection and Multimodel Inference: A Practical
Information-TheoreticApproach. 2nd Edition. New York: Springer, c2002.
36
34
1
AIC + 2k(k +1)
n - (k + 1)
where k is still the number of parameters in the model and n is the sample size.
Maximum likelihood estimation, the parameter estimation method, is widely known in
statistics3 9 and has several advantages over least-squares estimation including no distributional
assumptions, "complete information about the parameter of interest," and "lowest possible
variance of parameter estimates achieved asymptotically." 40 Hirotugu Akaike derived AIC in
1973, providing a new, straightforward method for model selection. AIC was also innovative in
its approach to model selection because of its basis in information theory41 . While maximum
likelihood and AIC are generally well known with many applications, there was no research
available that related AIC and SPC.
3.3
Continuous Improvement and Contract Manufacturing
With unprecedented change in the manufacturing industry over the last three decades and
increased pressure from customers and competitors in the global marketplace, manufacturing
firms can use continuous improvement to reduce cost, speed up processes and improve quality. 42
Continuous improvement is defined by Singh, et al., as "a culture of sustained improvement
aimed at eliminating waste in all organizational systems and processes." 43 There are many
methodologies aimed at continuous improvement including Lean, Six Sigma, Theory of
Eliason, Scott R. Maximum Likelihood Estimation: Logic and Practice. Sage University Papers Series.
Quantitative Applications in the Social Sciences: No. 07-096. Newbury Park, Calif.: Sage Publications, c1993.,
39
1993.
40
Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." JournalofMathematicalPsychology 47, no. 1
(February 2003): 90-100. doi:10.10 16/S0022-2496(02)00028-7.
41 Burnham, Kenneth P., and Anderson, David R. Model Selection and Multimodel Inference: A Practical
Information-TheoreticApproach. 2nd Edition. New York: Springer, c2002.
42 Singh, J. "Continuous Improvement Philosophy - Literature Review and Directions." Benchmarking 22, no. 1
(20150101): 75.
4
Ibid.
35
Constraints (TOC), Toyota Production System (TPS), Just-in-time (JIT) and all the variations of
TPS and JIT. Each of them focuses on the elimination of waste, improvement in quality and
overall improvement in operating efficiency through various tools. However, a commonality
among all successful continuous improvement implementations, no matter what methodology is
used, seems to be the buy-in of management. Lack of management commitment and lack of
leadership are among the barriers in implementation of continuous improvement 44 , and Deming's
14 points indicate the role of management is of dominant importance in implementing quality
and productivity improvement45.
Though no literature on continuous improvement efforts involving firms working with
contract manufacturers was found, continuous improvement efforts with suppliers are explored
due to similarities in the relationship between a firm and its suppliers and the firm and its
contract manufacturers. Similar to suppliers, contract manufacturers have an interest in providing
quality products and manufacture products for which the hiring firm does not have enough
capacity, sufficient expertise, or cost effectiveness. Additionally, contract manufactures do not
interact directly with consumers as does the hiring firm. Essentially, continuous improvement
efforts with suppliers serve as a parallel to continuous improvement efforts with contract
manufacturers.
Toyota is well known for its ability to build strong supplier relationships and implement
kaizen, or continuous improvement, at the supplier. The first four steps in implementing
continuous improvement at a supplier are awareness for improvement, analysis of current
situation, define the target, and decide on the kaizen strategy and tactic. Awareness for
Singh, J. "Continuous Improvement Philosophy - Literature Review and Directions." Benchmarking 22, no. 1
(20150101): 75.
45 Montgomery, Douglas C. Introduction to Statistical Quality Control. Hoboken, New Jersey: Wiley, c2013., 2013.
44
36
improvement involves commitment of top management and selection of a team that will take on
continuous improvement projects. Analysis of the current situation involves studying the flow of
materials and information. Definition of target involves setting a target based on the data and
information collected during the first two steps. Finally, deciding on a kaizen strategy and tactic
involves choosing the methods to a reach the targets set in the previous step. The final steps in
implementing continuous improvement at a supplier involve implementation of the methods that
were chosen in step four, evaluating the outcome, standardization and implementation of visual
management to identify abnormal performance.46
Rusli, Hazri M.1, Ahmedl Jaffar, Suzilawatil Muhamud-Kayat, and Mohd Tarmizil All. "Implementation of
Lean Manufacturing through Supplier Kaizen Framework - A Case Study." Proceedingsof the International
Conference on IndustrialEngineering& Operations Management, January 2014, 2221-28.
46
37
4
Application of Statistical Process Control
Amgen applies Statistical Process Control (SPC) in order to identify special causes of
variation to ensure the consistency of the manufacturing process and the quality of its products.
The primary SPC method applied for the identification of special causes of variation is control
charts. In order to develop a control chart for a commercial manufacturing process, one must first
have sufficient data that is representative of the process and make the assumption that the
process is in control. Next, the data is evaluated in order to determine a statistical model from
which control limits can be calculated. After control limits are established, SPC methods should
be applied by trending future data against the control limits.
4.1
Data Modeling
Amgen's policy for data modeling for the purpose of calculating control limits is
captured in the following steps.
Step 1: Check for normality.
To check for normality, Amgen uses normal quantile plots and the fat pencil test. The fat
pencil test is a crude check for normality that involves placing a pencil over the normal quantile
plot. In this test, if the pencil covers the plotted values on the normal quantile plot, the data is
determined to roughly follow a normal distribution4 7 . Amgen does not make use of AndersonDarling or Shapiro-Wilk or another more formal check on normality. Figure 6 below is an
example of a normal quantile plot that could possibly pass the fat pencil test, but the null
hypothesis that the data is normally distributed would be rejected by Shapiro-Wilk.
Scott Kowalski, and Vining, Geoffrey. Statistical Methodsfor Engineers. Third Edition. Boston, Massachusetts:
Brooks/Cole Cengage Learning, c201 1., n.d.
47
38
0.67-
o
67
Goodness-offt Test
Shapiro-Wilk WTest
W Prob<W
0.0006*
0.841932
Note: Ho = The data is from the Normal distribution. Small p-values
reject Ho.
Figure 1: Quantile Plot and Shapiro-Wilk Test Result
Step 2: If data is not normally distributed, determine a representative distribution to model
the data.
The determination of a representative distribution seems to be subjective. Under the
current process at Amgen, there is no objective way to consistently assign distributions to nonnormal data. The Quality Engineering (QE) group at Amgen is responsible for assigning
distributions and calculating control limits. There is a reliance on past distributions used for
certain parameters (Beta for yield parameters, Gamma or Weibull for parameters with 0 as a
threshold). There are certain parameters/types of data that will always conform to certain
distributions for scientific reasons. For example, if the FDA requires a test be done a certain way,
data from the test may always conform to a specific distribution. However, given that data from
many parameters is not normal and there is not a scientific reason to use a certain distribution for
many types of data, a more objective approach would be helpful in determining the best model
for data. Figure 2 below is an example of data that could be assigned three different distributions.
In this example, the reader cannot see the Log-normal curve because the Gamma curve
completely covers it. Table 1 provides the control limits associated with each distribution.
39
Despite the limits' equitableness, the slight differences could result in false signals or missed
signals.
1.280.
0.8
. 0.7
-- - -
---- 0.0
0
-0.6
0 .5 z
0.4
-0.3
-0.67
-0.2
0.1
-1.28
-1.64
98 98.5 99 99.5
100.5
101.5
102.5
-Normal(100.237,1.07197)
- Log-normal(4.60748,0.01041)
-
Gamma(9228.32,0.01086,0)
Data
Figure 2: Normal Quantile Plot, Box Plot and Histogram for Example Uncertain Normally-distributed
LCL
CL
UCL
Normal
97.02093
100.23684
103.45275
Gamma
97.1355
100.2332
103.3961
Table 1: Control Limits for Example Uncertain Normally-distributed Data
Determination of the correct distribution is so important because use of the wrong distribution
will lead
will not appropriately capture the intended 99.73% of values within control limits. This
40
to more false alarms and ensuing investigations that contribute to wasted resources or to quality
issues from missed signals.
Note that this step does not does describe the method by which to determine the most
representative distribution. Also note that data modeling for Onyx would be done in the same
way due to Onyx's reliance on Amgen's QE group for data modeling and control limit
calculation.
Step 3: If no distribution is representative, try to do a normal transformation of the data.
Since there is no meaningful back-calculation of the standard deviation after a
transformation, it makes sense to use the 0.135 and 99.865 percentiles to obtain the control
limits.
Step 4: If the transformation fails to produce a normal distribution, set control limits
through subjective means and document the justification.
After the data is determined to be non-normally distributed and no distribution can be
matched to the data, QE works with the monitoring groups to determine a) if control limits
should still be set and b) what the control limits should be. In this case, the control limits are not
statistically set.
4.2
Control Limits
There are two primary benefits for Onyx's use of control limits. The first is compliance
with the FDA's guidance on Process Validation and particularly Continued Process Verification
(CPV).
41
"The goal (/ [ContuiedProcess Verification] is continual assurance that the
process remains in a state of control (Ihe validated, state) during commercial
mianfiicture.
The ability to detect unwanted variability is critical to demonstrating control of the
manufacturing process. Control limits allow for the detection of unwanted variability and serve
as the means by which Onyx can demonstrate that the manufacturing process is in control.
The second benefit of using control limits is continuous improvement. Control limits
allow for continuous improvement in three ways. First, they provide the means to identify
problems and determine whether action must be taken. Second, in combination with other SPC
methods, they allow for correction, anticipation, and prevention of problems. Third, control
limits facilitate increased knowledge of the manufacturing process, leading to improvement of
the manufacturing process that will ensure product quality and patient safety.
4.2.1
Amgen Example: Effectiveness of Control Limits
The case below describes Amgen's ability to detect unwanted variability in its process
and thus satisfy the intent the FDA's guidance on CPV. The case also demonstrates a correction
in the manufacturing process, increased knowledge of the manufacturing process, and
improvement that ensures product quality and patient safety.
The control chart in Figure 8 below is taken from an existing Amgen report and is meant
to provide only a quick visual reference. This control chart indicates three violations of the lower
control limit, indicated by the lowest set of horizontal dashed lines, for a yield parameter.
Though the three points violate the control limit, they may not violate a specification or rejection
"Guidance for Industry; Process Validation: General Principles and Practices," January 2011.
http://www.fda.gov/downloads/Drugs/Guidances/UCM070336.pdf.
48
42
limit. Therefore these data points would not keep the entire batch from being processed.
However, the three points indicate that there is unwanted variability in the process.
~na
II 1 1
2
12
102
IM
103
10
07
P1 lOn
IM
I P2 P5 INl5 1 1 1 1 lie 112
INs IMs
IoT-52
lo IN s105 S107
12
10
12
UICL 100
100
10
-
X
X
X
98
97
CL4V
- - -
X'-3
..
...
-X
X
X
X
95
go
99
X
-
94
92
87
5
90~
-
98
0
SeQuenm-Ofla
Figure
The
8:
also
limit
reject
taken
an
from
as
shown
rate
an
for
limit
control
and
the
yield
existing
uppermost
(dependent
a
parameter,
independent
provides
and
report
set
of
horizontal
parameter)
Violations
to
relate
directly
observations
parameter
yield
three
a
quick
dashed
demonstrates
reference
visual
The
lines.
basic
Figure
in
shown
known
process
9
below.
with
relationship
the
upper
the
above
data
of
instances
three
rate,
reject
70
(smitd by 0eThm)
Limit
Control
with
Chart
Control
Parameter
Yield
Bakh
60
50
40
30
20
10
9
Figure
upper
is
control
between
understanding.
43
...............
ud
Defa Rounded o Pnecidon cfCoarn UMf (Disonneced Xe we NotCoUraed w I'am'- btA
LAL MA
LCL. 0
CL 12
IJCL 4.69
UAL MA
Red Dot =Ne=_1, N=3
Blue Squam 42. N=O
C
. Magenta DiaC
=Nei_3 N=0
I
12
I
i
PS
P4P
?
e
N
110
1R2
In
mbMfurft
n
2A3,r4s)
Groen
W7
3F
It
Triangte =Wl4, N=O
112
F
S
F
16
is
14
13
12
11
10
-
9
84
xx
-----
-
-
22 - _ L1.L
W333C9~
CL
I
3N
ff
0
10
20
I
HMM H
I
40
30
Sequnnce-One Blch (Id
5D
so
70
by DOMTh-)
Figure 9: Reject Rate Parameter Control Chart with Control Limit Violations
There was an investigation into the cause of the three control limit violations for the reject rate.
The investigation determined that these data points were the result of a material mismatch
between the calibration of a setting on a piece of equipment and the actual samples measured.
The calibration was done using metal standards, and the actual samples had a transparent
material. This mismatch led to a higher reject rate, ultimately leading to a lower yield. The
determination of the source of unwanted variability in the data allowed for the adjustment of the
setting on the equipment.
Without control limits, these points may have gone unnoticed, or their significance may
not have been recognized through investigation. Furthermore, the rejected vials would have been
lost due to the oversensitive equipment. The investigation determined that the majority of
44
samples rejected were in fact within the acceptable range, and these samples could be used,
saving money and ensuring supply for patients.
4.2.2
Onyx Hypothetical Example: Effectiveness of Control Limits
The case below is an example of how control limits could have been applied to
demonstrate the ability to identify unwanted variability, increase knowledge of the
manufacturing process, improve the manufacturing process, and possibly correct, anticipate or
prevent a problem.
The control chart below in Figure 10 is again meant to provide a quick visual reference.
This control chart shows the initial 24 batches of product without control limits for a reaction
completion parameter. Table 5 below includes the simple statistics of the raw data and the
calculated control limits.
Parameter
Mean
Median
Standard
Deviation
Upper Control
Limit
Lower Control
Limit
Table 5: Simple Statistics and Control Limits for Onyx Hypothetical Example Data
Reaction completion is a measure of the amount of a substance remaining after mixture. In this
case, a lower amount is better. The control chart in Figure 10 shows control limits starting with
batch 25 and beyond. The batch number is indicated by the x-axis. In other words, the first 24
batches were used to statistically determine the control limits that data for batches 25 and beyond
would be trended against. Notice that batch 25, the initial batch trended against control limits,
actually violates the upper control limit represented by the uppermost set of horizontal dashed
45
-
-- ---I-...
...............
.....................
-..
- - -
lines that begin at batch 25.49 Note that the solid line at the top of the control chart in Figure 10
indicates the upper action limit and is denoted by the "UAL: 2". A violation of the upper action
limit would trigger an immediate stop in the process.
Dm Rmdnt
WtPum
th,: NA
Red Dotz*464 1, N=I
511
105
UAL
U
10 e
I
isab
of Co"* Lamb @3hroumdm Ys an t"olem
Cot
n
Baum SquasNd 2.7
LUerAian
Lower
C0aw
Lne.1092
U
16
- 107fIN
amw
butar %mi WDfWF~U
2.3mAr)
uprOU Lka 0250315
Upper Actan Li 2
awod4I3
GmmTA~ne=Ntd.~N
~
10
112
If
1~
104
106o 107
ISO
2
0
0
2
4
6
a
10
12
14
16
18
20
22
24
26
28
SequneOra B8ah (santd by Da*Trm)
Figure 10: Control Chart for Onyx Hypothetical Example Data
Onyx's use of control limits would have allowed for the identification of unwanted variability.
Without control limits, the data point that violates the upper control limit in Figure 10 above is
not given attention. However, with control limits, Onyx can identify the opportunity to
investigate the data point in order to determine the possible cause for the poorer reaction
completion value. With a determination of the root cause, Onyx would then be able to make an
improvement to its manufacturing process, ensuring process stability and product quality. As this
49 Though control limits are not usually statistically calculated until >30 batches, the first 24 batches had sufficient
variability to calculate control limits for this illustration of the benefit of control limits.
46
is just one parameter, control limits for other parameters may allow Onyx to save vials or entire
batches, saving money and ensuring product is available for patients as demonstrated in the
Amgen example above. Furthermore, through the use of control limits, Onyx may determine
relationships between current parameters or determine that it should be monitoring additional
parameters. This would allow for increased control of the process. Finally, the use of control
limits and the resulting increase in understanding of the process will help prevent future batches
from experiencing the same causes of unwanted variability.
4.2.3 Control Limit Considerations
For normally distributed data, control limits are calculated using 3a control limits. With z
3, the corresponding a, or type I error, is approximately 0.27%, meaning that approximately
99.73% of normally observed values for a given parameter will fall within 3 standard deviations
of the mean. Assuming control limits are representatively set, this also means that 0.27% of
values will fall outside of the control limits but will be due to common cause variability or
variability inherent to the process and not due to any special cause. For non-normally distributed
data, the 0.135 and 99.865 percentiles are used to cover the same percent of values. For type I
error, there will be a resource intensive investigation that does not reveal a root cause. In other
words, there is the possibility of wasting resources for an investigation that will not lead to any
increased knowledge or improvement of the manufacturing process. However, for a company
with a mission to "Serve Patients," the threshold for time and resource consumption to
investigate type I error or false signals, is high. Therefore, calculating the most representative
control limits is not only paramount to providing the ability to detect special causes of variation
but also to ensure that resources are not wasted on investigating false signals. Additionally,
47
representative control limits help to minimize the possibility of a data point falling within the
control limits but having special cause or unwanted variation.
Amgen typically uses data from the first 15 commercial batches to calculate "preliminary
control limits." Amgen then trends data from batches 15-30 against the preliminary limits. If the
first 30 batches indicate that the process is in control, Amgen then takes data from batches 1-30
to calculate and statistically set control limits that all data after batch 30 will be trended against.
The only reason that there will be a reevaluation of the control limits later in the manufacturing
process' life cycle is if there is a change to the manufacturing process. The 30 batches guideline
is used for setting control limits because of the general acceptance that 30 samples is enough to
establish normality. However, this is just a guideline, and more or fewer batches may be required
to set control limits depending on the manufacturing process and the product. Additionally, it
would be helpful to test the statistical significance of 15 versus 30 samples or batches.
4.3
Application of Maximum Likelihood and Score Function
A case involving the reprocess of drug substance intermediates effectively demonstrates
the application of maximum likelihood and the Akaike information criteria (AIC) as a score
function to objectively select a model and calculate the most representative control limits.
4.3.1
Reprocessed Lot Background
Kyprolis is small molecule (compared to large molecule), meaning that it is produced by
a series of chemical reactions. On the other hand, large molecules are produced by
biotechnological processes, using genetically modified cells to produce proteins. Large
molecules are administered by injection or infusion, and most small molecules are administered
through tablets. However, Kyprolis is a small molecule that is administered by injection.
48
The Kyprolis manufacturing process consists of four drug substance intermediates
referred to here as Intermediate 1, Intermediate 2, Intermediate 3 and Intermediate 4 followed by
drug substance and the finished drug product. Intermediate 1 undergoes a chemical reaction and
becomes Intermediate 2. In turn, Intermediate 2 undergoes a chemical reaction and becomes
Intermediate 3 as indicated in Figure 11.
Intermediate 1
Intermediate 2
Chemicals
Chemicals
Intermediate 3
Intermediate 2
Figure 11: Drug Substance Intermediate Process for Intermediate 3
Intermediate 3 then combines with the fourth drug substance intermediate, Intermediate 4.
Together, they undergo another chemical reaction and yield the drug substance. The drug
substance mixes with chemicals to form a bulk solution which is then filtered and filled into vials
as drug product as indicated in Figure 12.
49
Intermediate 4
Intermediate 3
Drug Substance
Chemicals
Chemicals
Drug Product
Drug Substance
Figure 12: Drug Substance Intermediate Process for Drug Substance and Drug Product
There are two important notes about this process that are relevant to this discussion. First,
Kyprolis is currently manufactured solely by contract manufacturing organizations (CMOs). In
other words, none of the manufacturing is done by Onyx, though there are plans to move
production to Amgen facilities starting in 2016. Second, the relationship between a
biopharmaceutical company and a CMO is guided by the Quality Agreement, or contract,
between the two entities. The agreement includes what data and information the CMO is
required to provide to the biopharmaceutical company, or hiring company, and the timeline for
providing that data and information. This discussion will focus on Intermediate 1, Intermediate 2
and Intermediate 3, all manufactured by one CMO.
As mentioned above, the small molecule manufacturing process is a series of chemical
reactions, with each reaction producing a drug substance intermediate or the drug substance.
Each production stage essentially follows a similar process indicated in Figure 13.
50
Mixing 4 Filtering 4 Drying 4 Packaging
Figure 13: Production Stages for Small Molecule Drug Substance Intermediates and Drug Substance
When a lot is reprocessed, this essentially means that the wet cake is unpacked, re-mixed, refiltered, re-dried and re-packaged. The key step in reprocessing a lot is the filtering (or refiltering) step. This is because the filtering step removes impurities from the drug substance
intermediates. In the case below, the lot was reprocessed after lab testing discovered
unacceptably high levels of impurities in the drug substance intermediate. Reprocessing is
allowed by Good Manufacturing Practices (GMP), the practices required by the FDA to be
considered compliant. So although a lot may have unacceptable levels of impurities, it can be
reprocessed to remove impurities in order to meet specified impurity levels. Note that
reprocessing leads to a lower batch yield. Therefore, though there are operational and lostrevenue (due to lower yield) costs associated with reprocessing a lot, there is a benefit to
reprocessing a lot instead of completely discarding the batch and having to manufacture a batch
from scratch.
4.3.2
Intermediate 3 Reprocessed Lot Discussion
The lot of Intermediate 3 that was reprocessed will be referred to as batch 101. Batch 101
had an out of specification (OOS) observed for an impurity referred to as Impurity C. An
investigation into the root cause of the unacceptable level of the impurity identified water content
levels of a starting material, Starting Material 1 (a raw material input) and 1,4-dioxane as the
source of the OOS. Figure 14 indicates the flow of impurities through the drug substance
51
intermediate stages.
Intermediate 1
* 1,4-dioxane mixes with Starting
Material Ito form Intermediate 2
* Intermediate 2
Couplingof ImpurityA and
morpholinoacetic acid converts
Impurity A to Impurity B
eIntermediate 3
Impurity B carries over as
ImpurityC
Figure 14: Drug Substance Intermediate Flow of Impurities in Relation to Reaction Water Content
Note that the in-process control parameter, reaction water content, is a calculated value that
includes the water content level of Starting Material 1 (SM]) and of 1,4-dioxane. The equation to
determine the reaction water content is given by
(9.3 x SM1 water content) + (1,4 Dioxane water content) x (1,4 Dioxane qty HC + 1,4 Dioxane
qty reaction)
9.3 + 1,4 Dioxane qty HCI + 1,4 Dioxane qty reaction
Because the Reaction Water Content parameter includes the Starting Material 1 and 1,4-dioxane
water content levels and because these two levels were the root cause of the OOS, statistical
52
100
trending of this parameter could have prevented the reprocessed lot. The normal quantile plot in
Figure 15 indicates that the reaction water content data is not normally distributed.
*
/
1.64-
0.9
CY
0.67'
1'
0.8
0.7
a
;E
0
.7
-0.67'
4V26'
0.5 z
0.4
0.3
02
0.1
-1.64'
Figure 15: Normal Quantile Plot for Reaction Water Content Parameter Data
In order to calculate control limits, the data must be modeled using a non-normal distribution.
The density curves from several candidate distributions indicate that there are several possible
fits as shown in Figure 16.
53
0.4
64
-.0.8
.5
0.2
0.7
CL
E
.0150.020.0250.030.0350.040.045D.05
---
Norma(0.0215,0.0095)
LogNormal(-3.9126,0.35759)
Johso S(-7.4755,0.16427,0.01578,2.4)
Johnson SI(1.55425,0.16265,0.01578,1)
Normal2 Mixire
Normal3 Mixbre
Beta(0.18938,1.03742,0.01578,0.031)
Figure 16: Normal Quantile Plot, Box Plot and Histogram for Reaction Water Content Parameter Data
The issue is which distribution to use to model the data. Selection of the wrong distribution will
lead to non-representative control limits. This could lead to false signals (type I error) and wasted
resources investigating them or this could lead to special causes of variability going
unrecognized (type II error). In other words, the selection of the best distribution is critical to
calculating controls limits that allow for the identification of special causes of variation.
Maximum likelihood and AIC are methods that can be used to objectively select one
distribution from a group of candidate distributions. AIC provides a relative estimate of lost
54
..........
.- .. - - - ..........
....
....
information and uses the log-likelihood of each candidate distribution to calculate a respective
AIC value. As a relative estimate of lost information, a lower AIC, value indicates a better
model. Table 1 provides the AIC, values for the candidate distributions and indicates that the
Beta distribution is the model winner based on its low AICc value and relevant control limits.
Table 2 provides the calculated control limit values for the top candidate models. For this data,
the lower control limit would be 0.01578 and the upper control limit would be 0.04713.
AICe
Number of Parameters
-2*lo -likelihood
Johnson Su
4
-391.700
-32.2
Normal 3 Mixture
8
-297.61
-275.074
Lo -normal
2
-218.366
-213.937
Gamm
2
-213.559
-209.130
Extreme Value
2
-205.806
-2 1.37
Exponential
1
-176.063
-173.925
Distribution
Table 1: AIC, Values for all Candidate Distributions
JohnsnSI0.016
LCL
CL
UCL
0.00752
0.01860
0.04848
0.01675.6
Normal 2 Mixture
Table 2: Control Limit and Centerline Values for the Top Candidate Models
Note that the upper control limits for the Johnson distributions are not reasonable limits in this
case. The PDF of the Johnson Log-normal Distribution (Johnson Sl) and the Johnson Unbounded
Distribution (Johnson Su) is
55
f(x) =
6
g'91x
-
{ exp - 1 y + 6g(_x -
]2
with the elements of the PDF for each distribution given in Table 3.
Johnson Su
Johnson Si
Shape parameter y
Shape arameter y
Iocation narameter
2
Location narameter e
Table 3: PDF Elements for Johnson Sl and Johnson Su Distributions
These PDFs inherently include long tails as indicated by the Johnson curves over the histogram
in Error! Reference source not found.. The long tails of the Johnson distributions can lead to
irrelevant control limits as indicated by the upper control limits for Johnson Sl and Johnson Su in
Table 2. Also note that the Beta random variable is between 0 and 1, which matches the reaction
water content parameter data.
Finally, note that, based on a subjective assessment of the density curves, there is a
possibility that the Log-normal distribution could be chosen to model the data. However, given
the AICc values and that the Johnson distributions do not provide meaningful upper control
limits, the Beta distribution should be selected as the model for the data.
The reaction water content value for the batch that eventually led to the reprocessed lot
was 0.05. A subjective approach could have led to the Log-normal distribution as the model. The
reaction water content value of 0.05 would not have violated the upper control limit for the Log-
56
normal distribution and the batch would have required reprocessing just as it had without the use
of any control limits. However, the use of AICC to objectively select the Beta distribution as the
model would have led to a violation of the upper control limit and the batch possibly could have
avoided the need to be reprocessed if an investigation was conducted in time.
Had the Reaction Water Content data for batch 101 been trended against control limits
established through the selection of a non-normal distribution using maximum likelihood and
AICC, the value most likely would have been found to violate the upper control limit, leading to
an investigation into the root cause of the high water content. At this point, the high water
content could have possibly been addressed prior to processing the lot through the Intermediate 2
stage and Intermediate 3 stage. Intermediate 3 involves two crystallization steps in order to avoid
process impurities from moving forward. If the high water content had been identified after the
lot had been processed through the Intermediate 2 stage, it is possible that an additional
crystallization could have been implemented in the Intermediate 3 stage in order to prevent reprocessing of the lot. Note that the corrective action to address the root cause of the reprocessed
lot was to revise the batch record to include the selection of 1,4-dioxane with a specific water
content dependent on the water content of Starting Material 1 in order to avoid a high water
content of Intermediate 1. However, the use of AICc to select a model of the data and the
subsequent calculation of control limits may have identified the 0.05 value early enough to revise
the batch record before any other lots were exposed to the special cause of variation.
The objective selection of distributions to model data and set control limits is critical to
meeting demand and minimizing costs. As demand for Kyprolis increases, a reprocessed lot
could mean delays in ensuring that patients have the drug they need. Also, the associated costs
with reprocessing a lot are considerable. The application of maximum likelihood and AICc
57
provides an objective way to calculate the most representative control limits. Non-representative
control limits can lead to the failure to investigate a parameter value that includes assignable
cause variation and represents an exposure of the manufacturing process to variability outside the
normal variability of the process. In the Intermediate 3 case presented, the use of maximum
likelihood and AICc could have prevented the reprocessing of batch 101.
58
5
Management of Continuous Improvement with Contract
Manufacturing Organizations
The FDA's guidance on Process Validation and, specifically, Continued Process
Verification (CPV) is general and, therefore, does not describe requirements for implementing
CPV when working with contract manufacturing organizations (CMOs). Furthermore, the
guidance certainly does not discuss differences in management of the CPV program of firms that
have their own manufacturing sites, firms that use CMOs, or firms that have hybrid
manufacturing with their own manufacturing and CMOs. In relation to continuous improvement,
the Process Validation guidance only indicates that one general outcome of Process Validation
and CPV is "process improvements that can be evaluated and implemented" or "process
development and qualification work as well as manufacturing experience to continually
improve.. .process."50 Therefore, the issue of whether the hiring firm or the CMO has
responsibility for continuous improvement efforts remains unclear. Based on efforts to
implement a CPV program for a firm that utilizes CMOs for all manufactuirg, there are three
considerations for the hiring firm's role in continuous improvement of the manufacturing
process.
5.1
Flexible Contracts
Perhaps the most important consideration in implementing continuous improvement
efforts with CMOs is the Quality Agreement, or contract, that guides the relationship between
the hiring firm and the CMO. Contracts seem to limit continuous improvement efforts in three
ways.
"Guidance for Industry; Process Validation: General Principles and Practices," January 2011.
http://www.fda.gov/downloads/Drugs/Guidances/UCM070336.pdf.
50
59
The contract dictates which parameters will be monitored. However, in the application of
SPC for continuous improvement, the hiring firm may want to initiate monitoring of new
parameters or eliminate monitoring of some parameters. The hiring firm may want to initiate
monitoring of a new parameter based on the analysis of previous data in order to troubleshoot,
learn about and improve the manufacturing process. The hiring firm may want to eliminate
monitoring of a parameter because analysis of data has revealed that the parameter is extremely
stable and easily controlled. The elimination of monitoring stable parameters could also be one
source of decreased costs. However, requesting monitoring of an additional parameter, and
possibly, elimination of monitoring of a parameter, would require a revision to the contract, a
process that could take months as contracts often have annual renewal periods or longer. One
reason for such a long revision time is monitoring a new parameter could require a change to the
batch record, and the CMO would have to validate the change prior to approving it.
The contract not only stipulates which parameters will be monitored but could also
stipulate what information about a parameter the CMO is required to provide to the hiring firm.
For this reason, the CMO may not be obligated to provide data outside of what is required by the
contract. For example, the CMO may only be required by contract to provide average release
testing data and not data for all samples tested, or the CMO may be required to report only up to
a certain number of significant digits for a given parameter. Contracts that limit the data for
parameters could impede the hiring firm's ability to complete thorough analysis of process and
product data, resulting in less insight about the manufacturing process. To clarify, the hiring
company may request any data from the manufacturing process or from release testing.
However, any data that is not part of the formal report required by the contract would require a
special request of the CMO.
60
Consequently, the contract limits continuous improvement efforts by requiring the CMO
to provide data in a formal, structured report. Compiling the report requires data verification and
approval processes by the CMO. The hiring firm then receives the report and has to decompose
the content in order to extract the most pertinent information and data. The data has to go
through the hiring company's own verification process before it can be analyzed and before
Statistical Process Control (SPC) methods can be applied in order to uncover any signs of
unwanted variability. Therefore, the report is limiting in three ways. First, it may provide
extraneous information. Second, it may not provide some specific information that is outside the
scope of the content required by the contract but that the hiring company needs for analysis or
investigation. Finally, the CMO's verification and approval processes in addition to the hiring
company's own verification and analysis processes lead to long lead times for the hiring
company to receive data and even longer lead times for the hiring company to produce analysis
based on the data from the CMO.
Flexible contracts that provide the ability to add and remove the monitoring of some
parameters would be extremely beneficial in the hiring company's continuous improvement
efforts. Flexible contracts that allow the hiring company to dictate the format of the data for a
given parameter would also facilitate the hiring company's continuous improvement efforts.
Additionally, a flexible contract that allows for informal transfer of data ahead of the verification
and approval processes to be used for analysis would assist the continuous improvement efforts
of the hiring company. Of course, the possible range of flexibility in the contract is somewhat
constrained by Good Manufacturing Practices (GMPs). It is also important to note that a flexible
contract does increase the workload for the CMO.
61
Some hiring companies have strong relationships or leverage with their CMOs and the
contract between them may not require revision in order for the hiring company to make requests
outside the terms of the contract. However, even in a situation where the hiring company has
leverage with the CMO, the CMO is still under no obligation to meet requests that are outside the
scope of the contract. In this case, it seems that flexible contracts would still be a good
alternative with respect to continuous improvement efforts.
5.2
Timeliness of Data
The time between when a lot is manufactured and when the data from the lot is evaluated
using SPC methods is critical in preventing any future lots from exposure to unwanted
variability. In other words, the speed of continuous improvement efforts is critical to fast
identification of special cause variation and implementation of improvements. The speed of
continuous improvement efforts begins with the timeliness of data. The timeliness of data
directly affects the effectiveness of its evaluation. On one end of the time spectrum is receiving
data that is several weeks old. On the other end of the spectrum is real-time data. Unfortunately
for Onyx, it is on the several-weeks-old-data side. In this situation, after evaluation, root cause
analysis, and determination of the change or improvement to the manufacturing process, one or
several other batches may have been processed. With real-time data evaluation, corrections can
be made in real-time, leading to extreme control of the manufacturing process. The difference
between the two ends of the timeliness spectrum is reactive versus proactive feedback into the
manufacturing process.
In the case presented in Chapter 4, the key to preventing the erroneous processing of the
lot through the Intermediate 2 and Intermediate 3 stages is the timely evaluation of data. Figure
62
17 shows the flow of information and data from the CMO that manufactures Intermediate 1,
Intermediate 2, and Intermediate 3.
Week
/eek 4
Manufacturing Product
Analytical Testing*
Analytical Review, sends review form to
I
QA
2
Week
I
Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14
_
Batch Record Compilation via Hand
Written Notes and then PDF and Email
Onyx Review of Release Packet (batch
record, analytical data, deviations report,
investigations, etc)
CMO Produces Batch Record and CoA for
Final Release via POF and Email
O6nyx Releases OS via Email"*
eek 7
_i
_
_
I I I I
_
_
I I [
_
_
___
__
A_______
*Analytical testing is done at CMO
**lntermediate l and Intermediate 2 data is reviewed but no official release by Onyx; CMO sends batch record, analytical results, draft of CoA, copy of deviations, equipment cleaning
information, in PDF by email 11 weeks from manufacturing completion to release
Figure 17: Timeline for the Flow of Information from the CMO to Onyx for Drug Substance Intermediates 1,
2, and 3
The current procedure for evaluating information and data is that review of Intermediate 1 and
Intermediate 2 lots may not be done until review of Intermediate 3. Based on the timeline above,
this means that review of Intermediate 1 data could be anywhere between Week 6 and Week 7.
The three- to seven-week period after the completion of manufacturing does not include time to
trend data against control limits, identify root cause for a source of special cause variation, and
implement an improvement to the manufacturing process.
The length of time it takes to review data from the CMO indicates, at best, it will take
three to seven weeks to implement an improvement to the manufacturing process so that the next
batch is not exposed to the same unwanted variability that the original batch experienced. In the
case with batch 101, there was one batch of Intermediate 1 that would have been exposed to the
same issue that occurred with the water content level in batch 101. Note that this is the best case
scenario in which data is evaluated as soon as it is received and an improvement to address the
63
issue with water content level is implemented immediately. In reality, it is more likely that data
evaluation would occur after several more weeks, leading to additional lots being exposed to the
same unwanted variability that batch 101 experienced and possibly additional reprocessed lots.
There are several ways to address the issues with timely evaluation of data from CMOs.
1. Build more collaborative relationships with CMOs.
2. Revise Quality Agreements with CMOs to include trending of data against control
limits instead of just specification limits.
3. Create data and information review processes that are required as soon as a batch
is completed.
4. Work on decreasing the cycle times of each step in the entire review process
indicated in Figure 17.
5. Develop IT infrastructure to conduct simultaneous data verification and data
trending.
6. Develop IT infrastructure that provides the capability of real-time process and
product monitoring.
Cost is a critical factor in addressing issues with timely evaluation of data from CMOs,
especially for the development of IT infrastructure. However, the relationship with CMOs and
the timely processing of data are critical to ensuring that future batches are not exposed to the
same unwanted variability as a previous batch. The use of maximum likelihood and AICc is of
less impact if the evaluation of data does not lead to an improvement to the manufacturing
process prior to the next batch.
64
5.3
Location of Control Limits
The location of control limits in relation to where manufacturing takes place indicates
who has the burden of data evaluation when using SPC methods. The term, burden, is used in
order to indicate that the work of trending data against control limits and monitoring for special
cause variation resides with the control limits. In a situation involving a hiring firm that uses
only contract manufacturing, if the control limits reside with the hiring firm, the hiring firm does
the work of trending data against control limits, investigating causes of special cause variation,
and developing improvements to address causes of special cause variation. If the control limits
reside with the CMO, the CMO would likely be responsible for trending data against control
limits, investigating causes of special cause variation, and developing improvements to address
special cause variation. As the location of control limits approaches the manufacturing floor, the
burden of data evaluation shifts from the hiring firm to the CMO to the operator on the
manufacturing floor. This relationship between control limits and the work associated with their
location is important in determining ownership for continuous improvement efforts. The decision
to require a CMO to trend data against control limits implies additional work for the CMO. At
the same time, control limits residing with the CMO implies the CMO has responsibility for
continuous improvement of the product. Thus, the location of control limits is an important
consideration for hiring firms.
65
6
Conclusions and Recommendations
The conclusions and recommendations for Statistical Process Control (SPC) are based on
proven statistical processes and the case study involving the manufacture of downstream
intermediates. The key findings and recommendations for management of continuous
improvement with contract manufacturers are based on an accumulation of observations
involving the interaction between Onyx and its contract manufacturers, leading to several
important considerations.
6.1
Key Findings and Conclusions of the Application of Statistical Process
Control
Based on the case presented in this thesis, there is a need for a more objective approach to
modeling data for the purpose of calculating representative control limits. Non-representative
control limits lead to failure to identify signals or to false signals, which are a source of waste
and require unnecessary use of resources to investigate when no source of variation will be
found. The Akaike information criteria (AIC) provides an objective method for choosing a
distribution as the model for the data that minimizes information loss, providing the most
representative control limits. AIC should be used in conjunction with knowledge of the
manufacturing process and release tests in order to assist in the decision of how best to set the
most representative control limits. This method for setting control limits applies to many
applications of SPC, especially to processes or tests that produce non-normally distributed data.
Such processes or tests may be found in automotive, electronics and pharmaceutical industries.
The hypothesis that we tested is subjective data modeling contributes to nonrepresentative control limits. Based on the case involving drug substance intermediates, a
66
subjective approach to statistical modeling would have led to non-representative control limits.
Additionally, based on the subjectivity of model selection, it seems likely that control limits
contribute to false signals. Through the analysis of the drug substance Intermediate 3 case, the
use of AIC can help to provide more representative control limits.
6.2
Key Findings and Conclusions for Management of Continuous Improvement
with Contract Manufactures
There are several considerations with regard to continuous improvement efforts when
working with contract manufacturers. Traditional Quality Agreements, or contracts, with the
Contract Manufacturing Organization (CMO) seem to limit the quantity and format of data
available to the hiring firm. Additionally, the amount of time between the production of a batch
and completion of evaluation of data from the batch is several weeks, limiting the impact of
improvements developed through data evaluation. Finally, the location of control limits, either
with the hiring firm or with the CMO, indicates where the work to trend data and conduct
investigations related to the application of SPC will occur. The location of control limits with
respect to the manufacturing floor also indicates who has more responsibility for continuous
improvement efforts.
To address these considerations, flexible contracts, better collaboration with CMOs, more
IT integration with CMOs, and pushing control limits as close to the manufacturing floor as
possible seem likely to support continuous improvement efforts when working with CMOs.
The spirit of Continued Process Verification (CPV) is to maintain control of the
manufacturing process through the use of SPC. Continuous learning and continuous
improvement of the manufacturing process are also implied in the CPV guidance. The
67
application of maximum likelihood and AICC, timely evaluation of data, and the identification of
the most important parameters to monitor all contribute to the ability to meet the spirit and intent
of the CPV guidance. However, the CPV guidance by itself is not effective at producing
continuous improvement when working with CMOs due to the misaligned interests of the hiring
firm and the CMO. Though the hiring firm is interested in compliance with the CPV guidance,
and the CMO is interested in producing products within specification, neither party has a vested
interest in making the manufacturing process more efficient. Therefore, the CPV guidance is
ineffective at producing process improvements or continuous improvement. Flexible contracts,
timely data and locating the control limits as close to the manufacturing floor as possible allow
for the CPV guidance to be more effective in producing continuous improvement. Additionally,
using methods of continuous improvement with suppliers seems to provide a good model for
continuous improvement with contract manufacturers. Furthermore, the continuous improvement
with suppliers model seems to address many of the considerations for working with contract
manufacturers.
6.3
Next Steps
For the application of SPC, including the use of AIC to select a data model and calculate
control limits, the capability to track control limit performance in order to assess whether type I
or type II error has decreased to more expected levels is necessary. Additionally, a regression
study to verify that currently monitored parameters are the correct parameters is necessary. A
regression study will indicate whether additional parameters should be monitored and whether
some currently monitored parameters do not require monitoring. As part of a CPV program, it is
not effective to control parameters that do not have a relationship to other parameters. A
regression study should identify the relationships among currently monitored parameters and
68
identify whether any parameters are not currently monitored but should be. A regression study
would also identify whether any parameters are unnecessarily monitored. The study would also
be important in better overall understanding of the process in alignment with continuous learning
and continuous improvement, and would decrease the possibility of reprocessing lots going
forward.
For management of continuous improvement with CMOs, evaluation of quality
agreements with CMOs will allow the hiring company to determine where there are
opportunities for flexibility that facilitate continuous improvement. Additionally, collaboration
and with contract manufacturers will help to identify opportunities to reduce the time it takes to
apply SPC methods to evaluate data. Finally, pilots that involve locating the control limits on the
manufacturing floor and with the hiring company will provide insight into the best location for
control limits with respect to continuous improvement.
69
Bibliography
"Amgen Outlines Long-Term Strategy." Amgen News Release, February 7, 2013.
Auerbach, Mike. "Contract Manufacturing Trends This Year & Beyond." Pharmaceutical
Processing26, no. 1 (January 2011): 8-12.
Burnham, Kenneth P., and Anderson, David R. Model Selection and Multimodel Inference: A
PracticalInformation-TheoreticApproach. 2nd Edition. New York: Springer, c2002.
Chun, Soo Jin. "Brokering Strategic Partnerships between Asian and Western Biopharmaceutical
Companies in the Global Biologics Market: Assessment of Capabilities of Asian Participants
in the Biologics Contract Manufacturing Organization Marketplace." Thesis, Massachusetts
Institute of Technology, 2009. http://dspace.mit.edu/handle/1721.1/63014.
Eliason, Scott R. Maximum Likelihood Estimation: Logic and Practice. Sage University Papers
Series. Quantitative Applications in the Social Sciences: No. 07-096. Newbury Park,
California: Sage Publications, c1993., 1993.
"Excerpt from S-1/A SEC Filing, Biovex Group, Inc.," September 21, 2006. http://sec.edgaronline.com/biovex-group-inc/s- la-securities-registrationstatement/2006/09/2 1 /section2 .aspx.
Guha, Malini. "Amgen Swallows Onyx Whole." Nature Biotechnology 31, no. 10 (October
2013): 859-60. doi:10.1038/nbtO1l3-859.
"Guidance for Industry; Process Validation: General Principles and Practices," January 2011.
http://www.fda.gov/downloads/Drugs/Guidances/UCM070336.pdf.
Han, Chaodong, Tobin Porterfield, and Xiaolin Li. "Impact of Industry Competition on Contract
Manufacturing: An Empirical Study of U.S. Manufacturers." InternationalJournalof
ProductionEconomics 138, no. 1 (July 2012): 159-69. doi:10.1016/j.ijpe.2012.03.015.
Knodell, Jenny. "The Benefits and Disadvantages of Outsource Manufacturing." Accessed
January 21, 2015. http://blog.iqsdirectory.com/general-industry-articles/outsourcemanufacturing/.
Konishi, Sadanori, and Genshiro Kitagawa. Information Criteriaand Statistical Modeling.
Springer Series in Statistics, n.d.
Scott Kowalski, and Vining, Geoffrey. StatisticalMethodsfor Engineers. Third Edition. Boston,
Massachusetts: Brooks/Cole Cengage Learning, c201 1., n.d.
Montgomery, Douglas C. Introduction to StatisticalQuality Control. Hoboken, New Jersey:
Wiley, c2013., 2013.
Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journalof Mathematical
Psychology 47, no. 1 (February 2003): 90-100. doi:10.1016/S0022-2496(02)00028-7.
70
Noskievidovi, D., and R. Kucharczyk. "Effective Application of Statistical Process Control
(SPC) on the Lengthwise Tonsure Rolled Plates Process." DjelotvornaPrimjenaStatistijke
Kontrole Procesa (SKP) Rezanja Uzdu no Namotanih Limova Na Skarama. 51, no. 1
(January 2012): 137-40.
"Onyx Pharmaceuticals, Inc. SWOT Analysis." Onyx Pharmaceuticals,Inc. SWOT Analysis,
MarketLine, November 2013, 1-8.
Petersen, Peter B. "The Contribution of W. Edwards Deming to Japanese Management Theory
and Practice." In Academy of ManagementBest Papers Proceedings, 133-37. Academy of
Management, 1987. doi:10.5465/AMBPP.1987.17534034.
Phillips, Jocelyn. Biotechnology in the US. IBISWorld Industry Report. Melbourne, Australia:
IBISWorld Services, November 2014.
http://clients 1.ibisworld.com/reports/us/industry/default.aspx?entid=200 1.
Phillips, Jocelyn. Global Biotechnology. IBISWorld Industry Report. Melbourne, Australia:
IBISWorld Services, January 2015.
http://clients1.ibisworld.com/reports/gl/industry/default.aspx?entid=2010.
Rusli, Hazri M. 1, AhmedI Jaffar, Suzilawatil Muhamud-Kayat, and Mohd Tarmizil Ali.
"Implementation of Lean Manufacturing through Supplier Kaizen Framework - A Case
&
Study." Proceedingsof the InternationalConference on IndustrialEngineering
Operations Management, January 2014, 2221-28.
Singh, J. "Continuous Improvement Philosophy - Literature Review and Directions."
Benchmarking 22, no. 1 (20150101): 75.
Stec, David J. (David Joseph). "Performance Measures for Lean Manufacturing." Thesis,
Massachusetts Institute of Technology, 1998. http://dspace.mit.edu/handle/1721.1/9887.
Turk, Sarah. Brand Name PharmaceuticalManufacturingin the US. IBISWorld Industry Report
32541 a. Melbourne, Australia: IBISWorld Services, November 2014.
http://clients 1.ibisworld.com/reports/us/industry/default.aspx?entid=487.
Western Electric Co., Inc. Western Electric StatisticalQuality Control Handbook. First Edition
1956, Second Edition 1958. The Mack Printing Company, Easton, Pennsylvania, n.d.
Yin, Shing. "Bringing Quality into a Chinese Organization." Thesis, Massachusetts Institute of
Technology, 2000. http://dspace.mit.edu/handle/1721.1/86558.
71