Modelling and simulation in the
pharmaceutical industry
Reflections from a statistician’s
and engineer’s perspective
Carl-Fredrik Burman, PhD, Assoc Prof
Senior Principal Scientist
AstraZeneca R&D Mölndal
Agenda
• Seven theses about good modelling
1. It is about making better decisions
2. It is driven by the underlying question
3. It is based on applied sciences
4. It uses a diversity of information sources
5. It is not made unnecessarily complicated
6. It is a continuous process
7. It facilitates communication
• Some thoughts on simulation
• Concluding remarks
2
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Seven theses about good modelling
• Based on Burman & Wiklund (Pharm Stat 2011)
• The seven theses are partly overlapping
• The intention is rather prescriptive than descriptive
3
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Are the theses just saying the obvious?
4
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
1. Good modelling
is about making better decisions
Don’t model
unless you see which
decision could be
improved by your model
6
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Value of Modelling
(Cf. Value of Information)
• Data, information, models may have a larger or
smaller value depending on the situation
•
•
•
•
X
D
M
V
available data
decision
“modelling”
value (in e.g. Swiss franc, or total patient benefit)
• Value of Modelling =
VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ]
• Value only(?) through changing the decision
7
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Value of Modelling
A simple example
• Value of Modelling =
VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ]
• Say that we have a pure go / no go (dichotomous!)
phase III investment decision
• If it’s pretty obvious that we should “go”, modelling
doesn’t help
• However, if this is a tough decision, modelling (e.g.
predicting through biomarker-clinical endpoint relation,
extrapolating over time and population) may be very
valuable.
• Frontload: Start modelling ph III before ph II investment
8
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Value of Modelling
Example (cont’d)
• Value of Modelling =
VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ]
• Say that we face a phase III investment decision that is
not purely dichotomous, but concerns
- Go / No Go
- Dose selection
- Population
- Sample size
• Then, all these may add more or less to the overall value
• Highly dependent on specific situation
9
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Decision focus
Don’t necessarily have to be one single decision
• Useful to update model continuously (Thesis 6)
• Series of decisions
• Adaptive Programmes
• Cross-over to new projects
• Obvious for same indication
• But trial data for one drug can sometimes help to
give useful background information for completely
different indications (variability, disease
progression)
10
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
“Soft” but important?
• (Too) many modellers have been vague about
the aim of their work
- Describe the system
- Improve understanding
- Write scientific papers
- ...
• Modelling can be used for summarising
information, predicting, gaining insights, etc.
• However, the benefits of e.g. a ‘gained insight’
will be realised when the insight is reflected in
an actual decision.
11
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Industry vs. Academia
• I think the luxury of “non-purpose” modelling
should sometimes be allowed
- Cf. pure mathematics
- but more often in academia
12
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
13
2. Good modelling
is driven by the underlying question
“All models are wrong,
some are useful”
George Box
15
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Don’t search for the ultimate model
• “All models are wrong ...”
A model cannot incorporate all available information
and be capable of answering all relevant project
questions.
• “... some are useful”
The usefulness depends on the purpose of modelling,
on which decision we set out to support.
• Fit for purpose
The model should thus be tailored to the concrete
project question (Thesis 5).
16
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Different objectives  different models needed
• Example 1: Testing overall placebo-controlled efficacy
• Example 2: Is the drug less efficacious than a
competitor drug in elderly? If so, where’s the cut-off?
• Example 3: Dose adjustment possible based on age or
pharmacokinetic exposure (that correlates with age)
17
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Need to understand the real question
• Common consultancy experience:
- Clients may ask the wrong question
• The modeller should try to understand the
overall context
18
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
3. Good modelling
is based on applied sciences
Be a scientist,
not a narrow-minded
statistician!
20
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Modelling
“Reality”
Mathematics
To me,
modelling is about translating from the real problem to mathematics,
and going back from a mathematical solution to a practical solution.
21
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
• Note! Modelling in my narrow sense is not
about solving the mathematical problem!
• Modelling process:
- Understand the project problem
- Formulate objective
- Map reality onto mathematic
- Solve mathematical problem
- Translate back to give decision support
- Check robustness
- Communicate
22
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Good modelling requires the utilisation of
several scientific areas, not only statistics
• Statisticians often have excellent skill-sets for
modelling work.
• However, we may need to transcend traditional roles
and eagerly seek to understand the essence of the
project’s problem.
• Think and act as scientists in a wider sense, focussed
on providing useful decision support, irrespectively of
what kind of methods that are needed.
• Avoid “He that is skilled with a hammer tends to think
everything’s a nail”
• Collaborate!
23
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
4. Good modelling
uses a diversity of information sources
Combining different information sources
Several pieces to the puzzle
• Often, we need several components, e.g.
-
Dose-response
Time dependence
Biomarker – clinical endpoint
Variability (between patients, over time, day-to-day,
measure-to-measure)
• to build a useful model
• Information needed will often come from
different sources
25
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Different information sources
• In-house randomised clinical trials
• Pre-clinical data
• Competitor data
• Observational studies
• Literature information
• Expert knowledge
• ...
26
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Combining different information sources
… to estimate a common parameter
• Scientific insights needed to see the relevance
of information
• In principle, the Bayesian framework is readily
applicable as it treats all types of uncertainties
in the same way.
• In-house design decisions can be guided by
Bayesian decision theory, even if regulators
and other costumers require frequentist
analyses
27
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
5. Good modelling
is not made overly complicated
Not overly complicated
Ockham’s razor:
“Pluralitas non est ponenda sine neccesitate”
(“Entities should not be multiplied unnecessarily'‘)
William of Ockham, 1285–1347/49
“A model should be as simple as possible and yet no
simpler”
Albert Einstein, 1879-1955
29
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Efficiency of modelling work
• Don’t spend time on modelling that likely have
ignorable impact on the decision problem.
• The first question to ask is whether modelling
is worthwhile, i.e. to assess whether the net
‘Value of Modelling’ is positive.
30
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Marginal Value of Modelling
• How much effort to put into modelling?
• Engineering: Quick and dirty!
• Value of Modelling
VoM = E[ V( D(X,M) ) ]  E[ V( D(X) ) ]
31
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Example
• Pharmacokinetics / pharmacodynamics
(PK/PD) modelling can give great benefits.
• However, if it’s clear that one single dose is
sought, the PK component will typically not be
important.
• Why estimate f(PK(d)), when you only need f(d)?
32
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
6. Good modelling
is a continuous process
34
Continuous learning
• A model made for one decision point can often
be re-used, updated and applied to a later
decision.
• The ‘Learning Loop’ provides a description of
the continuous modelling process and the
interaction with (design) optimisations and
information retrieval.
• The greatest benefits will likely be achieved
with ’model-based drug development’, where
modelling is fully integrated in the process.
35
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
7. Good modelling
facilitates communication
Transparency
• Modelling is not a concern only for the
quantitative scientists.
• Modelling is not replacing the decisionmakers, but supporting them.
37
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Decision support
• Present not only the optimal solution, but
also
• Assumptions
• Robustness checks.
• The ideal is that the decision-makers can
challenge assumptions and interactively
study the consequences of altering them.
• Successful modelling processes provide
major benefits in transparency within the
teams regarding underlying assumptions,
and facilitate communication with
governance bodies and decision-makers.
38
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Some thoughts about simulation
Simulation
A piece of cake?
• We are considering “simple” clinical trial simulation, not
e.g. MCMC
• For most statisticians, it is trivial to simulate a simple
clinical trials, with sufficient precision
• However, many modelling problems are more
complicated, including
- a range of scenarios
- multidimensional optimisation.
• In our experience, simulation studies are often
ineffective, leading to inadequate precision in results.
40
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Give confidence intervals
for simulation results!
41
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Simulation
Alternatives
• Consider alternatives to simulation
- analytic solutions
- Numerical analysis
- Approximations
• In many cases, parts of the problem can be
solved by such means, leaving a much
simpler problem to stochastic simulations.
42
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Simulation
Simplify
• Don’t make your simulation model overly
complicated!
• Simulate sufficient statistics, not individual
data
• Approximate. E.g. central limit theorem.
43
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Simulation
Reduce simulation variability
• Simulations are often applied to compare the
efficiency of competing designs, e.g. of a
dose-finding trial.
• Two designs with different doses can
effectively be compared by using the same
simulated residuals for both designs.
• This can greatly reduce the variance of a
difference of estimates.
• A similar trick can be used when the doses
are the same but different allocations (e.g.
larger placebo group) are considered.
44
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Remember:
Confidence intervals!
45
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Concluding remarks
The role of the pharma statistician
• An arguably conservative attitude has served
us well
- Appropriate in the traditional core area: the analysis of
(confirmatory) clinical trial data.
• However, time they are a-changing.
- An increasing importance on complex issues in
programme design and decision support
• To more effectively add value, statisticians
need to adopt a more flexible mindset and be
willing to embrace new, useful methodology.
47
Carl-Fredrik Burman | 13 Sept 2012
Global Medicines Development | Biometrics & Information Science
Which designs are possible?
Alternative designs
What do we
know already?
Modelling
Where do we
want to go?
Optimise design,
based on model
& objectives
Simulations / Computations
Objectives
Confidentiality Notice
This file is private and may contain confidential and proprietary information. If you have received this file in error, please notify us and
remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorized use or
disclosure of the contents of this file is not permitted and may be unlawful. AstraZeneca PLC, 2 Kingdom Street, London, W2 6BD, UK,
T: +44(0)20 7604 8000, F: +44 (0)20 7604 8151, www.astrazeneca.com
49