Joakim Nyberg, Sebastian Ueckert, Andrew C. Hooker
Pharmacometrics research group
Department of Pharmaceutical Biosciences
Uppsala University

At PODE 2009 all Population Optimal Design (OD)
Software should evaluate the same simple Warfarin problem…
 1-compartment model, 1st order absorption, oral dose 70 mg
 Proportional error model ( σ 2 =0.01)
 32 subjects with 8 measurements at
0.5, 1, 2, 6 ,24, 36, 72,120 hours (evaluation)
Parameters Fixed effects ω 2 (IIV, exp)
CL/F (L/h)
V/F (L) ka (1/h)
0.15
8.0
1.0
0.07
0.02
0.6
2

FIM can be calculated in different ways:
Assuming var(y) w.r.t. the fixed effects ≠0
FIM
Full

*
A C
C B



Assuming var(y) w.r.t. the fixed effects=0
FIM
Reduced
 


A
0
0
B


A* is somewhat modified/updated if full is used, i.e.
A *
  

V
 
V

1

V
 
V

1
Different between full and reduced
3

The FIM
FIM

E



 ln
 

 ln
 
 
T

  
E


 2 ln





If we have correlation between fixed effects and random effects in like the FULL is the “ theoretically correct” method.
If not, the Reduced is “correct theoretically” but this is seldom the case in Pharmacometrics
4

Results from last PODE 2009
Uncertainty in fixed effect Ka
Reduced
Full
FOI/FOCE/FOCEI
Simulations
The “truth”
14
12
10
8
6
4
*
*
2
0
POPT PFIM PopED PopDes PopED PFIM PkStaMp PopDes PkStaMp PopDes PopED PopED NM FO NM FOCEI Monolix
* Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003
5

6
Results from last PODE 2009
Uncertainty in fixed effect CL
Reduced
Full
FOI/FOCE/FOCEI
Simulations
The “truth”
5
4
3
2
1
0
POPT PFIM PopED PopDes PopED PFIM PkStaMp PopDes PkStaMp PopDes PopED PopED NM FO NM FOCEI Monolix
6

Results from last PODE 2009
4000
3500
3000
2500
Reduced
Full
FOI/FOCE/FOCEI
Simulations
D-Criterion Possibly issues with the
Cramér-Rao inequality
FIM

1

COV
2000
1500
1000
500
0
POPT PFIM PopED PopDes PopED PFIM PkStaMp PopDes PkStaMp PopDes PopED PopED NM FO NM FOCEI Monolix
7

Results from last PODE 2009 summary



Software gave similar results with similar approximations
Reduced superior to Full in terms of predicting the “truth”
Even less predictive performance with higher order
FOCE-based FIM.
8






Possible reasons – Initial ideas
The derivation of Full or Red is wrong
Derive FIM with simulations, i.e. integrate over observed FIM
FO-approximation too poor
- FOCE is obviously not enough, try high order approximations
Asymptotic behavior (FIM -1 ≤COV)
- Increase data set x 2 => SE should decrease by 2 (1/2)
Numerical instability in Full but not in Red FIM
- Using automatic differentiation (AD) to avoid step length issues
Estimation software is not true ML-estimator, i.e. efficiency of estimator not accurate
- NM hard to know how the parameter search is performed but Monolix well documented
9

Investigations – Reducing the complexity


 ln-transform model to have additive res-error
(avoiding interaction terms)
Check that the problem holds for prop IIV structure
(FO approximation => proportional IIV = exp IIV)
Fix all parameters except fixed effect Ka
10

Results – Reducing the complexity ln model, add error, exp IIV = prop IIV
Θ ka
RSE(%) [CI]
“Truth” NONMEM FOCE SSE (1000) 13.59% * [13.06-13.88]
PopED Full (Analytic)
PopED Reduced (Analytic)
6.71%
13.90%
PopED Full FOCE 10 000 **
PopED Red FOCE 10 000 **
4.95%
6.49%
 Issues still remaining => work with simplified model
* 100 000 bootstrap samples
** Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003 11

Results – Full vs Reduced


Asymptotic behavior (FIM -1 ≤COV)
- Increase data set x 2 => SE should decrease by 2 (1/2)
Numerical instability in Full but not in Red FIM
- Using automatic differentiation (AD) to avoid step length issues
None of this affected the results (2 down 3 to go)
12

Next things to try...
 The derivation of Full or Red is wrong
Derive FIM with simulations, i.e. integrate over observed FIM
FIM
 

E x

 FIM


, x obs


    data
 2 ln like


, x obs

 T

SO
FO-approximation too poor
- FOCE is obviously not enough, try high order approximations
E( y ij
| x j
)
Var
 f i
 x j
,


 
0

1
2 tr


 j


L T
 x j
,

  x j
,



1
4
 diag tr


 j
,


 
0



 j
,

 diag
 h
 
0



 x j
,

  x j
,


T
 j
,




 j
,



T





0

0


13

Results – High order approximations & simulation based derivations
“Truth” NONMEM FOCE SSE (1000)
NONMEM Integrated (400) Full FOCE
PopED Full Analytic FO
PopED Integrated (100) Full FO
PopED Full Analytic SO
PopED Full FOCE 10 000
PopED Reduced Analytic FO
PopED Red Analytic SO
PopED Red FOCE 10 000
Θ ka
RSE(%)
13.59% [13.06-13.88] *
14.00%
6.71%
6.80%
8.94%
Similar
4.95%
SO – Closer to truth
13.90%
14.04%
6.49%
14

Results Full vs Red
 The derivation of Full or Red is wrong
Integration FIM ≈ Analytic FIM => Not the answer
 FO-approximation too poor
- SO shrinks the differences but still to poor of an approx.
- FOCE is worse but NONMEM integrated FOCE FIM is good?!
- Possibly issues with the FOCE method?
15

FOCE FIM – Differences & Improvements
 NONMEM FOCE assumes linearization around the mode of the distribution => correlation between the individual parameters and the population parameters.
 Analytic FIM
FOCE
* does not assume this
-To calculate individual mode data is needed
Update Analytic FIM
FOCE to include the correlation:
 Calculate Expected Empirical Bayes Estimates (EEBE)
EEBE are not data dependent
 Whenever PopED differentiates pop parameters; differentiate EEBE as well
* Retout, Mentré – Further developments of FIM in NLME-models…. J. BioPharm. Stat 2003
16

FOCE
Old FOCE method Full (1000) (4.46s)
Old FOCE method Red (1000) (2.6s)
New FOCE method (1000) Full (~40s)
New FOCE method (1000) Red (~23.7s)
Θ ka
RSE(%)
4.95%
6.49%
13.62%
12.5% - 13.8%
 The new FOCE method solves the problem!
17

 The Full FIM does not always work with the FO-approximation
18

19

Does Full/Red affect the optimal design?
Full
Optimal Full design
5
4
3
2
1
0
0
1
9
8
6
7
2
2
20 40
2
80 60 time (h)
5 support points
100
1
120
Reduced
Optimal reduced design
7
6
5
4
3
9
8
2
1
0
0
1
6
20 40 80 100 60 time (h)
3 support points
1
120
20

Does linearization method affect the optimal design?
Surface of |FIM| for SOCE-MC
SOCE>MC
SOCE=MC
SOCE<MC
Optimal MC
Optimal SOCE
Example from PAGE 2009 Nyberg et al
21

Full |FIM|
Reduced |FIM|
Similar results with the transit compartment model
Results from Nyberg et al PODE 2008
22

Surface of RSE(%) – Full FIM only different from reduced in some regions
Full Ka RSE(%) Reduced Ka RSE(%)
S8: (120h)
Full ≈ Red
S8: (120h)
23





The FO-approximation is not always enough for Full FIM
Possibly also too poor approx. for Reduced
Reduced FIM collapses occasionally
High order approximations stabilize differences
Different approximations give different optimal designs, e.g. different sampling times and different number of support points
24

1) If runtime allows – Use high order approximations
FOCE, SO, SOCE, MC etc.
2) If Red is stable – Use reduced to optimize but evaluate with both
3) If Red is unstable – Optimize with Full but evaluate with Red
Beware: No “golden” solution is presented
- The Cramer-Rao inequality does not hold comparing different methods when optimizing / estimating
To get “correct” SE from the FIM either sim/est needs to be performed or high order FIMs need to be evaluated
25

I would like to acknowledge Sergei Leonov for our interesting emails discussing these issues.
26