QRPEM, A Quasi-Random Parametric EM Method for
PK/PD NLME Estimationmultidimensional integrals
Robert H. Leary and Michael Dunlavey
Pharsight®, A Certara™ Company, St. Louis, MO, USA
INTRODUCTION
OBJECTIVE
Stochastic EM methods such as MCPEM in S-ADAPT and NONMEM 7 and
SAEM in MONOLIX and NONMEM 7 have become attractive alternatives to
traditional deterministic FO, FOCE, and LAPLACE NLME estimation
methods based on direct numerical optimization of a likelihood
approximation. The stochastic methods have two key advantages: a) they
do not make any likelihood approximations that fundamentally limit
statistical performance, and b) they avoid the reliability issues inherent
in use of numerical optimization methods .
Classical EM methods implement an E-step that requires the numerical
evaluation of multi-dimensional integrals. In the NLME context with
multivariate normal random effects, the integrals are the normalizing
factor for the posterior density of the random effects for each subject, as
well as the first and second moments of this density. The original Monte
Carlo EM method [1] proposed evaluating these integrals stochastically
with random samples. In the NLME case, usually the densities in the
integrand cannot be sampled directly and techniques such as importance
sampling (MCPEM) and MCMC sampling (SAEM) are used.
To improve the performance of stochastic EM methods in the context of
parametric PK/PD NLME estimation by utilizing the superior numerical
integration performance of quasi-random sampling relative to random
sampling. Specifically, QRPEM, based on the importance sampling parametric
EM method MCPEM, has been implemented with quasi-random sampling and the
RESULTS
performance is compared to the same algorithm with random sampling.
RESULTS
• The importance sampled integrals based on QR samples were far more accurate for a
given sample size than integrals based on random samples. Figure 1 (below left) shows
comparative results for a simple three dimensional variance integral for which the true
analytic value can be obtained. In agreement with theory, the relative error in the QR
case decays roughly twice as fast on a log-log scale as in the random case.
The accuracy of the numerical integrals is determined by the sample size
N, and typically the theoretical error magnitude decays relatively slowly
as O(N-1/2). Here we investigate the performance of quasi-random
samples from low discrepancy Sobol sequences as proposed in [2] , which
have a much faster error decay rate of approximately O(N-1) .
Random vs. Quasirandom Relative Errors vs Sample Size
1
IMPEM log likelihood convergence, N=500
10
-1166
-1166.5
0
10
random
quasirandom
-1
-1167
Relative error
10
METHODS
-1167.5
-1168
-2
10
-1168.5
-3
10
-1169
-4
sobol quasirandom sampling
random sampling
-1169.5
10
-1170
-5
10
-1170.5
-6
10
A new implementation QRPEM of EM importance sampling-based NLME
estimation has been developed for future inclusion in the Phoenix®
NLME™ application on the Pharsight Phoenix® software platform. The
method differs from stochastic versions such as MCPEM in that it samples
the relevant integrands at positions that are based on quasi-random
points rather than random points.
For a d-dimensional problem with d random effects, the base samples
are drawn from a d-dimensional Sobol sequence uniformly covering the
unit hypercube [0,1]d. This contrasts with the random case where the
unit hypercube is sampled with a d-dimensional vector whose
components are independent random variables uniformly distributed on
[0,1]. Other than this base sampling level difference, the remainder of
the QR importance sampling algorithm is identical to its stochastic
counterpart.
0
10
2
10
4
10
Sample size N
6
8
10
10
Figure 1: Error decays much faster with
quasi-random sampling than random
sampling
-1171
0
10
20
30
40
50
60
70
80
Figure 2: Importance sampling–based EM
convergence is faster and much smoother
with quasi-random sampling than random
sampling
• The graph in Figure 2 (above right) shows the contrasting convergence behavior of the
top level log likelihood objective function (summed over all 100 subjects) as a function
of the iteration number for a PD test problem with d=3 random effects from the Lyon
2004 and 2005 NLME estimation algorithm comparison exercises conducted by P.
Girard and F. Mentre’ [3]. The sample size N was fixed to 500 for both the random and
QR sampling versions of the algorithm. For the QR case the log likelihood increase with
iteration number is nearly monotonic in the initial convergence phase, and once
convergence has been achieved, the magnitude of the inter-iteration deviations from
the converged value of 1167.25 is less than 0.05 log likelihood units. The convergence
behavior of the random sampling version is much rougher and convergence more
difficult to detect, if indeed it has converged here. Note the QR-based PEM algorithm
discussed in [2], a precursor to the current QRPEM algorithm, finished first In the Lyon
comparison exercise in terms of least biased parameter estimates among a field that
also included random sample MCPEM, MCMC SAEM, FOCE, and LAPLACE.
RANDOM vs. QUASI-RANDOM SAMPLES
2000 2-dimensional Uniformly Distributed Random Points
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0
1000 2-dimensional Normally Distributed Random Points
4
3
3
2
2
1
1
0
0
-1
-1
-2
-2
-3
-3
-3
-2
-1
0
1
2
3
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Quasi-random sampling strongly outperformed conventional random sampling in an
MCPEM-like algorithm for NLME estimation in a series of PK/PD test problems. Generally,
the theoretical O(N-1) vs. O(N-1/2) error decay rate advantage of the QR technique has
been confirmed in practice. Improvements in speed of nearly 2 orders of magnitude have
been observed for some posterior density integrals where high accuracy is required, with
the sampling requirements being reduced from tens of thousands of points to a few
hundred. These results here confirm similar efficiency improvement results in a
stochastic EM application that have been reported by Jank [4] in a geostatistical
generalized linear mixed model.
1000 2-dimensional Normally Distributed Quasi-random Points
4
-4
-4
DISCUSSION / CONCLUSIONS
2000 2-dimensional Uniformly Distributed Quasi-random Points
4
-4
-4
Histogram of 5000 Normally Distributed Random Points
We are attempting to extend this work to SAEM. The use of quasi-random sequences for
MCMC algorithms such as SAEM is currently an area of active research in the academic
community. Raw Sobol sequences cannot be used directly because the components have
sequential correlations . However, these correlations can be removed while retaining
the essential low discrepancy property by using scrambling techniques such as proposed
in [5]. Unlike Monte Carlo integration, little is known regarding the theoretical behavior
of quasi-random sampling for MCMC, but empirical experiments suggest it may be a
useful accelerant.
-3
-2
-1
0
1
2
3
4
REFERENCES
Histogram of 5000 Normally Distributed Quasi-random Points
160
150
140
[1] C. Wei and M. Tanner. J. American Statistical Assoc., 85:699-704, 1990.
120
100
100
[2] R. H. Leary, R. Jelliffe, A. Schumitzky, and R. E. Port, PAGE 13, 2004, Abstract 491.
80
60
50
[3] P. Girard and F. Mentre’, PAGE 14, 2005, Abstract 834.
40
20
0
-4
-3
-2
-1
0
1
2
3
4
0
-4
-3
-2
-1
0
1
2
3
4
[4] W. Jank, Computational Statistics and Data Analysis, 48(4): 685-701, 2005.
[5] A .B. Owen, J. Complexity, 14(4): 466-489, 1998.