Modeling, Model Validation, Parameter Estimation, Model Reduction
Background
The purpose of a model is to solve a problem, such as either predicting or describing
something or to gain some kind of knowledge.
By a model we mean a triple consisting of (our understanding of) a real world system, a
mathematical system and a representation between the two. It is a point that information
is lost in this representation so the model reduce complexity in a way such that we are
better able of treating the model than the real world system. Another important point is
that we are only able of include a limited part of the real world, the part we think are
important for answering the problem. A third point is that one hardly sees the real world
as it really is but only sees our limited conceptual image of it.
Knowledge of the real world usually comes from observations, experiments and
modeling. When we want to put up a model trying to solve a problem, we usually base it
on all three elements. In physiology clinical experiments are central but never free of
interpretations, noise and errors. If we consider the example of measuring blood pressure
during a sit to stand transition one always got noise from the equipment and modulation
and noise from the intrinsic system under consideration. All measurements use
mathematical model to some degree to generate the measurements. Of cause some
measurements are fairly directly as measuring time etc. but even here models are
involved. Most of the time models are indirect and do use modeling in a non-trivial way
to generate measurements. Peripheral resistance is not measured directly; one measure
cardiac output, i.e., flow and pressure and calculate the resistance based on the model that
the two are proportional. The constant of proportionality is defined to be the resistance.
Notice, that we really could not do this model based calculation without interpreting the
measured quantities as physiological quantities. It is well known for experimentalists that
many measurements are wrong for different reasons. The sensors are not located
precisely as they should or the equipment does work correctly or are not calibrated well.
Beyond this general problems related to measurements there is a problem with classical
reproducibility, due to the fact that one can’t control all variable except for one or a few
during the experiment. The time series giving the pressure data for one person at one time
is usual not identical to the in principal same time series for the same person at another
time, in fact, it may deviate quite a lot. If the same person change position from sitting to
standing giving one time series and then sits down to repeat the experiment then the
initial conditions have changed. Moreover, muscle activity (central commend) and
physiological conditions done between the two experiments influence the second time
series. It is even worse if one would like to compare two time series from different
persons.
Modeling
We discussed modeling HR as proposed by Ottesen et al., comments included detailed
outline of the model and suggestions for model improvements in particular, we discussed
the presence and mechanism of central command, which according to both Eckberg and
Karemaker contributes to vagal withdrawal. This would be an accompanying signal from
the CNS to the motor command that might show up as suppression of the baroreflex
signal transmission from the nucleus tractus solitarii to the cardiac vagal motor nucleus.
Moreover the pressure sensitive afferents from the contracting muscle itself can have the
same effect. The ensemble of these ‘muscle-heart reflex’ effects will show up as fast
vagal withdrawal going together with the initiation of muscular effort. (refs:
Hollander/Bouman muscle-heart reflex in humans; Gelsema/Bouman/Karemaker same in
cats).
Other suggestions for modeling include “validation” of models against a number of
scenarios e.g. head-up tilt, sit-to-stand, bicycling ergometer tests, hand-grip. Karemaker
showed dataset (see fig. 1) with differentiated input indicating contributions from both
baroreceptors and modulation of atrial filling. Other topics for modeling include response
to slow tilt, which will induce syncope if the subject is tilted for long periods.
Suggestions by Eckberg to try to develop models that reproduce the scenario to try to
understand what happens just before syncope.
Model Validation, Parameter Estimation
Methodologies for model validation and parameter estimation were discussed by Fowler
and Gray. In particular, we discussed a range of derivative free optimization methods.
The methods considered were for designed simulation-based or “black-box”
minimization problems and use only function values to guide the minimization process.
In other words, derivatives are not required. These methods are well-equipped to handle
problems with low-amplitude noise, nonconvexity, nonsmoothness, and many local
minima. More specifically, they are ideal for the problems of parameter calibration that
arise from physiological models.
As a first attempt, we will perform a suite of parameter estimation optimization runs on
the heart rate model of Ottesen et al and compare the results therein obtained with the
Nelder Mead algorithm (see fig 2). We will use Matlab implementations of the implicit
filtering algorithm, DIRECT, and the genetic algorithm available in the Matlab
optimization toolbox as a first attempt. Later, we will apply the assychronous parallel
pattern search method as implemented in APPSPACK.
The group discussed the meaning of validation as it relates to physiological models using
examples from electrical engineering. Overall, we agreed that the goal is to quantify the
uncertainties and to explain the variations in the models. We presented some tools for
validation and sensitivity analysis including Minitab sensitivity plots, interaction effects,
the McKay correlation ratio, and comparison metrics. Our plan is to apply some of these
techniques to the heart rate model of Ottesen et al. to provide some added insight into the
role and importance of the parameters.
Model Reduction
This topic was not discussed in as much detail. Comments from Panerai indicated that the
focus on model reduction should be tied to limitations for parameter estimation and
model validation. An important point put forward by Karemaker was the importance of
making models follow the physiology as accurately as possible.
Figure 1: The first trace include head-up-tilt where subject was tilted 70 degrees over 2-3
sec. The cycling experiments had the subject sitting on the bike and then biking with a
maximum load for 2-3 seconds followed by rest. Both head-up-tilt and standing were
from supine position. For supine-to-stand, the subject was told to stand when ready, i.e.
the subject didn’t stand immediately following the cue. Sprangers and JJ Van Lieshout.
Figure 2: New optimizations with heart rate model by Ottesen et al. Left panel shows new
optimization with Nelder-Mead method, this converged with a cost J = 0.0071, the
optimization took 1-2 hours on a Beowulf cluster.
To be discussed & scrutinized:
A framework for model validation.
Model validation follows different stages, depending whether models are compartmental
(mathematical, structural), or ‘data-driven’ (‘black-box’, input-output). The former
involves a preliminary stage of optimization for parameter estimation (as mentioned
above), whilst for the latter, model parameters are usually obtained from system
identification procedures such as frequency-domain analysis or time-domain
identification (e.g. MA or ARMA models). Despite these differences in the initial stages
of model development, the validation process can follow similar routes, which in most
cases can be seen as a hierarchical process, aimed at addressing the following questions:
1. What is the sensitivity of model prediction errors, due to:
i)
the same data used to estimate parameters (or fit the model)
ii)
another set of very similar data from the same subject, a short time before or
after the development set of data (test set)
iii)
similar set of data, same subject, but for longer time differences in relation to
the development set (longitudinal variability).
iv)
similar set of data for different subjects (intra-subject variability)
v)
physiologically different sets of data (e.g. different manouevres), for same
subject (physiological latitude, generalizability of model).
2. What is the performance of the model in clinical applications, regarding:
i)
group consistency (e.g. normal vs. pathology)
ii)
diagnostic value for individual patients
iii)
sensitivity/specificity
iv)
safety