Computational Sustainability
March 12, 2010 at 1:25PM – 2:40PM
1150 Snee Hall
Optimizing Intervention Strategies in Food
Animal Systems: modeling production, health
and food safety
Elva Cha, DVM, PhD candidate, Rebecca Smith, DVM, PhD
candidate, Zhao Lu, PhD, Research Associate and Cristina
Lanzas, DVM, PhD, Research Associate and Yrjö T. Gröhn,
DVM, MPVM, MS, PhD
Department of Population Medicine and Diagnostic Sciences,
College of Veterinary Medicine, Cornell University
Perhaps I should start with “ a disclaimer”…
If I understood correctly, your approach is to assume the
model is 'correct', then optimize the system.
As veterinarians, our responsibility is to build a
model based on subject matter representing reality.
Of course, our goal is to find the optimal way to control
disease (not necessarily eradication). And sometimes we
need to learn the economically optimal way to coexist
with them.
Food Supply Veterinary Medicine
….all aspects of veterinary medicine's involvement in
food supply systems, from traditional agricultural
production to consumption.
Modeling production, health and food safety:
1. Optimizing health and management decisions
2. Mathematical modeling of zoonotic infectious
diseases (such as L. monozytogenes, E. coli, MDR salmonella and
paratuberculosis).
Three examples …
1. Modeling production and health:
Project 1. “Optimal Clinical Mastitis Management in Dairy cows.”
Elva Cha’s PhD research
Project 2. “Cost Effective Control Strategies for The Reduction of
Johne’s Disease on Dairy Farms.” Zhao Lu, Research Associate and
Becky Smith’s PhD research
2. Modeling Food Safety:
Project 3. “Food Animal Systems-Based Mathematical Models of
Antibiotic Resistance among Commensal Bacteria.” Cristina
Lanzas, Research Associate
Let’s start with…
1. Modeling production and health:
Our overall goal is to develop a comprehensive
economic model, dynamic model (DP), to assist
farmers in making treatment and culling
decisions.
Our 1st example:
Elva Cha’s PhD research: “Optimal Clinical Mastitis
Management in Dairy cows”
Mastitis (inflammation in mammary gland)
Common, costly disease (major losses: milk yield,
conception rates, and culling).
The question we address is whether it is better to treat animals at
the time CM is first observed or whether it is worthwhile collecting
more information related to the nature of the pathogen involved
(Gram-positive or Gram-negative, or even the actual pathogen), and
then make a treatment decision.
More information would seem to be of benefit when deciding what
to do with diseased cows, but unless the additional tests provide a
different recommended outcome, they are only an extra cost.
Objective:
• To determine the economically optimal amount of
information needed to make mastitis treatment
decisions
To do this
• We will build upon our existing Dynamic
Programming model.
• This work requires us to determine the risk and
consequences of CM (milk loss, delayed conception,
mortality) as functions of cow characteristics,
including the disease history and disease prognosis
of the cow.
How do we address our objective?
• We will compare 2 models
– Model 1 which will identify cows based on generic
(non-specific) mastitis
– And Model 2, whereby if a cow has mastitis, it will
be specified as gram-positive, gram-negative or
other types of mastitis
– We will then compare the profit of each model
What is this model?
The models
• The models will calculate the optimal policy
for each individual cow
• This can be done by ‘dynamic programming’
Dynamic programming
• Numerical method for solving sequential
decision problems
– Based on the ‘Bellman principal of optimality’
– In our case, the ‘system’ is dairy cows, observed
over an infinite time horizon split into ‘stages’ i.e.
months of a cow’s life
Dynamic programming
At each stage, the state of the system is observed
• E.g. pregnancy status, disease status, milk yield
And a decision is made
• Replace, keep (treat and inseminate) or sell
This decision influences stochastically the state to
be observed at the next stage
Depending on the state and decision, a reward is
gained
Dynamic programming
– Value function is the expected total rewards from
the current stage until the end of the horizon
– Optimal decisions depending on stage and state
are determined backwards step by step
Structure of the hierarchic Markov process optimization model
Founder
Process
a
Stage
a
a
State
Child
a
1
a
Action
2
3
b
Grandchild
4
Lactation
5
b
1
6
2
7
8
a states of permanent milk yield
0
Month in
lactation
3
c
4
5
c
6
c
b states of mastitis in previous lactation
c states of temporary milk yield, pregnancy, and mastitis
c
c
c
Replace
Keep
Inseminate
Parameters in our model
• For each type of mastitis
– Risk of mastitis
– Repeated risk of mastitis
– Effect on milk yield
– Effect on conception
– Effect on mortality and culling
Limitations
• Now we can only study 3 different types of
mastitis, although we have data for very
specific types of mastitis! (i.e. >3!)
Why is this a limitation? Can’t we simply expand
the model?
Implications of a larger model
• By expanding the model, we will encounter
the ‘curse of dimensionality’
– An opportunity cost of including another disease,
and hence the parameters associated with it
– The model increases as a power function, not by a
factor of 1, 2 or 3…
– This makes computations even more challenging
and time consuming
The curse of dimensionality
example: Houben et al. 1994
 State
variables:
• Age (monthly intervals, 204 levels)
• Milk yield, present lactation (15 levels)
• Milk yield, previous lactation (15 levels)
• Length of calving interval (8 levels)
• Mastitis, present lactation (4 levels)
• Mastitis, previous lactation (4 levels)
• Clinical mastitis (yes/no)
 Total state space 6,821,724 states
The curse of dimensionality
example: Gröhn et al. 2003
 State
variables:
• Parity (12 levels)
• Conception in month (10 levels)
•Stage of lactation (20 levels)
•Milk yield (5 levels)
• Month of calving (12 levels)
• Disease index (212 levels)
 Total state space 144,000 x 212= 30,528,000
Structure of the hierarchic Markov process optimization model:
First level:
5 milk yield levels
Second level:
8 possible lactations and 2 carry-over mastitis states from previous
lactation (yes/no).
Third level:
20 lactation stages (max calving interval of 20 months).
5 temporary milk yield levels (relative to permanent milk yield),
9 pregnancy status levels (0=open, 1-7 months pregnant, and 8=to be
dried off), one involuntary culled state
13 mastitis states:
0=no mastitis
1 = 1st occurrence of CM (observed at the end of the stage),
2, 3, 4 = 1, 2, 3 and more months after 1st CM,
5 = 2nd CM,
6, 7, 8 = 1, 2, 3 and more months after 2nd CM,
9 = 3rd CM,
10, 11, 12 = 1, 2, 3 and more months after 3rd CM,
CM events > 3 assigned same penalties as if they were 3rd occurrence.
After deleting impossible stage-state combinations, the model
described 560,725 stage-state combinations.
If we were able to overcome the curse
of dimensionality …
No longer only generic guidelines for the
generic cow.
The DP recommendations could be tailored to
the individual cow in real time according to her
cow characteristics and economics of the herd.
Project 2: “Cost Effective Control Strategies for The
Reduction of Johne’s Disease on Dairy Farms”
Zhao Lu, PhD, Research Associate, and
Becky Smith, DVM, PhD student
Johne’s disease (paratuberculosis)
• Johne’s disease is a chronic, infectious, intestinal disease
caused by infection with Mycobacterium avium subspecies
paratuberculosis (MAP).
• Infection process of paratuberculosis on a dairy cow:
Infection status
Infection
Transient
shedding
Latency
No clinical signs
Low shedding
Sub-clinical
Disease status
High shedding
Clinical
Issues of Johne’s disease
• Economic loss: > 200 million $ per year (Ott, 1999) due to the reduced
milk production, lower slaughter value, etc.
• Public health: a potential association between Johne’s disease and human
Crohn’s disease has been debated.
• Control of Johne’s disease:
– Test and cull strategies, i.e., to cull/remove infectious animals form herd by
test-positive results using diagnostic testing methods, such as culture and
ELISA tests.
– Improved hygiene management;
– Vaccination.
However, it is difficult to control JD spread:
– Long incubation period;
– Low diagnostic test sensitivity for animals shedding low levels of MAP;
– Cross reactivity of Johne’s disease vaccines with tuberculosis (TB).
Modeling of MAP transmission on a dairy herd
• A deterministic compartmental model (Mitchell et al., 2008).
The test-based culling rates of low and high shedders are
denoted by δ1 and δ2, respectively.
-

()
X1
Susceptible

d
X2
Resistant
d
Tr

H
Y1
Latent
Transient
d

d
Low
d
1

Y2
High
d  
2
Evaluation of effectiveness of test-based culling
in Johne’s disease control
• The reproduction ratio
R0 was derived and a
global parameter
uncertainty analysis was
performed to determine
the effectiveness of testbased culling
intervention (Lu et al.,
2008)
A stochastic multi-group model
(Evaluation of effectiveness of test-based culling)
Calves
Heifers
Cows
Optimal control of Johne’s disease:
economic model
• Objective function (cost function):
– Profit: selling milk, culling cows (meat);
– Cost: raising first and second-year calves/heifers; diagnostic testing;
– Lost: false-positive testing results.
• Control: various scenarios of test-based culling rates and
management
• Constraints: a system of dynamic equations.
– Compartment model providing numbers of calves, heifers, and cows in
each compartment, which are needed in the objective functional (cost
function).
• A deterministic, discrete time economic model has been developed to
find optimal test-based culling rates with large (6 and 12 month) time
steps.
What do we want?
(Optimal control of test-based culling rates)
• A deterministic, continuous time economic model.
– Analytical studies of linear controls (test-based culling rates);
– Numerical search of optimal test-based culling rates.
• A stochastic economic model.
– Reasons: more realistic (variable prevalence and fadeout due to random
events)
– Optimal culling rates using stochastic differential equations;
– Numerical simulations of optimal culling rates for the mean of cost
function.
• Economic analysis of Johne’s disease vaccines.
– Mathematical modeling of imperfect Johne’s vaccines is in progress.
Modeling the efficacy of an imperfect
vaccine with multiple effects
• Vaccines are often
imperfect
– They may not prevent all
infections
– They may have effects other
than decreasing susceptibility
• Efficacy can be considered
as the proportional effect
on a rate or probability
parameter in a
compartmental model
5 vaccine effects:
1.
2.
3.
4.
5.
Vertical transmission
Horizontal transmission
i. Susceptibility
ii. Infectiousness
Duration of latency
Duration of low-infectious
period
Progression of clinical
symptoms
Modeling vaccine efficacy against JD
(1-p)(-)
()
X1
Susceptible

d
(1-p)
Tr
Transient
d


Y1
Low
d
Y2
High
d

 t b )   1Y1 t b )   2 Y 2 t b )   3 Tr t b )  e1  1V Y 1 t b )   2V Y 1 t b )   3V Tr t b )
d
 t ) 
p-)
 1Y1 t )   2 Y 2 t )   3 Tr t )  e 21  1V Y 1 t )   2 V Y 2 t )   3V Tr t )
N
p
VX1 e20()
VTr
Susceptible
Transient

VX2
Resistant
d
H
Latent
d
X2
Resistant
d

d

VH
Latent
d
e3
VY1
Low
d
e4
VY2
High
d
e5
Estimating vaccine efficacy against JD
with field data
• Known information:
– birth date
– death date
– annual test dates and
results
– vaccination status
• Missing information:
–
–
–
–
Date of infection
Onset of low-shedding
Onset of high-shedding
True infection status (if
all tests results were
negative)
To estimate vaccine efficacy,
missing information must also be estimated
Estimating vaccine efficacy with
Markov Chain Monte Carlo models
MCMC models are Bayesian statistical models, useful for disease
modeling because they
• Can account for nonlinear systems
– parameters may be inter-related
• Can account for time-dependence
– i.e. infectious pressure
• Have a mechanism for missing-data problems:
– Missing information can be estimated probabilistically, given a set of
parameters drawn from a prior distribution
– The full dataset can then be used to determine the relative likelihood
of a different set of parameters drawn from the prior
• The new set of parameters may be accepted or rejected, based on its
relative likelihood
– This process is iterated until it converges on a posterior distribution for
all parameters
Validating MCMC models
• In order to test that an MCMC model predicts the
true parameter distribution, we feed it data
simulated with known parameters
• In the case of the JD model, the full model requires
individual animal data:
– Infection status
– Vaccination status
– Dates of birth, compartment transitions, death
• We need an individual-animal stochastic model
Optimal control of Johne’s disease using an
individual-based (agent-based) modeling approach
• Controls: test-based culling, farm management, JD vaccines
• Advantages of individual-based modeling (IBM)
– Providing a general framework to model infectious disease transmission
in a dairy herd;
– Integrating all individual information together to predict the dynamics on
farms;
– Adding controls on farm level and/or individual animals easily.
– Economic analysis based on IBM would be more accurate.
• Disadvantages of IBM
– Individual information collection: a detailed profile for each animal in a
herd. (also an advantage)
– Simulations: efficient algorithm and powerful computers are necessary.
Project 3: “Develop, evaluate and improve food
animal systems-based mathematical models
of antimicrobial resistance among
commensal bacteria”
Cristina Lanzas, DVM, PhD, Research Associate
• At least 200,000 people suffer
from hospital acquired infection
every year, and at least 90,000
die in US.
• Economical burden of
antimicrobial resistance in
clinical settings in US is
estimated to be as high as $ 80
billion annually.
• Pathogens outside the hospital
are also becoming progressively
resistant to common
antimicrobials.
Bergstrom and Feldgarden, 2008
• Antimicrobial resistance is also
considered a food safety issue
because infections with drug
resistant foodborne pathogens
(e.g. Salmonella) can be
particularly serious.
• Reservoirs of resistant genes are
found in commensal bacteria in the
human and animal gastrointestinal
tracts (small intestine supports ~
1010 bacterial cells/g) .
• Commensal bacteria can transfer
mobile genes coding antimicrobial
resistance among themselves and
to pathogen bacteria (e.g. plasmid
transfer between Salmonella and E.
coli)
Salyers et al., 2004
Molecular mechanisms
involved in the spread of
antimicrobial resistance. Intercellular movement (horizontal
spread) is the main cause of
acquisition of resistance
genes.
Boerlin, 2008
WITHIN HOST
BETWEEN HOSTS
-
S
S
R
Population dynamics of
antibiotic-sensitive and –
resistant bacteria
Linked to antibiotic exposure
Emergence of resistance
during antibiotic treatment
Fitness cost linked to
microbial growth
+
I
The host population is divided
according its epidemiological
status (e.g. susceptible,
infectious)
“Binary response”: Animal
carries the bacteria carrying the
resistance or not
Transmission of resistant
clones
Individuals colonized with
either susceptible or resistant
strains
Within host dynamics of antimicrobial resistance dissemination
Microbial growth for sensitive and resistant strains with horizontal
gene transfer
dt
dN r
dt
 rN s - r
(N s  N r )
K
 r (1 -  ) N r - r (1 -  )
Ns - 
(N s  N r )
K
Logistic
Growth
E max
E d (C )  1 
EC
H
50
NsNr
 C 


MIC


N
- uE d ( C ) N s
Nr  
NsNr
N
Plasmid
transfer
- uE d ( C ) N r - r (1 -  ) pN r
Antibiotic
effect
Plasmid loss
during
segregation
H
 C 


 MIC 
100
90
H
Percentage of resistant bacteria 24 h
after the end of the antimicrobial
treatment
80
% bacteria with resistance
dN s
70
60
50
40
30
20
10
0
1
2
3
4
Duration of treatment, days
5
Between host dynamics of antimicrobial resistance
dissemination
S
S
γ
susceptible
IS
IR
ISR
I
infectious
πη
λ
W
environment

Integrating within and between host
antimicrobial resistance dynamics
• Interventions to minimize the dissemination of antimicrobial resistance
can be applied at different organizational levels (e.g. within
host/between hosts and environment):
•Optimize antimicrobial dosage regimes to mitigate the
dissemination of antimicrobial resistance within enteric commensal
bacteria.
•Reduce the exposure of animals to antimicrobial resistant bacteria.
• Mathematical approaches that integrate within and between host
dynamics are necessary to optimize mitigation strategies acting at
different hierarchical scales:
• Agent-based/Individual-based models
• Dynamic nested models
Modeling On-farm Escherichia coli O157:H7
population dynamics
• Metapopulation models has allowed us to investigate the
potential role of non-bovine habitats (i.e., water troughs,
feedbunks, and the surrounding pen environment) on the
persistence and loads of E. coli O157:H7 in feedlots.
•O157:H7 survive and reproduce in water troughs, feed, slurry,
pen floors.

C 
 rc C 1  -   c  p )C  m ec E  m wc W  m fc F
dt
K
c 

dC
dW
dt

W 
 rw W 1  -   w  m wc  m we )W  m ew E  m cw pC
K
w 


F 
 r f F 1   m cf pC - 
dt
K
f 


dF
f
m
fc
)F

E 
 re E 1   m we W  m ce pC -   e  m ew  m ec ) E
dt
K
e 

dE
Ayscue et al., Foodborne Pathog Dis, 6:461-470 (2009)
This metapopulation approach is suitable for modeling the dynamics of antimicrobial
resistance dissemination. Pharmacokinetics and pharmacodynamics and biological
fitness of antimicrobial resistance can be integrated.
Monogastric calf
S, R
S, R
Growing ruminant
heifer
Animal
patches
S, R
Selective
S, R
Bred ruminant heifer
pressures
Lactating cow
Dry cow
Water
Environment
(S,R)
(S, R)
Cull cow
Dairy beef
Assuming three types of ecological patches (water, environment and n
animals) and assuming indirect transmission (bacteria are transmitted to
animals through water and environment):
For the j animal:
dN
s j
 r j N sj - r j
dt
dN
r j
dt
(N s j  N r j )
K
Nsj - 
Ns jNr j
N
j
 r j (1 -  ) N rj - r j (1 -  )
j
( N s j  N rj )
K
- u j E d ( C ) N s j  m wj W s  m ej E s - ( m jw  m je ) N sj
N sj N rj
Nr j  
N
j
- u j E d ( C ) N rj - r j (1 -  ) pN r j
j
 m wj W r  m ej E r - ( m jw  m je ) N rj
Water patch:
dW s
dt
dW r
dt
 rwW s - rw
(W s  W r )
Kw
 rw (1 -  )W r - rw (1 -  )
Ws - 
(W s  W r )
Kw
jn
Ws Wr
- u wW s 
W
Wr - 
jn

m jw N s j  m ew E s -  m wj W s - m we W s
j 1
Ws Wr
W
j 1
jn
- u wW r 

j 1
jn
m jw N r j  m ew E r -  m wj W r - m we W r - rw (1 -  ) pW r
j 1
Environmental patch:
dE s
dt
dE r
dt
 re E s - re
(Es  Er )
 re (1 -  ) E r - re (1 -  )
Ke
Es - 
(Es  Er )
Ke
Er - 
Es Er
E
Es Er
E
jn
- ue Es 

j 1
jn
- ue Er 

j 1
jn
m je N s j  m we W s -  m ej E s - m ew E s
j 1
jn
m je N r j  m we W r -  m ej E r - m ew E r - re (1 -  ) pW e
j 1
Potential students projects
• Application of optimal control to evaluate strategies in metapopulation
models
• Development of agent based models to address antimicrobial resistance
dissemination.
• Optimization in agent based models
• Optimization in hierarchical models