Planning for Gene Regulatory
Network Intervention
Daniel Bryce
Arizona State University
Seungchan Kim
Arizona State University &
Translational Genomics Research Institute
Prior Work

Planning for Finding Pathways



S. Khan, K. Decker, W. Gillis, and C. Schmidt. “A multi-agent system-driven AI
planning approach to biological pathway discovery.” In Proceedings of ICAPS’03,
2003.
Fifth International Planning Competition, 2006.
Reasoning about change in cellular processes

N. Tran and C. Baral. “Issues in reasoning about interaction networks in cells:
necessity of event ordering knowledge.” In Proceedings of AAAI’05, 2005.
Extracting and Expressing Transition Functions from
Micro-array experiments, Markov chain analysis.
S. Kim, H. Li, E. Dougherty, N. Cao, Y. Chen, M. Bittner,
and E. Suh. “Can Markov chain models mimic biological
regulation?” Journal of Biological Systems, 10(4):337–357,
2002.
I. Shmulevich, E. Dougherty, S. Kim, and W.
Zhang.”Probabilistic boolean networks: a rule-based
uncertainty model for gene regulatory networks.”
Bioinformatics 18(2):261–274, 2002.
Non-AI work on planning interventions.
A. Datta, A. Choudhary, M. Bittner, and E. Dougherty.
“External control in Markovian genetic regulatory networks:
the imperfect information case.” Bioinformatics, 20(6):924–
930, 2004.
1/10/07
Bryce & Kim -- IJCAI-07
2
Gene Regulatory Networks (GRNs)
Tissue
 Questions of interest:
 How does cancer occur?
 How can we prevent cancer?
Gene Correlations
g1How do
cells?
array
g2 we kill specificMicro
Data
 Can we control Differentiation?
Cell Type
(Phenotype,
e.g., liver cell)
 e.g., Program stem cell to become
g4Cell
g3 Liver
 Can we change Phenotype?
Dynamics Model
 e.g., Revert liver cell to back to
g2 g3
g1 cell,
stem
then g4
differentiate to heart
cell
g1’ g2’ g3’ g4’
From: [Wuensche, PSB-98]
1/10/07
Bryce & Kim -- IJCAI-07
3
Gene Regulatory Network Behavior
Edge Thickness == Pr(s | s’)
Extra cellular signals can effect the cell state
transitions (e.g., Chemotherapy, Pharmaceuticals,
and Stress)
Cancer
Phenotype
Steady
Transient
Undesirable
States
States
State
(normal)
(intermediate)
1/10/07
Partial
Observations of
molecular
components or
physiology are
available
Bryce & Kim -- IJCAI-07
4
GRN Intervention Planning
 Datta et. al. Assumptions
 Synchronous Events
 Exact Representation
 Optimal Bounded Length
Plans
Non-Intervention
Observation
 Datta et. al. Approach
 Enumerate Reachable
Belief States
 Dynamic Programming
Intervention
 Our Approach
Observation
 AI Planning
 AO* Search
1/10/07
Bryce & Kim -- IJCAI-07
5
Evaluation
 WNT5A GRN
 Highly active WNT5A
indicates proliferation
of cancer
 2 (non)interventions
 2 variations: direct and
indirect control
 2 observations
 7 genes (binary valued)
 Compare AI
Planning with
Datta et. al.
 Scaling horizon
 Sensitivity to
Reward Function
 Metric: Total Time
 Randomly Generated
GRN
 4 (non)interventions
 2 observations
 7 genes (binary valued)
1/10/07
Bryce & Kim -- IJCAI-07
6
WNT5A GRN (from TGEN dataset)
AO*
Total Time
Datta
1200
 Intevene Pirin gene
 Observe WNT5A gene
1000
Time(s)
 Indirect Control
800
600
400
200
0
2
4
6
8
10
12
Horizon
AO*
Total Time
Datta
Time(s)
2000
 Intervene WNT5A
gene
 Observe Pirin gene
1000
0
2
1/10/07
 Direct Control
4
6
8
10
Horizon
12
14
16
Bryce & Kim -- IJCAI-07
7
Random GRN (4 acts)
-10
-1
1
10
Datta
Total Time
Time(s)
2000
1000
0
2
4
6
8
Horizon
1/10/07
Bryce & Kim -- IJCAI-07
10
12
Goal
Reward
(AO*)
Enumeration
AO*
exploits
Reward
Function for
Pruning
(Improved
Scalability
In Some
Cases)
8
Assumptions Revisited
 Finite Horizon
 Not all treatments require same length
 Synchronous Change
 Actions overloaded to include GRN change
 7 Genes and 1 intervention
 Within human comprehension
1/10/07
Bryce & Kim -- IJCAI-07
9
Indefinite or Finite Horizon?
 Indefinite Horizon: If goal state is a steady state,
then no need to plan more actions to meet a given
horizon
1/10/07
Bryce & Kim -- IJCAI-07
10
Asynchronous Change
 Decouple Intervention from Gene
Regulatory Network Simulation
 Triggers (Tran and Baral, AAAI’05)
 Probabilistic Exogenous Events (Blythe,
UAI’94)
1/10/07
Bryce & Kim -- IJCAI-07
11
Larger GRNs
 50-5000 genes
 More Interventions and Observations
 Representation:
 ADD for transition relation blows up
 DBN is better, but exact inference can be costly
 Extensions of Thrun’s MC-POMDP’s, sample based
representation, is in the right direction
 Search Heuristics:
 McLUG: Planning Graphs with Probabilistic Actions
1/10/07
Bryce & Kim -- IJCAI-07
12
Conclusion
 Off-the-shelf AI planning improves upon state of
the art in Intervention Problems
 Future Research Needed:
 Scaling




Indefinite Horizon
Extra Actions and Observations
Sample-based Representation
Search Heuristics
 Modeling
 Asynchronous Probabilistic Change
 Plan Explanation
1/10/07
Bryce & Kim -- IJCAI-07
13
Extra Slides
1/10/07
Bryce & Kim -- IJCAI-07
14
Empirical Comparison
Datta
Enumeration
AO*
With no heuristics
Search performance
Correlates with Reward
Function
1/10/07
Total Time and
Expanded Nodes
Better in all Cases
Bryce & Kim -- IJCAI-07
15
The Network
1/10/07
Bryce & Kim -- IJCAI-07
18
The Parameters and Functions
1/10/07
Bryce & Kim -- IJCAI-07
19
Computational Biology
 Bioinformatics
 Knowledge Discovery & Data-mining
 Manage and Analyze Biological Data
 Systems Biology
 Simulation
 Model Dynamic Systems
1/10/07
Bryce & Kim -- IJCAI-07
20
Representing State Distributions
Algebraic Decision Diagram
Explicit Vector









g1 g 2 
_ _
g1 g 2
_
g1 g 2
_
g1 g 2
1/10/07
g1
.25 

.35 

.2 
.2



g2
.2
Bryce & Kim -- IJCAI-07
g2
.25
.35
21
Representing State Distributions
Algebraic Decision Diagram
Explicit Vector









g1 g 2 
_ _
g1 g 2
_
g1 g 2
_
g1 g 2
1/10/07
g1
.25 

.35 

.2 
.2



g2
.2
Bryce & Kim -- IJCAI-07
.25
.35
22
Representing Probabilistic Actions
Explicit Transition Matrix
_ _
g '1 g '2
 .2

 .1
0


g1 g 2  1
_ _
g1 g 2
_
g1 g 2
_
g1 g 2
1/10/07
_
g '1 g '2
.3
.6
.2
0
_
g '1 g '2
.5
0
.8
0
Algebraic Decision Diagram
g1
g '1 g '2
0

.3 
0

0 
g’1
g’1
g2
g2
g2
g2
g’2
g’2
g’2
g’2
g’2
g’2
g’2
g’2
0
.1
.2
.3
.5
.6
.8
1
Bryce & Kim -- IJCAI-07
23
Representing Probabilistic Actions
Explicit Transition Matrix
_ _
g '1 g '2
 .2

 .1
0


g1 g 2  1
_ _
g1 g 2
_
g1 g 2
_
g1 g 2
_
g '1 g '2
.3
.6
.2
0
_
g '1 g '2
.5
0
.8
0
Algebraic Decision Diagram
g1
g '1 g '2
0

.3 
0

0 
g’1
g2
0
1/10/07
g’1
g2
g2
g2
g’2
g’2
g’2
g’2
g’2
g’2
g’2
.1
.2
.3
.5
.6
.8
1
Bryce & Kim -- IJCAI-07
24
Modeling Network Dynamics
(- (influence1 ?g1 ?g2 ?g)
(noise))
(predicts ?g1 ?g2 ?g)
?g1
?g2
(- (influence2 ?g3 ?g4 ?g)
(noise))
(noise)
(predicts ?g3 ?g4 ?g)
?g3
?g4
?g1
(not (up-regulated ?g1))
(not (up-regulated ?g1))
(up-regulated ?g1)
(up-regulated ?g1)
?g1
(not (up-regulated ?g1))
(not (up-regulated ?g1))
(up-regulated ?g1)
(up-regulated ?g1)
1/10/07
(noise)
?g2
(not (up-regulated ?g2))
(up-regulated ?g2)
(not (up-regulated ?g2))
(up-regulated ?g2)
0
1
?g
(pred-fn ?g1 ?g2 ?g zz)
(pred-fn ?g1 ?g2 ?g zo)
(pred-fn ?g1 ?g2 ?g oz)
(pred-fn ?g1 ?g2 ?g oo)
?g2
?g
(not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g zz)
(up-regulated ?g2) (pred-fn ?g1 ?g2 ?g zo)
(not (up-regulated ?g2)) (pred-fn ?g1 ?g2 ?g oz)
(up-regulated ?g2) (pred-fn ?g1 ?g2 ?g oo)
Bryce & Kim -- IJCAI-07
?g
25
Network Dynamics Encoding <dynamics>
1/10/07
(forall (?g ?g1 ?g2 ?g3 ?g4 - gene)
;;constraint for grounding that binds only those genes ?g1 - ?g4 that
;;predict ?g. External control actions add predicates to the
;;antecedent below so that ?g does not bind to controlled genes.
(when (and (predicts1 ?g1 ?g2 ?g) (predicts2 ?g3 ?g4 ?g))
(probabilistic
(- (influence1 ?g1 ?g2 ?g) (noise)) ;;predictor 1 probability
(and (when (or
;;conditions to set ?g up
(and (not (up-regulated ?g1)) (not (up-regulated ?g2))
(pred-fn ?g1 ?g2 ?g zz))
(and (not (up-regulated ?g1))
(up-regulated ?g2)
(pred-fn ?g1 ?g2 ?g zo))
(and
(up-regulated ?g1) (not (up-regulated ?g2))
(pred-fn ?g1 ?g2 ?g oz))
(and
(up-regulated ?g1)
(up-regulated ?g2)
(pred-fn ?g1 ?g2 ?g oo))
)
(up-regulated ?g)) ;;set ?g up
(when (or
;;conditions to set ?g down
(and (not (up-regulated ?g1)) (not (up-regulated ?g2))
(not (pred-fn ?g1 ?g2 ?g zz)))
(and (not (up-regulated ?g1))
(up-regulated ?g2)
(not (pred-fn ?g1 ?g2 ?g zo)))
(and
(up-regulated ?g1) (not (up-regulated ?g2))
(not (pred-fn ?g1 ?g2 ?g oz)))
(and
(up-regulated ?g1)
(up-regulated ?g2)
(not (pred-fn ?g1 ?g2 ?g oo)))
)
(not (up-regulated ?g))) ;;set ?g down
)
(- (influence2 ?g3 ?g4 ?g) (noise)) ;;predictor 2 probability
(and [...])
;;predictor 2, similar to predictor 1
(noise) (up-regulated ?g)
;;noise to set ?g up
(noise) (not (up-regulated ?g))
;;noise
to set ?g
Bryce
& Kim
--down
IJCAI-07
)
Bind all genes
to variables
Binding constraints
probability
Of using
predictor1
conditions to
up-regulate
with predictor1
up-regulate
with predictor1
Conditions to
down-regulate
down-regulate
with predictor1
Rules for
predictor2
26
Network Parameters
.68
.30
(predicts wnt5a ret2 s100p)
wnt5a
ret2
.01
(predicts stc2 ret2 s100p)
stc2
ret2
?g1
(not (up-regulated ?g1))
(not (up-regulated ?g1))
(up-regulated ?g1)
(up-regulated ?g1)
?g2
(not (up-regulated ?g2))
(up-regulated ?g2)
(not (up-regulated ?g2))
(up-regulated ?g2)
0
Bryce & Kim -- IJCAI-07
1
?g
(pred-fn ?g1 ?g2 ?g zz)
(pred-fn ?g1 ?g2 ?g zo)
(pred-fn ?g1 ?g2 ?g oz)
(pred-fn ?g1 ?g2 ?g oo)
wnt5a
ret2
s100p
(not (up-regulated wnt5a)) (not (up-regulated ret2)) (not (up-regulated s100p))
(not (up-regulated wnt5a))
(up-regulated ret2)
(up-regulated s100p)
(up-regulated wnt5a) (not (up-regulated ret2))
(up-regulated s100p)
(up-regulated wnt5a)
(up-regulated ret2)
(up-regulated s100p)
1/10/07
.01
s100p
27
Predictor Encoding
(= (noise) .01)
;s100p predictor1
wnt5a
(predicts1 wnt5a ret2 s100p)
(not (up-regulated wnt5a))
wnt5a))
(pred-fn wnt5a ret2 s100p zo) ;1 (not (up-regulated
(up-regulated wnt5a)
(pred-fn wnt5a ret2 s100p oz) ;1 (up-regulated wnt5a)
(pred-fn wnt5a ret2 s100p oo) ;1
(= (influence1 wnt5a ret2 s100p)
.69)
stc2
(not (up-regulated stc2))
(not (up-regulated stc2))
(up-regulated stc2)
(up-regulated stc2)
;s100p predictor2
(predicts2 stc2 ret2 s100p)
(pred-fn stc2 ret2 s100p oz) ;1
(pred-fn stc2 ret2 s100p oo) ;1
(= (influence2 stc2 ret2 s100p) .31)
1/10/07
Bryce & Kim -- IJCAI-07
ret2
s100p
(not (up-regulated ret2)) (not (up-regulated s100p))
(up-regulated ret2)
(up-regulated s100p)
(not (up-regulated ret2))
(up-regulated s100p)
(up-regulated ret2)
(up-regulated s100p)
ret2
s100p
(not (up-regulated ret2)) (not (up-regulated s100p))
(up-regulated ret2) (not (up-regulated s100p))
(not (up-regulated ret2))
(up-regulated s100p)
(up-regulated ret2)
(up-regulated s100p)
28
Control Encoding <control>, perfect/partial
obeservation
(:action down-regulate
:parameters (?gr ?go - gene)
:precondition (and (observed ?go)
(controlled ?gr)
(started))
:effect
(and (decrease (reward) 1)
(when (up-regulated ?gr) (not (up-regulated ?gr)))
<dynamics + (not (= ?g ?gr))>
)
:observation (
((up-regulated ?go) (up-regulated ?go) 1)
((not (up-regulated ?go)) (not (up-regulated ?go)) 1)
)
)
Could be better
model!?
(when (up-regulated ?gr) (probabilistic .75 (not (up-regulated ?gr))))
1/10/07
Bryce & Kim -- IJCAI-07
29