Thor Whalen
Metron, Inc.
whalen@metsci.com
Research Notes
Subject:
Bayesian Networks
The purpose of this research is to investigate problems that arise in designing the
Decision Architecture (DA) being developed for MDA within project Hercules. This is
the second phase of the project. Prior work:

Final report for Phase I

Proposal for Phase II
Directions:



Find ways to integrate evidence in distributed Bayesian networks (connect or
reconnect a subnet to rest of the decision architecture in such a manner that
information from the subnet is correctly and optimally integrated into the Bayes’
net representing the rest of the DA)
Dynamically reconfigure the net to account for new targets
Incorporate Gaussian and non-Gaussian, continuous measurement information at
leaf nodes
Key words:

Distributed Bayesian networks
Websites:
Papers:
Causal Decomposition

“A Factorized Representation of Independence of Causal Influence and Lazy
Propagation”
(Anders L. Madsen, et al.(1999))
Abstract:
The efficiency of algorithms for probabilistic inference in Bayesian networks can be improved by
exploiting independence of causal influence. The factorized representation of independence of causal
influence offers a factorized decomposition of certain independence of causal influence models. We
describe how lazy propagation - a junction tree based inference algorithm - easily can be extended to take
advantage of the decomposition offered by the factorized representation. We introduce two extensions to
the factorized representation easing the knowledge acquisition task and reducing the space complexity of
the representation exponentially in the state space size of the effect variable of an independence of causal
influence model. Finally, we describe how the factorized representation can be used to solve tasks such as
calculating the maximum a posteriori hypothesis, the maximum expected utility, and the most probable
configuration.

“Exploiting Functional Dependence in Bayesian Network Inference”
(Anders L. Madsen, et al.(2002))
Abstract:
This paper explores the role of independence of causal influence (ICI) in Bayesian network
inference. ICI allows one to factorize a conditional probability table into smaller pieces. We describe a
method for exploiting the factorization in clique tree propagation (CTP) --- the state-of-the-art exact
inference algorithm for Bayesian networks. We also present empirical results showing that the resulting
algorithm is significantly more efficient than the combination of CTP and previous techniques for
exploiting ICI.

“Exploiting Causal Independence in Bayesian Network Inference”
(Zhang and Poole (1996))
Abstract:
A new method is proposed for exploiting causal independencies in exact Bayesian network
inference. A Bayesian network can be viewed as representing a factorization of a joint probability into the
multiplication of a set of conditional probabilities. We present a notion of causal independence that enables
one to further factorize the conditional probabilities into a combination of even smaller factors and
consequently obtain a finer-grain factorization of the joint probability.
The new formulation of causal independence lets us specify the conditional probability of a
variable given its parents in terms of an associative and commutative operator, such as ``or'', ``sum'' or
``max'', on the contribution of each parent. We start with a simple algorithm VE for Bayesian network
inference that, given evidence and a query variable, uses the factorization to find the posterior distribution
of the query. We show how this algorithm can be extended to exploit causal independence. Empirical
studies, based on the CPCS networks for medical diagnosis, show that this method is more efficient than
previous methods and allows for inference in larger networks than previous algorithms.

“Intercausal independence and heterogeneous factorization”
(Zhang and Poole (1994))
Abstract:
A constructive definition of intercausal independence is given. It is well known that conditional
independence implies factorization of joint probability. Under the constructive definition, intercausal
independence implies factorization of conditional probability. An inference algorithm is developed, which
makes use of both conditional independence and intercausal independence to reduce inference complexity
in Bayesian networks.

“Partition-based Anytime Approximation for Belief Updating”
(Mateescu, Dechter, Kask (2001))
Abstract:
The paper presents a parameterized approximation scheme for probabilistic inference. The
scheme, called Mini-Clustering (MC), extends the partition-based approximation offered by mini-bucket
elimination, to tree decompositions. The benefit of this extension is that all single-variable beliefs are
computed (approximately) at once, using a two-phase message-passing process along the cluster tree. The
resulting approximation scheme allows adjustable levels of accuracy and efficiency, in anytime style.
Empirical evaluation against competing algorithms such as iterative belief propagation and Gibbs sampling
demonstrates the potential of the MC approximation scheme for several classes of problems.

“On the impact of causal independence”
(Rish and Dechter (1998))
Abstract:
Reasoning in Bayesian networks is exponential in a graph parameter called induced-width (also
known as tree-width and max-clique size). In this paper, we investigate how a property called causal
independence can improve this performance. We show that the "effective" induced-width of algorithms
exploiting this property can be significantly reduced: it can be as small as the induced width of the
unmoralized network's graph, and will never exceed the induced-width of the network's moral graph. For
example, for poly- trees, causal independence reduces complexity from exponential to linear in the family
size. Our analysis is presented for belief updating first, and is then extended to three other tasks: finding a
most probable explanation (MPE), finding the maximum a posteriori hypothesis (MAP) and finding the
max- imum expected utility (MEU). We show that, while causal independence can considerably reduce the
complexity of belief updating, MAP and MEU, it may have no effect on MPE. For the first three tasks, we
present variable-elimination algorithm that exploit causal independence.

“”
(())
Abstract:
The paper, in anytime style. Empirical evaluation against competing algorithms such as iterative
belief propagation and Gibbs sampling demonstrates the potential of the MC approximation scheme for
several classes of problems.
Non-technical articles:
Presentations:

“Advances in Approximate and Hybrid Reasoning for Decision Making
Under Uncertainty”
(Dechter (Muri progress report, 2001))
Abstract:
Mini-clustering: a universal anytime approximation scheme. Applied to probabilistic inference
and to Optimization, decision making tasks
Hybrid processing of beliefs and constraints
REES: Reasoning Engine Evaluation Shell.
Online algorithms (S. Irani)
.
Contacts:
Nevin Lianwen Zhang
lzhang@cs.ust.hk
Department of Computer Science,
University of Science & Technology, Hong Kong
David Poole
poole@cs.ubc.ca
Department of Computer Science, University of British Columbia,
2366 Main Mall, Vancouver, B.C., Canada V6T 1Z4
Irina Rish and Rina Dechter
Department of Information and Computer Science
University of California, Irvine firinar,
dechterg@ics.uci.edu