Bayesian Network and Influence
Diagram
A Guide to Construction
And Analysis
Chapter 1
Introduction
• This chapter provides brief accounts on the
context of Probabilistic Networks, what they
are and when to use them
• The main topic covered are
– Expert Systems
– Rule-Based Systems
– Bayesian Networks
Expert Systems
• A system that is able to perform tasks that are
intellectually demanding often said to exhibit an
Expert System if the system’s problem solving
ability is restricted to particular area
• The techniques that enable us to construct
devices and services that are able to
– Perform reasoning and decision making under
uncertainty
– Acquire knowledge from data/experience
– Solve problems efficiently and respond to new
situation
Representation of Uncertainty
• Randomness and uncertain judgment is inherent in
most real world decision problems
• We need a method to automate reasoning and
decision making under uncertain statements and
conditions and method to combining the measures
such that reasoning and decision making under
uncertainty can be automated
• Probability theory is the prevailing method for dealing
with uncertainty and is the focus of this book
• Alternative method to probability theory are Belief
Theory and Fuzzy Methods
Normative Expert Systems
• Earlier the expert system is design to mimic
human behavior
• Today a model of the problem domain is
created instead of model of the expert
• Normative expert systems uses classical
probability calculus and decision theory as
their basis of reasoning and decision making
under uncertainty
Rule-Based Systems
• One of the earliest methods of knowledge
representation and manipulation was logical rules
of the form
• R1: if s1 then s2
– S2 can be concluded with certainty when s1 is
observed
• R2: if s2 then s3
– S3 can be concluded through forward chaining
involving R1 and R2 once s1 is known
• These rules are asymmetric
Causality
• Occurrence of some event c is known to cause
the effect e, relationship between c & e is
deterministic
• If c then e, rather then e then c
• We can conclude that rules like R1 and R2
express causal relationship
• Rule based system represent the problem
domain only up to some precision
• Smoking → bronchitis → dyspnoea
Causality Contd.
• R3: if smoking then bronchitis
• R4: if bronchitis then dyspnoea
• R4’: if dyspnoea then bronchitis
Uncertainty in Rule-Based Systems
• The vast majority of cause-effect mechanism
of interest in our attempts to model parts of
the world in the expert systems are uncertain
• A method of rule-based system with
uncertainty was developed in the 1970s
• Certainty Factor CF [-1,+1] indicates the
strength of the conclusion of the rule
whenever it’s condition is satisfied
Explaining Away
C1
E1
C2
E2
C1 can cause event E1 & E2, and C2 can cause event E2
Bayesian Networks
• Due to serious limitations in the rule-based
systems with certainty factor as a method for
knowledge representation, researchers turned
their attention towards probabilistic
interpretation of certainty factor, leading to
Bayesian Network
• Bayesian can be defined as acyclic directed graph
DAG which defines a factorization of a joint
probability distribution over the variables that are
represented by nodes of the DAG, factorization is
given by directed links of the DAG
Bayesian Networks Contd.
• A joint probability distribution over the set of
variables can be represented as
• P(U) = ∏ P(Ai | pa(Ai)),
Inference in Bayesian Networks
• Contrary to rule-base systems, inference in
Bayesian network is consistent
• Efficiency of inference is highly dependent on
structure of DAG
• Posterior Probability distribution P(X|Y=y)
– P(X|Y=y) = P(Y=y|X)P(X)
P(Y=y)
Construction of Bayesian Networks
• Bayesian networks can be describe in terms of a
qualitative component consisting of a DAG, and a
quantitative component, consisting of joint
probability distribution
• The construction of Bayesian networks runs in
two phases
– Identification of relevant variables and causal relation
among them
– The resulting DAG specifies a set of dependent and
independent assumptions that will be enforced on
joint probability distribution specifying a set of
conditional probability distributions
An Example
Spark-Plugs
Fuel
Fuel-gauge
Start?